Text classification model optimization method and device, text classification method and device
By mapping the text classification model to hyperbolic space and performing contrastive learning and Riemann optimization, the problem of insufficient learning of hierarchical information in Euclidean space is solved, and higher classification accuracy is achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- MASHANG CONSUMER FINANCE CO LTD
- Filing Date
- 2022-11-28
- Publication Date
- 2026-06-02
Smart Images
Figure CN116127063B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of neural network technology, specifically to an optimization method and apparatus for a text classification model, and a text classification method and apparatus. Background Technology
[0002] When archiving texts, they are usually categorized first. A single text may belong to multiple categories, and these categories may have a hierarchical structure. In related technologies, text classification models are commonly used for text classification.
[0003] Specifically, text classification models typically perform classification in Euclidean space. However, due to the poor distance preservation properties of Euclidean space, embedding hierarchical text labels into it can cause the distances between different hierarchical categories to become either too close or too far apart. This prevents the text classification model from effectively learning hierarchical structure information, leading to inaccurate classification. Therefore, improving the classification accuracy of text classification models is a pressing issue that needs to be addressed. Summary of the Invention
[0004] To address the aforementioned technical problems, this application is proposed. Embodiments of this application provide a method and apparatus for optimizing a text classification model, and a text classification method and apparatus.
[0005] In a first aspect, one embodiment of this application provides an optimization method for a text classification model. The method includes: performing hyperbolic space mapping processing on a first text classification model to obtain a hyperbolic classification model; determining the actual positive text and actual negative text in the labeled text based on the labeled text and the hierarchical text labels corresponding to the labeled text, wherein the hierarchical text labels represent the category hierarchy of the categories contained in the labeled text in a preset text category structure; performing comparative learning processing on the hyperbolic classification model based on the labeled text, the actual positive text, and the actual negative text to determine the loss data of the hyperbolic classification model; and performing Riemann optimization processing on the hyperbolic classification model using the loss data of the hyperbolic classification model to obtain a second text classification model.
[0006] Secondly, one embodiment of this application provides an optimization apparatus for a text classification model. The apparatus includes: a first processing module for performing hyperbolic space mapping processing on a first text classification model to obtain a hyperbolic classification model; a first determining module for determining the actual positive text and actual negative text in the labeled text based on the labeled text and the hierarchical text labels corresponding to the labeled text, wherein the hierarchical text labels represent the category hierarchy of the categories contained in the labeled text in a preset text category structure; a second determining module for performing comparative learning processing on the hyperbolic classification model based on the labeled text, the actual positive text, and the actual negative text to determine the loss data of the hyperbolic classification model; and a second processing module for performing Riemann optimization processing on the hyperbolic classification model using the loss data of the hyperbolic classification model to obtain a second text classification model.
[0007] Thirdly, one embodiment of this application provides a text classification method, which includes: in response to a text classification request, obtaining unlabeled text and a text classification model, wherein the text classification model is an optimized text classification model using the method described in the first aspect above; and processing the unlabeled text using the text classification model to obtain a classification result for the unlabeled text.
[0008] Fourthly, one embodiment of this application provides a text classification apparatus, which includes: an acquisition module, configured to acquire unlabeled text and a text classification model in response to a text classification request, wherein the text classification model is an optimized text classification model using the method described in the third aspect above; and a processing module, configured to process the unlabeled text using the text classification model to obtain a classification result for the unlabeled text.
[0009] Fifthly, one embodiment of this application provides a computer-readable storage medium storing a computer program for performing the methods described in the first and third aspects.
[0010] In a sixth aspect, one embodiment of this application provides an electronic device comprising: a processor; a memory for storing processor-executable instructions; and the processor for performing the methods described in the first and third aspects.
[0011] The text classification model optimization method provided in this application determines the actual positive and negative text in the labeled text based on the labeled text and its corresponding hierarchical text labels. Based on the labeled text, actual positive text, and actual negative text, a contrastive learning process is performed on the hyperbolic classification model to determine its loss data. The loss data of the hyperbolic classification model is then used to perform Riemann optimization on the hyperbolic classification model to obtain a second text classification model. Therefore, this application embodiment improves the classification accuracy of the final second text classification model by mapping a first text classification model to a hyperbolic space and optimizing the first text classification model using the labeled text, actual positive text, and actual negative text for contrastive learning. Attached Figure Description
[0012] The above and other objects, features, and advantages of this application will become more apparent from the more detailed description of the embodiments of this application in conjunction with the accompanying drawings. The drawings are provided to further illustrate the embodiments of this application and form part of the specification. They are used together with the embodiments of this application to explain this application and do not constitute a limitation thereof. In the drawings, the same reference numerals generally represent the same components or steps.
[0013] Figure 1 The diagram shown is a scenario applicable to an embodiment of this application.
[0014] Figure 2 The diagram shown is a flowchart illustrating an optimization method for a text classification model provided in an exemplary embodiment of this application.
[0015] Figure 3 The diagram shown is a schematic representation of an exemplary embodiment of this application, illustrating the process of determining the actual positive text and the actual negative text in the annotation text based on the annotation text and the hierarchical text labels corresponding to the annotation text.
[0016] Figure 4 The diagram shown is a schematic representation of an exemplary embodiment of this application, illustrating the process of determining the actual positive text and the actual negative text in the labeled text based on the feature vector corresponding to the hierarchical text label and the encoding vector of the labeled text.
[0017] Figure 5a The diagram shows a flowchart illustrating how a hyperbolic classification model is subjected to comparative learning processing based on labeled text, actual positive text, and actual negative text to determine the loss data of the hyperbolic classification model, according to an exemplary embodiment of this application.
[0018] Figure 5b The diagram shown is a flowchart of the comparative learning process.
[0019] Figure 6The diagram shown is a schematic representation of an exemplary embodiment of this application, illustrating the process of performing Riemann optimization on a hyperbolic classification model using loss data from the hyperbolic classification model to obtain a second text classification model.
[0020] Figure 7 The diagram shown is a flowchart illustrating a text classification method provided in an exemplary embodiment of this application.
[0021] Figure 8 The diagram shown is a flow structure schematic of an optimization device for a text classification model provided in an exemplary embodiment of this application.
[0022] Figure 9 The diagram shown is a flow structure schematic of a text classification device provided in an exemplary embodiment of this application.
[0023] Figure 10 The diagram shown is a structural schematic of an electronic device provided in an embodiment of this application. Detailed Implementation
[0024] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.
[0025] For ease of understanding, the following explanations are provided for the keywords appearing in this solution.
[0026] Hyperbolic space: Hyperbolic space is a space with constant curvature, negative curvature, infinite parallel lines, and triangles with a sum of degrees less than 180 degrees. It has high distance preservation, which means that the distances between nodes in hyperbolic space are similar. The higher the distance preservation, the more similar the distances between nodes.
[0027] Hyperbolic geometry refers to geometric operations in hyperbolic space, which differ from geometric operations in Euclidean space. Through certain formula transformations, geometric operations in Euclidean space can be converted to geometric operations in hyperbolic space.
[0028] Normalized Temperature-scaled Cross Entropy Loss (NT-Xentloss): It mainly calculates the similarity between positive samples divided by the similarity between negative samples, so that positive samples are more similar to themselves and negative samples are less similar to themselves.
[0029] Hierarchical text classification: Hierarchical text classification is a challenging task in multi-label classification. In common text classification, categories are usually orthogonal, meaning there is no inclusion relationship. Hierarchical classification, however, is a special type of text classification task where categories have a hierarchical structure, which can generally be represented as a tree or an undirected graph.
[0030] Riemann optimization: used to solve a class of structured constraint optimization problems.
[0031] The embodiments of this application can be applied to hierarchical text classification, in which there is a hierarchical structure relationship between categories. A text tag may be included in two different categories simultaneously, and in more complex cases, a text may be included in even more different categories. In related technologies, classification models for hierarchical text classification generally perform classification in Euclidean space. The hierarchical text tags representing categories are usually implemented as tree diagrams or undirected graphs. Taking a tree diagram as an example, embedding hierarchical text tags into Euclidean space usually causes distortion, resulting in poor distance preservation. That is, the distance between different hierarchical categories will become closer or farther apart, leading to the classification of different categories together or the same category into different categories. This makes it difficult for the text classification model to effectively learn hierarchical structure information, resulting in inaccurate classification.
[0032] To improve the classification accuracy of text classification models, the inventors conducted a series of studies and proposed an optimization method for text classification models. Specifically, a hyperbolic space mapping process is applied to the first text classification model to obtain a hyperbolic classification model. Based on the labeled text and its corresponding hierarchical text labels, the actual positive and negative texts in the labeled text are determined. Based on the labeled text, actual positive text, and actual negative text, a contrastive learning process is performed on the hyperbolic classification model to determine its loss data. The loss data of the hyperbolic classification model is then used to perform Riemann optimization on the hyperbolic classification model to obtain a second text classification model. By mapping the first text classification model to hyperbolic space to obtain the hyperbolic classification model, and by performing a contrastive learning process on the hyperbolic classification model using labeled text, actual positive text, and actual negative text, the first text classification model is optimized, thereby improving its classification accuracy.
[0033] Figure 1 The diagram shown illustrates a scenario applicable to an embodiment of this application. The scenario includes a text classification model 101, a text classification model optimization device 102, an optimized text classification model 103, unlabeled text 104, and the classification result of the unlabeled text 105.
[0034] For example, text classification model 101 is used to classify unlabeled text 104.
[0035] For example, the specific optimization method of the text classification model optimization device 102 for the text classification model 101 will be described in detail in the following embodiments, and will not be repeated here.
[0036] In practical applications, after the text classification model 101 is optimized by the text classification model optimization device 102, the optimized text classification model 103 is obtained. The optimized text classification model 103 is used to classify the unlabeled text 104 to obtain the classification result 105 of the unlabeled text, thereby ultimately improving the accuracy of the classification result corresponding to the unlabeled text.
[0037] Figure 2 The diagram shown is a flowchart illustrating an optimization method for a text classification model provided in an exemplary embodiment of this application. Figure 2 As shown, the optimization method for the text classification model provided in this application embodiment may include the following steps.
[0038] Step S201: Perform hyperbolic space mapping on the first text classification model to obtain a hyperbolic classification model.
[0039] For example, the first text classification model is used for hierarchical text classification. By performing hyperbolic space mapping on the first text classification model, it can be converted into a hyperbolic classification model. Then, by utilizing the high distance preservation property of hyperbolic space, the hyperbolic classification model also has high distance preservation property, thus using high distance preservation property as the basis for optimizing the first text classification model.
[0040] Step S202: Based on the annotation text and the corresponding hierarchical text labels, determine the actual positive text and the actual negative text in the annotation text.
[0041] For example, hierarchical text labels represent the category hierarchy of the categories contained in the labeled text within a preset text category structure. This preset text category structure can be a tree diagram, an undirected graph, etc., and the category hierarchy in the hierarchical text labels is the accurate category hierarchy.
[0042] For example, the labeled text can be manually annotated by staff or roughly annotated using a model with annotation capabilities.
[0043] For example, the actual positive text is the text or sentence in the annotation text that is strongly related to the corresponding category label. The actual negative text is the annotation text in the annotation text that is not the actual positive text.
[0044] Step S203: Based on the labeled text, actual positive text, and actual negative text, perform comparative learning processing on the hyperbolic classification model to determine the loss data of the hyperbolic classification model.
[0045] Step S204: Use the loss data of the hyperbolic classification model to perform Riemann optimization on the hyperbolic classification model to obtain the second text classification model.
[0046] For example, the loss data of the hyperbolic classification model is obtained by using normalized temperature cross loss, which represents the ratio of the similarity between the labeled text and the actual positive text to the similarity between the labeled text and the actual negative text. The closer the ratio is to infinity, the higher the classification accuracy.
[0047] For example, Riemann optimization can restore the first text classification model (i.e., the second text classification model) that has been mapped and optimized in hyperbolic space to the space before hyperbolic space mapping, thereby ensuring that the structure of the second text classification model is the same as that of the first text classification model. This avoids the problem that the second text classification model cannot be compatible with the text content and cannot be classified when using it for text classification.
[0048] Specifically, the hyperbolic classification model is continuously adjusted through comparative learning, thereby changing its loss data. When the loss data of the hyperbolic classification model reaches a preset loss threshold, it is considered to have met the optimization requirements. Riemann optimization is then used to transform the structure of the hyperbolic classification model and update the parameters of the first text classification model after comparative learning, resulting in the second text classification model, which is the optimized first text classification model.
[0049] In this embodiment, a hyperbolic classification model is obtained by mapping the first text classification model to a hyperbolic space, thereby enabling the hyperbolic classification model to have high distance preservation. Based on the high distance preservation, the loss data of the hyperbolic classification model is adjusted by comparing and learning with labeled text, actual positive text and actual negative text. Then, Riemann optimization is performed on the hyperbolic classification model to complete the optimization of the first text classification model, thereby improving the classification accuracy of the first text classification model.
[0050] Figure 3 The diagram illustrates a process for determining the actual positive and negative text within an annotation text based on the annotation text and its corresponding hierarchical text labels, according to an exemplary embodiment of this application. Figure 2 Extending from the basis Figure 3 The embodiments shown are described in detail below. Figure 3 The illustrated embodiments and Figure 2 The differences between the embodiments shown are not repeated here, and the similarities are not repeated here.
[0051] like Figure 3 As shown, determining the actual positive and negative text in the annotation text based on the annotation text and the corresponding hierarchical text labels can include the following steps.
[0052] Step S301: Perform hyperbolic space mapping on the labeled text to obtain the encoding vector of the labeled text.
[0053] For example, performing hyperbolic space mapping on the labeled text to obtain the encoded vector of the labeled text may include performing hyperbolic space mapping on the category labels in the labeled text to obtain the encoded vector of the labeled text.
[0054] Step S302: Extract the category hierarchy of the hierarchical text labels in the hyperbolic space to obtain the feature vectors corresponding to the hierarchical text labels.
[0055] Specifically, extracting the category hierarchy of hierarchical text labels in hyperbolic space and obtaining the feature vectors corresponding to the hierarchical text labels may include the following steps.
[0056] Step A: Perform hyperbolic space mapping on the graph convolution model to obtain a hyperbolic graph convolution model.
[0057] For example, since the first text classification model has been transformed into a hyperbolic classification model through hyperbolic space mapping, while the space containing the labeled text remains the same as the space before the hyperbolic space mapping of the first text classification model, it is necessary to also map the labeled text to hyperbolic space to avoid incompatibility issues when inputting it into the hyperbolic classification model, thereby obtaining the encoded vector of the labeled text. Similarly, the graph convolution model also needs to be mapped to hyperbolic space to obtain a hyperbolic graph convolution model.
[0058] Step B: Use the hyperbolic graph convolution model to determine the feature vectors corresponding to the hierarchical text labels.
[0059] For example, determining the feature vector corresponding to the hierarchical text label using a hyperbolic graph convolution model may include determining the feature vector of each category label represented by the hierarchical text label using a hyperbolic graph convolution model.
[0060] Step S303: Determine the actual positive text and actual negative text in the labeled text based on the feature vectors corresponding to the hierarchical text labels and the encoding vectors of the labeled text.
[0061] For example, the feature vectors corresponding to the hierarchical text labels can be used to select the actual positive text that is strongly correlated with the labeled text in the encoding vector of the labeled text, as well as the actual negative text that is not in the labeled text.
[0062] In this embodiment, based on the feature vector corresponding to the hierarchical text label, the actual positive text strongly correlated with the labeled text in the encoding vector of the labeled text, as well as the actual negative text other than the actual positive text in the labeled text, can be determined, thereby completing the classification of the actual positive text and the actual negative text, laying the foundation for subsequent calculation of loss data.
[0063] Figure 4 The diagram illustrates a process for determining the actual positive and negative text in the labeled text based on the feature vectors corresponding to the hierarchical text labels and the encoding vectors of the labeled text, according to an exemplary embodiment of this application. Figure 3 Extending from the basis Figure 4 The embodiments shown are described in detail below. Figure 4 The illustrated embodiments and Figure 3 The differences between the embodiments shown are not repeated here, and the similarities are not repeated here.
[0064] like Figure 4 As shown, determining the actual positive and negative text in the labeled text based on the feature vectors corresponding to the hierarchical text labels and the encoding vectors of the labeled text can include the following steps.
[0065] Step S401: Convert the format of the feature vector corresponding to the hierarchical text label. The format of the converted feature vector is the same as the encoding vector format of the labeled text.
[0066] For example, since the feature vector and the encoding vector are two vectors with different dimensions, it is necessary to convert the format of the feature vector corresponding to the hierarchical text label so that the format of the converted feature vector is the same as the format of the encoding vector of the labeled text. This avoids the problem of incompatibility between the feature vector and the encoding vector, as well as the problem of misclassification when the labeled text is classified as actual positive text or actual negative text. Of course, the format of the encoding vector of the labeled text can also be converted, and the format of the converted encoding vector of the labeled text is the same as that of the feature vector corresponding to the hierarchical text label.
[0067] Step S402: Based on the transformed feature vector and the encoding vector of the labeled text, use an attention mechanism to determine the actual main text in the labeled text.
[0068] Step S403: Filter the labeled text based on the actual positive text, and take the text in the labeled text other than the actual positive text as the actual negative text.
[0069] For example, the attention mechanism can select the information to focus on. It can select sentences or words that are strongly related to the labeled text as the information to focus on. After using the attention mechanism, the strongly related sentences in the labeled text can be filtered out as the actual positive text, and the rest can be used as the actual negative text.
[0070] In this embodiment, by transforming the feature vector into a vector with the same dimension as the encoding vector, the problem of misclassification caused by the incompatibility between the feature vector and the encoding vector is avoided, which leads to the classification of the labeled text as actual positive text or actual negative text. Furthermore, by using an attention mechanism to classify the labeled text, the accuracy of the classification of the labeled text is ensured.
[0071] Figure 5a The diagram illustrates a flowchart of an exemplary embodiment of this application, illustrating the process of performing comparative learning on a hyperbolic classification model based on labeled text, actual positive text, and actual negative text to determine the loss data of the hyperbolic classification model. Figure 2 Extending from the basis Figure 5a The embodiments shown are described in detail below. Figure 5a The illustrated embodiments and Figure 2 The differences between the embodiments shown are not repeated here, and the similarities are not repeated here.
[0072] like Figure 5a As shown, the hyperbolic classification model is subjected to comparative learning processing based on labeled text, actual positive text, and actual negative text to determine the loss data of the hyperbolic classification model, which may include the following steps.
[0073] Step S501: Perform hyperbolic space mapping on the actual text to obtain the encoding vector of the actual text.
[0074] Step S502: Perform hyperbolic space mapping on the actual negative text to obtain the encoding vector of the actual negative text.
[0075] For example, since the first text classification model has been transformed into a hyperbolic classification model through hyperbolic space mapping, while the space containing the labeled text remains the same as the space before the hyperbolic space mapping of the first text classification model, in order to avoid incompatibility issues when inputting labeled text into the hyperbolic classification model, it is necessary to also map the actual positive text to hyperbolic space to obtain the encoding vector of the actual positive text. Similarly, it is necessary to also map the actual negative text to hyperbolic space to obtain the encoding vector of the actual negative text.
[0076] Step S503: Using the hyperbolic classification model, process the encoding vectors of the labeled text, the actual positive text, and the actual negative text respectively to obtain the feature vectors of the labeled text, the actual positive text, and the actual negative text.
[0077] For example, after the encoding vectors of the labeled text, the actual positive text, and the actual negative text are processed by the hyperbolic classification model, feature vectors of the labeled text, the actual positive text, and the actual negative text can be obtained. The loss data of the hyperbolic classification model can be calculated using the feature vectors of the labeled text, the actual positive text, and the actual negative text.
[0078] Step S504: Use the feature vectors of the labeled text and the feature vectors of the actual text to perform learning processing to obtain the learning results.
[0079] For example, the learning process calculates the similarity between the feature vectors of the labeled text and the feature vectors of the actual text, and can increase the similarity between the labeled text and the actual text.
[0080] Step S505: Compare and distinguish the feature vectors of the labeled text with the feature vectors of the actual negative text to obtain the distinction result.
[0081] For example, the distinguishing processing calculates the similarity between the feature vector of the labeled text and the feature vector of the actual negative text, and can reduce the similarity between the labeled text and the actual negative text.
[0082] Step S506: Determine the loss data for the hyperbolic classification model based on the learning results and the discrimination results.
[0083] Specifically, the learning and differentiation processing is the actual calculation method of normalized temperature cross loss. By increasing the similarity between the labeled text and the actual positive text and decreasing the similarity between the labeled text and the actual negative text, a comparative learning is performed to adjust the loss data of the hyperbolic classification model. Then, comparative learning is performed again until the loss data of the hyperbolic classification model meets the preset loss data threshold to complete the comparative learning.
[0084] To better deepen our understanding of contrastive learning processing, the following will combine... Figure 5b Please elaborate. Figure 5b The diagram shown is a flowchart of the comparative learning process.
[0085] In practical applications, hyperbolic space mapping can be performed on labeled text, actual positive text, and actual negative text. The resulting encoded vectors of labeled text, actual positive text, and actual negative text are then input into a hyperbolic classification model. After processing by the hyperbolic classification model, feature vectors of labeled text, actual positive text, and actual negative text are obtained. These feature vectors are then used to obtain the loss data of the hyperbolic classification model through normalized temperature cross loss, thus completing one comparative learning operation.
[0086] In this embodiment, by adjusting the hyperbolic classification model using loss data for the hyperbolic classification model and then performing comparative learning again, the loss data of the hyperbolic classification model can be adjusted to meet the preset loss data threshold, thereby achieving the optimization requirements of the text classification model and improving the accuracy of the optimized first text classification model.
[0087] Figure 6 The diagram shown is a schematic representation of an exemplary embodiment of this application, illustrating the process of performing Riemann optimization on a hyperbolic classification model using loss data from the hyperbolic classification model to obtain a second text classification model. Figure 2 Extending from the basis Figure 6 The embodiments shown are described in detail below. Figure 6 The illustrated embodiments and Figure 2 The differences between the embodiments shown are not repeated here, and the similarities are not repeated here.
[0088] like Figure 6 As shown, the second text classification model is obtained by performing Riemann optimization on the hyperbolic classification model using the loss data of the hyperbolic classification model, which may include the following steps.
[0089] Step S601: Use the loss data of the hyperbolic classification model to perform Riemann optimization on the hyperbolic classification model to obtain a preliminary classification model.
[0090] Step S602: Determine whether the loss data of the hyperbolic classification model meets the preset conditions.
[0091] For example, the loss data of the hyperbolic classification model is the ratio of the similarity between the labeled text and the actual positive text to the similarity between the labeled text and the actual negative text. The loss data of the hyperbolic classification model is used to determine whether it meets the preset conditions. If the loss data of the hyperbolic classification model is greater than or equal to the preset loss data, it is considered to meet the preset conditions. Meeting the preset conditions indicates that the optimization of the first text classification model can be completed, and the accuracy of the optimized first text classification model is sufficient for the task of labeling text.
[0092] For example, in actual application, if the judgment result of step S602 is yes, that is, the loss data of the hyperbolic classification model meets the preset conditions, then step S603 is executed; if the judgment result of step S602 is no, that is, the loss data of the hyperbolic classification model does not meet the preset conditions, then step S604 is executed.
[0093] Step S603: Use the pre-classification model as the second text classification model.
[0094] For example, if the loss data of the preparatory classification model already meets the preset conditions, then the preparatory classification model can be used as the second text classification model, that is, the optimized first text classification model.
[0095] Step S604: Based on the labeled text, actual positive text, and actual negative text, perform comparative learning processing on the pre-classification model to determine the loss data of the pre-classification model.
[0096] For example, if the loss data of the pre-classification model does not meet the preset conditions, the pre-classification model needs to be adjusted and compared again to adjust the loss data of the pre-classification model until the loss data of the pre-classification model is greater than or equal to the preset loss data threshold.
[0097] Step S605: Use the loss data of the pre-classification model to perform Riemann optimization on the pre-classification model to obtain the second text classification model.
[0098] For example, by using Riemann optimization, the structure of the hyperbolic classification model is transformed and the parameters of the preparatory classification model after contrastive learning are updated to obtain the second text classification model, which is the optimized first text classification model.
[0099] In this embodiment, the preliminary classification model that does not meet the preset loss data threshold is adjusted and continuously compared and learned until the preset loss data threshold is met, thereby satisfying the conditions for optimization completion, so that the obtained second text classification model has higher classification accuracy.
[0100] Figure 7 The diagram shown is a flowchart illustrating a text classification method provided in an exemplary embodiment of this application.
[0101] like Figure 7 As shown, the text classification method provided in this application embodiment may include the following steps.
[0102] Step S701: In response to the text classification request, obtain the unlabeled text and the text classification model.
[0103] For example, the text classification model utilizes the above... Figures 2 to 6 The text classification model optimized by the method of any of the mentioned embodiments.
[0104] Step S702: Use a text classification model to process the unlabeled text and obtain the classification results of the unlabeled text.
[0105] For example, the classification result of unlabeled text is the category label corresponding to each sentence or word in the unlabeled text.
[0106] In this embodiment of the application, by utilizing an optimized text classification model, unlabeled text is classified, making the category labels corresponding to each sentence or word in the unlabeled text more accurate.
[0107] The above text combined Figures 2 to 6 The following describes in detail the implementation of the optimization method for the text classification model of this application, in conjunction with... Figure 8 This application describes in detail an embodiment of the optimization apparatus for a text classification model. It should be understood that the description of the method embodiment for optimizing the text classification model corresponds to the description of the apparatus embodiment; therefore, any parts not described in detail can be found in the preceding method embodiments.
[0108] Figure 8 The diagram shown is a flow structure schematic of an optimization device for a text classification model provided in an exemplary embodiment of this application. Figure 8 As shown, the optimization device for the text classification model provided in this application embodiment may include the following modules.
[0109] The first processing module 801 is used to perform hyperbolic space mapping processing on the first text classification model to obtain a hyperbolic classification model.
[0110] The first determining module 802 is used to determine the actual positive text and the actual negative text in the annotation text based on the annotation text and the hierarchical text labels corresponding to the annotation text.
[0111] For example, hierarchical text labels represent the category hierarchy of categories contained in manually annotated text within a predefined text category structure.
[0112] The second determination module 803 is used to perform comparative learning processing on the hyperbolic classification model based on labeled text, actual positive text, and actual negative text to determine the loss data of the hyperbolic classification model.
[0113] The second processing module 804 is used to perform Riemann optimization on the hyperbolic classification model using the loss data of the hyperbolic classification model to obtain a second text classification model.
[0114] In one embodiment of this application, the first determining module 802 is further configured to: perform hyperbolic space mapping processing on the labeled text to obtain the encoding vector of the labeled text; extract the category hierarchy of the hierarchical text labels in the hyperbolic space to obtain the feature vector corresponding to the hierarchical text labels; and determine the actual positive text and the actual negative text in the labeled text based on the feature vector corresponding to the hierarchical text labels and the encoding vector of the labeled text.
[0115] In one embodiment of this application, the first determining module 802 is further configured to: convert the format of the feature vector corresponding to the hierarchical text label, wherein the format of the converted feature vector is the same as the encoding vector format of the labeled text; determine the actual positive text in the labeled text using an attention mechanism based on the converted feature vector and the encoding vector of the labeled text; and filter the labeled text based on the actual positive text, taking the text other than the actual positive text in the labeled text as the actual negative text.
[0116] In one embodiment of this application, the second determining module 803 is further configured to: perform hyperbolic space mapping processing on the positively labeled text to obtain the encoding vector of the positively labeled text; perform hyperbolic space mapping processing on the negatively labeled text to obtain the encoding vector of the negatively labeled text; use a hyperbolic classification model to process the encoding vectors of the manually labeled text, the positively labeled text, and the negatively labeled text respectively to obtain the feature vectors of the manually labeled text, the positively labeled text, and the negatively labeled text; perform learning processing using the feature vectors of the manually labeled text and the positively labeled text to obtain a learning result, the learning processing being used to increase the similarity between the manually labeled text and the positively labeled text; perform comparison and differentiation processing using the feature vectors of the manually labeled text and the negatively labeled text to obtain a differentiation result, the differentiation processing being used to reduce the similarity between the manually labeled text and the negatively labeled text. Based on the learning result and the differentiation result, the loss data of the hyperbolic classification model is determined.
[0117] In one embodiment of this application, the second processing module 804 is further configured to: perform Riemann optimization on the hyperbolic classification model using the loss data of the hyperbolic classification model to obtain a preliminary classification model; if the loss data of the hyperbolic classification model meets preset conditions, use the preliminary classification model as a second text classification model; if the loss data of the hyperbolic classification model does not meet preset conditions, perform comparative learning on the preliminary classification model based on labeled text, actual positive text, and actual negative text to determine the loss data of the preliminary classification model; and perform Riemann optimization on the preliminary classification model using the loss data of the preliminary classification model to obtain a second text classification model.
[0118] It should be understood that Figure 8The operation and function of the first processing module 801, the first determining module 802, the second determining module 803, and the second processing module 804 in the optimization device of the provided text classification model can be referred to the above. Figures 2 to 6 To avoid repetition, the optimization methods for the provided text classification model will not be elaborated here.
[0119] The above text combined Figure 7 The text classification method embodiments of this application are described in detail below, in conjunction with... Figure 9 This application describes in detail the embodiments of the text classification apparatus. It should be understood that the descriptions of the text classification method embodiments correspond to the descriptions of the text classification apparatus embodiments; therefore, any parts not described in detail can be found in the preceding method embodiments.
[0120] Figure 9 The diagram shown is a schematic representation of the flow structure of a text classification apparatus provided in an exemplary embodiment of this application. Figure 9 As shown, the text classification device provided in this application embodiment may include the following modules.
[0121] The acquisition module 901 is used to acquire unlabeled text and a text classification model in response to a text classification request.
[0122] For example, the text classification model utilizes the above... Figure 8 The mentioned embodiment is an optimized text classification model.
[0123] Processing module 902 is used to process unlabeled text using a text classification model to obtain the classification results of the unlabeled text.
[0124] It should be understood that Figure 9 The operation and function of the acquisition module 901 and processing module 902 in the provided text classification device can be referred to the above. Figure 7 The provided text classification methods will not be elaborated upon here to avoid repetition.
[0125] Below, for reference Figure 10 This describes an electronic device according to embodiments of the present application. Figure 10 The diagram shown is a structural schematic of an electronic device provided in an exemplary embodiment of this application.
[0126] like Figure 10 As shown, the electronic device 100 includes one or more processors 1001 and memory 1002.
[0127] The processor 1001 may be a central processing unit (CPU) or other form of processing unit with data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device 100 to perform desired functions.
[0128] The memory 1002 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and / or cache memory. Non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 1001 may execute the program instructions to implement the methods of the various embodiments of this application described above and / or other desired functions. Various content, such as annotated text, actual positive text, and actual negative text, may also be stored in the computer-readable storage medium.
[0129] In one example, the electronic device 100 may also include an input device 1003 and an output device 1004, which are interconnected via a bus system and / or other forms of connection mechanism (not shown).
[0130] The input device 1003 may include, for example, a keyboard, a mouse, etc.
[0131] The output device 1004 can output various types of information to the outside, including labeled text, actual positive text, actual negative text, etc. The output device 1004 may include, for example, a display, a speaker, a printer, and a communication network and its connected remote output devices, etc.
[0132] Of course, for the sake of simplicity, Figure 10 Only some of the components of the electronic device 100 relevant to this application are shown in this illustration; components such as buses, input / output interfaces, etc., are omitted. In addition, the electronic device 100 may include any other suitable components depending on the specific application.
[0133] In addition to the methods and apparatus described above, embodiments of this application may also be computer program products, which include computer program instructions that, when executed by a processor, cause the processor to perform the steps of the methods according to the various embodiments of this application described above.
[0134] Computer program products can be written in any combination of one or more programming languages to perform the operations of the embodiments of this application. The programming languages include object-oriented programming languages such as Java and C++, as well as conventional procedural programming languages such as C or similar languages. The program code can be executed entirely on the user's computing device, partially on the user's computing device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.
[0135] Furthermore, embodiments of this application may also be computer-readable storage media storing computer program instructions thereon, which, when executed by a processor, cause the processor to perform the steps of the methods according to the various embodiments of this application described above.
[0136] Computer-readable storage media may take the form of any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may, for example, include, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatuses, or devices, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: electrical connections having one or more wires, portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.
[0137] The basic principles of this application have been described above with reference to specific embodiments. However, it should be noted that the advantages, benefits, and effects mentioned in this application are merely examples and not limitations, and should not be considered as essential features of each embodiment of this application. Furthermore, the specific details disclosed above are for illustrative and facilitative purposes only, and are not limitations. These details do not limit the application to the necessity of employing the aforementioned specific details for implementation.
[0138] The block diagrams of devices, apparatuses, devices, and systems involved in this application are merely illustrative examples and are not intended to require or imply that they must be connected, arranged, or configured in the manner shown in the block diagrams. As those skilled in the art will recognize, these devices, apparatuses, devices, and systems can be connected, arranged, and configured in any manner. Words such as “comprising,” “including,” “having,” etc., are open-ended terms meaning “including but not limited to,” and are used interchangeably with them. The terms “or” and “and” as used herein refer to the terms “and / or,” and are used interchangeably with them unless the context clearly indicates otherwise. The term “such as” as used herein refers to the phrase “such as but not limited to,” and is used interchangeably with it.
[0139] It should also be noted that in the apparatus, equipment, and methods of this application, the components or steps can be disassembled and / or recombined. These disassemblies and / or recombinations should be considered as equivalent solutions of this application.
[0140] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use this application. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein can be applied to other aspects without departing from the scope of this application. Therefore, this application is not intended to be limited to the aspects shown herein, but rather to be accorded the widest scope consistent with the principles and novel features disclosed herein.
[0141] The above description has been given for purposes of illustration and description. Furthermore, this description is not intended to limit the embodiments of this application to the forms disclosed herein. Although numerous exemplary aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, alterations, additions, and sub-combinations thereof.
Claims
1. An optimization method for a text classification model, characterized in that, include: The first text classification model is subjected to hyperbolic space mapping to obtain a hyperbolic classification model. Based on the labeled text and the corresponding hierarchical text tags, the actual positive text and actual negative text in the labeled text are determined. The hierarchical text tags represent the category hierarchy of the categories contained in the labeled text in the preset text category structure. Based on the encoding vectors of the labeled text, the actual positive text, and the actual negative text, a contrastive learning process is performed using the hyperbolic classification model to determine the loss data of the hyperbolic classification model. Each encoding vector is obtained by performing hyperbolic space mapping on the labeled text, the actual positive text, and the actual negative text respectively. The contrastive learning is used to increase the similarity between the labeled text and the actual positive text, and to decrease the similarity between the labeled text and the actual negative text. The hyperbolic classification model is subjected to Riemann optimization using the loss data of the hyperbolic classification model to obtain a second text classification model.
2. The method according to claim 1, characterized in that, The step of determining the actual positive text and actual negative text in the labeled text based on the labeled text and the corresponding hierarchical text tags includes: Extract the category hierarchy of the hierarchical text labels in hyperbolic space to obtain the feature vector corresponding to the hierarchical text labels; Based on the feature vectors corresponding to the hierarchical text labels and the encoding vectors of the labeled text, the actual positive text and the actual negative text in the labeled text are determined.
3. The method according to claim 2, characterized in that, Determining the actual positive text and the actual negative text in the labeled text based on the feature vector corresponding to the hierarchical text label and the encoding vector of the labeled text includes: The feature vectors corresponding to the hierarchical text labels are formatted and converted so that the format of the converted feature vectors is the same as that of the encoding vectors of the labeled text. Based on the transformed feature vector and the encoding vector of the labeled text, the actual text in the labeled text is determined using an attention mechanism; The labeled text is filtered based on the actual positive text, and the text in the labeled text other than the actual positive text is taken as the actual negative text.
4. The method according to claim 1, characterized in that, The hyperbolic classification model is subjected to comparative learning processing based on the encoded vector of the labeled text, the encoded vector of the actual positive text, and the encoded vector of the actual negative text to determine the loss data of the hyperbolic classification model, including: Using the hyperbolic classification model, the encoding vectors of the labeled text, the actual positive text, and the actual negative text are processed respectively to obtain the feature vectors of the labeled text, the actual positive text, and the actual negative text. The feature vectors of the labeled text and the actual positive text are then used for learning to obtain a learning result. The learning process is used to increase the similarity between the labeled text and the actual positive text. The feature vectors of the labeled text and the feature vectors of the actual negative text are compared and distinguished to obtain a distinction result. The distinction process is used to reduce the similarity between the labeled text and the actual negative text. The loss data for the hyperbolic classification model is determined based on the learning results and the discrimination results.
5. The method according to claim 1, characterized in that, The step of performing Riemann optimization on the hyperbolic classification model using the loss data of the hyperbolic classification model to obtain a second text classification model includes: The hyperbolic classification model is subjected to Riemann optimization using the loss data of the hyperbolic classification model to obtain a preliminary classification model; If the loss data of the hyperbolic classification model meets the preset conditions, the pre-classification model is used as the second text classification model; If the loss data of the hyperbolic classification model does not meet the preset conditions, the pre-classification model is subjected to comparative learning processing based on the labeled text, the actual positive text, and the actual negative text to determine the loss data of the pre-classification model. The Riemann optimization process is performed on the pre-classification model using the loss data of the pre-classification model to obtain the second text classification model.
6. A text classification method, characterized in that, include: In response to a text classification request, unlabeled text and a text classification model are obtained, wherein the text classification model is an optimized text classification model using the method described in any one of claims 1 to 5; The unlabeled text is processed using the text classification model to obtain the classification result of the unlabeled text.
7. An optimization device for a text classification model, characterized in that, include: The first processing module is used to perform hyperbolic space mapping processing on the first text classification model to obtain a hyperbolic classification model. The first determining module is used to determine the actual positive text and the actual negative text in the labeled text based on the labeled text and the hierarchical text labels corresponding to the labeled text. The hierarchical text labels represent the category hierarchy of the categories contained in the labeled text in the preset text category structure. The second determining module is used to determine the loss data of the hyperbolic classification model by performing contrastive learning processing on the hyperbolic classification model based on the encoding vector of the labeled text, the encoding vector of the actual positive text, and the encoding vector of the actual negative text; wherein each encoding vector is obtained by performing hyperbolic space mapping processing on the labeled text, the actual positive text, and the actual negative text respectively; the contrastive learning is used to increase the similarity between the labeled text and the actual positive text and decrease the similarity between the labeled text and the actual negative text; The second processing module is used to perform Riemann optimization on the hyperbolic classification model using the loss data of the hyperbolic classification model to obtain a second text classification model.
8. A text classification device, characterized in that, include: The acquisition module is used to acquire unlabeled text and a text classification model in response to a text classification request, wherein the text classification model is a text classification model optimized using the method described in any one of claims 1 to 5; The processing module is used to process the unlabeled text using the text classification model to obtain the classification result of the unlabeled text.
9. An electronic device, characterized in that, include: processor; Memory used to store the processor's executable instructions; The processor is configured to perform the method described in any one of claims 1 to 6.
10. A computer-readable storage medium, characterized in that, The storage medium stores a computer program for performing the method described in any one of claims 1 to 6.