A hierarchical text classification method based on label geometry construction and alignment

Abstract

This invention belongs to the field of hierarchical text classification and discloses a hierarchical text classification method based on label geometry construction and alignment. Based on the label set and label hierarchy structure, the embeddings of each label are obtained. Global orthogonalization is performed on the embeddings of the head labels, and recursive local orthogonalization is performed on the embeddings of parent and child labels. The orthogonalized embeddings are used as initial label embeddings. A hierarchical awareness cue fine-tuning module is trained based on the initial label embeddings and text samples. After training, the output serves as a multi-task classification model. During training, a hierarchical geometry regularization loss is calculated for hierarchical geometry alignment. The hierarchical geometry regularization loss includes a global orthogonal loss calculated based on the head label embeddings, and parent-child similarity loss, norm decay loss, and sibling residual orthogonal loss calculated based on the parent-child label embeddings. This invention can significantly improve text classification accuracy.

Description

Technical Field

[0001] This invention belongs to the field of hierarchical text classification, specifically relating to a hierarchical text classification method based on label geometry construction and alignment. Background Technology

[0002] Hierarchical text classification (HTC) is an important task aimed at assigning text to a predefined hierarchical structure of labels, and it is widely used in fields such as academic paper classification and product classification. However, real-world classification systems are often characterized by deep hierarchies, large scale, and complex parent-child and sibling relationships between labels, making accurate HTC a challenging task to achieve.

[0003] To address these challenges, researchers explored various methods for modeling label hierarchy structures. Among them, a cue-based fine-tuning approach reconstructs the classification problem into a cloze test by constructing hierarchical text templates, thus injecting the label hierarchy as prior knowledge into pre-trained language models (PLMs). Furthermore, a contrastive learning-based approach constructs positive and negative sample pairs at the instance or label level and uses the label hierarchy structure to determine the similarity between samples, thereby learning label embeddings and hierarchical relationships in the feature space.

[0004] However, when modeling the complex label hierarchy in HTC, the aforementioned methods often focus on implicit learning of label embeddings, neglecting the hierarchical geometric relationships between labels. This modeling approach results in generated representation vectors that fail to accurately reflect the true distribution of label embeddings in the feature space (i.e., label geometry); in long-tail distribution scenarios, it can even lead to label geometry distortion. This distortion manifests in two main ways: 1) Structural confusion: The feature space distribution learned by the model fails to construct a topological relationship consistent with the label hierarchy structure. Semantically related labels (especially fine-grained sibling labels) are often mapped to positions that are too close together in the feature space, making them difficult to distinguish. 2) Representation collapse: Due to the limited training samples, tail labels often struggle to learn clear embedding representations. During optimization, the representations of these labels are easily absorbed by their corresponding parent or head labels, leading to a decrease in the discriminative power of tail labels. Although recent studies have attempted to improve the feature space distribution by introducing additional constraints, such as learnable decision boundaries or structural entropy, these constraints are either limited to local areas and highly dependent on data distribution, or, while global, lack hierarchical awareness. Since these constraints are all based on implicitly learned label embeddings, they cannot provide a hierarchical geometric structure, thus making it difficult to fundamentally solve the problems of structural confusion and representation collapse. Summary of the Invention

[0005] The purpose of this invention is to provide a hierarchical text classification method based on label geometry construction and alignment, which significantly improves the accuracy of text classification.

[0006] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0007] A hierarchical text classification method based on label geometry construction and alignment includes the following steps:

[0008] Hierarchical geometry construction: Based on the tag set and tag hierarchy, the embeddings of each tag are obtained, and global orthogonalization is performed on the embeddings of the header tag, and recursive local orthogonalization is performed on the embeddings of parent and child tags.

[0009] Multi-task training: The orthogonalized embedding of the labels is used as the initial label embedding. The hierarchical perception prompt fine-tuning module is trained based on the initial label embedding and text samples. After training, the output is used as the multi-task classification model. During training, the hierarchical geometric regularization loss is calculated for hierarchical geometric alignment. The hierarchical geometric regularization loss includes the global orthogonal loss calculated based on the embedding of the head label, and the parent-child similarity loss, norm decay loss and sibling residual orthogonal loss calculated based on the embedding of the parent and child labels.

[0010] Classification reasoning application: Input the text to be classified into a multi-task classification model to obtain hierarchical text classification results.

[0011] Several alternative methods are provided below, but they are not intended as additional limitations on the overall solution above. They are merely further additions or optimizations. Provided there are no technical or logical contradictions, each alternative method can be combined individually with respect to the overall solution above, or multiple alternative methods can be combined with each other.

[0012] Preferably, the step of obtaining the embedding of each tag based on the tag set and tag hierarchy includes:

[0013] The label adjacency matrix is ​​obtained from the directed acyclic graph corresponding to the label set;

[0014] A BERT word embedding layer is used to generate semantic embeddings for each tag;

[0015] The label adjacency matrix and the semantics of the labels are embedded into the input graph attention network to obtain the embeddings of each label.

[0016] Preferably, the global orthogonalization of the embedding of the header tags includes:

[0017] For the embedding of header labels, Schmidt orthogonalization is applied to obtain an orthonormal basis;

[0018] The orthonormal basis is rescaled back to the original norm corresponding to the embedding of the header label to obtain the orthogonalized embedding of the header label.

[0019] Preferably, the recursive local orthogonalization of the embedding of parent and child tags includes:

[0020] For a child tag in a parent-child tag pair, it is split into projection embedding and residual embedding along the direction of the parent tag;

[0021] Apply Schmidt orthogonalization to the residual embedding to obtain the orthonormal basis. Scale the orthonormal basis back to the original norm corresponding to the residual embedding and concatenate it with the projected embedding along the parent label direction to obtain the orthogonalized embedding of the child label.

[0022] Preferably, the global orthogonal loss is obtained by minimizing the cosine similarity between the embeddings of the head labels, as shown in the following formula:

[0023]

[0024] In the formula, Indicates global orthogonal loss. The number of permutations representing the number of header tags. , Represents a collection of header tags Size, Indicates the first A tag, Indicates the first A tag, Indicates the first step in the multi-task training process. Embedding of header tags Indicates the first step in the multi-task training process. Embedding of header tags Describes the Euclidean norm. It is a constant.

[0025] Preferably, the parent-child similarity loss is obtained by constraining the embedding distribution of child labels to a narrow conical region centered on the embedding of the parent label, as shown in the following formula:

[0026]

[0027] In the formula, Indicates the loss of similarity between father and son. Represents the set of all parent-child tag pairs Size, Indicates the first step in the multi-task training process. Embedding of parent tags Indicates the first step in the multi-task training process. The first parent tag Embedding of individual tags This represents the target angle that serves as the tolerance boundary.

[0028] Preferably, the norm decay loss applies a one-sided penalty using the ReLU activation function only when the norm of the child tag's embedding is lower than the decay ratio of the parent tag's embedding norm. The specific formula is as follows:

[0029]

[0030] In the formula, For norm decay loss, It is the ReLU activation function. The preset attenuation ratio, Indicates the first step in the multi-task training process. Embedding of parent tags Indicates the first step in the multi-task training process. The first parent tag Embedding of individual tags.

[0031] Preferably, the sibling residual orthogonal loss includes first subtracting the components parallel to the parent label's embedding from the child label's embedding to obtain the residual embedding representing the sibling label information; then, orthogonality constraints are applied only to the residual embeddings to obtain the sibling residual orthogonal loss, as shown in the following formula:

[0032]

[0033]

[0034] In the formula, Indicates the first step in the multi-task training process. The first parent tag Residual embedding of individual labels Indicates the first step in the multi-task training process. The first parent tag Embedding of individual tags This indicates an operation that embeds the input and projects it onto the corresponding parent label. This indicates the orthogonal loss of the brother's residual. This represents a set of parent tags that have at least two child tags. Size, Indicates the first A collection of child tags of a parent tag. Size, Indicates the first A tag, Indicates the first step in the multi-task training process. The first parent tag Residual embedding of individual tags.

[0035] This invention addresses the HTC task by providing a hierarchical text classification method based on label geometry structuring and aligning (LGSA). It proposes a two-stage framework for label geometry structuring and alignment, transforming the passively learned implicit prior label hierarchy into a clearly structured explicit embedding representation. In the model initialization phase, hierarchical geometric structuring (HGS) is proposed. Specifically, HGS first generates initial semantic embeddings for each label; then, by introducing a general orthogonal frame (GOF) and applying a Schmidt orthogonalization process, an explicit label geometry conforming to the label hierarchy structure is reconstructed in a top-down manner. This process provides a stable and structurally meaningful geometric position for each label in the feature space, and the orthogonally reconstructed label geometry serves as a label prototype to guide subsequent model training. In the model training phase, hierarchical geometric alignment (HGA) is proposed, aiming to guide the model to learn predefined label geometry. Unlike methods that only optimize global spectral properties (such as singular spectral smoothing), HGA enforces explicit structural alignment, ensuring that the learned embedding representations strictly follow the directional relationships in the hierarchical structure, rather than merely smoothing feature differences. Specifically, HGA aligns the learned label geometry with the explicit label prototypes by applying complementary geometric constraints as geometric regularization terms for label embeddings. In practical applications, global geometric constraints guarantee the geometric orthogonality between head labels, while a series of local geometric constraints continuously calibrate specific hierarchical topological relationships (such as parent-child and sibling relationships), effectively avoiding structural confusion and representation collapse, thus significantly improving classification accuracy. Attached Figure Description

[0036] Figure 1 This is a framework diagram of the hierarchical text classification method based on label geometry construction and alignment of the present invention;

[0037] Figure 2 This is a schematic diagram showing the performance comparison of the model at all layer depths in the NYT dataset during the experiments of this invention;

[0038] Figure 3 This is a schematic diagram showing the performance comparison of the model at all layer depths in the RCV1-V2 dataset during the experiments of this invention;

[0039] Figure 4 This is a schematic diagram comparing the label frequencies of the model at all levels of depth in the NYT dataset during the experiments of this invention;

[0040] Figure 5 This is a schematic diagram comparing the label frequencies of the model at all layer depths in the RCV1-V2 dataset during the experiment of this invention. Detailed Implementation

[0041] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0042] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein in the description of the invention is for the purpose of describing particular embodiments only and is not intended to limit the invention.

[0043] Existing hierarchical text classification methods typically employ cue-based fine-tuning or contrastive learning, injecting the label hierarchy as prior knowledge into the model to implicitly learn label embeddings for classification. However, this implicit learning approach fails to model the hierarchical geometric relationships between labels, resulting in the model's inability to accurately reflect label geometric features (i.e., the spatial distribution of label embeddings in the feature space). To address this issue, this embodiment proposes a hierarchical text classification method based on label geometry construction and alignment, featuring a two-stage label geometry construction and alignment framework designed to transform the label hierarchy from implicit priors into explicit embeddings. First, hierarchical geometry construction is proposed, using a generalized orthogonal framework to reconstruct an explicit label geometry space that fits the label hierarchy. Subsequently, this label geometry is used as a label prototype to guide model training. To further enhance this process, hierarchical geometry alignment is proposed as a regularization term, aiming to align the label geometry learned by the model with the explicit label prototype.

[0044] In HTC tasks, tag sets Organized as a hierarchical structure and formally defined as a tree-like directed acyclic graph (DAG). Each node represents a label. The edges in the text represent parent-child relationships. Given the text... The goal of a classification task is to predict a subset of labels. The prediction results must satisfy the hierarchical consistency constraint, that is, if the prediction results contain the first... If a tag is defined, then all its ancestor tags must also be included. In this context, this constraint ensures that the final predicted set of labels corresponds to one or more valid paths from the root node to a specific node in the hierarchical structure. .

[0045] like Figure 1 As shown in this embodiment, a hierarchical text classification method based on label geometry construction and alignment is presented. LGSA first constructs a prior label prototype in the feature space using the HGS module; subsequently, the HGA module is used to align the learned label embeddings with this prior label prototype. The specific execution process is as follows:

[0046] (1) Initialization process: hierarchical geometry construction.

[0047] To address the two major challenges of structural confusion and representation collapse, label geometry must meet three key conditions: (1) Head labels need to be globally orthogonal to achieve clear geometric separability, thus laying the foundation for distinguishing different hierarchical branches; (2) Sibling labels need to maintain orthogonality in their unique semantic features to ensure accurate differentiation under the same parent label; (3) Parent and child labels need to maintain directional consistency in the feature space to accurately reflect hierarchical relationships. Based on these conditions, this embodiment proposes an HGS module for constructing a priori label prototypes during the model initialization phase. The geometry construction process includes the following steps:

[0048] Step 1: Tag Initialization. This embodiment first utilizes the BERT (as described in the reference "BERT: pre-training of deep bidirectional transformers for language understanding") word embedding layer to generate semantic embeddings for each tag. Secondly, the label hierarchy structure is integrated into the semantic embedding using a graph attention network (GAT) (described in the reference "Graph attention networks") to obtain the label embedding. The label initialization operation is shown in formulas (1) and (2).

[0049] (1)

[0050] (2)

[0051] in, For the first Semantic embedding of each label and These represent the word embedding operation of BERT and the graph encoding operation of GAT, respectively. The adjacency matrix represents the labels. Represents a set of tags Size.

[0052] Step 2: Global Orthogonalization. While the rich semantic information captured in Step 1 is indispensable, clear geometric separation between head labels is equally crucial for well-defined hierarchical branches. Therefore, this embodiment orthogonalizes the head label embeddings to enhance their independence in the feature space and establish clear boundaries for the main branches of the hierarchical structure. Specifically, given a set of head labels... and its embedding First, Schmidt orthogonalization is applied to obtain the standard orthogonal basis. Subsequently, to preserve the semantic strength represented by the original embeddings, these basis vectors are rescaled back to their original norms. The orthogonalization process of the head tag embeddings is shown in equations (3) and (4):

[0053] (3)

[0054] (4)

[0055] in, Indicates the first Embedding of header tags Indicates the first The orthonormal basis corresponding to each header label Indicates the first The orthonormal basis corresponding to each header label Represents the i-th after orthogonalization and norm rescaling. The tag prototype of each header tag (i.e., the orthogonalized embedding of header tags). This indicates the number of header tags in the hierarchical structure. Denotes the Euclidean (L2) norm. Represents the square of the Euclidean norm; This represents the Schmitt orthogonalization function.

[0056] Step 3: Recursive Local Orthogonalization. After completing the global orthogonalization of the header tags, this process propagates from top to bottom along the tag hierarchy, aiming to maintain semantic consistency between parent and child tags while decoupling sibling tags. For the... The parent tag, its first Sub-tag embedding is represented as It is decomposed into two semantically distinct components: the projection embedding along the parent tag direction. (representing inherited public semantics) and unique semantic features known as residual embeddings. Subsequently, regarding The Schmidt orthogonalization method (same as formulas (3) and (4) in step 2) is applied to ensure that sibling labels remain orthogonal within the subspace defined by the parent label. To prevent semantic loss, the norm of the original residual embedding is preserved during the construction process, so that each child label retains its independent geometric features while inheriting the semantics of the parent label. The calculation process is shown in formulas (5) to (7):

[0057] (5)

[0058] (6)

[0059] (7)

[0060] in, Indicates the first The first parent tag The projection of each child tag along the direction of the parent tag is embedded. and They represent the first and second digits obtained after orthogonalization, respectively. The parent tag and its first The tag prototypes of each sub-tag (i.e., the orthogonalized embedding of sub-tags), in the tag hierarchy structure, the top-level tag corresponds to... The value obtained in step 2 ; Indicates the first The first parent tag Residual embedding of sub-labels; function This indicates an operation that projects the input embedding onto its corresponding parent tag embedding.

[0061] (2) Multi-task training process.

[0062] This embodiment uses hierarchy-aware prompt tuning (HPT; derived from the paper "HPT: hierarchy-aware prompt tuning for hierarchical text classification") as the optimization base model. As a representative prompt-based tuning model, it reconstructs the classification task into a masked language modeling task, and the output after training serves as a multi-task classification model. Specifically, the input text is optimized by including soft cue words. and masked words The hierarchical perception template is formatted, where, , Representing the depth of the tag hierarchy, these tokens are first mapped to dense vectors through a BERT embedding layer, and then processed by the BERT encoder to generate context-sensitive embedding representations. The model is supervised by two objective functions: derived from... Classification loss ( ) and masked language modeling loss derived from input text ( ).

[0063] It should be noted that the formatting, embedding generation, and loss calculation are all involved. and All references to the aforementioned literature are provided, and this embodiment will not elaborate further. The tag prototypes obtained during the initialization process are used for tag embedding in the hierarchical awareness cue fine-tuning module's initialization. In subsequent iterations, the hierarchical awareness cue fine-tuning module iteratively updates the tag embeddings. Furthermore, to facilitate the embedding of symbols during the initialization phase... To distinguish them, this embodiment denotes the embedding of tags during the iteration process as symbols. .

[0064] Finally, multi-task training objectives Defined as and and hierarchical geometric regularization loss ( The weighted combination of ) is shown in formula (8).

[0065] (8)

[0066] in, Indicates control The hyperparameters of the weights. The hierarchical geometric regularization loss is used to achieve hierarchical geometric alignment, and its details are as follows:

[0067] To explicitly incorporate the constructed label prototypes into the representation learning process, this embodiment proposes an HGA module. Specifically, this module employs hierarchical geometric regularization loss. The loss consists of four specific geometric constraint losses that collectively regularize the label embeddings. Crucially, these constraints are organized in a complementary manner to handle different topological ranges. Global orthogonal loss. The orthogonality between header labels is enforced, thus achieving geometric separation and providing independent subspaces for local geometric constraints. Subsequently, these local geometric constraints recursively calibrate specific hierarchical topological relationships. Specifically, parent-child similarity loss... With norm decay loss They jointly maintain longitudinal consistency (including orientation and modulus), while sibling residuals exhibit orthogonal loss. This ensures the lateral discriminative power between sibling tags. Therefore, It is the sum of the above four types of losses, formally defined as shown in formula (9):

[0068] (9)

[0069] Global orthogonal loss To ensure geometric separation between header labels, the loss function enforces explicit orthogonality by minimizing the cosine similarity between label embeddings. Specifically, a squared penalty term is used to reduce any deviations from orthogonality (including positive and negative correlations), thereby ensuring that the main branches of the label hierarchy remain mathematically independent. Its formula is expressed as Equation (10):

[0070] (10)

[0071] in, Indicates global orthogonal loss. The number of permutations representing the number of header tags. , Represents a collection of header tags Size, Indicates the first One tag, Indicates the first One tag, Indicates the first step in the multi-task training process. Embedding of header tags Indicates the first step in the multi-task training process. Embedding of header tags Denotes the Euclidean norm. It is a very small constant used to prevent the denominator from being zero.

[0072] Father-son similarity loss The "is-a" relationship in the hierarchical structure geometrically requires consistency in direction to better capture the inherent semantic dependencies. To explicitly model this inheritance relationship during training, the loss constraint places the sub-label embeddings within a narrow conical region centered on their parent label embeddings. This spatial constraint establishes a geometric tolerance boundary for semantic evolution. Specifically, by applying a squared penalty to deviations beyond this boundary, it ensures that the sub-label embeddings closely follow the direction of their parent label embeddings, thus effectively modeling the parent-child label inheritance relationship in the geometric dimension. Its formula is expressed as Equation (11):

[0073] (11)

[0074] in, Indicates the loss of similarity between father and son. It is the set of all parent-child tag pairs. Represents the set of all parent-child tag pairs Size, Indicates the first step in the multi-task training process. Embedding of parent tags Indicates the first step in the multi-task training process. The first parent tag Embedding of individual tags This represents the target angle that serves as the tolerance boundary.

[0075] Norm decay loss To prevent the tail label embedding from losing its discriminative power due to excessively low norm caused by data imbalance, this embodiment introduces a norm decay constraint. Specifically, this paper uses the ReLU activation function to apply a one-sided penalty: the loss term only takes effect when the norm of the child label embedding is lower than the preset proportion of its parent label embedding. This mechanism explicitly prevents semantic degradation (i.e., norm vanishing). Its formula is expressed as shown in formula (12):

[0076] (12)

[0077] in, For norm decay loss, It is the ReLU activation function. The preset attenuation ratio, Indicates the first step in the multi-task training process. Embedding of parent tags Indicates the first step in the multi-task training process. The first parent tag Embedding of individual tags.

[0078] Brother residual orthogonal loss Although sibling labels share the same parent label, their effective differentiation depends on achieving clear geometric separation. Therefore, to mitigate the confusion caused by the dense clustering of tail labels in the feature space, this loss requires that their residual embeddings remain orthogonal, thereby ensuring that each label has independent and distinguishable features. Specifically, the component parallel to the parent label embedding is first subtracted from the child label embedding to obtain the residual embedding representing the information unique to the sibling label. Subsequently, orthogonality constraints are enforced only on these residual embeddings, thereby maximizing the discriminative power between labels without compromising vertical consistency (i.e., alignment of parent and child directions). The formula is expressed as shown in formulas (13) to (14):

[0079] (13)

[0080] (14)

[0081] in, Indicates the first step in the multi-task training process. The first parent tag Residual embedding of individual labels Indicates the first step in the multi-task training process. The first parent tag Embedding of individual tags This indicates an operation that embeds the input and projects it onto the corresponding parent label. This indicates the orthogonal loss of the brother's residual. This represents a set of parent tags that have at least two child tags. Size, Indicates the first A collection of child tags of a parent tag. Size, Indicates the first A tag, Indicates the first step in the multi-task training process. The first parent tag Residual embedding of individual tags.

[0082] (3) Classification Reasoning Application: Input the text to be classified into the multi-task classification model to obtain hierarchical text classification results. The complete workflow is as follows:

[0083] 1. First, obtain the text to be classified, and according to the maximum hierarchical depth of the target label system, dynamically concatenate a specific number of virtual prompt words and placeholders specifically used for classification at the end of the text sequence to form a complete input sequence.

[0084] 2. The constructed input sequence is fed into the pre-trained language model (i.e., a multi-task classification model) that incorporates a graph encoder for forward propagation. During this process, the underlying network not only extracts deep contextual semantic features of the text but also effectively utilizes the label hierarchy topology and orthogonalized semantic representation learned during training through the graph encoder. Finally, the model generates a predicted representation of the entire sequence in the feature space through its language model output.

[0085] 3. During the inference generation stage, the model accurately locates and extracts feature vectors corresponding to the classification placeholder positions. Then, the dimensions of these features are mapped and truncated to the total number of target labels, and pre-trained multi-class bias parameters are superimposed to calculate the final prediction score of the current text for all candidate labels.

[0086] 4. Utilizing the pre-established mapping relationship between hierarchical depth and corresponding labels, the predicted scores are structurally reorganized. The decoding mechanism traverses all valid candidate labels layer by layer downwards according to the tree-like hierarchical depth: when the predicted score of a certain label is determined to be greater than a preset threshold, the model triggers a determination, considering that the current input text contains that label category. Through this top-down, layer-by-layer determination, a complete set of multi-label prediction results that conforms to hierarchical logical relationships is finally aggregated and output.

[0087] To verify the effectiveness of the method of the present invention, the following experiments were conducted in this embodiment:

[0088] I. Experimental setup.

[0089] Datasets and Evaluation Metrics. To verify the effectiveness of the proposed LGSA framework, experiments were conducted on three datasets: Web-of-Science (WOS), RCV1-V2, and NYT. To ensure fair comparison, the data preprocessing techniques described in the papers "Hierarchy-aware label semantics matching network for hierarchical text classification" and "HPT: hierarchy-aware prompt tuning for hierarchical text classification" were used to clean the data and divide it into training, validation, and test sets. Table 1 shows the statistical results for these three datasets. Indicates the total number of tags. This represents the average number of labels assigned to each sample. Following established evaluation conventions, this experiment selects two standard metrics, Macro-F1 and Micro-F1, to evaluate model performance.

[0090] Table 1: Statistical information of three HTC datasets

[0091]

[0092] Implementation details. This experiment uses the HPT model as the base model for the proposed LGSA framework. The batch size for model training is set to 16, and the Adam optimizer is used with a learning rate of 1e-4. To ensure the stability of geometric regularization, the hyperparameters are adjusted accordingly. A warm-up strategy was adopted, in which the parameter was linearly increased from zero to a preset value during the first 10% of training steps. The specific hyperparameter experimental settings are shown in Table 2.

[0093] Table 2: Hyperparameter Settings

[0094]

[0095] Baseline Models. To evaluate the performance of LGSA, this experiment compares it with the following state-of-the-art baseline models: (1) Hierarchy-aware models: HiAGM (hierarchy-aware global model), HTCInfoMax (hierarchical text classification via information maximization), and HiMatch (hierarchy-aware label semantics matching network) use a dual encoder architecture to model the interaction between text and label hierarchical structures. (2) Contrastive learning-based models: HGCLR (hierarchy-guided contrastive learning model), HJCL (hierarchy-aware joint supervised contrastive learning method), HILL (hierarchy-aware information lossless contrastive learning), and HiSR (hierarchical sequence ranking method) enhance feature representation by introducing hierarchical information and utilizing contrastive learning. (3) Model based on prompt fine-tuning: HPT (hierarchy-aware prompt tuning method) was the first to apply prompt fine-tuning to HTC tasks.Following this research line, subsequent work has extended HPT in several directions. For example, DPT (dual prompt tuning method) proposed a dual prompt mechanism, COPHTC (contrastive-enhanced prompt based model) introduced contrast constraints, NERHTC (NER-guided comprehensive hierarchy-aware prompt tuning) integrated named entity recognition, and SIHTC (hierarchical text classification optimization method via structural entropy and singular spectrum smoothing) utilized structural entropy.

[0096] Main Results. The main experimental results are shown in Table 3. In each column, the best value is shown in bold, and the second-best value is indicated by an underline. Among all the baseline models, LGSA achieved the best performance on all three HTC datasets, and introduced only a very small number of additional parameters compared to the base model HPT.

[0097] On the WOS dataset with a label depth of 2, LGSA outperforms the strongest baseline model, achieving a 0.14% higher Micro-F1 score than SIHTC and a 0.38% higher Macro-F1 score than COPHTC. On the RCV1-V2 dataset with a label depth of 4, LGSA demonstrates a significant advantage, especially in handling imbalanced data scenarios. Its Micro-F1 score is 0.64% higher than HPT, and its Macro-F1 score is significantly improved by 1.82%. Furthermore, LGSA's Macro-F1 score is 0.57% higher than the previously best-performing baseline model, DPT. This significant improvement in Macro-F1 strongly demonstrates that LGSA effectively mitigates representation collapse caused by tail labels. On the NYT dataset with a label depth of 8, LGSA exhibits superior robustness in modeling complex dependencies, consistently outperforming all baseline models. Specifically, its Micro-F1 and Macro-F1 scores exceed the strongest baseline model, NERHTC, by 0.22% and 0.36%, respectively. This demonstrates that the proposed explicit geometric modeling method remains effective in mitigating structural confusion even in deep hierarchical structures.

[0098] Table 3: Experimental results on three HTC datasets

[0099]

[0100] II. Ablation Experiment.

[0101] (1) Key modules in LGSA.

[0102] The ablation experiments for key modules are shown in Table 4. First, removing both the HGS and HGA modules simultaneously leads to the most severe performance degradation, with a 2.27% drop in Macro-F1 on the RCV1-V2 dataset and a 1.35% drop on the NYT dataset. This strongly confirms that without explicit geometric modeling, the model degenerates into a simple classifier, unable to capture complex hierarchical dependencies. Second, removing only the HGS module also results in a significant drop in Macro-F1 (e.g., a 1.15% drop on the RCV1-V2 dataset and a 0.92% drop on the NYT dataset), indicating that HGS provides high-quality label prototypes, which are crucial for the effective functioning of the subsequent HGA module. Similarly, removing the HGA module also leads to a significant performance decline on both datasets, highlighting the necessity of continuous geometric alignment during optimization. In summary, these results validate the complementary enhancement relationship between the HGS and HGA modules.

[0103] Table 4: Ablation study of key components of LGSA based on RCV1-V2 and NYT datasets.

[0104]

[0105] (2) Geometry in HGS.

[0106] Table 5 shows the experimental results, validating the effectiveness of the HGS design strategy. "Random" indicates that the label embeddings are initialized from a standard normal distribution; "Frozen" indicates that the label prototypes are not updated during training. First, the Random initialization variant shows the largest reduction in Macro-F1 performance (1.95% on the RCV1-V2 dataset and 2.80% on the NYT dataset). This indicates that without the semantic embeddings provided by the label initialization process, simple geometric structuring is ineffective. Second, on both the RCV1-V2 and NYT datasets, removing recursive local orthogonalization has a more severe impact on Macro-F1 than removing global orthogonalization. For example, on the RCV1-V2 dataset, the former results in a 1.24% reduction, while the latter only shows a 0.67% reduction. This suggests that while global orthogonalization can establish clear geometric separation boundaries for head labels, decoupling fine-grained sibling label dependencies is the more critical bottleneck limiting model performance when dealing with imbalanced or deep hierarchical structures. Finally, the learnable HGS outperforms the frozen variant. This demonstrates that while the label prototype provides a robust foundation, fine-tuning the label embeddings during training remains crucial for achieving optimal alignment.

[0107] Table 5: Ablation experiments of HGS internal geometry construction strategy

[0108]

[0109] (3) Geometric constraints in HGA.

[0110] This experiment further investigated the specific contributions of the four geometric constraints in the HGA module, and the experimental results are shown in Table 6. It is worth noting that on the NYT dataset, removing global constraints... This results in the largest performance drop (0.79% drop in Macro-F1). This validates that enforcing global orthogonality among header labels is crucial for providing independent subspaces and mitigating structural confusion. Similarly, removing... This leads to a significant decrease in the Macro-F1 score on the RCV1-V2 dataset (a decrease of 0.53%), confirming that this constraint has a significant effect on enhancing the lateral discriminative power of difficult-to-distinguish sibling labels. In contrast, removing... or This leads to a modest decrease in Macro-F1 (e.g., a 0.38% decrease on RCV1-V2). The above results indicate the importance of constructing clear decision boundaries (through...). and The contribution of discrimination to model performance is more significant because discriminative power is a prerequisite for accurate classification. In contrast, and As an auxiliary regularization term, it mainly plays a role in maintaining vertical consistency and calibrating parent-child dependencies.

[0111] Table 6: Ablation study of four geometric constraints in HGA.

[0112]

[0113] III. Exploratory Research.

[0114] (1) Research on unbalanced hierarchies.

[0115] To evaluate the robustness of the LGSA framework in dealing with structural complexity and data imbalance, this experiment references the experimental setup in the paper "HPT: hierarchy-aware prompt tuning for hierarchical text classification" and analyzes its performance at different hierarchy depths and label frequencies. Figure 2 and Figure 3 As shown, LGSA outperforms or maintains comparable performance to HPT and SIHTC across all hierarchical depths in the NYT and RCV1-V2 datasets. This demonstrates the model's superior stability in deep structures. Furthermore, analysis of labels grouped by frequency (…) Figure 4 and Figure 5 The labels were divided into five intervals based on their frequency (>80% represents the top 20% of labels, and so on for the remaining intervals). The results show that LGSA achieved significant gains, especially for the data-scarce tail labels where baseline models typically perform poorly. These results confirm that by introducing explicit geometric priors and constraints, the long-tail effect can be effectively mitigated, enabling the learning of more generalizable and discriminative feature representations for tail labels, even in complex label hierarchies.

[0116] (2) Computational efficiency.

[0117] This experiment evaluated the computational efficiency of LGSA compared to the base model HPT and the strong baseline model SIHTC. To ensure a fair comparison, all models were trained on a single NVIDIA GeForce RTX 3090 GPU with a fixed batch size of 16. As shown in Table 7, LGSA exhibits superior parameter efficiency, introducing only a negligible number of learnable parameters compared to HPT. LGSA's computational efficiency is comparable to or even better than the base model HPT, while significantly outperforming SIHTC in terms of running speed and memory usage. This indicates that the proposed geometry module is highly lightweight with minimal architectural overhead. Crucially, LGSA's GPU memory usage is completely consistent with HPT across all datasets. In contrast, SIHTC exhibits significant memory overhead (nearly 2000MB more memory usage on the RCV1-V2 dataset). Furthermore, LGSA's training speed is comparable to HPT and consistently outperforms SIHTC. These findings demonstrate that LGSA achieves an effective balance between high performance and low computational cost without increasing hardware resource requirements.

[0118] Table 7: Comparison of computational efficiency on the three datasets

[0119]

[0120] This invention aims to address the long-standing structural confusion and representation collapse issues in HTC datasets. Unlike existing implicitly learned label embedding methods, LGSA decouples label geometry from data distribution. During model initialization, the HGS module introduces GOF (Go of Good and Fault) to reconstruct the label embeddings into a stable geometric structure that reflects the label hierarchy. During model training, LGSA uses the HGA module as a geometric regularization term to align the label embeddings with the predefined geometric structure, ensuring the model maintains discriminative power even with long-tailed distributions. Experimental results on the large-scale real-world HTC dataset demonstrate that LGSA achieves state-of-the-art performance, confirming that proactively constructing geometric structures is an effective paradigm.

[0121] In another embodiment, the present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of a hierarchical text classification method based on label geometry construction and alignment.

[0122] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0123] In another embodiment, the present invention also provides a computer device including a processor and a memory storing a plurality of computer instructions, which, when executed by the processor, implement the steps of a hierarchical text classification method based on label geometry construction and alignment.

[0124] The memory and processor are electrically connected directly or indirectly to enable data transmission or interaction. For example, these components can be electrically connected to each other via one or more communication buses or signal lines. The memory stores a computer program that can run on the processor, which implements the method of the present invention by running the computer program stored in the memory.

[0125] The memory may be, but is not limited to, Random Access Memory (RAM), Read Only Memory (ROM), Programmable Read-Only Memory (PROM), Erasable Programmable Read-Only Memory (EPROM), Electrically Erasable Programmable Read-Only Memory (EEPROM), etc. The memory stores the program, and the processor executes the program upon receiving an execution instruction.

[0126] The processor may be an integrated circuit chip with data processing capabilities. The aforementioned processor can be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this invention. The general-purpose processor can be a microprocessor or any conventional processor.

[0127] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0128] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements all fall within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the appended claims.

Claims

1. A hierarchical text classification method based on label geometry construction and alignment, characterized in that, Includes the following steps: Hierarchical geometry construction: Based on the tag set and tag hierarchy, the embeddings of each tag are obtained, and global orthogonalization is performed on the embeddings of the header tag, and recursive local orthogonalization is performed on the embeddings of parent and child tags. Multi-task training: The orthogonalized embedding of the labels is used as the initial label embedding. The hierarchical perception prompt fine-tuning module is trained based on the initial label embedding and text samples. After training, the output is used as the multi-task classification model. During training, the hierarchical geometric regularization loss is calculated for hierarchical geometric alignment. The hierarchical geometric regularization loss includes the global orthogonal loss calculated based on the embedding of the head label, and the parent-child similarity loss, norm decay loss and sibling residual orthogonal loss calculated based on the embedding of the parent and child labels. Classification reasoning application: Input the text to be classified into a multi-task classification model to obtain hierarchical text classification results.

2. The hierarchical text classification method based on label geometry construction and alignment according to claim 1, characterized in that, The process of obtaining the embedding of each tag based on the tag set and tag hierarchy includes: The label adjacency matrix is ​​obtained from the directed acyclic graph corresponding to the label set; A BERT word embedding layer is used to generate semantic embeddings for each tag; The label adjacency matrix and the semantics of the labels are embedded into the input graph attention network to obtain the embeddings of each label.

3. The hierarchical text classification method based on label geometry construction and alignment according to claim 1, characterized in that, The global orthogonalization of the embedding of the header tags includes: For the embedding of header labels, Schmidt orthogonalization is applied to obtain an orthonormal basis; The orthonormal basis is rescaled back to the original norm corresponding to the embedding of the header label to obtain the orthogonalized embedding of the header label.

4. The hierarchical text classification method based on label geometry construction and alignment according to claim 1, characterized in that, The recursive local orthogonalization of the embedding of parent and child tags includes: For a child tag in a parent-child tag pair, it is split into projection embedding and residual embedding along the direction of the parent tag; Apply Schmidt orthogonalization to the residual embedding to obtain the orthonormal basis. Scale the orthonormal basis back to the original norm corresponding to the residual embedding and concatenate it with the projected embedding along the parent label direction to obtain the orthogonalized embedding of the child label.

5. The hierarchical text classification method based on label geometry construction and alignment according to claim 1, characterized in that, The global orthogonal loss is obtained by minimizing the cosine similarity between the embeddings of the head labels, and the specific formula is as follows: In the formula, Indicates global orthogonal loss. The number of permutations representing the number of header tags. , Represents a collection of header tags Size, Indicates the first One tag, Indicates the first A tag, Indicates the first step in the multi-task training process. Embedding of header tags Indicates the first step in the multi-task training process. Embedding of header tags Describes the Euclidean norm. It is a constant.

6. The hierarchical text classification method based on label geometry construction and alignment according to claim 1, characterized in that, The parent-child similarity loss is obtained by constraining the embedding distribution of child labels to a narrow conical region centered on the embedding of the parent label, as shown in the following formula: In the formula, Indicates the loss of similarity between father and son. Represents the set of all parent-child tag pairs Size, Indicates the first step in the multi-task training process. Embedding of parent tags Indicates the first step in the multi-task training process. The first parent tag Embedding of individual tags This represents the target angle that serves as the tolerance boundary.

7. The hierarchical text classification method based on label geometry construction and alignment according to claim 1, characterized in that, The norm decay loss applies a one-sided penalty using the ReLU activation function only when the norm of the child tag's embedding is lower than the decay ratio of the norm of the parent tag's embedding. The specific formula is as follows: In the formula, For norm decay loss, It is the ReLU activation function. The preset attenuation ratio, Indicates the first step in the multi-task training process. Embedding of parent tags Indicates the first step in the multi-task training process. The first parent tag Embedding of individual tags.

8. The hierarchical text classification method based on label geometry construction and alignment according to claim 1, characterized in that, The sibling residual orthogonal loss includes first subtracting the components parallel to the parent label's embedding from the child label's embedding to obtain the residual embedding representing the sibling label information; then, orthogonality constraints are applied only to the residual embeddings to obtain the sibling residual orthogonal loss, as shown in the following formula: In the formula, Indicates the first step in the multi-task training process. The first parent tag Residual embedding of sub-labels, Indicates the first step in the multi-task training process. The first parent tag Embedding of individual tags This indicates an operation that embeds the input and projects it onto the corresponding parent label. This indicates the orthogonal loss of the brother's residual. This represents a set of parent tags that have at least two child tags. Size, Indicates the first A collection of child tags of a parent tag. Size, Indicates the first A tag, Indicates the first step in the multi-task training process. The first parent tag Residual embedding of individual tags.

Images