Civil aviation operation unit problem automatic classification method and system

Abstract

The application discloses a kind of civil aviation operating unit problem automatic classification method and system, it is related to natural language processing technical field, including the following steps: S1, constructs three levels hierarchical label tree, collects historical problem text and labels hierarchical label path to form training sample set, and data enhancement is carried out to the label category with insufficient sample amount;S2, continue pre-training is carried out to pre-training language model using civil aviation field corpus, and obtains field self-adapting language model.The civil aviation operating unit problem automatic classification method and system proposed in the application adopt three levels hierarchical label tree structure to model civil aviation operating unit problem, accurately reflect the hierarchical from belonging relationship between problem category, so that classification result meets civil aviation business organization structure;Through label embedding recursive propagation mechanism, parent node semantic information is layer by layer transferred to child node, so that each level label representation vector fuses hierarchical dependence information, improves the description ability of model to the semantic correlation between label.

Description

Technical Field

[0001] This invention relates to the field of natural language processing technology, specifically to an automatic classification method and system for problems of civil aviation operating units. Background Technology

[0002] Civil aviation operators engage in multiple business areas in their daily operations, including safety management, operational control, maintenance engineering, and ground support, generating a massive volume of complex problem reports, incident logs, and safety information. Accurate and efficient classification of this information is crucial for civil aviation regulatory authorities and operators to conduct risk identification, hazard investigation, and safety decision-making. Automatic text classification technology, as an important research direction in natural language processing, has been widely applied in various industries. Multi-label classification technology can handle scenarios where a single sample corresponds to multiple categories, while hierarchical classification technology further considers the hierarchical relationships between categories. Combining these two approaches can better model complex classification needs. In recent years, methods based on pre-trained language models have made significant progress in feature extraction and semantic understanding, providing technical support for the automation of civil aviation problem classification.

[0003] Existing text classification methods in the civil aviation field mostly employ a flat, single-label classification framework, treating various questions as independent categories without fully considering the hierarchical structure and semantic relationships between question categories. Civil aviation operational unit questions typically possess clear hierarchical characteristics; for example, safety management questions can be further divided into subcategories such as operational safety, personnel qualifications, and equipment status. This hierarchical structure implies rich category dependencies. Flat classification methods struggle to effectively utilize parent-child constraint information between labels, potentially leading to logical inconsistencies in classification results. Furthermore, existing methods typically use standard loss functions during model training, failing to explicitly constrain label hierarchy consistency, leaving room for performance improvement even with uneven category distributions. In addition, the domain adaptability of general pre-trained language models in the civil aviation field needs further enhancement.

[0004] In response, we propose an automatic problem classification method and system for civil aviation operators. Summary of the Invention

[0005] To address the aforementioned technical issues, an automatic problem classification method for civil aviation operators is provided, comprising the following steps: S1. Construct a three-level hierarchical label tree, collect historical question texts and label the hierarchical label paths to form a training sample set, and perform data augmentation on label categories with insufficient sample size. S2. Use civil aviation domain corpus to further pre-train the pre-trained language model to obtain a domain-adaptive language model; S3. Generate an initial label embedding vector for each label node. From the root node to the leaf node, the parent node fusion embedding is weighted and superimposed to the current node's initial embedding by a preset coefficient to obtain the fusion embedding vector of each node. S4. Construct a hierarchical classification model that includes a domain-adaptive language model encoder and three-level classification heads. The input text is encoded and then input into the three-level classification heads to obtain the predicted probabilities of each level of label. S5. Generate a hierarchical constraint mask for the third-level label based on the predicted probabilities of the first-level and second-level labels, and multiply the mask by the original predicted probabilities of the third-level labels to obtain the constrained predicted probabilities. S6. Calculate the joint loss function, which includes classification loss and hierarchical consistency regularization term, and train the model. S7. Input the text to be classified into the trained model, output the predicted probability of each level of label, and determine the label attribution after hierarchical constraint mask correction.

[0006] Preferably, S1 includes: Based on the business area distribution of historical problem data of civil aviation operators, primary label categories are determined, including safety management, operation control, maintenance engineering, and ground support. Based on the business sub-domains under each primary tag category, secondary tag categories are determined, and the secondary tags cover the specific business scenarios under each primary tag. Based on the specific problem types under each secondary tag category, a tertiary tag category is determined, and the tertiary tag covers the specific operational problems under each secondary tag. A unique hierarchical code is established for all tag nodes. The hierarchical code reflects the hierarchical position of the tag in the hierarchical tag tree and the parent-child relationship.

[0007] Preferably, S1 further includes: Based on the annotation results of the training sample set, count the number of samples for each third-level label category; Class 3 label categories with a sample size below a preset sufficient threshold are marked as categories to be enhanced; For each category to be enhanced, candidate enhanced samples are generated using three enhancement methods: synonym substitution, back translation, and noise insertion. Calculate the semantic similarity between each candidate augmented sample and its corresponding original sample; Candidate augmentation samples with semantic similarity not lower than a preset semantic threshold are retained as valid augmentation samples; This will effectively enhance the training sample set by adding samples and updating the sample distribution of the training sample set.

[0008] Preferably, S2 includes: Collect textual data from civil aviation regulations, accident reports, maintenance records, and operations manuals; The collected text data was cleaned and deduplicated to construct a corpus for the civil aviation field; Using BERT-base-chinese as the basic pre-trained model, further pre-training was performed on a civil aviation corpus using a masked language modeling task; The AdamW optimizer and linear learning rate scheduling strategy are used to set the base learning rate and number of training rounds. After further pre-training, a domain-adaptive language model is obtained. The parameters of the domain-adaptive language model are used as the initialization parameters of the text encoder in the hierarchical classification model.

[0009] Preferably, S3 includes: Create a learnable embedding vector for each label node in the three-level hierarchical label tree, mapping the unique hierarchical encoding of the label node to the corresponding initial embedding vector; Organize the initial embedding vectors of all first-level tags into a first-level embedding matrix, organize the initial embedding vectors of all second-level tags into a second-level embedding matrix, and organize the initial embedding vectors of all third-level tags into a third-level embedding matrix. Based on the parent-child relationship of the three-level hierarchical label tree, a parent node index mapping table is established. The parent node index mapping table records the position index of the parent node corresponding to each label node in the previous level embedding matrix.

[0010] Preferably, S3 further includes: The initial embedding vector of the first-level label is directly used as the fused embedding vector of the first-level label; Traverse each tag node in the secondary tag, obtain the position index of the parent node of the current secondary tag node in the primary embedding matrix according to the parent node index mapping table, extract the corresponding primary tag fusion embedding vector, and add the primary tag fusion embedding vector to the initial embedding vector of the current secondary tag node after weighting by a preset propagation coefficient α to obtain the fusion embedding vector of the current secondary tag node. Traverse each tag node in the three-level tags, obtain the position index of the parent node of the current three-level tag node in the two-level embedding matrix according to the parent node index mapping table, extract the corresponding two-level tag fusion embedding vector, and add the two-level tag fusion embedding vector to the initial embedding vector of the current three-level tag node after weighting by a preset propagation coefficient α to obtain the fusion embedding vector of the current three-level tag node.

[0011] Preferably, S4 includes: A domain-adaptive language model is used as a text encoder to receive input text and output a fixed-dimensional text representation vector. Construct a first-level classification head, a second-level classification head, and a third-level classification head. Each classification head contains a fully connected layer and a sigmoid activation function. The output dimension of the fully connected layer of the first-level classification head is the same as the number of first-level label categories; the output dimension of the fully connected layer of the second-level classification head is the same as the number of second-level label categories; and the output dimension of the fully connected layer of the third-level classification head is the same as the number of third-level label categories. The text representation vector is input into the first-level classification head, the second-level classification head, and the third-level classification head, respectively. After mapping by a fully connected layer and calculation by the sigmoid activation function, the predicted probability distributions of the first-level label, the second-level label, and the original predicted probability distributions of the third-level label are obtained.

[0012] Preferably, S5 includes: The prediction probability distribution of the first-level label is used as the prediction probability of the first-level parent node, and the prediction probability distribution of the second-level label is used as the prediction probability of the second-level parent node. Iterate through each tag node in the three-level tags and determine the position index of the parent node of the current three-level tag node in the predicted probability distribution of the two-level tags according to the parent node index mapping table; Extract the predicted probability value of the corresponding position index from the predicted probability distribution of the secondary label, and use the predicted probability value as the hierarchical constraint mask value of the current tertiary label node; According to the arrangement order of the three-level label nodes, the hierarchical constraint mask values ​​of all three-level label nodes are combined into a hierarchical constraint mask vector.

[0013] Preferably, S5 further includes: Perform element-wise multiplication between the hierarchical constraint mask vector and the original predicted probability distribution vector of the three-level labels; For each third-level label node, when the predicted probability value of the parent node of the node is lower than the preset parent label probability threshold, the constrained predicted probability of the node is suppressed to near zero. The result of element-wise multiplication is used as the constrained probability distribution for the three-level label prediction.

[0014] An automatic problem classification system for civil aviation operators is also provided, including: The hierarchical label construction and data augmentation module constructs a three-level hierarchical label tree for civil aviation operating unit issues, including first-level labels, second-level labels, and third-level labels. It collects historical issue text data of civil aviation operating units and marks the hierarchical label paths to form a training sample set. For third-level label categories with a sample quantity lower than a preset sufficient threshold, it uses synonym replacement, back translation, or noise insertion to perform data augmentation. It retains augmented samples with semantic similarity not lower than a preset semantic threshold. The tag embedding recursive propagation module generates an initial tag embedding vector for each tag node in the three-level hierarchical tag tree, recursively propagates the parent node embedding information from the root node to the leaf node, and calculates the fused embedding vector for each tag node. The hierarchical classification model construction and training module constructs a hierarchical classification model that includes a domain-adaptive language model encoder and three-level classification heads. It generates a hierarchical constraint mask for the third-level label based on the predicted probability distribution of the first-level and second-level labels. The mask is multiplied by the original predicted probability of the third-level label to obtain the constrained predicted probability of the third-level label. The joint loss function, which includes classification loss, hierarchical consistency regularization term and class imbalance weight, is calculated to train the hierarchical classification model. The classification output module receives the question text of civil aviation operating units to be classified, inputs the trained hierarchical classification model, outputs the predicted probability of each level of label, and determines the attribution of each level of label according to the preset threshold after correction by hierarchical constraint mask.

[0015] Compared with the prior art, the beneficial effects of the present invention are as follows: This invention proposes an automatic classification method and system for civil aviation operating unit issues. It employs a three-layer hierarchical label tree structure to model civil aviation operating unit issues, accurately reflecting the hierarchical relationships between issue categories and ensuring the classification results conform to the organizational structure of civil aviation operations. Through a label embedding recursive propagation mechanism, semantic information from parent nodes is passed layer by layer to child nodes, enabling the fusion of hierarchical dependency information in the label representation vectors at each level, thus enhancing the model's ability to characterize semantic relationships between labels. A hierarchical constraint mask is generated based on the predicted probability of parent labels to constrain and correct the predicted probabilities of child labels, ensuring that the classification results meet the requirements of hierarchical structure logic consistency and avoiding logical contradictions. A joint loss function integrates classification loss and hierarchical consistency regularization terms, explicitly constraining hierarchical consistency during training to improve classification performance under conditions of imbalanced category distribution. Domain-adaptive pre-training utilizes civil aviation professional corpus to further train the general language model, adapting it to civil aviation terminology and textual features, enhancing its ability to understand professional issues. Hierarchical-aware data augmentation expands label categories with insufficient samples, filtering high-quality augmented samples through semantic similarity verification to alleviate the problem of imbalanced category distribution and improve the ability to identify minority categories. Attached Figure Description

[0016] Figure 1 This is a flowchart of the method of the present invention; Figure 2 This is a system module connection diagram of the present invention. Detailed Implementation

[0017] The following description is intended to disclose the invention and enable those skilled in the art to implement it. The preferred embodiments described below are merely examples, and other obvious variations will occur to those skilled in the art.

[0018] Reference Figure 1 As shown, an automatic problem classification method for civil aviation operators includes the following steps: A three-level hierarchical label tree is constructed. Historical question texts are collected and hierarchical label paths are labeled to form a training sample set. Data augmentation is performed on label categories with insufficient sample size.

[0019] Specifically, based on the business area distribution of historical problem data of civil aviation operators, primary label categories are determined, including safety management, operation control, maintenance engineering, and ground support.

[0020] Based on the business sub-domains under each primary tag category, secondary tag categories are determined, each secondary tag covering the specific business scenarios under each primary tag. Based on the specific problem types under each secondary tag category, tertiary tag categories are determined, each tertiary tag covering the specific operational problems under each secondary tag. A unique hierarchical code is established for all tag nodes, reflecting the tag's hierarchical position and parent-child relationship in the hierarchical tag tree.

[0021] During the data augmentation phase, based on the annotation results of the training sample set, the number of samples for each third-level label category is counted. Third-level label categories with a sample count below a preset sufficiency threshold are marked as categories to be augmented. The preset sufficiency threshold is set based on the following criteria: In hierarchical multi-label classification tasks, when the number of training samples for a certain category is less than a certain amount, the model struggles to learn the effective feature distribution of that category, leading to a significant drop in classification performance. Based on the actual distribution characteristics of civil aviation operating unit problem data and experimental verification, when the number of samples for a third-level label category is less than 50, the model's classification accuracy for that category will decline significantly. Therefore, the preset sufficient threshold is set to 50 samples.

[0022] The threshold setting takes into account both the category distribution characteristics of civil aviation issue data and the data requirements for model training, ensuring that each category has sufficient samples to support model learning.

[0023] For each category to be augmented, candidate augmented samples are generated using three augmentation methods: synonym substitution, back-translation, and noise insertion. The semantic similarity between each candidate augmented sample and its corresponding original sample is calculated. Candidate augmented samples with a semantic similarity not lower than a preset semantic threshold are retained as valid augmented samples.

[0024] The preset semantic threshold is set based on the principle that data augmentation aims to increase the number of samples while maintaining the original semantic information. Augmented samples with excessively low semantic similarity will introduce noise, thus reducing model performance. Based on the characteristics of civil aviation texts and the semantic representation capabilities of pre-trained language models, experiments have shown that setting the semantic similarity threshold to 0.85 achieves good data augmentation results while ensuring semantic consistency of the augmented samples. This threshold setting is based on the semantic features of civil aviation issue texts and the effectiveness verification of the augmentation strategy. Effective augmented samples are then added to the training sample set, updating the sample distribution of the training sample set.

[0025] The pre-trained language model was further pre-trained using corpus from the civil aviation domain to obtain a domain-adaptive language model.

[0026] Specifically, textual data is collected from civil aviation regulations, accident reports, maintenance records, and operations manuals.

[0027] The collected text data was cleaned and deduplicated to construct a corpus for the civil aviation field.

[0028] Using BERT-base-chinese as the base pre-trained model, further pre-training was performed on a civil aviation corpus using a masked language modeling task. The AdamW optimizer and a linear learning rate scheduling strategy were employed, with the base learning rate and number of training epochs set accordingly.

[0029] The base learning rate is set to: 2 × 10⁻⁶ 5 The basis for setting this value is that in the continued pre-training phase of the BERT-base-chinese model, an excessively large learning rate will cause the model to forget the general language ability learned in the pre-training phase, while an excessively small learning rate will lead to a slow convergence speed and may get stuck in a local optimum.

[0030] Based on the size of the civil aviation corpus and the empirical parameters for further pre-training of the BERT model, 2×10 was selected. 5 As a base learning rate, it can maintain the model's general language capability while adapting to the characteristics of the civil aviation field. The number of training rounds is set to three, and this value is based on the following: The purpose of continued pre-training is to adapt the model to the terminology and text features of the civil aviation domain, rather than learning language knowledge from scratch. Based on experimental observations of the size of the civil aviation domain corpus and the model's convergence, three rounds of training enable the model to fully learn the features of the civil aviation domain while avoiding overfitting. After completing the continued pre-training, a domain-adaptive language model is obtained. The parameters of the domain-adaptive language model are used as the initialization parameters for the text encoder in the hierarchical classification model.

[0031] For each tag node, an initial tag embedding vector is generated. From the root node to the leaf node, the parent node fusion embedding is weighted and superimposed to the current node's initial embedding by a preset coefficient to obtain the fusion embedding vector of each node.

[0032] Specifically, a learnable embedding vector is created for each label node in the three-level hierarchical label tree, mapping the unique hierarchical encoding of the label node to the corresponding initial embedding vector.

[0033] The learnable embedding vector is set to 256 dimensions. This dimension is set based on the principle that the embedding vector dimension needs to be consistent with the dimension of the text representation vector in the subsequent hierarchical classification model in order to enable effective feature interaction.

[0034] Considering the balance between the output dimension and computational resources of the BERT-base-chinese model, the embedding vector dimension is set to 256. This dimension satisfies the requirements of label semantic representation and matches the overall model architecture.

[0035] The initial embedding vectors of all first-level tags are organized into a first-level embedding matrix, the initial embedding vectors of all second-level tags are organized into a second-level embedding matrix, and the initial embedding vectors of all third-level tags are organized into a third-level embedding matrix. Based on the parent-child relationship of the three-level hierarchical tag tree, a parent node index mapping table is established, which records the position index of the parent node corresponding to each tag node in the previous level embedding matrix.

[0036] During the recursive propagation of tag embedding, the initial embedding vector of the first-level tag is directly used as the fused embedding vector of the first-level tag.

[0037] Traverse each tag node in the secondary tags, obtain the position index of the parent node of the current secondary tag node in the primary embedding matrix according to the parent node index mapping table, extract the corresponding primary tag fusion embedding vector, and add the primary tag fusion embedding vector to the initial embedding vector of the current secondary tag node after weighting by a preset propagation coefficient α to obtain the fusion embedding vector of the current secondary tag node.

[0038] The preset propagation coefficient α is set to 0.3. This value is based on the fact that the propagation coefficient controls the strength of the semantic information transmitted from the parent node to the child node. If the coefficient is too large, the semantic representation of the child node will rely excessively on the parent node and lose its own distinctiveness; if the coefficient is too small, the hierarchical structure information cannot be effectively utilized. Experiments have verified that when the propagation coefficient is set to 0.3, the semantic information of the parent node can be fully utilized while maintaining the semantic independence of the child node, thus achieving optimal classification performance.

[0039] The value is set based on the depth of the tag hierarchy and the effectiveness of semantic transmission. Each tag node in the three-level tags is traversed, and the position index of the parent node of the current three-level tag node in the two-level embedding matrix is ​​obtained according to the parent node index mapping table. The corresponding two-level tag fusion embedding vector is extracted, and the two-level tag fusion embedding vector is weighted by a preset propagation coefficient α and added to the initial embedding vector of the current three-level tag node to obtain the fusion embedding vector of the current three-level tag node.

[0040] A hierarchical classification model is constructed, which includes a domain-adaptive language model encoder and three-level classification heads. The input text is encoded and then input into the three-level classification heads to obtain the predicted probabilities of each level of label.

[0041] Specifically, a domain-adaptive language model is used as the text encoder, which receives input text and outputs a fixed-dimensional text representation vector. The text representation vector has a dimension of 768, consistent with the hidden layer dimension of the BERT-base-chinese model.

[0042] A first-level, second-level, and third-level classification head are constructed. Each classification head contains a fully connected layer and a sigmoid activation function. The fully connected layer in the first-level classification head has an input dimension of 768 and an output dimension of four, corresponding to the four first-level label categories.

[0043] The fully connected layer in the secondary classification header has an input dimension of 768 and an output dimension of 12, corresponding to 12 secondary label categories.

[0044] The fully connected layer in the three-level classification head has an input dimension of 768 and an output dimension of 48, corresponding to 48 three-level label categories.

[0045] The dimensionality of each classification head is kept consistent with the number of label categories at the corresponding level to ensure that the model output matches the label system. The text representation vector is input into the first-level, second-level, and third-level classification heads respectively. After mapping through a fully connected layer and calculation using the sigmoid activation function, the predicted probability distributions of the first-level, second-level, and third-level labels are obtained, respectively.

[0046] A hierarchical constraint mask for the third-level label is generated based on the predicted probabilities of the first-level and second-level labels. The mask is then multiplied by the original predicted probabilities of the third-level labels to obtain the constrained predicted probabilities.

[0047] The predicted probability distribution of the first-level labels is used as the predicted probability of the first-level parent nodes, and the predicted probability distribution of the second-level labels is used as the predicted probability of the second-level parent nodes. For each label node in the third-level labels, the position index of the parent node of the current third-level label node in the second-level label predicted probability distribution is determined according to the parent node index mapping table. The predicted probability value corresponding to the position index is extracted from the second-level label predicted probability distribution, and this predicted probability value is used as the hierarchical constraint mask value of the current third-level label node. Following the arrangement order of the third-level label nodes, the hierarchical constraint mask values ​​of all third-level label nodes are combined into a hierarchical constraint mask vector.

[0048] In the application of hierarchical constraint masks, the hierarchical constraint mask vector is multiplied element-wise with the original predicted probability distribution vector of the three-level labels. For each three-level label node, when the predicted probability value of its parent node is lower than the preset parent label probability threshold, the constrained predicted probability of that node is suppressed to a value close to 0.

[0049] The preset parent label probability threshold is set to 0.5. This value is based on the fact that in binary classification tasks, 0.5 is a natural dividing point between positive and negative samples. When the predicted probability of a parent label is below 0.5, it indicates that the model considers the parent label not to be a positive example, and therefore all its child labels should not be classified as positive examples either. This conforms to the logical constraints of hierarchical labels. This threshold setting is based on the basic principles of probabilistic classification and the logical consistency requirements of hierarchical labels. The result of element-wise multiplication is used as the constrained tertiary label prediction probability distribution.

[0050] Calculate the joint loss function, which includes classification loss and hierarchical consistency regularization term, and train the model.

[0051] A classification loss function is constructed, using weighted binary cross-entropy as the classification loss for the three-level labels. The weight of each three-level label is calculated based on the number of samples for that label in the training sample set. The smaller the number of samples, the larger the weight of the category, in order to alleviate the problem of imbalanced class distribution.

[0052] A hierarchical consistency regularization term is constructed. For each third-level label node, the difference between its constrained predicted probability and the predicted probability of its corresponding parent node is calculated. When the constrained predicted probability of a child label exceeds the predicted probability of its parent node, a positive penalty is applied to ensure that the predicted probability of a child label does not exceed the predicted probability of its parent label, thus maintaining the logical consistency of the label hierarchy. The classification loss and the hierarchical consistency regularization term are weighted and summed according to preset regularization coefficients to obtain the joint loss function.

[0053] The regularization coefficient controls the proportion of hierarchical consistency constraints in the total loss. If the coefficient is too large, it will overemphasize hierarchical consistency and ignore classification accuracy. If the coefficient is too small, the hierarchical constraint effect will be insufficient. The preset regularization coefficient of 1.6 can achieve a balance between classification accuracy and hierarchical consistency.

[0054] The hierarchical classification model was trained using the AdamW optimizer with an initial learning rate of 3×10⁻⁶. 5 A cosine annealing learning rate scheduling strategy is adopted, with the number of training rounds set to 50. The model performance is monitored on the validation set, and training is terminated early when the validation set loss no longer decreases for five consecutive rounds to prevent the model from overfitting.

[0055] The text to be classified is input into the trained model, which outputs the predicted probabilities of labels at each level. After hierarchical constraint mask correction, the label attribution is determined.

[0056] Obtain the text of the civil aviation operating unit questions to be classified, and perform the same text preprocessing as in the training phase, including word segmentation, adding special tags, and truncation and padding.

[0057] The preprocessed text is input into the trained hierarchical classification model. The text encoder produces a text representation vector, which is then input into the first-level classification head, the second-level classification head, and the third-level classification head to obtain the first-level label prediction probability distribution, the second-level label prediction probability distribution, and the third-level label original prediction probability distribution.

[0058] Based on the first-level and second-level label prediction probability distributions, a hierarchical constraint mask vector for third-level labels is generated using the aforementioned method. The hierarchical constraint mask vector is then multiplied element-wise with the original third-level label prediction probability distribution vector to obtain the constrained third-level label prediction probability distribution.

[0059] Threshold determinations are applied to the predicted probability distributions of first-level, second-level, and third-level tags. If the predicted probability of a tag is greater than or equal to a preset classification threshold, the tag is classified as a positive example; otherwise, it is classified as a negative example. The preset classification threshold is set to 0.5. Based on the determination results, the complete hierarchical tag path of the text to be classified is output, including the attribution determinations of all first-level, second-level, and third-level tags.

[0060] Reference Figure 2 As shown, this invention also proposes an automatic classification system for civil aviation operating unit problems, including a hierarchical label construction and data augmentation module, a label embedding recursive propagation module, a hierarchical classification model construction and training module, and a classification output module.

[0061] The hierarchical labeling and data augmentation module constructs a three-level hierarchical label tree for civil aviation operating unit issues, including first-level labels, second-level labels, and third-level labels. It collects historical issue text data of civil aviation operating units and marks the hierarchical label paths to form a training sample set. For third-level label categories with a sample quantity lower than a preset sufficient threshold, it uses synonym replacement, back translation, or noise insertion to perform data augmentation. Augmented samples with semantic similarity not lower than a preset semantic threshold are retained.

[0062] The tag embedding recursive propagation module generates an initial tag embedding vector for each tag node in the three-level hierarchical tag tree, and recursively propagates the parent node embedding information from the root node to the leaf node, calculating the fused embedding vector for each tag node.

[0063] The hierarchical classification model construction and training module constructs a hierarchical classification model that includes a domain-adaptive language model encoder and three-level classification heads. Based on the predicted probability distributions of the first-level and second-level labels, it generates a hierarchical constraint mask for the third-level labels. The mask is multiplied by the original predicted probabilities of the third-level labels to obtain the constrained predicted probabilities of the third-level labels. The module calculates a joint loss function that includes classification loss, hierarchical consistency regularization term, and class imbalance weights, and then trains the hierarchical classification model.

[0064] The classification output module receives the question text of the civil aviation operating unit to be classified, inputs the trained hierarchical classification model, outputs the predicted probability of each level of label, and determines the attribution of each level of label according to the preset threshold after hierarchical constraint mask correction.

[0065] The data flow between the modules is as follows: the training sample set and hierarchical label tree structure output by the hierarchical label construction and data augmentation module are passed to the label embedding recursive propagation module. The fused embedding vectors of each label node output by the label embedding recursive propagation module are passed to the hierarchical classification model construction and training module.

[0066] The hierarchical classification model construction and training module uses fused embedding vectors to build and train a hierarchical classification model, and then passes the trained model to the classification output module. The classification output module receives the text to be classified, performs classification using the trained model, and outputs the final hierarchical label assignment results.

[0067] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely principles of the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the claimed invention. The scope of protection claimed by the appended claims and their equivalents is defined.

Claims

1. An automatic classification method for problems of civil aviation operating units, characterized in that, Includes the following steps: S1. Construct a three-level hierarchical label tree, collect historical question texts and label the hierarchical label paths to form a training sample set, and perform data augmentation on label categories with insufficient sample size. S2. Use civil aviation domain corpus to further pre-train the pre-trained language model to obtain a domain-adaptive language model; S3. Generate an initial label embedding vector for each label node. From the root node to the leaf node, the parent node fusion embedding is weighted and superimposed to the current node's initial embedding by a preset coefficient to obtain the fusion embedding vector of each node. S4. Construct a hierarchical classification model that includes a domain-adaptive language model encoder and three-level classification heads. The input text is encoded and then input into the three-level classification heads to obtain the predicted probabilities of each level of label. S5. Generate a hierarchical constraint mask for the third-level label based on the prediction probabilities of the first-level and second-level labels, and multiply the mask by the original prediction probabilities of the third-level labels to obtain the constrained prediction probabilities. S6. Calculate the joint loss function, which includes classification loss and hierarchical consistency regularization term, and train the model. S7. Input the text to be classified into the trained model, output the predicted probability of each level of label, and determine the label attribution after hierarchical constraint mask correction.

2. The automatic classification method for problems of civil aviation operating units according to claim 1, characterized in that, S1 includes: Based on the business area distribution of historical problem data of civil aviation operators, primary label categories are determined, including safety management, operation control, maintenance engineering, and ground support. Based on the business sub-domains under each primary tag category, secondary tag categories are determined, and the secondary tags cover the specific business scenarios under each primary tag. Based on the specific problem types under each secondary tag category, a tertiary tag category is determined, and the tertiary tag covers the specific operational problems under each secondary tag. A unique hierarchical code is established for all tag nodes. The hierarchical code reflects the hierarchical position of the tag in the hierarchical tag tree and the parent-child relationship.

3. The automatic classification method for problems of civil aviation operating units according to claim 1, characterized in that, S1 further includes: Based on the annotation results of the training sample set, count the number of samples for each third-level label category; Class 3 label categories with a sample size below a preset sufficient threshold are marked as categories to be enhanced; For each category to be enhanced, candidate enhanced samples are generated using three enhancement methods: synonym substitution, back translation, and noise insertion. Calculate the semantic similarity between each candidate augmented sample and its corresponding original sample; Candidate augmentation samples with semantic similarity not lower than a preset semantic threshold are retained as valid augmentation samples; This will effectively enhance the training sample set by adding samples and updating the sample distribution of the training sample set.

4. The automatic classification method for problems of civil aviation operating units according to claim 1, characterized in that, S2 includes: Collect textual data from civil aviation regulations, accident reports, maintenance records, and operations manuals; The collected text data was cleaned and deduplicated to construct a corpus for the civil aviation field; Using BERT-base-chinese as the basic pre-trained model, further pre-training was performed on a civil aviation corpus using a masked language modeling task; The AdamW optimizer and linear learning rate scheduling strategy are used to set the base learning rate and number of training rounds. After further pre-training, a domain-adaptive language model is obtained. The parameters of the domain-adaptive language model are used as the initialization parameters of the text encoder in the hierarchical classification model.

5. The automatic classification method for problems of civil aviation operating units according to claim 1, characterized in that, S3 includes: Create a learnable embedding vector for each label node in the three-level hierarchical label tree, mapping the unique hierarchical encoding of the label node to the corresponding initial embedding vector; Organize the initial embedding vectors of all first-level tags into a first-level embedding matrix, organize the initial embedding vectors of all second-level tags into a second-level embedding matrix, and organize the initial embedding vectors of all third-level tags into a third-level embedding matrix. Based on the parent-child relationship of the three-level hierarchical label tree, a parent node index mapping table is established. The parent node index mapping table records the position index of the parent node corresponding to each label node in the previous layer embedding matrix.

6. The automatic classification method for problems of civil aviation operating units according to claim 1, characterized in that, S3 further includes: The initial embedding vector of the first-level label is directly used as the fused embedding vector of the first-level label; Traverse each tag node in the secondary tag, obtain the position index of the parent node of the current secondary tag node in the primary embedding matrix according to the parent node index mapping table, extract the corresponding primary tag fusion embedding vector, and add the primary tag fusion embedding vector to the initial embedding vector of the current secondary tag node after weighting by a preset propagation coefficient α to obtain the fusion embedding vector of the current secondary tag node. Traverse each tag node in the three-level tags, obtain the position index of the parent node of the current three-level tag node in the two-level embedding matrix according to the parent node index mapping table, extract the corresponding two-level tag fusion embedding vector, and add the two-level tag fusion embedding vector to the initial embedding vector of the current three-level tag node after weighting by a preset propagation coefficient α to obtain the fusion embedding vector of the current three-level tag node.

7. The automatic classification method for problems of civil aviation operating units according to claim 1, characterized in that, S4 includes: A domain-adaptive language model is used as a text encoder to receive input text and output a fixed-dimensional text representation vector. Construct a first-level classification head, a second-level classification head, and a third-level classification head. Each classification head contains a fully connected layer and a sigmoid activation function. The output dimension of the fully connected layer of the first-level classification head is the same as the number of first-level label categories; the output dimension of the fully connected layer of the second-level classification head is the same as the number of second-level label categories; and the output dimension of the fully connected layer of the third-level classification head is the same as the number of third-level label categories. The text representation vector is input into the first-level classification head, the second-level classification head, and the third-level classification head, respectively. After mapping by a fully connected layer and calculation by the sigmoid activation function, the predicted probability distributions of the first-level label, the second-level label, and the original predicted probability distributions of the third-level label are obtained.

8. The automatic classification method for problems of civil aviation operating units according to claim 1, characterized in that, S5 includes: The prediction probability distribution of the first-level label is used as the prediction probability of the first-level parent node, and the prediction probability distribution of the second-level label is used as the prediction probability of the second-level parent node. Traverse each tag node in the three-level tags and determine the position index of the parent node of the current three-level tag node in the predicted probability distribution of the two-level tags according to the parent node index mapping table; Extract the predicted probability value of the corresponding position index from the predicted probability distribution of the secondary label, and use the predicted probability value as the hierarchical constraint mask value of the current tertiary label node; According to the arrangement order of the three-level label nodes, the hierarchical constraint mask values ​​of all the three-level label nodes are combined into a hierarchical constraint mask vector.

9. The automatic classification method for problems of civil aviation operating units according to claim 1, characterized in that, The S5 also includes: Perform element-wise multiplication between the hierarchical constraint mask vector and the original predicted probability distribution vector of the three-level labels; For each third-level label node, when the predicted probability value of the parent node of the node is lower than the preset parent label probability threshold, the constrained predicted probability of the node is suppressed to near zero. The result of element-wise multiplication is used as the constrained probability distribution for the three-level label prediction.

10. An automatic problem classification system for civil aviation operating units, characterized in that, include: The hierarchical label construction and data augmentation module constructs a three-level hierarchical label tree for civil aviation operating unit issues, including first-level labels, second-level labels, and third-level labels. It collects historical issue text data of civil aviation operating units and marks the hierarchical label paths to form a training sample set. For third-level label categories with a sample quantity lower than a preset sufficient threshold, it uses synonym replacement, back translation, or noise insertion to perform data augmentation. It retains augmented samples with semantic similarity not lower than a preset semantic threshold. The tag embedding recursive propagation module generates an initial tag embedding vector for each tag node in the three-level hierarchical tag tree, recursively propagates the parent node embedding information from the root node to the leaf node, and calculates the fused embedding vector for each tag node. The hierarchical classification model construction and training module constructs a hierarchical classification model that includes a domain-adaptive language model encoder and three-level classification heads. It generates a hierarchical constraint mask for the third-level label based on the predicted probability distribution of the first-level and second-level labels. The mask is multiplied by the original predicted probability of the third-level label to obtain the constrained predicted probability of the third-level label. The joint loss function, which includes classification loss, hierarchical consistency regularization term and class imbalance weight, is calculated to train the hierarchical classification model. The classification output module receives the question text of civil aviation operating units to be classified, inputs the trained hierarchical classification model, outputs the predicted probability of each level of label, and determines the attribution of each level of label according to the preset threshold after correction by hierarchical constraint mask.

Images