A domain knowledge-guided air traffic control unsafe event classification method

Abstract

The present invention discloses a method for classifying air traffic control unsafe events guided by domain knowledge, classifies pre-acquired unsafe event text data, and pre-processes the text data; detects and classifies terms in an unsupervised mode based on a byte pair encoding algorithm and a term frequency method, and constructs an unsafe event domain knowledge base; constructs a text classification deep learning model to classify air traffic control safety text events under the guidance of domain knowledge; the text classification deep learning model includes an embedding module, a wide module, a deep module and a classifier; uses the historical unsafe event records of the air traffic control hazard source database as model input samples for model training, and iteratively updates the model parameters, thereby realizing accurate, domain knowledge-guided classification of air traffic control unsafe events. The present invention can greatly reduce the difficulty of acquiring domain knowledge, reduce the interaction loss between domain knowledge and input text, and is conducive to realizing accurate classification of air traffic control unsafe events guided by domain knowledge.

Description

Technical Field

[0001] The present invention belongs to the technical field of civil aviation air traffic control digitization and intelligence, and specifically relates to a method for classifying air traffic control unsafe events guided by domain knowledge. Background Art

[0002] In recent years, with the rapid growth of civil aviation transportation demand, in order to improve safety assurance capabilities, make full use of the collected civil aviation safety information, evaluate safety conditions and trends, and realize information-driven safety management, it is necessary to achieve civil aviation safety transportation. In order to achieve the goal of information-driven, it is crucial to automatically classify the event data in the safety information. The automatic classification of events is a prerequisite for applying statistical methods such as causal inference analysis to civil aviation safety management, and it has a great role in promoting the timely discovery of safety hazards, risk control, and prevention of civil aviation accidents. As the "central nervous system" of civil aviation, air traffic control should first conduct intelligent analysis of its unsafe events.

[0003] Air traffic control unsafe incident reports are generally recorded in the form of short texts. Existing short text classification technologies can be divided into three categories: (1) Rule-based methods: This type of method divides the text to be classified into different categories by matching the text to be classified with a set of manually constructed language rules. Rule-based text classification is useful when limited training data is available, and it allows a more transparent and explainable decision-making process. However, this type of method relies on a lot of manual work to create and maintain rules. (2) Machine learning-based methods: This type of method usually breaks down the text classification problem into a two-step task: first, design appropriate feature extraction and dimensionality reduction methods based on the characteristics of the text, and then train a variety of machine learning models to complete the regression from features to categories. For text classification problems, the biggest disadvantage of this type of method is that its feature engineering is relatively cumbersome, and it is heavily dependent on domain knowledge and difficult to generalize. (3) Deep learning-based methods: By introducing an embedding model to map the text into a low-dimensional continuous feature vector, the process of manually extracting features is replaced. For the text embedding vector, various deep learning models such as LSTM, CNN, and their combination can be connected as classifiers to complete the text classification task.

[0004] The existing technology currently has the following problems: (1) In highly professional fields, such as air traffic control, the performance is reduced due to the existence of special domain knowledge. In these fields, deep learning models require more training data than general models to fit domain causal relationships and learn the embedding of domain terms. (2) It is difficult to solve the problem of domain drift caused by the diversity of training sample expressions. For example, for the category "military and civil aviation conflicts", most of the text expressions of its samples will contain "military aircraft" or its synonyms, while a small number will not mention "military" but contain descriptions of specific models such as "bombers". The above two expressions actually predict the category through the latent variable "military aircraft". However, what is actually observed in the sample is some expression mapping of the latent variable. Under this influence, the model does not learn the real latent variable, resulting in misjudgment. Summary of the invention

[0005] Purpose of the invention: In view of the above-mentioned problems existing in the method of extracting key features of dangerous sources of unsafe events, the present invention proposes a method for classifying unsafe air traffic control events guided by domain knowledge, so as to achieve accurate and domain knowledge-guided classification of unsafe air traffic control events.

[0006] Technical solution: The method for classifying air traffic control unsafe events guided by domain knowledge of the present invention comprises the following steps:

[0007] (1) Classify the pre-acquired unsafe event text data and pre-process the text data;

[0008] (2) Based on the byte pair encoding algorithm and term frequency method, the terms are detected and classified in an unsupervised mode to build a knowledge base in the field of unsafe events;

[0009] (3) constructing a text classification deep learning model to classify air traffic control safety text events under the guidance of domain knowledge; the text classification deep learning model includes an embedding module, a wide module, a deep module and a classifier;

[0010] (4) The historical unsafe event records in the air traffic control hazard source database are used as model input samples for model training, and the model parameters are iteratively updated to achieve accurate, domain knowledge-guided air traffic control unsafe event classification.

[0011] Furthermore, the process of classifying the pre-acquired unsafe event text data in step (1) is as follows:

[0012] The text data of data insecurity incidents are divided into hazard source number, unit, profession, hazard source description, triggering factors, consequences, possibility, severity, initial risk level, major hazard sources, existing control measures, mitigation measures, control unit, expected severity, expected possibility, expected risk level, residual or derivative risks, safety performance goals and control status.

[0013] Furthermore, the preprocessing of the text data in step (1) is implemented as follows:

[0014] Invalid characters such as garbled characters, redundant spaces, and line breaks in the text data are deleted; descriptions of spatial entities are replaced with unified marks; professional abbreviations in the hazard source database are replaced with corresponding full names; and the unsafe events corresponding to each unsafe event consequence text are identified and labeled to form an air traffic control unsafe event classification data set.

[0015] Furthermore, the implementation process of step (2) is as follows:

[0016] Based on the byte pair encoding algorithm, the civil aviation regulations text corpus is traversed to find a pair of adjacent Chinese characters with the highest frequency of occurrence, and they are replaced with a new word tag; the adjacent Chinese characters with the highest frequency of occurrence are continued to be found, including a combination of Chinese characters and a combination of Chinese characters and word tags; the above operations are repeatedly performed until a preset stop condition is met, and a regulation text word segmentation dictionary containing frequencies is obtained; the regulation document corpus is segmented according to the dictionary;

[0017] If a term appears more frequently than other terms in a certain category of text and rarely appears in other categories of text, then the term is related to this category; the KF-IDF value is calculated based on this principle:

[0018]

[0019] Where docs(term,cat) is the number of documents in the category containing the term, cats(term) is the number of categories in which the term appears, and α is the smoothing factor;

[0020] The redundant items in the results obtained using the KF-IDF value are further screened by the parts of speech of these term phrases; other phrases in the list that do not meet the part of speech requirements are removed.

[0021] Furthermore, the embedding module in step (3) converts the input text and the domain knowledge text into an embedding vector; the specific implementation process is as follows:

[0022] For the text sequence input {w1,w2,...}, the lemma analyzer first lemmatizes it to obtain a string of lemma sequences of length N. Then BERT calculates the word-level embedding sequence S w :

[0023]

[0024] in, represents the embedding vector of the nth word, and h is the hidden layer dimension of BERT;

[0025] For each domain knowledge text, the average value of the embedding vector of each word source is used to represent it, and all domain knowledge texts are represented as a complete sequence;

[0026] For a domain knowledge text set L = {P1, P2, ..., P M}, using the same word analyzer to obtain a word sequence of length l(m) Then calculate P j Embedding:

[0027]

[0028] When M is not equal to N, it is padded or truncated to make its length M; L is converted into domain knowledge text embedding sequence S p :

[0029]

[0030] Among them, e li S represents the phrase-level embedding vector of the i-th domain knowledge text; w With S p The matrix forms are:

[0031]

[0032]

[0033] Furthermore, the wide module in step (3) realizes the memory and call of various domain knowledge of event tags; the specific implementation process is as follows:

[0034] The embedding vector of each word of the input text is averaged; for the input text M in matrix form w The embedding sequence M of the domain knowledge text p , respectively represented as query matrix Q W With the key matrix K W :

[0035]

[0036] K W =Mp

[0037]

[0038] Among them, r is the half-step length of the sliding average, h is the dimension of the embedding vector, and the value matrix is ​​the one-hot matrix of the category to which the domain knowledge text belongs, that is:

[0039]

[0040] Among them, K represents the dimension of the unique hot encoding of the label, that is, the number of categories; the matrix Q W The nth row represents the embedding vector of the nth word of the input text Matrix K W The mth row represents the phrase-level embedding of the mth domain knowledge text V W The mth row is the one-hot encoding of the category to which the mth domain knowledge text belongs;

[0041] The attention score is calculated through the regularized compatible attention mechanism:

[0042] Β=(Q W K W ) T

[0043]

[0044]

[0045]

[0046] Among them, Β is the element-by-element compatibility between the token sequence and the domain knowledge text sequence, β j is the j-th column vector of Β;

[0047] The maximum pooling is used to pool the regularization compatible attention scores; for A W , the probability that the input text belongs to the kth category is:

[0048]

[0049] Furthermore, the deep module in step (3) includes a deep text attention block and a deep domain attention block; the deep text attention block focuses on important words in the sentence through a self-attention module training model; the deep domain attention block focuses on important domain knowledge through a common attention module training model; and the outputs of multiple layers of deep text attention blocks and deep domain attention blocks are merged to obtain the output of the deep module, thereby achieving deep extraction of text information.

[0050] Furthermore, the deep text attention block uses a self-attention module to process the input text; a learnable parameter is added to the attention mechanism and Calculate the self-attention of the input text:

[0051]

[0052]

[0053]

[0054] A X =Attention(Q X ,K X ,V X )

[0055] Next, A X Perform residual connection and LayerNormalization (LN):

[0056] A X′ =M w +LayerNorm(A X )

[0057] A X′ It is then nonlinearized by the feedforward layer and enters the next deep text attention block; at the end of all deep text attention blocks, average pooling is calculated for each token in the input sequence:

[0058]

[0059] Among them, d X Represents the sentence-level embedding of the input text after being processed by the self-attention mechanism, that is, the output of the deep text attention block.

[0060] Furthermore, the deep domain attention block adds learnable parameters to the attention mechanism, and value is the embedding vector of the domain knowledge text. The input embedding sequence is matrix-multiplied with the parameter to calculate the attention:

[0061]

[0062]

[0063]

[0064] Alpha D =Attention(Q D ,K D ,V D )

[0065] A D′ =M p +LayerNorm(Α D )

[0066] Likewise for A D′ Calculate average pooling:

[0067]

[0068] Among them, the vector d D That is the output of the deep domain attention block, d D It also undergoes nonlinearization through the feedforward layer and then enters the next deep domain attention block.

[0069] Furthermore, the classifier implementation process in step (3) is as follows:

[0070] The output w of the wide module has the dimension of the number of labels K; the sentence-by-sentence deep text features d of the deep module have the dimension of the embedding vector h; the classifier integrates these two vectors and outputs the predicted probability of the category to which the text belongs; first reduce the dimension of d and integrate the two vectors into one vector x:

[0071] d′=W1d+b1

[0072] x=concat(w,d′)

[0073] Among them, W1 and b1 are the parameters of the linear layer that reduces the dimension of d; concat represents the connection of vectors; x is input into a fully connected layer with tanh as the activation function for classification, and the result is converted into a predicted probability through the softmax function:

[0074]

[0075] Among them, W2 and b2 are the parameters of the fully connected layer;

[0076] Minimize the binary cross entropy loss function between the predicted value and the true value:

[0077]

[0078] Among them, y is the true category of the sample.

[0079] Beneficial effects: Compared with the prior art, the present invention has the following beneficial effects:

[0080] 1. The present invention represents domain prior knowledge in the form of text, and maps them to the same embedding space as the input text, so as to achieve interaction with the input text and guide classification; this method directly records domain knowledge in the form of terms (phrases), so it does not require structured data storage techniques; it can greatly reduce the difficulty of obtaining domain prior knowledge and reduce the interaction loss between domain prior knowledge and input text;

[0081] 2. The present invention introduces wide modules and deep modules to respectively complete the tasks of memorizing and applying prior knowledge or mining potential classification patterns, thereby realizing the application of domain prior knowledge and solving the domain drift problem;

[0082] 3. The present invention adopts an unsupervised approach to extract different forms of expressions related to air traffic control unsafe incidents from a large number of civil aviation regulations and documents as alternatives to the domain knowledge dictionary; compared with experience-based expert knowledge acquisition methods, the present invention is conducive to obtaining more comprehensive and accurate domain knowledge texts. BRIEF DESCRIPTION OF THE DRAWINGS

[0083] Figure 1 It is a flow chart of the present invention. DETAILED DESCRIPTION

[0084] The present invention will be further described below in conjunction with the accompanying drawings.

[0085] like Figure 1 As shown, the present invention proposes a method for classifying air traffic control unsafe events guided by domain knowledge, which specifically includes the following steps:

[0086] Step 1: Classify the pre-acquired unsafe event text data and pre-process the text data.

[0087] First, the unsafe incident text data is read in, the classification basis is determined, and then the text data is preprocessed by text denoising, spatial entity labeling, and abbreviation replacement.

[0088] The air traffic control hazard source database stores data in 19 attributes, including hazard source number, unit, specialty, hazard source description, triggering factor, consequence, possibility, severity, initial risk level, major hazard source, existing control measures, mitigation measures, control unit, expected severity, expected possibility, expected risk level, residual / derivative risk, safety performance target and control status. The present invention classifies air traffic control unsafe events through the "consequence" attribute data. "Consequence" is the description of the possible or already caused consequences of the unsafe event by the reporting personnel, and is the best attribute for identifying the category of unsafe events.

[0089] Preprocess the unsafe event text data:

[0090] Text denoising: The text data contains invalid characters such as garbled characters, redundant spaces, and line breaks, which interfere with the training process. These characters are deleted.

[0091] Spatial entity labeling: Some descriptions involve spatial entities, such as city names, airport names, waypoint names, etc. Replace these entities with unified labels to eliminate bias caused by spatial factors.

[0092] Abbreviation replacement: The hazard database contains a large number of highly specialized abbreviations, which are replaced with their full names.

[0093] Data labeling is performed on the data set. According to the ICAO Safety Management Manual (Doc 9859) and Civil Aircraft Incidents (MH / T 2001-2015), 21 types of standard dangerous events are defined. The consequence description corresponds to multiple unsafe events, that is, one data instance can correspond to multiple labels. For each unsafe event consequence text, the type of unsafe event is determined and labeled according to the provisions of the ICAO Safety Management Manual (Doc 9859) and Civil Aircraft Incidents (MH / T 2001-2015), forming an air traffic control unsafe event classification data set.

[0094] Step 2: Extract terms containing category labels from massive regulatory documents, first use byte pair encoding algorithm to implement word segmentation, then use a term frequency-based method to detect and classify terms in an unsupervised mode to build a knowledge base in the field of unsafe events. The specific steps are as follows:

[0095] Traverse the civil aviation regulations text corpus, find the pair of adjacent Chinese characters that appear most frequently, and replace them with a new word tag; continue to find the adjacent Chinese characters that appear most frequently (including the combination of Chinese characters-Chinese characters and the combination of Chinese characters-word tags). Repeat the above operation until the stop condition is met, and obtain the regulation text word segmentation dictionary containing the frequency. Segment the regulation document corpus according to this table.

[0096] A method based on Keyword Frequency-Inverse Document Frequency (KF-IDF) is used to extract knowledge terms in the field of unsafe events. If a term appears more frequently than other terms in the text of a certain category and rarely appears in the text of other categories, then the term is a term related to this category. The KF-IDF value is calculated based on this principle:

[0097]

[0098] Among them, docs(term,cat) is the number of documents in the category containing the term, cats(term) is the number of categories in which the term appears, and α is the smoothing factor. In the regulatory document, each entry is defined as a document, and the unsafe incident described by the entry corresponds to the category, which is obtained by regular matching the title and content of the entry.

[0099] The redundant items in the results obtained using KF-IDF values ​​are further screened. The screening is achieved through the part of speech of these term phrases. Nouns (phrases) (such as "go-around") or combinations of nouns (phrases) and verbs (such as "deviate from control instructions") have a high probability of forming valid terms. Only these terms are retained, and other phrases in the list that do not meet the part of speech requirements are eliminated. Finally, manual review and addition and deletion are performed to ensure the quality of the domain knowledge base.

[0100] Step 3: Build a text classification deep learning model to accurately classify air traffic control safety text events under the guidance of domain knowledge; the text classification deep learning model includes an embedding module, a wide module, a deep module and a classifier.

[0101] The embedding module converts the input text and domain knowledge text into embedding vectors, so that the deep learning model can understand natural language text. For the text sequence input {w1,w2,...}, the lemma analyzer first lemmatizes it to obtain a string of lemma sequences of length N. Then BERT calculates the word-level embedding sequence S w ,Right now:

[0102]

[0103] in, Represents the embedding vector of the nth word, and h is the hidden layer dimension of BERT.

[0104] For each domain knowledge text, the average value of the embedding vector of each word source is used to represent it, so that all domain knowledge texts are represented as a complete sequence, rather than a separate sequence for each domain knowledge text. This method can reduce the amount of calculation to a few hundredths of the original amount.

[0105] For a domain knowledge text set L = {P1, P2, ..., P M}, with P m Take the example to calculate the representation of each domain knowledge text. Use the same word element analyzer to obtain a word element sequence of length l(m) Then calculate P j Embedding:

[0106]

[0107] When M is not equal to N, it needs to be padded or truncated to make its length M. Convert L into domain knowledge text embedding sequence S p ,Right now:

[0108]

[0109] Among them, e li S represents the phrase-level embedding vector of the i-th domain knowledge text. w With S p The matrix forms are:

[0110]

[0111]

[0112] The wide module focuses on the memory and call of various domain knowledge of event labels. First, the embedding vector of each word of the input text is averaged. Specifically, for the input text M in matrix form w The embedding sequence M of the domain knowledge text p , respectively represented as query matrix Q W With the key matrix K W ,have:

[0113]

[0114] K W =M p

[0115]

[0116] Among them, r is the half-step length of the sliding average, h is the dimension of the embedding vector. The value matrix is ​​the one-hot matrix of the category to which the domain knowledge text belongs, that is:

[0117]

[0118] Among them, K represents the dimension of the unique hot encoding of the label, that is, the number of categories. The figurative explanation of the above three matrices is that the matrix Q W The nth row represents the embedding vector of the nth word of the input text Matrix K W The mth row represents the phrase-level embedding of the mth domain knowledge text And V W The mth row of is the one-hot encoding of the category to which the mth domain knowledge text belongs.

[0119] Next, the attention score is calculated through the regularized compatible attention mechanism:

[0120] Β=(Q W K W ) T

[0121]

[0122]

[0123]

[0124] Among them, Β is the element-by-element compatibility between the token sequence and the domain knowledge text sequence, β j is the jth column vector of Β. Since V W The one-hot encoding of the category is used. The calculation result of this step actually represents the probability that each input token belongs to a certain category.

[0125] The maximum pooling is used to pool the regularization compatible attention scores. W , the probability that the input text belongs to the kth category is:

[0126]

[0127] The deep module uses the generalization ability of the deep network to mine the deep features of the input text. The deep module is divided into two types of blocks: the deep text attention block uses the self-attention module to train the model to focus on important words in the sentence, and the deep domain attention block uses the ordinary attention module to train the model to focus on important domain knowledge. By stacking multiple blocks, deep extraction of text information is achieved. The results of the two parts are merged at the end of the wide module.

[0128] Deep Text Attention Block: Uses a self-attention module to process input text. Unlike the class attention mechanism in the wide module, the attention module in the deep text attention block adds learnable parameters. and Compute the self-attention of the input text:

[0129]

[0130]

[0131]

[0132] A X =Attention(Q X ,K X ,V X )

[0133] Next, A XPerform residual connection and LayerNormalization (LN):

[0134] A X′ =M w +LayerNorm(A X )

[0135] A X′ It is then nonlinearized by a feed-forward layer before entering the next deep text attention block.

[0136] Because A X′ It is token-by-token, so at the end of all deep text attention blocks, average pooling needs to be calculated for each token in the input sequence:

[0137]

[0138] Among them, d X Represents the sentence-level embedding of the input text after being processed by the self-attention mechanism, and is also the output of the deep text attention block.

[0139] Deep Domain Attention Block: The essence of the deep domain attention block is to enable the model to learn the potential association between the input text and domain knowledge, and attach the corresponding domain knowledge text embedding to the sentence embedding of the input text.

[0140] The deep domain attention block is similar to the wide module. The difference between the two is that the attention mechanism of the deep domain attention block has learnable parameters, and the value is the embedding vector of the domain knowledge text, rather than the one-hot matrix in the wide module. The input embedding sequence is matrix-multiplied with the parameters to calculate the attention, that is:

[0141]

[0142]

[0143]

[0144] Alpha D =Attention(Q D ,K D ,V D )

[0145] A D′ =M p +LayerNorm(Α D )

[0146] Likewise for A D′ Calculate average pooling:

[0147]

[0148] Vector d D That is the output of the deep domain attention block, d D It also undergoes nonlinearization through the feedforward layer and then enters the next deep domain attention block.

[0149] By merging the outputs of the multi-layer deep text attention block and the deep domain attention block, the output d of the deep module is obtained:

[0150] d=d X +d D

[0151] In the above three modules, two vectors are obtained: the output w of the wide module, whose dimension is the number of labels K; the sentence-by-sentence deep text feature d of the deep module, whose dimension is the dimension h of the embedding vector. The classifier integrates these two vectors and outputs the predicted probability of the category to which the text belongs. First, reduce the dimension of d and integrate the two vectors into one vector x:

[0152] d′=W1d+b1

[0153] x=concat(w,d′)

[0154] Among them, W1 and b1 are the parameters of the linear layer that reduces the dimension of d; concat represents the connection of vectors. x is input into a fully connected layer with tanh as the activation function for classification, and the result is converted into a predicted probability through the softmax function:

[0155]

[0156] Among them, W2 and b2 are the parameters of the fully connected layer.

[0157] For the multi-label classification problem involved in this patent, it is necessary to minimize the binary cross entropy (BCE) loss function between the predicted value and the true value:

[0158]

[0159] Among them, y is the true category of the sample.

[0160] Step 4: Use the historical unsafe event records in the air traffic control hazard source database as model input samples for model training, and iteratively update the model parameters to achieve accurate, domain knowledge-guided air traffic control unsafe event classification.

[0161] Set the training parameters, training mode and other properties of the text classification deep learning model, and train it on the training set. Use AdamW from the transformers library as the optimizer, and use different learning rates for BERT and downstream modules: BERT's initial learning rate is 2e-5, and the initial learning rate of other modules is 1e-4. The batch size is 16, the number of training rounds is 5, and the maximum length of the word sequence is 256. The dataset is randomly divided into a training set and a test set in a ratio of 8:2. Core TM Training takes about 7.5 hours on a server with an i9-13900K 3.00GHz CPU, NVIDIA GeForce RTX 4090GPU, and 128GBRAM.

[0162] In this implementation, taking 8,000 hazard source records of the Safety Department of the Civil Aviation Administration of China from 2012 to 2022 as an example, the following evaluation indicators are used:

[0163] Multi-label classification accuracy: Calculate the frequency of the prediction and the true label completely match, let n be the number of samples, the accuracy is:

[0164]

[0165] in is the predicted category of the i-th sample, y i is the true category of the ith sample, and 1(x) is the indicator function: if x is true, the value of the function will be 1, otherwise it will be 0.

[0166] Top-k accuracy: Top-k accuracy calculates the number of instances that contain the correct label among the top k predicted labels with the highest scores. Top-k accuracy is:

[0167]

[0168] in, is the class with the jth highest prediction score in the ith sample. Commonly used k values ​​are 3 (denoted as P@3) and 5 (denoted as P@5).

[0169] Table 1 Comparative experimental results of the present invention and the mainstream method

[0170]

[0171] Table 1 is a comparison of the experimental results of the method of the present invention and the mainstream text classification method. It can be seen from the table that the prediction model of the present invention is more accurate than the mainstream text classification model and has a better prediction effect.

Claims

1. A domain knowledge-guided air traffic control unsafe event classification method, characterized in that: The following steps are involved: (1) Classify the pre-acquired unsafe event text data and pre-process the text data; (2) Based on the byte pair encoding algorithm and term frequency method, the terms are detected and classified in an unsupervised mode to build a knowledge base in the field of unsafe events; (3) constructing a text classification deep learning model to classify air traffic control safety text events under the guidance of domain knowledge; the text classification deep learning model includes an embedding module, a wide module, a deep module and a classifier; (4) The historical unsafe event records of the air traffic control hazard source database are used as model input samples for model training, and the model parameters are iteratively updated to achieve accurate and domain knowledge-guided air traffic control unsafe event classification; The wide module in step (3) realizes the memory and call of various domain knowledge of event tags; the specific implementation process is as follows: The embedding vector of each word of the input text is averaged; the matrix M w With M p , respectively represented as query matrix Q W With the key matrix K W : K W =M p Among them, r is the half-step size of the sliding average, h is the dimension of the embedding vector, and the value matrix V W is the one-hot matrix of the category to which the domain knowledge text belongs, Among them, K is the number of labels; the matrix Q W The nth row represents the embedding vector of the nth word of the input text Matrix K W The mth row represents the phrase-level embedding of the mth domain knowledge text V W The mth row is the one-hot encoding of the category to which the mth domain knowledge text belongs; The attention score is calculated through the regularized compatible attention mechanism: B=(Q w K w ) T Among them, Β is the element-by-element compatibility between the token sequence and the domain knowledge text sequence, β j is the j-th column vector of Β; The maximum pooling is used to pool the regularization compatible attention scores; for A W , the probability that the input text belongs to the kth category is: The deep module in step (3) includes a deep text attention block and a deep domain attention block; the deep text attention block focuses on important words in the sentence through a self-attention module training model; the deep domain attention block focuses on important domain knowledge through a common attention module training model; the output of the deep module is obtained by merging the outputs of multiple layers of deep text attention blocks and deep domain attention blocks, thereby realizing deep extraction of text information.

2. According to the domain knowledge-guided air traffic control unsafe event classification method of claim 1, it is characterized in that: The process of classifying the pre-acquired unsafe event text data in step (1) is as follows: The text data of data insecurity incidents are divided into hazard source number, unit, profession, hazard source description, triggering factors, consequences, possibility, severity, initial risk level, major hazard sources, existing control measures, mitigation measures, control unit, expected severity, expected possibility, expected risk level, residual or derivative risks, safety performance goals and control status.

3. The method for classifying air traffic control unsafe events guided by domain knowledge according to claim 1 is characterized in that: The implementation process of preprocessing the text data in step (1) is as follows: Delete invalid characters such as garbled characters, redundant spaces, and line breaks in text data; replace descriptions of spatial entities with unified marks; replace professional abbreviations in the hazard source database with corresponding full names; Each unsafe event consequence text is labeled with the corresponding unsafe events to form an air traffic control unsafe event classification dataset.

4. The method for classifying air traffic control unsafe events guided by domain knowledge according to claim 1 is characterized in that: The implementation process of step (2) is as follows: Based on the byte pair encoding algorithm, the civil aviation regulations text corpus is traversed to find a pair of adjacent Chinese characters with the highest frequency of occurrence, and they are replaced with a new word tag; the adjacent Chinese characters with the highest frequency of occurrence are continued to be found, including a combination of Chinese characters and a combination of Chinese characters and word tags; the above operations are repeatedly performed until a preset stop condition is met, and a regulation text word segmentation dictionary containing frequencies is obtained; the regulation document corpus is segmented according to the dictionary; If a term appears more frequently than other terms in a certain category of text and rarely appears in other categories of text, then the term is related to this category; the KF-IDF value is calculated based on this principle: Where docs(term,cat) is the number of documents in the category containing the term, cats(term) is the number of categories in which the term appears, and α is the smoothing factor; The redundant items in the results obtained using the KF-IDF value are further screened by the parts of speech of these term phrases; other phrases in the list that do not meet the part of speech requirements are removed.

5. The method for classifying air traffic control unsafe events guided by domain knowledge according to claim 1 is characterized in that: The embedding module in step (3) converts the input text and the domain knowledge text into an embedding vector; the specific implementation process is as follows: First, the word analyzer tokenizes the text sequence input to obtain a word sequence of length N. Then BERT calculates the word-level embedding sequence S w : in, Represents the embedding vector of the nth word; For each domain knowledge text, the average value of the embedding vector of each word is used to represent it, and all domain knowledge texts are represented as a complete sequence; For a domain knowledge text set L = {P1, P2, ..., P m ,...,P M }, using the same word analyzer to obtain a word sequence of length l(m) Then calculate P m Embed When M is not equal to N, it is padded or truncated to make its length M; L is converted into domain knowledge text embedding sequence S p : in, S represents the phrase-level embedding vector of the mth domain knowledge text; w With S p The matrix forms are:

6. The method for classifying air traffic control unsafe events guided by domain knowledge according to claim 1 is characterized in that: The deep text attention block uses a self-attention module to process the input text; a learnable parameter is added to the attention mechanism and Compute the self-attention of the input text: A X =Attention(Q X ,K X ,V X ) Next, A X Perform residual connection and LayerNormalization: A X′ =M w +LayerNorm(A X ) A X′ It is then nonlinearized by the feedforward layer and enters the next deep text attention block; at the end of all deep text attention blocks, average pooling is calculated for each token in the input sequence: Among them, d X Represents the sentence-level embedding of the input text after being processed by the self-attention mechanism, which is the output of the deep text attention block.

7. The method for classifying air traffic control unsafe events guided by domain knowledge according to claim 1 is characterized in that: The deep domain attention block adds learnable parameters to the attention mechanism, and value is the embedding vector of the domain knowledge text. The input embedding sequence is matrix-multiplied with the parameter to calculate the attention: A D =Attention(Q D ,K D ,V D ) A D′ =M p +LayerNorm(A D ) Likewise for A D′ Calculate average pooling: Among them, the vector d D That is the output of the deep domain attention block, d D It also undergoes nonlinearization through the feedforward layer and then enters the next deep domain attention block.

8. The method for classifying air traffic control unsafe events guided by domain knowledge according to claim 1 is characterized in that: The classifier implementation process in step (3) is as follows: The output w of the wide module has the dimension of the number of labels K; the sentence-by-sentence deep text features d of the deep module have the dimension of the embedding vector h; the classifier integrates these two vectors and outputs the predicted probability of the category to which the text belongs; first reduce the dimension of d and integrate the two vectors into a vector x: d'=W1d+b1 x=concat(w,d′) Among them, W1 and b1 are the parameters of the linear layer that reduces the dimension of d; concat represents the connection of vectors; x is input into a fully connected layer with tanh as the activation function for classification, and the result is converted into a predicted probability through the softmax function: Among them, W2 and b2 are the parameters of the fully connected layer; Minimize the binary cross entropy loss function between the predicted value and the true value: Among them, y is the true category of the sample.

Images