Electronic warfare equipment relation extraction method based on BERT and Bayesian classifier

Abstract

The invention provides an electronic warfare equipment relation extraction method based on BERT and a Bayesian classifier, and belongs to the technical field of knowledge graph relation extraction. The method comprises the following steps: S1, preprocessing text data, and adding a label based on a classification result; s2, constructing an embedding layer, and converting the Chinese data into vector representation by using the embedding layer; s3, constructing a bidirectional encoder, and extracting context features by using the bidirectional encoder; s4, a naive Bayes classifier is constructed, and classification is carried out; and S5, carrying out model training and testing to obtain a relation extraction result. According to the method, the accuracy and practicability of electronic confrontation text relation extraction can be improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of knowledge graph relationship extraction and relates to an electronic warfare equipment relationship extraction method based on BERT and Bayesian classifier. Background Art

[0002] With the rapid development of information technology, electronic warfare has become a crucial component of modern warfare. The signal environments involved in electronic warfare are complex and ever-changing. Quickly and accurately extracting valuable information from massive amounts of data is crucial for understanding battlefield trends and formulating combat strategies. Relationship extraction, a key technology in natural language processing (NLP), aims to identify semantic relationships between entities in text, providing an important research direction for information processing in the field of electronic warfare.

[0003] In recent years, deep learning technology has achieved remarkable results in the field of natural language processing. The BERT (Bidirectional Encoder Representations from Transformers) model, a pre-trained language representation model, has achieved excellent performance in multiple NLP tasks due to its powerful language understanding capabilities. However, in the field of electronic warfare, due to the extensive technical terminology and complex text structure, the traditional BERT model may have certain limitations in relation extraction tasks. Summary of the Invention

[0004] In response to the above technical problems, the present invention provides an electronic warfare equipment relationship extraction solution based on BERT and Bayesian classifier.

[0005] The first aspect of the present invention proposes a method for extracting electronic warfare equipment relationships based on BERT and a Bayesian classifier, the method comprising:

[0006] Step S1: pre-process the text data and add labels based on the classification results;

[0007] Step S2: construct an embedding layer and use the embedding layer to convert Chinese data into vector representation;

[0008] Step S3: construct a bidirectional encoder and use the bidirectional encoder to extract context features;

[0009] Step S4: construct a naive Bayes classifier to perform classification;

[0010] Step S5: Perform model training and testing to obtain relationship extraction results.

[0011] According to the method of the first aspect of the present invention, in step S1:

[0012] Use web crawler technology to collect text data; apply data cleaning methods to pre-process the data, including removing special characters and non-text characters;

[0013] All collected data are merged to create an electronic countermeasure equipment entity recognition and relationship extraction dataset, which contains entity categories, equipment types, electronic countermeasure professional terms, and corresponding relationships. The entity categories are labeled using the BIO entity labeling scheme, and the dataset is divided according to proportion.

[0014] According to the method of the first aspect of the present invention, in step S2:

[0015] Use word embedding models to convert the original Chinese text discrete variables into continuous vectors. The tokens used include CLS and SEP. CLS indicates the beginning of a sentence, and SEP is used to distinguish different sentences.

[0016] The input Chinese text uses position embedding to represent the position information of each word in the input sequence; segment embedding is used to represent the separation between different sentences; and token embedding is used to map each word in the text to its corresponding word vector. The above three types of embedding are added together to generate the output vector of the bidirectional encoder.

[0017] According to the method of the first aspect of the present invention, in step S3:

[0018] The bidirectional encoder is composed of multiple stacked encoder layers, which increases the model's representational power and depth. By stacking multiple encoder layers, the bidirectional encoder learns the semantics and associations of the input sequence. Each encoder layer consists of a multi-head self-attention mechanism and a feedforward neural network. The self-attention mechanism allows the model to interact at different positions in the sequence, associating information at each position with other positions.

[0019] A feedforward neural network consisting of two fully connected layers is configured after the self-attention mechanism; the feedforward neural network provides representation capabilities by performing nonlinear transformations on the output of the self-attention mechanism; the output after multiple encoder layers contains bidirectional encoding information of the entire input sequence and is classified by a naive Bayes classifier based on the extracted contextual features.

[0020] According to the method of the first aspect of the present invention, in step S4:

[0021] The Naive Bayes classifier is expressed as:

[0022]

[0023] Among them, y represents the category, x i represents the i-th feature of the sample, P(y) represents the prior probability of the category, P(x i|y) represents the conditional probability of the i-th feature under category y.

[0024] According to the method of the first aspect of the present invention, in step S5:

[0025] Each training batch contains 64 samples, the initial learning rate of the model is set to 2e-5, the maximum input sequence length of the model is set to 128, the hidden layer size of the bidirectional encoder is 768, and the training process is repeated for 100 rounds.

[0026] During the testing process, the evaluation indicators include precision, recall, and F1 value;

[0027] The accuracy calculation formula is:

[0028]

[0029] The recall calculation formula is:

[0030]

[0031] The formula for calculating the F1 value is:

[0032]

[0033] Among them, TP means that the classifier prediction result is a positive sample, FP means that the classifier prediction result is a positive sample, and FN means that the classifier prediction result is a negative sample.

[0034] A second aspect of the present invention provides an electronic warfare equipment relationship extraction system based on BERT and a Bayesian classifier, the system comprising a processing unit configured to execute:

[0035] Preprocess text data and add labels based on classification results;

[0036] Build an embedding layer and use it to convert Chinese data into vector representation;

[0037] Build a bidirectional encoder and use it to extract context features;

[0038] Construct a naive Bayes classifier for classification;

[0039] Perform model training and testing to obtain relationship extraction results.

[0040] A third aspect of the present invention discloses an electronic device. The electronic device includes a memory and a processor. The memory stores a computer program. When the processor executes the computer program, it implements the electronic warfare equipment relationship extraction method based on BERT and Bayesian classifier according to the first aspect of the present disclosure.

[0041] A fourth aspect of the present invention discloses a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the electronic warfare equipment relationship extraction method based on BERT and a Bayesian classifier according to the first aspect of the present disclosure.

[0042] In summary, the present invention significantly improves the accuracy of electronic countermeasure equipment relationship extraction by introducing BERT's word embedding model and bidirectional encoder, combined with a naive Bayes classifier. It uses the word embedding model to effectively handle the polysemy and abbreviation problems in the electronic countermeasure equipment text, and uses a bidirectional encoder to capture electronic countermeasure-related terms and their contextual information. At the same time, the naive Bayes classifier is combined to further optimize the classifier's decision-making process. By combining the knowledge and experience of domain experts, the prior probability of the naive Bayes classifier is fine-tuned. This innovative measure not only improves the robustness of the model when processing electronic countermeasure equipment relationship extraction, but also ensures that the model can maintain high classification performance even with limited labeled data. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the specific embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0044] Figure 1 A flowchart of the model relationship extraction of the present invention;

[0045] Figure 2 It is the overall model diagram of the present invention;

[0046] Figure 3 is a schematic diagram of a bidirectional encoder of the present invention;

[0047] Figure 4 Schematic diagram of the accuracy results of the model comparison experiment of the present invention;

[0048] Figure 5 Schematic diagram of the recall rate 1 result of the comparative experiment of the model of the present invention;

[0049] Figure 6 Schematic diagram of the F1 value results of the comparative experiment of the model of the present invention. DETAILED DESCRIPTION

[0050] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.

[0051] The proposed solution combines the advantages of the BERT model in text representation with the simplicity and efficiency of the naive Bayes classifier in classification. This approach, verified in practical applications in the field of electronic countermeasures equipment, has shown that combining the BERT model with the naive Bayes classifier significantly improves the F1 value of relation extraction, reduces misclassifications and missed recognitions, and provides strong technical support for information processing and analysis of electronic countermeasures equipment, further promoting the development of informatization in the field.

[0052] First, the needs of the relationship extraction task in the field of electronic countermeasures are analyzed; secondly, the basic principles of the BERT model and the naive Bayes classifier are briefly analyzed; then, the BERT-NB relationship extraction model and its training process are explained; finally, experiments are conducted to verify the effectiveness of this method in the relationship extraction task in the field of electronic countermeasures.

[0053] The first aspect of the present invention proposes an electronic warfare equipment relationship extraction method based on BERT and Bayesian classifier, such as Figure 1-2 As shown, the method includes:

[0054] Step S1: pre-process the text data and add labels based on the classification results;

[0055] Step S2: construct an embedding layer and use the embedding layer to convert Chinese data into vector representation;

[0056] Step S3: construct a bidirectional encoder and use the bidirectional encoder to extract context features;

[0057] Step S4: construct a naive Bayes classifier to perform classification;

[0058] Step S5: Perform model training and testing to obtain relationship extraction results.

[0059] Step S1: Preprocess the text data and label it according to classification. There is a lack of publicly available Chinese datasets for electronic countermeasure equipment relationship extraction. Given the high degree of expertise in the field of electronic countermeasures, the large number of target and classification categories, and the sensitive nature of the relevant information, collecting relevant data is challenging, making the construction of an electronic countermeasure equipment dataset somewhat challenging. To address this, we used web crawling technology to collect text data from sources such as Baidu Baike, Wikipedia, major public defense websites, military news websites, and WeChat official accounts. We then applied data cleaning methods, including removing special characters and non-text characters, and preprocessed the data. Finally, all collected data was merged to create a 210,000-character electronic countermeasure equipment entity recognition and relationship extraction dataset, named ELWF. ELWF contains eight entity categories, encompassing over 2,000 equipment types, over 200 electronic countermeasure terminology, and 15 corresponding relationships. Entity categories were labeled using the BIO entity labeling scheme, and the dataset was partitioned into an 8:1:1 ratio.

[0060] Step S2: Build the Embedding layer to convert Chinese data into vector representations; in the specific implementation process, embedding is used to convert the original Chinese text discrete variables into continuous vectors. The special markers used are CLS, which indicates the beginning of a sentence, and SEP, which is used to distinguish different sentences. The input Chinese text uses position embedding to represent the position information of each word in the input sequence. Segment embedding is used to represent the separation between different sentences. Token embedding is used to map each word in the text to its corresponding word vector. Finally, these three types of embedding are added together to generate the output vector representation of the Bi-Encoder module.

[0061] Step S3: Build a bidirectional encoder (Bi-Encoder) to extract context features; the Bi-Encoder module is composed of multiple encoder layers (a form of Transformer) stacked together, such as Figure 3 As shown in the figure. The purpose of stacking encoders is to increase the representation power and depth of the model. By stacking multiple encoder layers, the Bi-Encoder module is able to better learn the complex semantics and associations of the input sequence. Each encoder layer consists of a multi-head self-attention mechanism and a feedforward neural network. The self-attention mechanism allows the model to interact at different positions in the sequence, associating the information at each position with other positions. The self-attention mechanism is followed by a feedforward neural network, which consists of two fully connected layers. The feedforward neural network provides richer representation capabilities by performing nonlinear transformations on the output of the self-attention mechanism. The output after multiple encoder layers contains the bidirectional encoded information of the entire input sequence and is classified by a naive Bayes classifier.

[0062] Step S4: Construct a naive Bayes classifier for classification. The naive Bayes classifier is based on Bayes' theorem and adopts the "attribute conditional independence assumption" for known categories, that is, it assumes that all attributes are independent of each other. This theorem describes the conditional probability relationship between two events A and B:

[0063]

[0064] In classification problems, sample features and categories are used as conditions and conditional probabilities. The core of the Naive Bayes classifier is to calculate the probability of sample features corresponding to each category and select the category with the highest probability as the classification output. The mathematical model of the Naive Bayes classifier can be expressed as:

[0065]

[0066] Among them, (y) represents the category, (x_i) represents the (i)th feature of the sample, (P(y)) represents the prior probability of the category, and (P(x_i|y)) represents the conditional probability of the (i)th feature under category (y).

[0067] The Naive Bayes classifier is widely used in many fields, especially in text classification. Due to its simplicity and efficiency, the Naive Bayes classifier has significant advantages in processing large-scale datasets.

[0068] Step S5: training and testing the model extraction effect;

[0069] Parameter settings and evaluation indicators

[0070] In this research paper, the experimental section uses an Embedding layer to encode text data. Subsequently, a Bi-Encoder module is used to learn the complex semantics and associations of the input sequence from the encoded text, and finally a Bayesian classifier is used for classification. The key parameters of the model are set according to Table 1, which lists their respective values.

[0071] Table 1 Model parameter settings

[0072] Parameter Value Batchsize 64 Epoch 100 Maxlength 128 Lr 2e-5 Hiddensize 768

[0073] In the experimental setup, each training batch contained 64 samples. The model's initial learning rate was set to 2e-5. The model's maximum input sequence length was set to 128. The hidden layer size of the BiEncoder module was 768. Training was performed for a total of 100 epochs. This paper uses commonly used evaluation metrics in the field of natural language processing: precision, recall, and F1 score. Precision measures the accuracy of the model's predictions for positive examples and is calculated as follows:

[0074] Precision measures the accuracy of the positive examples in the model's prediction results. Its calculation formula is as follows:

[0075]

[0076] Recall evaluates the model's ability to cover positive instances. Its calculation formula is as follows:

[0077]

[0078] The F1 value is a comprehensive metric that evaluates the overall performance of the model by considering precision and recall.

[0079]

[0080] Among them, TP, FP, and FN have the following meanings:

[0081] TP: The classifier predicts that the result is a positive sample.

[0082] FP: The classifier predicts that the result is a positive sample.

[0083] FN: The classifier predicts a negative sample.

[0084] In this study, we conducted a comparative analysis of four named entity recognition models, namely BERT, BERT-LSTM, BERT-BiLSTM, and BERT-NB. These models were evaluated on the ELWF dataset. Figure 4-6 The graphs corresponding to the various indicators of all models during the training process are shown (the horizontal axis unit is 10 epochs), such as Figure 4-6 As shown in Figure 2, on the ELWF dataset, various metrics stabilize and become smoother after approximately 20 epochs of training. The goal is to comprehensively evaluate and understand the performance of these models across different text types. Evaluation metrics include precision, recall, and F1 score, which serve as benchmarks to measure the accuracy and completeness of the models in identifying named entities. The results are shown in Table 2.

[0085] Table 2 Model comparison results

[0086] Model P R F1 BERT 88.87％ 92.12％ 90.46％ BERT-LSTM 86.48％ 87.86％ 87.16％ BERT-BiLSTM 92.60％ 93.86％ 93.22％ BERT-NB 94.86％ 94.97％ 94.93％

[0087] On the ELWF dataset, the BERT-NB model demonstrated exceptional performance, achieving a precision of 94.86%, a recall of 94.97%, and an F1 score of 94.93%. Compared to the baseline BERT model, the BERT-NB model, which integrates a Bayesian classifier, achieved an F1 score improvement of 4.5 percentage points. Furthermore, compared to two common BERT-based improvements, BERT-LSTM and BERT-BiLSTM, BERT-NB outperformed the other models in all test metrics. This demonstrates that the BERT model significantly improves both precision and recall by integrating a naive Bayesian classifier.

[0088] In summary, the BERT and Naive Bayes classifier model combination effectively leverages BERT's context-aware feature extraction capabilities, improving generalization and reducing reliance on feature engineering, thereby enhancing classification accuracy. BERT is particularly effective when processing long sequences and complex semantic relationships, capturing long-range dependencies and addressing the limitations of Naive Bayes' feature independence assumption. This combination makes the model highly adaptable, enabling rapid adaptation to different languages ​​and domains, while maintaining the computational efficiency and a degree of interpretability of the Naive Bayes classifier.

[0089] Please note that the technical features of the above embodiments can be combined arbitrarily. In order to make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification. The above embodiments only express several implementation methods of the present application. The description is relatively specific and detailed, but it cannot be understood as a limitation on the scope of the invention patent. It should be pointed out that for ordinary technicians in this field, without departing from the concept of this application, several variations and improvements can be made, which all fall within the scope of protection of this application. Therefore, the scope of protection of the patent in this application shall be based on the attached claims.

Claims

1. A method for extracting electronic warfare equipment relations based on BERT and Bayesian classifier, characterized in that: The method comprises: Step S1: pre-process the text data and add labels based on the classification results; Step S2: construct an embedding layer and use the embedding layer to convert Chinese data into vector representation; Step S3: construct a bidirectional encoder and use the bidirectional encoder to extract context features; Step S4: construct a naive Bayes classifier to perform classification; Step S5: Perform model training and testing to obtain relationship extraction results.

2. The method for extracting electronic warfare equipment relationships based on BERT and Bayesian classifier according to claim 1, characterized in that: In step S1: Use web crawler technology to collect text data; apply data cleaning methods to pre-process the data, including removing special characters and non-text characters; All collected data are merged to create an electronic countermeasure equipment entity recognition and relationship extraction dataset, which contains entity categories, equipment types, electronic countermeasure professional terms, and corresponding relationships. The entity categories are labeled using the BIO entity labeling scheme, and the dataset is divided according to proportion.

3. The method for extracting electronic warfare equipment relationships based on BERT and Bayesian classifier according to claim 2, characterized in that: In step S2: Use word embedding models to convert the original Chinese text discrete variables into continuous vectors. The tokens used include CLS and SEP. CLS indicates the beginning of a sentence, and SEP is used to distinguish different sentences. The input Chinese text uses position embedding to represent the position information of each word in the input sequence; segment embedding is used to represent the separation between different sentences; and token embedding is used to map each word in the text to its corresponding word vector. The above three types of embedding are added together to generate the output vector of the bidirectional encoder.

4. The method for extracting electronic warfare equipment relationships based on BERT and Bayesian classifier according to claim 3, characterized in that: In step S3: The bidirectional encoder is composed of multiple stacked encoder layers, which increases the model's representational power and depth. By stacking multiple encoder layers, the bidirectional encoder learns the semantics and associations of the input sequence. Each encoder layer consists of a multi-head self-attention mechanism and a feedforward neural network. The self-attention mechanism allows the model to interact at different positions in the sequence, associating information at each position with other positions. A feedforward neural network consisting of two fully connected layers is configured after the self-attention mechanism; the feedforward neural network provides representation capabilities by performing nonlinear transformations on the output of the self-attention mechanism; the output after multiple encoder layers contains bidirectional encoding information of the entire input sequence and is classified by a naive Bayes classifier based on the extracted contextual features.

5. The method for extracting electronic warfare equipment relationships based on BERT and Bayesian classifier according to claim 4, characterized in that: In step S4: The Naive Bayes classifier is expressed as: Among them, y represents the category, x i represents the i-th feature of the sample, P(y) represents the prior probability of the category, P(x i |y) represents the conditional probability of the i-th feature under category y.

6. The method for extracting electronic warfare equipment relationships based on BERT and Bayesian classifier according to claim 5, characterized in that: In step S5: Each training batch contains 64 samples, the initial learning rate of the model is set to 2e-5, the maximum input sequence length of the model is set to 128, the hidden layer size of the bidirectional encoder is 768, and the training process is repeated for 100 rounds. During the testing process, the evaluation indicators include precision, recall, and F1 value; The accuracy calculation formula is: The recall calculation formula is: The formula for calculating the F1 value is: Among them, TP means that the classifier prediction result is a positive sample, FP means that the classifier prediction result is a positive sample, and FN means that the classifier prediction result is a negative sample.

7. An electronic warfare equipment relationship extraction system based on BERT and Bayesian classifier, characterized by: The system comprises a processing unit configured to perform: Preprocess text data and add labels based on classification results; Build an embedding layer and use it to convert Chinese data into vector representation; Build a bidirectional encoder and use it to extract context features; Construct a naive Bayes classifier for classification; Perform model training and testing to obtain relationship extraction results.

8. An electronic device, characterized in that: The electronic device includes a memory and a processor, the memory stores a computer program, and when the processor executes the computer program, it implements the electronic warfare equipment relationship extraction method based on BERT and Bayesian classifier as described in any one of claims 1-6.

9. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, and when the computer program is executed by the processor, it implements the electronic warfare equipment relationship extraction method based on BERT and Bayesian classifier as described in any one of claims 1-6.

Images