A data oversampling method based on Gaussian mixture model

Abstract

A data oversampling method based on Gaussian mixture model, comprising the following steps: First, for an imbalanced data set, use the Gaussian mixture model to denoise and cluster the minority class samples; Next, according to the idea of cost-sensitive learning, assign more oversampling quantities to the clusters with higher classifier prediction error rates under inter-cluster cross-validation; Finally, use the standard SMOTE algorithm to synthesize corresponding quantities of new samples inside each minority class sample cluster; The present invention precisely controls the original samples and quantities of oversampling by using the distribution information of the data itself and the idea of cost-sensitive learning on the premise of ensuring the integrity of data information, increases the generalization information of the oversampled samples, and reduces the classification difficulty of the classifier.

Description

Technical Field

[0001] The present invention belongs to the technical field of unbalanced data processing, and particularly relates to a data oversampling method based on a Gaussian mixture model. Background Art

[0002] Traditional machine learning algorithms assume that the data is roughly balanced in terms of categories. When encountering an imbalanced dataset, they tend to classify uncertain samples as majority-class samples, which affects the recall rate of minority-class samples. Although the number of minority-class samples is small, their importance is higher, and the cost of misclassifying minority-class samples is much higher than that of misclassifying majority-class samples. In real life, the vast majority of datasets people come into contact with are imbalanced datasets, and the data to be processed often has more imbalanced data, such as financial fraud transaction detection, radar signal recognition, medical diagnosis, some coding information in gene sequences, text classification, and so on. Therefore, it is necessary to study how to solve the class imbalance problem in machine learning. Data oversampling is an important method to solve the data imbalance problem in the field of machine learning. Data resampling mainly has two strategies: undersampling and oversampling. Existing research has shown that undersampling may cause the dataset to lose important information about the majority class, and its improvement of the overall performance of the classifier is often lower than that of oversampling. Summary of the Invention

[0003] In order to solve the defects and deficiencies of the existing oversampling technology, the present invention proposes a data oversampling method based on a Gaussian mixture model. First, use the Gaussian mixture model to model the probability of minority-class samples, and use the EM algorithm to iteratively solve the model parameters; then cluster the minority-class samples according to the obtained GMM probability distribution and filter out low-probability noise points; then use cross-validation and use relevant classifiers to train and predict each cluster respectively, introduce the idea of cost-sensitive learning, correspond the error rate with the number of oversampling, and give relatively higher sampling weights to the clusters with higher prediction error rates; finally, use the SMOTE algorithm to oversample each minority-class sample cluster to generate new data so that the number of minority-class and majority-class samples in the dataset is equal.

[0004] On the premise of ensuring the integrity of data information, the present invention uses the distribution information of the data itself and the idea of cost-sensitive learning to precisely control the original samples and the quantity of oversampling, increases the generalization information of the oversampled samples, and reduces the classification difficulty of the classifier.

[0005] A data oversampling method based on a Gaussian mixture model, characterized in that the steps are as follows:

[0006] Step 1: Model the generation probability of minority-class samples in the sample set with a Gaussian mixture model;

[0007] Step 2: After completing the clustering process, calculate the classification error rate of each cluster through cross-validation between clusters using a pre-trained classifier, and determine the oversampling quantity for each cluster according to the relative magnitude of the error rate of each cluster;

[0008] Step 3: According to the oversampling quantity determined in the previous step, use the SMOTE algorithm to synthesize new samples for each cluster in turn.

[0009] Furthermore, Step 1 specifically includes: Assume that the mixture model consists of M single Gaussian models, and the model form is as follows:

[0010]

[0011] where \(p(x|\theta i )\) is the \(i\)-th Gaussian sub-model, \(\alpha i \) represents the weight of the \(i\)-th sub-model, \(\theta i =(u i ,\(\Sigma i )\) represents the mean and covariance parameters of the \(i\)-th sub-model, \(\theta = (\alpha_1,\alpha_2,...,\alpha M ;\(\theta_1,\theta_2,...,\theta M )\), and use the EM algorithm to solve the parameter \(\theta\).

[0012] After determining the parameter \(\theta\), calculate the generation probability of each minority-class sample under this parameter, filter a certain number of low-probability samples (preset a probability threshold) to improve the generalization of the overall samples; then perform clustering division on the remaining samples in the following way: Assume that the total number of samples to be clustered is N, introduce the variable \(\gamma jm \), representing the posterior probability that the \(j\)-th sample is generated by the \(m\)-th sub-model:

[0013]

[0014] where \(j\in\{1,2,...,N\}\), \(i\in\{1,2,...,M\}\); Gaussian mixture clustering divides the minority-class sample set into M clusters \(C = \{C_1,C_2,...,C M \}\), and the cluster label \(\lambda j \) of each sample \(x j \) is determined in the following way:

[0015]

[0016] Furthermore, the value of M is determined according to the "elbow" rule: Fit the samples by taking values of K from 2 to 8 in turn and draw the change curve of the logarithmic likelihood loss value of the model, and select the value corresponding to the "elbow" position point of the curve as the final value of M.

[0017] Furthermore, Step 2 specifically includes:

[0018] Step 2-1: Use the minority class samples of M-1 clusters and all majority class samples to train a classifier. Use the samples of the remaining one minority class cluster to evaluate the prediction effect of the classifier, and calculate the classification error rate (the error rate is the ratio of the number of misclassified samples to the total number of classified samples). Repeat this process M times and record the classification error rates of each cluster (denoted as ε1, ε2,..., ε M );

[0019] Step 2-2: Calculate the Class Ratio (CR) according to the following formula:

[0020]

[0021] where N j is the number of samples in the minority class cluster C j , and N is the total number of minority class samples in the population; then multiply the above formula by the mixing weight to obtain the description degree d j of the minority class samples in the population for the cluster C j , and the calculation formula is as follows:

[0022] d j =α j ·CR j (5)

[0023] where α j is the weight of the cluster C j calculated in Step 1;

[0024] Step 2-3: According to the idea of cost-sensitive learning, calculate the oversampling weight of cluster j according to the following formula:

[0025]

[0026] where ε j is the classification error rate of the cluster C j ;

[0027] Finally, allocate the oversampling quantity according to the above sampling weight, and the calculation formula is as follows:

[0028] δ j =(S maj -S min )×w j (7)

[0029] where δ j is the final oversampling quantity of the cluster C j , S maj is the total number of majority class samples in the dataset, and S min is the total number of minority class samples.

[0030] Furthermore, Step 3 specifically includes: First, in the cluster Cj Randomly select δ j samples. If δ j is greater than the total number of samples in cluster C j , the situation of selecting the same sample multiple times will occur. Then for each of these samples, find its K-nearest neighbor samples in cluster C j . The value of K is determined according to the sampling effect, and any one of them is selected as the source of the synthetic sample. Finally, linearly interpolate between two minority-class sample points to synthesize new samples. The position of the new sample is calculated according to the following formula:

[0031] x syn = x + rand(0, 1) × (x t - x) (8)

[0032] where x is any sample in cluster C j , x syn is the newly synthesized sample corresponding to x, x t , t ∈ {1, 2,..., K} is one of the K-nearest neighbor samples of x, and rand(0, 1) represents a random number between 0 and 1.

[0033] The beneficial effects of the present invention are as follows: Since the present invention filters out noise points according to the probability distribution function of the samples, these noise points are avoided from reducing the prediction performance of the classifier. Then, the idea of cost sensitivity is used to assign different sampling weights to the samples in different clusters, optimizing the training direction of the classifier. Moreover, the present invention controls the selection range of the original samples for synthesizing new samples within each cluster, which can reduce the classification difficulty of the classifier because there may be some majority-class samples distributed between different clusters, and the newly synthesized samples sourced from inter-cluster samples may be closer to these majority-class samples in terms of attributes. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 is a flowchart of the method of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0035] The present invention will be further described below in conjunction with embodiments. The present invention includes but is not limited to the following embodiments.

[0036] In the field of electronic reconnaissance, identifying the radiation source type of intercepted radar signals is a basic task. After an electronic reconnaissance system detects a radar signal, it first separates various useful signals from the dense signal stream, and then precisely measures the characteristic parameters of the signals through telecommunications technology, including pulse width, radio frequency, power level, and scan period, etc. In the past, the task of further identifying the signal type based on the characteristic parameters often relied on manual experience comparison and analysis. In recent years, with the development of computer and artificial intelligence technologies, more and more research has applied machine learning methods to signal type recognition to establish a quantitative model for accurately identifying signal types. However, in reality, due to technical limitations and the scarcity of samples, etc., the number of detected signals is unbalanced in terms of categories. Signals for target detection, long-range early warning, etc. are more likely to be detected, while signals for guidance and fire control, etc. are rarely detected. Such signals often pose a greater threat and have a higher cost of misjudgment.

[0037] This embodiment provides a radar signal recognition method applying the oversampling method described above. This method includes processes such as signal feature dimensionality reduction, oversampling, and establishing an SVM recognition model, and the specific steps are as follows:

[0038] Step 1: Use the signal characteristics and types that have been identified in the electronic reconnaissance system database as the original data set. The signal characteristics are obtained by measuring the signals collected in the past through telecommunications technology, including radio frequency and its change characteristics, power level, pulse repetition interval and its modulation characteristics, pulse width and its modulation characteristics, intra-pulse frequency or phase modulation characteristics, antenna scan type, scan period, radiation pattern, and polarization characteristics. The signal types are obtained through manual analysis. Set a quantity threshold, divide all signals into two categories, divide the signals of each type with a quantity greater than the threshold into the majority class, and vice versa into the minority class. After dividing the categories, use the LLE (Locally Linear Embedding) algorithm to perform dimensionality reduction processing on the signal characteristics, so that the characteristics are reduced to a three-dimensional space;

[0039] Step 2: Model the generation probability of the oversampled minority-class signal samples using a Gaussian mixture model. Assume that this mixture model consists of M single Gaussian models (the value of M is determined according to the "elbow" rule: the value of K can be taken from 2 to 8 in turn to fit the samples and draw the change curve of the model log-likelihood loss value, and select the value corresponding to the "elbow" position point of the curve as the final value of M). The model form is as follows:

[0040]

[0041] where p(x|θ i ) is the i-th Gaussian sub-model, α i represents the weight of the i-th sub-model, and θ i =(u i ,Σ i) represents the mean and covariance parameters of the ith sub-model, θ=(α1,α2,...,α M ;θ1,θ2,...,θ M ), use the EM algorithm to solve the parameter θ.

[0042] After the parameter θ is determined, the generation probability of each minority class signal sample under the parameter is calculated, and a certain number of low-probability samples are filtered to improve the generalization of the overall signal samples; then the remaining signals are clustered as follows: Assuming that the total number of samples to be clustered is N, the variable γ is introduced jm , represents the posterior probability that the jth sample is generated by the mth sub-model:

[0043]

[0044] Where j∈{1,2,...,N},i∈{1,2,...,M}; Gaussian mixture clustering divides the minority class sample set into M clusters C={C1,C2,...,C M}, each sample x j The cluster label λ j Determined as follows:

[0045]

[0046] Step 3: After the clustering process is completed, the classification error rate of each signal sample cluster is calculated by performing inter-cluster cross-validation on the pre-trained classifier, and the number of oversampling is determined according to the relative size of the error rate of each cluster. j For example, the specific process is as follows:

[0047] Step 3-1: Use the minority class signals of K-1 clusters and all the majority class signal samples to train the SVM classification model (the kernel function is the radial basis function), use the samples of the remaining minority class cluster to evaluate the prediction effect of the SVM classifier, and calculate the classification error rate (the error rate is the ratio of the number of samples with classification errors to the total number of classification samples). Repeat this cycle M times and record the classification error rate of each cluster (denoted as ε1, ε2, ..., ε M );

[0048] Step 3-2: Calculate the Class Ratio (CR) according to the following formula:

[0049]

[0050] where N j It is a minority cluster C j The number of samples in, N is the number of total minority class samples; then multiply the above formula by the mixing weight to get cluster C j Descriptive degree d for the total minority class samplesj , the calculation formula is as follows:

[0051] d j =α j ·CR j (5)

[0052] where α j is the weight of cluster C calculated in step 2; j

[0053] Step 3-3: According to the cost-sensitive learning idea, calculate the oversampling weight of cluster j according to the following formula:

[0054]

[0055] where ε j is the classification error rate of cluster C; j

[0056] Finally, allocate the oversampling quantity according to the above sampling weights, and the calculation formula is as follows:

[0057] δ j =(S maj -S min )×w j (7)

[0058] where δ j is the final oversampling quantity of cluster C, S j is the total number of majority-class samples in the dataset, and S maj is the total number of minority-class samples; min

[0059] Step 4: According to the oversampling quantity determined in the previous step, use the SMOTE algorithm to synthesize new signal samples for each cluster in turn. Taking cluster C j as an example, the specific process is as follows: First, randomly select δ j samples in cluster C j (if δ j is greater than the total number of samples in cluster C j , there will be a situation where the same sample is selected multiple times); then for each of these samples, find its K nearest neighbor samples in cluster C j (the value of K can be determined according to the sampling effect), and randomly select one as the source of the synthesized sample; finally, linearly interpolate between two minority-class sample points to synthesize a new sample, and the position of the new sample is calculated according to the following formula

[0060] x syn =x+rand(0,1)×(x t -x) (8)

[0061] where x is in cluster C jAny signal sample in, x syn is the newly synthesized signal sample corresponding to x, x t , t ∈ {1, 2,..., K} is one of the K-nearest neighbor samples of x, and rand(0, 1) represents a random number between 0 and 1;

[0062] Step 5: Use the SVM model to build a model for signal recognition. Since the radial basis function (RBF) can intuitively reflect the clustering between data and has a better classification effect than other kernel functions, the RBF is selected as the SVM kernel function in this embodiment; then use the dataset processed by the previous step of balancing to train the model, and perform a grid search on the two model hyperparameters, the SVM penalty factor C and the kernel parameter σ, within a certain range, and select the one with the best effect. Finally, use the trained model for the recognition of newly detected signals.

[0063] In this embodiment, on the premise of ensuring data integrity, for the imbalance problem existing in radar signal data, an oversampling method based on the Gaussian mixture model is used to augment the data to improve the recognition accuracy.

[0064] The above embodiments are only used to illustrate the technical concept and features of the present invention, and the purpose is to enable those who are familiar with this technology to understand the content of the present invention and implement it accordingly, and cannot be used to limit the protection scope of the present invention. Any equivalent changes or modifications made according to the spirit and essence of the present invention should be covered within the protection scope of the present invention.

Claims

1. A radar signal recognition method applying a data oversampling method based on Gaussian mixture model, including the processes of signal feature dimensionality reduction, oversampling, and establishing an SVM recognition model. The specific steps are as follows: Step 1: Use the signal features and types already recognized in the electronic reconnaissance system database as the original data set. The signal features are obtained by measuring the signals collected in the past through telecommunications technology, including radio frequency and its change characteristics, power level, pulse repetition interval and its modulation characteristics, pulse width and its modulation characteristics, in-pulse frequency or phase modulation characteristics, antenna scanning type, scanning period, radiation pattern, and polarization characteristics. The signal types are obtained through manual analysis. Set a quantity threshold, divide all signals into two categories, divide the signals of each type with a quantity greater than the threshold into the majority class, and vice versa into the minority class. After classifying, use the locally linear embedding algorithm to perform dimensionality reduction processing on the signal features, reducing the features to a three-dimensional space. Step 2: Use the Gaussian mixture model to model the generation probability of the oversampled minority-class signal samples. Assume that the mixture model consists of M single Gaussian models, and the value of M is determined according to the "elbow" rule: The value of K can be successively taken from 2 to 8 to fit the samples and draw the change curve of the model log-likelihood loss value, and select the value corresponding to the "elbow" position point of the curve as the final value of M. The model form is as follows: where p(x|θ i ) is the i-th Gaussian mixture model, and α i represents the weight of the i-th mixture model. θ i =(u i , Σ i ) represents the mean and covariance parameters of the i-th mixture model. θ = (α1, α2,..., α M ; θ1, θ2,..., θ M ). The EM algorithm is used to solve for the parameter θ; After the parameter θ is determined, calculate the generation probability of each minority-class signal sample under this parameter, filter a certain number of low-probability samples to improve the generalization of the overall signal samples; then cluster and divide the remaining signals in the following way: assume that the total number of samples to be clustered is N, and introduce the variable γ jm , representing the posterior probability that the j-th sample is generated by the m-th sub-model: where \(j\in\{1,2,\cdots,N\}\), \(i\in\{1,2,\cdots,M\}\); Gaussian mixture clustering divides the minority class sample set into \(M\) clusters \(C = \{C_1,C_2,\cdots,C\) M \}\), and the cluster label \(\lambda\) of each sample \(x\) j is determined as follows: j ​ Step 3: After completing the clustering process, calculate the classification error rate of each signal sample cluster through cross-validation between clusters by a pre-trained classifier, and determine the oversampling quantity according to the relative magnitude of the error rate of each cluster. The specific process is as follows: Step 3-1: Train an SVM classification model with the minority class signals of K - 1 clusters and all majority class signal samples, and use the samples of the remaining one minority class cluster to evaluate the prediction effect of the SVM classifier, calculate the classification error rate, loop M times in this way, record the classification error rates of each cluster, denoted as ε1, ε2,..., ε M ; Step 3-2: Calculate the class ratio CR according to the following formula: where N j is the number of samples in the minority cluster C j , and N is the total number of minority class samples; then multiply the above formula by the mixing weight to obtain the description degree d j of the total minority class samples j , and the calculation formula is as follows: d j = α j ·CR j (5) where α j is the weight of cluster C j calculated in step 2; Step 3-3: According to the cost-sensitive learning idea, calculate the oversampling weight of cluster C j as follows: where ε j is the classification error rate of cluster C j ; Finally, allocate the oversampling quantity according to the above sampling weights. The calculation formula is as follows: δ j = (S maj - S min ) × w j (7) where δ j is the final oversampling number of cluster C j , S maj is the total number of samples of the majority class in the dataset, and S min is the total number of samples of the minority class; Step 4: According to the oversampling quantity determined in the previous step, use the SMOTE algorithm to synthesize new signal samples for each cluster in turn. The specific process is as follows: First, randomly select δ j samples in cluster C j ; then for each of these samples, find its K nearest neighbor samples in cluster C j , where the value of K is determined according to the sampling effect, and randomly select one as the source of the synthesized sample; finally, linearly interpolate between two minority-class sample points to synthesize new samples, and the position of the new sample is calculated according to the following formula: x syn = x + rand(0,1)×(x t - x) (8) where x is any signal sample in cluster C j and x syn is the newly synthesized signal sample corresponding to x t , t ∈ {1, 2,..., K} is one of the K-nearest neighbor samples of x, and rand(0, 1) represents a random number between 0 and 1; Step 5: Use the SVM model to model signal recognition. Select RBF as the SVM kernel function, train the model with the balanced data set, and perform grid search on the two model hyperparameters, the SVM penalty factor C and the kernel parameter σ, and select the one with the best effect. Finally, use the trained model for the recognition of newly intercepted signals.

Images