Class boundary prediction method based on mode selection and generation
By using the class boundary prediction method based on pattern selection and generation when data is insufficient, the boundary degree of the sample is calculated and data is generated in different directions of the geometric structure is generated, and the problem of difficulty in obtaining boundary patterns in the prior art is solved, and more accurate anomaly detection and incremental learning performance are achieved.
Patent Information
- Application Number
- CN202411597058.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-11
- Publication Date
- 2025-03-11
AI Technical Summary
The prior art is difficult to accurately obtain boundary patterns when data is insufficient, resulting in the generation of edge patterns in tasks such as anomaly detection, incremental learning, and clustering that is limited to the existing distribution range and may increase the risk of catastrophic forgetting.
A class boundary prediction method based on pattern selection and generation is proposed. By calculating the boundary degree of the sample, building a diffusion model and generating data in different directions of the geometric structure, iteratively evaluate and generate until the boundary missing degree condition is met, class boundary is predicted.
This method can generate more representative samples, improve the understanding of data structures and obtain key details, significantly improve the performance of anomaly detection, and effectively alleviate catastrophic forgetting in incremental learning.
Smart Images
Figure CN119598120A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical fields of pattern recognition, intelligent information processing and data mining. Background Art
[0002] Boundary patterns often have special properties because they may have characteristics of different categories or clusters at the same time. The prediction of boundary patterns is important for many machine learning and data mining tasks. For example, in anomaly detection, Li et al. proposed a method to identify boundary points and outliers by using the inaccessibility of points based on the self-representation of data. In incremental learning, Liu et al. proposed a new automatic framework called "mnemonics" to overcome catastrophic forgetting by parameterizing and optimizing examples end-to-end. Experiments show that "mnemonics" examples are often located on the boundaries between different categories.
[0003] By identifying and understanding boundary patterns, we can better understand the structure of the data, improve the performance of the model, and help discover potential anomalies or important data. However, it is difficult to accurately obtain boundary patterns. At present, edge patterns are used as key information in tasks such as anomaly detection, incremental learning, and clustering. For example, in anomaly detection, Lee et al. proposed a method for generating pseudo target data and pseudo anomaly data based on geometric edge probability for the selection of OCSVM hyperparameters. Xiao et al. proposed a method for selecting hyperparameters for GP OCC based on the prediction difference between edge and internal positive training samples. Xiao et al. proposed a Gaussian kernel parameter selection method based on the difference between the distance between edge and internal samples to the OCSVM hyperplane in the feature space. Wang et al. proposed a data shift mechanism that generates pseudo outliers by moving edge patterns along the negative direction of the data density gradient, thereby realizing the selection of OCSVM hyperparameters. In incremental learning, Dang et al. proposed an incremental learning method based on class boundary sample selection (CBesIL), which preserves previous knowledge by using class boundary samples as key examples. Dang et al. proposed a method for selecting key examples, which retains the high-density and inter-class overlap areas in known samples as the prior distribution to evaluate the classification reliability of unknown samples. Li et al. proposed a density-based sample selection strategy, which selects samples that are closer to the classification boundary and samples with larger likelihood values. Gao et al. proposed a classifier correction method based on boundary features, which uses the boundary features of the old and new categories to fine-tune the classifier parameters and correct the bias of the classifier in incremental learning. In clustering, Punit et al. used Maximin sampling to identify boundary points in clusters to improve the partition quality. In practice, due to insufficient sampling and the difficulty in obtaining real environment samples, data sets usually have insufficient data, making the boundary mode different from the edge mode. Since the edges formed by insufficient data do not have key features between categories, it may be inappropriate to generate negative samples based on them in anomaly detection, because this is likely to cause the generated negative samples to fall on the original distribution. Similarly, in incremental learning, the risk of catastrophic forgetting may increase due to the selection of non-boundary samples as key examples.
[0004] At present, for question 1, Li et al. proposed that the number of negative components can be used to evaluate the degree to which the data is an outlier and a boundary point. This provides a good idea for the analysis of data structure, but does not give accurate judgment conditions for different attribute data such as outliers, boundary points, and internal points. For question 2, as far as we know, there are no relevant results yet. When data is insufficient, most existing methods use sampling methods and minimum feature lines to generate pseudo targets. Although these methods can increase the amount of data, the pseudo samples generated are limited to the existing distribution range. Summary of the invention
[0005] The present invention provides a class boundary prediction method based on pattern selection and generation, and the method is implemented by the following steps:
[0006] Step 1: Class data input
[0007] Select and determine the target class data that requires class boundary prediction from the data set;
[0008] Step 2: Model building and training
[0009] The model has a constraint based on which the diffusion model is determined on the geometry of the selected edge pattern and its neighborhood, and data are generated in three different directions;
[0010] The specific implementation process is as follows:
[0011] Boundary degree calculation: The negative components of points on the manifold boundary help reduce the reconstruction error in the affine space. At the same time, the number of negative components increases with the distance from the point to the center. The number of negative components is defined as the boundary degree; for the data set X = {x1, x2, ..., x n}∈R d×n , each column vector represents an observation value, d represents the number of features, and n represents the number of observation values; the boundary degree calculation formula is as follows:
[0012]
[0013] Among them, D(x i ) refers to the observed value x i The boundary degree of i, j∈{1, 2, ..., n}, when W i，j <0, otherwise Where W represents the self-representation matrix, which is obtained by the following constraints:
[0014]
[0015] Among them, st means subject to...;
[0016] Definition 1: Given a dataset X∈R d×n , x i ∈X,i∈{1,2,...,n};x i The nearest neighbor point of The calculation formula is as follows;
[0017]
[0018] Definition 2: Given a dataset X∈R d×n , x i The nearest neighbor The relationship is defined as three geometric structures:
[0019] (a) If It extends outwards;
[0020] (b) If It extends inwards;
[0021] (c) If It is parallel extension;
[0022] Definition 3: Given a dataset X∈R d×n , x i A triplet of is The calculation formula is as follows;
[0023]
[0024] Where j1, j2∈{1, 2, ..., n};
[0025] For the dataset X∈R d×n , according to Definition 3, we get a set of triples in Contains x i The geometric structure information of the two data in the neighborhood includes outward extension, inward extension and parallel extension; further, the present invention encodes outward extension, inward extension and parallel extension as 2, 0, 1 respectively; when the diffusion model input x i ∈X, condition c is added in the reverse process, which consists of c1 and c2. At this time, a neighborhood data information in the triple data is used, c1 = x i1 , c2 is x j1 With x i The geometric structure information between c1 and x i The output of the fully connected layer is 128 dimensions. c2 and t are also mapped to 128 dimensions through the Embedding layer, and then their vectors are added to obtain a new fusion feature. Finally, the diffusion model is made according to x through three layers of fully connected layers. i(t) And condition c to learn x i(t-1) The probability distribution of x i(t) represents x in the forward process of the diffusion model i The result after adding noise at time t, x i(t-1) represents x in the forward process of the diffusion model i The result after adding noise at time t-1; when using the other neighborhood data information in the triplet, c1 = x j2 , c2 is x j2 With x iThe geometric structure information between them; when all the data in the dataset X are trained, random noise and condition c1=x are given in the reverse process of the diffusion model g ∈X, g∈{1, 2, ..., n} and c2∈{0, 1, 2}, the model outputs x g Inward extension, parallel extension and outward extension data;
[0026] Step 3: Edge mode estimation
[0027] The local gradient method BEPS is widely used in edge mode estimation to determine the data sample x i Whether it is an edge sample depends mainly on the distribution of its neighboring samples on both sides of the tangent plane of its imaginary class surface; the direction of the normal vector is x i The gradient direction at x i The average vector direction of the neighboring samples is consistent; the normal vector v i The calculation formula is as follows:
[0028]
[0029] in is the k nearest neighbors e samples; when the neighboring sample x j When on one side of the tangent plane, x i to x j The direction vector and normal vector v i If the angle is less than 90 degrees, the dot product is positive. Therefore, the proportion γ of neighboring samples with non-negative dot products is calculated to determine whether the sample is an edge sample, and finally the edge mode of the data set is obtained.
[0030] Step 4: Mode selection
[0031] For data set X, we get edge mode X based on BEPS evaluation. E and non-edge mode X I , X I =XX E , where the subscripts E and I represent the edge mode and internal mode of the data set respectively; their boundary degree value sets are and in, represents the boundary value of the edge mode and represents the boundary degree value of the internal mode and dictionary Key is the internal mode boundary value, is the number of samples corresponding to the boundary value; at the same time, o(l i )=n i ,ζ(n i )=li ;
[0032] When data is missing, X E It can be divided into two modes: active mode and non-active mode. The calculation formula is as follows:
[0033] X AP ={x i |D(x i )<k x , x i ∈X E}
[0034] X NAP ={x i |D(x i )≥k x , x i ∈X E}
[0035] In an ideal situation, the threshold However, affected by the noise in the dataset and the choice of neighborhood size in the local geometric gradient method, k x Should usually be less than This method is effective for k x The value is calculated as follows:
[0036]
[0037] Step 5: Calculation of Boundary Missing Degree
[0038] The number of samples in the active pattern and the non-active pattern are statistically analyzed, and the boundary missing degree bmd is calculated as follows:
[0039]
[0040] Where num(.) represents the number of samples in the corresponding mode; if bmd≤k, the class boundary can be predicted based on the current data and jump to step 7; otherwise, jump to step 6, usually k=0.1;
[0041] Step 6: Data Generation
[0042] For the data generation process, this method proposes an iterative generation method based on the condition c1 = X AP and c2∈{0, 1, 2} generate new data points X in three different directions G1 , where the subscript G1 represents the data generated for the first time,
[0043] Then aggregate the data X+XG1 Then perform edge mode estimation and jump to step 3;
[0044] Step 7: Class Boundary Prediction
[0045] Predict the class boundary based on all the data generated and obtained at the end, that is, take the samples evaluated as edge patterns in the data generated in the last step as the class boundary X B =(X+X G1 +X G2 , ..., +X Gn ) E .
[0046] The present invention predicts class boundaries based on pattern selection and generation. The obtained class boundaries have more representative samples than the edges of the original data, which is helpful for understanding the data structure and obtaining key details. In anomaly detection, the use of aggregated data for training can significantly improve the performance of anomaly detection. At the same time, generating potential abnormal samples based on class boundaries can also more accurately estimate the hyperparameter values of the anomaly detection model. In incremental learning, using class boundaries as key examples avoids the cumbersome sample selection process and can effectively alleviate the model's catastrophic forgetting of old knowledge. In addition, the method can be widely applied to various data types, including single-mode data, multi-mode data, single-category data, and multi-category data, and the data dimension will not limit the application of the method.
[0047] The scarcity of representative samples of data distribution often leads to the lack of key boundary information, which hinders the understanding of data structure and the extraction of key details. The present invention proposes a new class boundary prediction method based on pattern selection and generation, called CBPM. It distinguishes different edge patterns based on the negative component of the sample. Then, a new condition of the diffusion model is proposed, according to which data can be generated in three different directions determined by the geometric structure of the selected edge pattern and its neighborhood. Then, the aggregated data is evaluated and generated in a loop to obtain more potential representative samples reflecting the distribution of class data. The final edge pattern derived from this comprehensive data set is regarded as the class boundary. In addition, the method of the present invention can process different types of data. In the actual application process, it only needs to modify the input form according to the specific situation and appropriately configure the relevant parameters to achieve the class boundary prediction task, and has strong scene adaptability. BRIEF DESCRIPTION OF THE DRAWINGS
[0048] Figure 1 The technical flow chart of the method of the present invention is Example
[0049] The technical flow chart of the method of the present invention is as follows Figure 1As shown in the figure. The first step is to select and determine the class data that needs class boundary prediction. The second step is to build and train the conditional diffusion model based on the neighborhood geometry structure in the class data. The third step is to estimate the edge mode of the class data based on BEPS. The fourth step is to calculate the boundary degree value of the estimated edge mode and non-edge mode, and determine the threshold k based on the boundary degree value of the non-edge mode x , the boundary value in the edge mode is less than k x The samples with are considered as active patterns, and the rest are non-active patterns. In the fifth step, the number of samples in the active pattern and non-active pattern is statistically analyzed to calculate the boundary missing degree. If the boundary missing degree condition is met, jump to the seventh step. In the sixth step, new data is generated based on the active pattern and geometric structure, and then jump to the third step. When the boundary missing degree condition is met, the seventh step is performed to predict the class boundary.
[0050] Taking drone signal recognition as an example, the class boundary prediction method proposed in the present invention can be used to improve the accuracy of drone signal recognition. Specifically, for each type of drone signal, according to the method of the present invention, a conditional diffusion model is first built and trained based on the neighborhood geometry structure, and then the edge pattern of the class is estimated and the edge pattern is divided into two categories: active pattern and non-active pattern. Then, new data is generated based on the active pattern and the geometry structure, and then the aggregated data is evaluated and generated in a cycle. Finally, when the boundary missing degree condition is met, the class boundary is predicted for the class. Training the recognition model with the final aggregated data or class boundary as the training set can significantly improve the accuracy of drone signal recognition.
[0051] The method of the present invention can also be used in other application scenarios, such as signal anomaly detection, incremental learning, small sample learning, etc.
[0052] The method of the present invention can be implemented by programming in any computer programming language (such as Python language), and the recognition model and detection model based on the method can be implemented in any PC or embedded system.
Claims
1. A class boundary prediction method based on pattern selection and generation, characterized by: The method is implemented by the following steps: Step 1: Class data input Select and determine the target class data that requires class boundary prediction from the data set; Step 2: Model building and training The model has a constraint based on which the diffusion model is determined on the geometry of the selected edge pattern and its neighborhood, and data are generated in three different directions; The specific implementation process is as follows: Boundary degree calculation: The negative components of points on the manifold boundary help reduce the reconstruction error in the affine space. At the same time, the number of negative components increases with the distance from the point to the center. The number of negative components is defined as the boundary degree; for the data set X = {x1, x2, ..., x n }∈R d×n , each column vector represents an observation value, d represents the number of features, and n represents the number of observation values; The boundary degree calculation formula is as follows: Among them, D(x i ) refers to the observed value x i The boundary degree of i, j∈{1, 2, ..., n}, when W i，j <0, otherwise Where W represents the self-representation matrix, which is obtained by the following constraints: Among them, st means subject to...; Definition 1: Given a dataset X∈R d×n , x i ∈X,i∈{1,2,...,n};x i The nearest neighbor point of The calculation formula is as follows; Definition 2: Given a dataset X∈R d×n , x i The nearest neighbor The relationship is defined as three geometric structures: (a) If It extends outwards; (b) If It extends inwards; (c) If It is parallel extension; Definition 3: Given a dataset X∈R d×n , x i A triplet of is The calculation formula is as follows; Where j1, j2∈{1, 2, ..., n}; For the dataset X∈R d×n , according to Definition 3, we get a set of triples in Contains x i The geometric structure information of the two data in the neighborhood includes outward extension, inward extension and parallel extension; further, the present invention encodes outward extension, inward extension and parallel extension as 2, 0, 1 respectively; when the diffusion model input x i ∈X, condition c is added in the reverse process, which consists of c1 and c2. At this time, a neighborhood data information in the triple data is used, c1 = x i1 , c2 is x j1 With x i The geometric structure information between c1 and x i The output of the fully connected layer is 128 dimensions. c2 and t are also mapped to 128 dimensions through the Embedding layer, and then their vectors are added to obtain a new fusion feature. Finally, the diffusion model is made according to x through three layers of fully connected layers. i(t) And condition c to learn x i(t-1) The probability distribution of x i(t) represents x in the forward process of the diffusion model i The result after adding noise at time t, x i(t-1) represents x in the forward process of the diffusion model i The result after adding noise at time t-1; when using the other neighborhood data information in the triplet, c1 = x j2 , c2 is x j2 With x i The geometric structure information between them; when all the data in the dataset X are trained, random noise and condition c1=x are given in the reverse process of the diffusion model g ∈X, g∈{1, 2, ..., n} and c2∈{0, 1, 2}, the model outputs x g Inward extension, parallel extension and outward extension data; Step 3: Edge mode estimation The local gradient method BEPS is widely used in edge mode estimation to determine the data sample x i Whether it is an edge sample depends mainly on the distribution of its neighboring samples on both sides of the tangent plane of its imaginary class surface; the direction of the normal vector is x i The gradient direction at x i The average vector direction of the neighboring samples is consistent; the normal vector v i The calculation formula is as follows: in is the k nearest neighbors e samples; when the neighboring sample x j When on one side of the tangent plane, x i to x j The direction vector and normal vector v i If the angle is less than 90 degrees, the dot product is positive. Therefore, the proportion γ of neighboring samples with non-negative dot products is calculated to determine whether the sample is an edge sample, and finally the edge mode of the data set is obtained. Step 4: Mode selection For data set X, we get edge mode X based on BEPS evaluation. E and non-edge mode X I , X I =XX E , where the subscripts E and I represent the edge mode and internal mode of the data set respectively; their boundary degree value sets are and in, represents the boundary value of the edge mode and represents the boundary degree value of the internal mode and dictionary Key is the internal mode boundary value, is the number of samples corresponding to the boundary value; at the same time, o(l i )=n i ,ζ(n i )=l i ; When data is missing, X E It can be divided into two modes: active pattern and non-active pattern. The calculation formula is as follows: X AP ={x o |D(x i )<k x ,x i ∈x E } X NAP ={x i |D(x i )≥k x ,x i ∈x E } In an ideal situation, the threshold However, affected by the noise in the dataset and the choice of neighborhood size in the local geometric gradient method, k x Should usually be less than This method is effective for k x The value is calculated as follows: Step 5: Calculation of Boundary Missing Degree The number of samples in the active pattern and the non-active pattern are statistically analyzed, and the boundary missing degree bmd is calculated as follows: Where num(.) represents the number of samples in the corresponding mode; if bmd≤k, the class boundary can be predicted based on the current data and jump to (7); otherwise, jump to (6), usually k = 0.1; Step 6: Data Generation For the data generation process, this method proposes an iterative generation method based on the condition c1 = X AP and c2∈{0, 1, 2} generate new data points X in three different directions G1 , where the subscript G1 represents the data generated for the first time, and then the aggregate data X+X G1 Then perform edge mode estimation and jump to (3); Step 7: Class Boundary Prediction Predict the class boundary based on all the data generated and obtained at the end, that is, take the samples evaluated as edge patterns in the data generated in the last step as the class boundary X B =(X+X G1 +X G2 , ..., +X Gn ) E .