Intrusion detection method based on data equalization algorithm framework

Abstract

The invention provides an intrusion detection method based on a data equalization algorithm framework, and the method comprises the steps: obtaining original network data, and carrying out the preprocessing of the original network data, and obtaining the preprocessed network data; performing data balancing processing on the preprocessed network data based on a data balancing algorithm framework to obtain balanced network data; performing feature selection processing on the balanced network data through an information gain algorithm to obtain low-dimensional network data; a CNN network model is improved in combination with a KAN to obtain an improved CKAN network model, and low-dimensional network data is classified through the improved CKAN network model to obtain a classification result. The improved CKAN model can better predict and classify the data, the classification accuracy is improved, the problem of data set imbalance can be solved to the greatest extent, and the training efficiency and the detection precision of the network model are improved while the redundancy of the network data to be detected is reduced.

Description

Technical Field

[0001] The present invention relates to the field of network information security and deep learning technology, and in particular to an intrusion detection method based on a data balancing algorithm framework. Background Art

[0002] The rapid development of big data, cloud computing, and 5G technologies, along with the widespread adoption of the Internet of Things (IoT), has ushered in a new era of the interconnectedness of everything. This, in turn, has introduced security risks from virtual networks into the physical world. Intrusion detection, as an effective proactive defense technology that can effectively safeguard network security, has been widely researched.

[0003] Intrusion detection problems can be categorized as binary or multi-classification. Based on this, many traditional machine learning classification methods have been applied to intrusion detection. However, traditional machine learning has limitations when processing massive, high-dimensional network data.

[0004] To address these issues, deep learning has been widely used in network intrusion detection. While deep learning can efficiently learn intrinsic data features and excels at processing complex, high-dimensional, and noisy data, it still has several drawbacks. First, collecting valid attack samples from the network is extremely difficult and requires significant human and material resources. This results in the number of attack samples in experiments being far smaller than the number of valid samples, leading to class imbalance in the dataset. Second, with such a small number of samples, traditional methods for processing imbalanced data and deep learning methods struggle to achieve effective classification. Summary of the Invention

[0005] In response to the shortcomings of the existing technology, the present invention provides an intrusion detection method based on a data balancing algorithm framework. The present invention can not only solve the problem of data set imbalance to the greatest extent, but also reduce the redundancy of the network data to be detected while improving the training efficiency and detection accuracy of the network model.

[0006] The technical solution of the present invention is: an intrusion detection method based on a data balancing algorithm framework, comprising the following steps:

[0007] S1) obtaining original network data and preprocessing it to obtain preprocessed network data;

[0008] S2), performing data balancing on the pre-processed network data based on the data balancing algorithm framework to obtain balanced network data;

[0009] S3) Perform feature selection on the balanced network data using the information gain algorithm to obtain low-dimensional network data;

[0010] S4) The CNN network model is improved by combining KAN to obtain an improved CKAN network model. The low-dimensional network data is classified by the improved CKAN network model to obtain the classification results.

[0011] Preferably, in step S2), data balancing is performed on the pre-processed network data based on a data balancing algorithm framework, specifically comprising the following steps:

[0012] S21) Constructing a data balancing sample model based on a data balancing algorithm framework;

[0013] S22) Performing data balancing on the pre-processed network data using a balanced sample model.

[0014] Preferably, in step S21), the data balanced sample model includes two sub-models, namely a VAGCUS undersampling model constructed based on a hierarchical clustering undersampling method and an attack sample generation model constructed based on a WGAN-GP framework; the majority class label samples are subjected to hierarchical clustering undersampling processing by the VAGCUS undersampling model; and the minority class label samples are oversampled using the attack sample generation model.

[0015] Preferably, in step S3), feature selection processing is performed on the balanced network data using an information gain algorithm, specifically:

[0016] S31), based on the size of the balanced network data, performing information entropy calculation processing on the balanced network data to obtain the information entropy value of the balanced network data;

[0017] S32), based on the dimensional characteristics of the balanced network data, performing conditional entropy calculation processing on the balanced network data to obtain a conditional entropy value of the balanced network data;

[0018] S33), performing a difference process on the information entropy value of the balanced network data and the conditional entropy value to obtain the information gain value of the balanced network data;

[0019] S34), sorting the balanced network data according to the information gain value and eliminating the balanced network data with an information gain value less than a preset threshold to obtain low-dimensional network data.

[0020] Preferably, in step S4), the improved CKAN network model includes a convolutional layer, a nonlinear activation layer, an adaptive pooling layer, and a KAN layer;

[0021] The KAN layer is a neural network layer inspired by the Kolmogorov-Arnold theorem.

[0022] Preferably, in step S4), the low-dimensional network data is convolved through a convolution layer to obtain the spatiotemporal features of the low-dimensional network data; and the local information is summarized through an adaptive pooling layer to reduce the dimension of the data to remove irrelevant features of the data; then the spatiotemporal features of the low-dimensional traffic data are flattened and classified through a KAN layer, and the classification results are output through the output layer.

[0023] The beneficial effects of the present invention are:

[0024] 1. The improved CKAN network model of the present invention can better predict and classify data, improve the accuracy of classification, and can maximize the solution to the problem of data set imbalance, while reducing the redundancy of the network data to be detected and improving the training efficiency and detection accuracy of the network model;

[0025] 2. The present invention uses the VAGCUS undersampling model to not only eliminate redundant data but also make the sample boundaries clearer, thereby improving the model's performance in predicting minority class samples. The VAGCUS undersampling model is used to undersample the majority class samples in the network data to obtain undersampled majority class samples.

[0026] 3. The present invention uses an attack sample generation model built based on the WGAN-GP framework to fill the minority class samples in the network data so that their number is the same as the undersampled majority class samples, converting the unbalanced dataset into a balanced dataset, thereby improving the performance of subsequent model predictions. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 Schematic diagram of the process of the present invention;

[0028] Figure 2 It is a flow chart of the method of the present invention;

[0029] Figure 3 This is a structural framework diagram of the improved CKAN network model of the present invention. DETAILED DESCRIPTION

[0030] The specific embodiments of the present invention will be further described below with reference to the accompanying drawings:

[0031] like Figure 1 and 2 As shown, this embodiment provides an intrusion detection method based on a data balancing algorithm framework, comprising the following steps:

[0032] S1) obtaining original network data and preprocessing it to obtain preprocessed network data;

[0033] In this embodiment, the pre-processing specifically includes the following steps:

[0034] S11), converting non-numerical data in the original network data into numerical features using entity embedding method;

[0035] S12) Use the Min-Max numerical method to normalize the converted numerical features; thereby scaling the data to a specific range, making the distribution closer to the normal distribution, and alleviating the problem of gradient explosion in subsequent training.

[0036] This embodiment divides the preprocessed network data into minority class label samples and majority class label samples.

[0037] S2) Performing data balancing on the pre-processed network data based on the data balancing algorithm framework to obtain balanced network data; specifically comprising the following steps:

[0038] S21) Constructing a data balancing sample model based on a data balancing algorithm framework;

[0039] In this embodiment, the data balanced sample model includes two sub-models, namely the VAGCUS undersampling model based on the hierarchical clustering undersampling method and the attack sample generation model based on the WGAN-GP framework;

[0040] S22) Performing data balancing on the pre-processed network data using the balanced sample model; specifically:

[0041] The majority class label samples are undersampled using the VAGCUS undersampling model to obtain undersampled data; the minority class label samples are oversampled using the attack sample generation model to obtain oversampled data; the undersampled data and oversampled data are fused to obtain balanced network data.

[0042] In this embodiment, the attack sample generation model includes a generator G and a discriminator D; and the generator G and the discriminator D are both feedforward neural networks; the generator G includes an input layer, an output layer and a hidden layer, the activation function of the output layer is Linear, and the activation function of the hidden layer is ReLu; wherein the number of neurons in the input layer is equal to the noise dimension, and the number of neurons in the output layer is equal to the real sample dimension.

[0043] The discriminator D includes an input layer, an output layer and a hidden layer, wherein the number of neurons in the input layer is equal to the real sample dimension, the number of neurons in the output layer is 1; the activation function of the output layer is Linear, and the activation function of the hidden layer is ReLu.

[0044] The objective function L of the discriminator of the attack sample generation model is:

[0045]

[0046] In the formula, E represents the mathematical expectation of the distribution function; D represents the discriminator; x represents the real sample; Indicates the generated sample; represents the random interpolation sampling points between the real sample and the generated sample; P represents the data distribution; P data is the distribution of real sample data; P G To generate the sample distribution, Defined as the distribution of real sample data P data And the generated sample distribution P G The uniform distribution of the sampled point pairs along the straight line; λ is the gradient penalty coefficient; Represents the gradient.

[0047] Pre-train the attack sample generation model, specifically:

[0048] S2111), initializing the parameters of the generator G and the discriminator D networks, and defining a noise distribution that obeys a normal distribution;

[0049] S2112), prepare real data and noise data; the real data is obtained from a few attack classifications in the preprocessed network dataset; first take one of the attack subsets DS attack , the noise is obtained from the noise distribution and DS attack Attack noise with the same amount of data;

[0050] S2113), fix the generator G and train the discriminator D; the noise generates the same number of samples through the generator G attack , using DS attack and Sample attack Train the discriminator D so that the discriminator D can distinguish whether the data comes from DS attack The real data still comes from Sample attack ;

[0051] S2114), fix the discriminator D and train the generator G; train the generator G with the discriminator D obtained after 100 rounds of training according to step S2113), so that the discriminator D is as unable to distinguish whether the data comes from DS as possible. attack The real data still comes from Sample attack ;

[0052] S2115) After 1000 update iterations according to steps S2113) and S2114), the final attack sample generator is obtained;

[0053] S2116) Use each attack subset in turn to train the generative adversarial network according to the above steps, and finally obtain different attack sample generators.

[0054] In this embodiment, the VAGCUS undersampling model divides the majority class samples into subclasses through a hierarchical clustering algorithm, and then extracts samples from the subclasses according to different strategies to reduce the number of majority class samples while retaining their diversity and alleviating the sample imbalance problem.

[0055] The VAGCUS undersampling model initially considers each sample as an independent cluster; then, in each iteration, the two clusters with the smallest distance are merged by calculating the variance and sample sum gain VSS.

[0056] The variance change of samples after cluster merging is measured by the sample sum gain VSS. The distance cluster(C1,C2) between two clusters C1 and C2 is defined as the sample sum gain VSS before and after their merging, that is:

[0057]

[0058] Where, It represents the sample sum gain VSS after clusters C1 and C2 are merged; VSS(C1) and VSS(C2) represent the sample sum gain VSS of clusters C1 and C2 respectively.

[0059] The sample sum gain VSS passes through each point to the cluster center X c It is calculated by the sum of the squares of the Euclidean distance, that is:

[0060]

[0061] In the formula, m represents the number of samples; X i represents the i-th sample in the cluster; X c Represents the center point of the cluster; d(X i ,X c ) represents the sample X i The Euclidean distance to the center point; n represents the total dimension of the feature; x ij Represents sample X i The value of the j-th feature dimension; x cj Represents the value of the cluster center on the jth feature dimension.

[0062] In this embodiment, during the clustering iteration process, it is necessary to determine the target number of clusters, that is, to find the inflection point by weighted average standard deviation:

[0063]

[0064] Where wastd(S) represents the weighted average standard deviation of the current cluster set S; k represents the total number of clusters; Indicates the weight of the i-th cluster; num(C i ) represents the number of samples in the i-th cluster; σ(C i ) is cluster C i The standard deviation of

[0065] The iteration is completed until all data are merged into a large cluster.

[0066] In this embodiment, the VAGCUS undersampling model identifies hidden subclasses within the majority class using a hierarchical clustering undersampling approach, thereby providing more information for the undersampling process. A target number of samples is then selected from these subclasses using different sampling strategies, including:

[0067] Truncated sampling strategy: ensures that the same number of samples are extracted from each subclass;

[0068] Boundary-preserving sampling strategy: retain boundary points, which are very important for distinguishing different classes;

[0069] Equal proportion sampling strategy: ensure that the proportion of each subclass remains unchanged.

[0070] S3) Perform feature selection processing on the balanced network data using an information gain algorithm to obtain low-dimensional network data; specifically, the steps include:

[0071] S31), based on the size of the balanced network data, the information entropy of the balanced network data is calculated and processed to obtain the information entropy value H(Y) of the balanced network data; that is:

[0072]

[0073] Where Y is the feature data set; N is the number of different values in the feature; p i represents the probability of the i-th value appearing in the set;

[0074] S32), based on the dimensional characteristics of the balanced network data, the balanced network data is subjected to conditional entropy calculation processing to obtain the conditional entropy value H(Y|X) of the balanced network data; namely:

[0075] H(Y|X)=∑ x∈X p(x)H(Y|x)=-∑ x∈X p(x)∑ y∈Y p(y|x)log2p(y|x);

[0076] Where X represents the target feature; Y represents the candidate feature; p(x) represents the probability that the target feature is x; p(y|x) represents the numerical probability of Y under the condition that X occurs;

[0077] S33), performing a difference process on the information entropy value of the balanced network data and the conditional entropy value to obtain the information gain value IG of the balanced network data; namely:

[0078] IG = H(Y) - H(Y|X);

[0079] S34), sorting the balanced network data according to the information gain value and eliminating the balanced network data with an information gain value less than a preset threshold to obtain low-dimensional network data.

[0080] S4) Improve the CNN network model by combining KAN to obtain an improved CKAN network model, and use low-dimensional network data to train the improved CKAN network model; classify the low-dimensional network data through the trained improved CKAN network model to obtain a classification result.

[0081] like Figure 3 As shown, in this embodiment, the improved CKAN network model includes a convolutional layer, a nonlinear activation layer, an adaptive pooling layer, and a KAN layer.

[0082] The KAN layer is a neural network layer inspired by the Kolmogorov-Arnold theorem.

[0083] In this embodiment, the convolution layer is used to perform convolution processing on the low-dimensional network data to obtain the spatiotemporal features of the low-dimensional network data; the convolution layer is used to perform spatial feature extraction and dimensionality reduction processing on the low-dimensional network data; the nonlinear activation layer adopts the activation function ReLU, that is:

[0084] ReLU(A)=max(0,A);

[0085] Where A represents the input value of the activation function;

[0086] And through the adaptive pooling layer, local information is summarized to reduce the dimension of the data and remove irrelevant features of the data, thereby reducing the noise of the data.

[0087] The adaptive pooling layer achieves a fixed output size by dynamically dividing the input area. If the input size is H in ×W in , the output size is H out ×W out ,but:

[0088]

[0089] Where h start Indicates the starting index of the window in the height direction of the input feature map; h end Indicates the end index of the window in the height direction of the input feature map; wstart Indicates the starting index of the window in the width direction of the input feature map; w end Represents the end index of the window in the width direction of the input feature map; i and j represent the horizontal and vertical coordinates of the output position (i, j);

[0090] Then perform adaptive average pooling operation and adaptive maximum pooling operation, namely:

[0091]

[0092] Where Output1(i,j) represents the average of all pixels in the region; Output2(i,j) represents the maximum value of all pixels in the region; Input(h,w) represents the value of the input feature map at the (h,w) position.

[0093] Adaptive pooling achieves flexible mapping of input to fixed-size output by dynamically adjusting the pooling area, solving the problem that traditional pooling layers are sensitive to input size.

[0094] The KAN layer is based on the Kolmogorov-Arnold representation theorem, for any multivariate continuous function f(x1,x2,…,x n ); can be expressed as a combination of superposition and addition of single variable functions, namely:

[0095]

[0096] Where M is the number of continuous functions; Φ q and φ q,p is a learnable univariate function; each φ 1,p (x p ) is responsible for processing input x p ; and Φ q Perform nonlinear transformation on the combined results.

[0097] In this embodiment, the KAN layer uses a spline function as the activation function s(t), that is:

[0098]

[0099] Where B i (t) is a predefined spline basis function; c i is a trainable coefficient, which is optimized by back-propagation; K is the order of the spline, which controls the smoothness of the curve; t is the scalar input value passed into the spline basis function.

[0100] After flattening the spatiotemporal features of the low-dimensional traffic data, classification processing is performed based on the KAN layer, and the classification results are output through the output layer, namely:

[0101]

[0102] Where f(x) is the mapping output of the KAN layer to the input x; x represents the variable feature vector of the input KAN layer; J is the number of branches in the KAN layer; σ j is a spline-based activation function; w j and b j are input weights and bias terms respectively; a j is the output weight, which is used to adjust the contribution of each path; T is the transpose operation.

[0103] In this embodiment, the number of parameters of the KAN layer is determined by the following formula:

[0104] KAN_params=in_dim×(grid_size+K+2)×out_dim;

[0105] Where KAN_params represents the number of parameters of the KAN layer; in_dim and out_dim represent the input and output dimensions respectively; grid_size is the number of intervals in the spline grid; and K is the order of the spline.

[0106] In this embodiment, a CKAN network model improved by K-fold cross-training is used; K-fold cross-training divides the training data set into K parts on average, and then makes a validation set for each subset of data, and the remaining K-1 groups of subset data are used as training sets. Unlike K-fold cross-validation, each K-fold cross-validation will obtain K models, and the average classification accuracy of the final validation set of these K models is used as the performance indicator of this classifier, while K-fold cross-training will only obtain 1 model. Each training of the model continues to be tuned on the basis of the previous training, which is similar to the idea of pre-training, so that the model has strong prior knowledge before each training, which can make the model converge faster and avoid falling into the local optimal situation. The specific steps of K-fold cross-training are as follows:

[0107] Define the model and learning rate, and divide the dataset into training and testing datasets;

[0108] The training data set is divided into K parts, 1 part is used as the validation set, and the remaining K-1 parts are used as the training set;

[0109] Define a gradient optimizer with a learning rate decay strategy. K-1 copies of the data are used for model training, and the remaining copy of the data is used for testing the model.

[0110] Repeat the above steps continuously to obtain the optimal model after K training times and obtain the optimal model performance index in the test data set.

[0111] The above embodiments and descriptions are only for explaining the principles and best embodiments of the present invention. Without departing from the spirit and scope of the present invention, the present invention may be subject to various changes and improvements, which shall fall within the scope of the invention to be protected.

Claims

1. An intrusion detection method based on a data balancing algorithm framework, characterized in that: The following steps are involved: S1) obtaining original network data and preprocessing it to obtain preprocessed network data; S2), performing data balancing on the pre-processed network data based on the data balancing algorithm framework to obtain balanced network data; S3) Perform feature selection on the balanced network data using the information gain algorithm to obtain low-dimensional network data; S4) The CNN network model is improved by combining KAN to obtain an improved CKAN network model. The low-dimensional network data is classified by the improved CKAN network model to obtain the classification results.

2. The intrusion detection method based on the data balancing algorithm framework according to claim 1 is characterized in that: In step S2), the pre-processed network data is subjected to data balancing processing using a data balancing sample model constructed based on a data balancing algorithm framework.

3. The intrusion detection method based on the data balancing algorithm framework according to claim 2 is characterized in that: In step S2), the data balanced sample model includes two sub-models, namely, a VAGCUS undersampling model constructed based on a hierarchical clustering undersampling method, and an attack sample generation model constructed based on a WGAN-GP framework; the VAGCUS undersampling model is used to perform hierarchical clustering undersampling processing on the majority class label samples to obtain undersampling data; The attack sample generation model is used to oversample the minority class label samples to obtain oversampled data, and then the undersampled data and oversampled data are merged to obtain balanced network data.

4. The intrusion detection method based on the data balancing algorithm framework according to claim 3 is characterized in that: In step S2), the objective function L of the discriminator of the attack sample generation model is: In the formula, E represents the mathematical expectation of the distribution function; D represents the discriminator; x represents the real sample; Indicates the generated sample; represents the random interpolation sampling points between the real sample and the generated sample; P represents the data distribution; P data is the distribution of real sample data; P G To generate the sample distribution, Defined as the distribution of real sample data P data And the generated sample distribution P G The uniform distribution of the sampled point pairs along the straight line; λ is the gradient penalty coefficient; Represents the gradient.

5. The intrusion detection method based on the data balancing algorithm framework according to claim 3 is characterized in that: In step S2), the VAGCUS undersampling model divides the majority class samples into subclasses using a hierarchical clustering algorithm, and then extracts samples from the subclasses according to different strategies; The VAGCUS undersampling model initially considers each sample as an independent cluster; then, in each iteration, the two clusters with the smallest distance are merged by calculating the variance and the sample sum gain VSS; The variance change of samples after cluster merging is measured by the sample sum gain VSS. The distance cluster(C1,C2) between two clusters C1 and C2 is defined as the sample sum gain VSS before and after their merging, that is: Where, It represents the sample sum gain VSS after clusters C1 and C2 are merged; VSS(C1) and VSS(C2) represent the sample sum gain VSS of clusters C1 and C2 respectively.

6. The intrusion detection method based on the data balancing algorithm framework according to claim 5 is characterized in that: In step S2), the sample sum gain VSS is calculated from each point to the cluster center X c It is calculated by the sum of the squares of the Euclidean distance, that is: In the formula, m represents the number of samples; X i represents the i-th sample in the cluster; X c Represents the center point of the cluster; d(X i ,X c ) represents the sample X i The Euclidean distance to the center point; n represents the total dimension of the feature; x ij Represents sample X i The value of the j-th feature dimension; x cj Represents the value of the cluster center on the jth feature dimension.

7. The intrusion detection method based on the data balancing algorithm framework according to claim 5 is characterized in that: In step S2), during the clustering iteration process, it is necessary to determine the target number of clusters, that is, to find the inflection point by weighted average standard deviation: Where wastd(S) represents the weighted average standard deviation of the current cluster set S; k represents the total number of clusters; Indicates the weight of the i-th cluster; num(C i ) represents the number of samples in the i-th cluster; σ(C i ) is cluster C i The standard deviation of The iteration is completed until all data are merged into a large cluster.

8. The intrusion detection method based on the data balancing algorithm framework according to claim 1 is characterized in that: In step S3), feature selection processing is performed on the balanced network data using the information gain algorithm, specifically: S31), based on the size of the balanced network data, performing information entropy calculation processing on the balanced network data to obtain the information entropy value of the balanced network data; S32), based on the dimensional characteristics of the balanced network data, performing conditional entropy calculation processing on the balanced network data to obtain a conditional entropy value of the balanced network data; S33), performing a difference process on the information entropy value of the balanced network data and the conditional entropy value to obtain the information gain value of the balanced network data; S34), sorting the balanced network data according to the information gain value and eliminating the balanced network data with an information gain value less than a preset threshold to obtain low-dimensional network data.

9. The intrusion detection method based on the data balancing algorithm framework according to claim 1 is characterized in that: In step S4), the improved CKAN network model includes a convolution layer, a nonlinear activation layer, an adaptive pooling layer, and a KAN layer; wherein the KAN layer is a neural network layer inspired by the Kolmogorov-Arnold theorem; the low-dimensional network data is convolved by the convolution layer to obtain the spatiotemporal features of the low-dimensional network data; and the local information is summarized by the adaptive pooling layer to reduce the dimension of the data to remove irrelevant features of the data; then the spatiotemporal features of the low-dimensional traffic data are flattened and classified by the KAN layer, and the classification results are output by the output layer.

10. The intrusion detection method based on the data balancing algorithm framework according to claim 9, characterized in that: In step S4), the adaptive pooling layer achieves a fixed output size by dynamically dividing the input area. If the input size is H in ×W in , the output size is H out ×W out ,but: Where h start Indicates the starting index of the window in the height direction of the input feature map; h end Indicates the end index of the window in the height direction of the input feature map; w start Indicates the starting index of the window in the width direction of the input feature map; w end Represents the end index of the window in the width direction of the input feature map; i and j represent the horizontal and vertical coordinates of the output position (i, j); Then perform adaptive average pooling operation and adaptive maximum pooling operation, namely: Where Output1(i,j) represents the average of all pixels in the region, Output 2 (i, j) means taking the maximum value of all pixels in the area; Input (h, w) means the value of the input feature map at the (h, w) position; The KAN layer is based on the Kolmogorov-Arnold representation theorem, for any multivariate continuous function f(x1,x2,…,x n ); express it as a superposition and addition combination of single variable functions, namely: Where M is the number of continuous functions; φ q and φ q,p is a learnable univariate function; each φ q,p (x p ) is responsible for processing input x p ; and Φ q Perform nonlinear transformation on the combined results; The KAN layer uses a spline function as the activation function s(t), namely: Where B i (t) is a predefined spline basis function; c i is a trainable coefficient, which is optimized by back propagation; K is the order of the spline, which controls the smoothness of the curve; t is the scalar input value of the spline basis function; After flattening the spatiotemporal features of the low-dimensional traffic data, classification processing is performed based on the KAN layer, and the classification results are output through the output layer, namely: Where f(x) is the mapping output of the KAN layer to the input x; x represents the variable feature vector of the input KAN layer; J is the number of branches in the KAN layer; σ j is a spline-based activation function; w j and b j are input weights and bias terms respectively; a j is the output weight, which is used to adjust the contribution of each path; T is the transpose operation.

Images