Improved Kolmogorov-Arnold neural network method

Abstract

The invention is suitable for the technical field of artificial intelligence and signal processing, and provides an improved Kolmogorov-Arnold neural network method, comprising the following steps: constructing a neural network infrastructure based on KANs, an activation function on a network edge being a learnable B-spline function; introducing continuous wavelet transform into a neural network basic structure; constructing a MexiKAN framework, and carrying out weighted additive fusion on Mexican cap wavelets and the B spline function to form a mixed activation function; and normalization processing is applied to the input before fusion. The method has the smoothness of KANs, efficient parameter utilization and global feature extraction capability, and also has the multi-level signal decomposition characteristic of Mexican cap wavelets, so that low-frequency and high-frequency components can be captured at the same time. While important information is reserved, irrelevant noise is effectively removed, and robustness and accuracy of signal approximation are ensured.

Description

Technical Field

[0001] This invention belongs to the field of artificial intelligence and signal processing technology, and particularly relates to an improved Kolmogorov-Arnold neural network method. Background Technology

[0002] With the rapid development of artificial intelligence, neural networks have been widely used in tasks such as signal processing, image classification, and sequence prediction. Traditional multilayer perceptrons (MLPs) typically rely on fixed activation functions and linear weights, which limits their ability to model high-dimensional data and fit complex signals. To overcome these limitations, the KAN proposed in recent years introduces learnable edge activation functions and uses univariate B-spline functions to replace traditional linear weights, thus demonstrating significant advantages in interpretability and prediction accuracy.

[0003] However, existing Kolmogorov-Arnold network (KAN) frameworks primarily rely on B-spline basis functions, which suffer from inefficiency and excessive smoothing when handling high-dimensional complex data and fitting abrupt signal changes. To address this shortcoming, several improvement schemes have been proposed, such as FastKAN using radial basis functions to improve computational efficiency, and ReLU-KAN introducing ReLU-based polynomial functions to enhance parallel computing performance. Nevertheless, these methods still struggle to achieve efficient and accurate modeling when dealing with abrupt signal changes and complex function variations common in real-world data.

[0004] Wavelet transform, as a multi-resolution analysis tool, can effectively capture the characteristics of abrupt signal changes. However, common wavelets often exhibit oversensitivity when faced with high-frequency noise, leading to decreased model stability. Summary of the Invention

[0005] The purpose of this invention is to provide an improved Kolmogorov-Arnold neural network method, which aims to solve the problems mentioned in the background art.

[0006] This invention is implemented as follows: an improved Kolmogorov-Arnold neural network method, named MexiKAN, is based on the weighted fusion of the Mexican hat wavelet and the original B-spline activation function, and the introduction of a normalization mechanism. This achieves high-precision fitting of smooth and abrupt signals while effectively suppressing noise interference. Specifically, it includes the following steps: Step 1: Construct the basic neural network structure based on KANs, where the activation functions on the network edges are learnable B-spline functions; Step 2: Introduce Continuous Wavelet Transform (CWT) into the neural network infrastructure; Step 3: Construct the Mexican Hat wavelet framework, perform weighted additive fusion of the Mexican Hat wavelet and the B-spline function to form a hybrid activation function, and apply normalization processing to the input before fusion.

[0007] A further technical solution is that, in step 1, for a generalized L-layer KANs network... Its output is obtained through the input. get: (1); in, Indicates the KANs network's th Activation function matrix and operators in the layer Indicates matrix composition; No. A layer contains several nodes, the first... The first in the layer Each node is denoted as From node arrive Each edge is assigned an activation function. These functions act on the first... The output of the layer is aggregated to generate the first layer. layer activation value As shown below: (2); In the formula, the activation function The details are as follows: (3); in, and All of these are trainable coefficients, used to control the optimization process of the activation function over a wider range; equal Activation function: (4); It consists of N B-spline basis functions The B-spline function, constructed from a linear combination of , has the following form: (5); in, These are the trainable control point coefficients.

[0008] In a further technical solution, in step 2, the mathematical form of the continuous wavelet transform is expressed as follows: (6); in, This represents the signal to be fitted; The mother wavelet can be used as a template to apply translation and scaling to any local region of the signal. Indicates the scale factor; Indicates the translation factor; Indicates time; Used to measure wavelets and signals at scale factors Translation factor The relationship between them; By applying a positive scaling factor Translation factor By approximating different parts of the signal, the region that needs to be emphasized can be determined.

[0009] In a further technical solution, in step 3, the mathematical definition of the Mexican hat wavelet is as follows: (7); in, Used to control the width of the wavelet. This represents the input after translation and scale normalization; As a standardization process, formula (7) is further simplified to: (8); While keeping the structure of equation (3) unchanged, a weighted additive fusion with the Mexican hat wavelet is introduced, the definition of which is shown in the following equation: (9); in, is a trainable parameter used to control the influence of the Mexican hat wavelet in signal fitting, and its initial value is set to 0; This represents the normalization function.

[0010] Further technical solutions, normalized functions The definition is as follows: (10); in, and These represent the mean and standard deviation of the input along the specified dimension, respectively. It is a minimal constant introduced for numerical stability.

[0011] In a further technical solution, the MexiKAN network consists of two layers, where the first layer contains 116 activation functions and the second layer contains 58 activation functions.

[0012] This invention provides an improved Kolmogorov-Arnold neural network method. This method extends the original KANs by introducing weighted Mexican hat wavelets, combining the smoothness, efficient parameter utilization, and global feature extraction capabilities of KANs with the multi-level signal decomposition characteristics of Mexican hat wavelets, thus enabling the simultaneous capture of low-frequency and high-frequency components. Therefore, this method effectively removes irrelevant noise while retaining important information, ensuring the robustness and accuracy of signal approximation. Attached Figure Description

[0013] Figure 1 This is a schematic diagram of fitting a real signal based on B-splines; Figure 2 This is the overall framework of the MexiKAN network. Detailed Implementation

[0014] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0015] The specific implementation of the present invention will be described in detail below with reference to specific embodiments.

[0016] An embodiment of this invention provides an improved Kolmogorov-Arnold neural network method, named MexiKAN. Its core lies in the weighted fusion of the Mexican cap wavelet and the original B-spline activation function, and the introduction of a normalization mechanism. This achieves high-precision fitting of smooth and abrupt signals while effectively suppressing noise interference. Specifically, it includes the following steps: Step 1: KANs basic structure preserved; KANs is a novel neural network architecture based on the Kolmogorov-Arnold representation theorem. This method applies adaptive, learnable activation functions to the edges, thereby overcoming the limitation of fixed activation functions that only apply to nodes in traditional MLPs.

[0017] For a generalized L-layer KANs network Its output can be obtained through input. get: (1); in, Indicates the KANs network's th Activation function matrix and operators in the layer This indicates matrix composition.

[0018] Specifically, the A layer contains several nodes, the first... The first in the layer Each node is denoted as From node arrive Each edge is assigned an activation function. These functions act on the first... The output of the layer is aggregated to generate the first layer. layer activation value As shown below: (2); In the formula, the activation function Specifically, the details are as follows: (3); in, and All of these are trainable coefficients, used to control the optimization process of the activation function over a wider range; equal Activation function: (4); It consists of N B-spline basis functions The B-spline function, constructed from a linear combination of , has the following form: (5); in, These are the trainable control point coefficients.

[0019] The B-spline in equation (5) is a piecewise polynomial function defined by a set of control points, possessing a smooth and flexible fitting mechanism. However, when dealing with signals containing abrupt changes or high-dimensional irregular data, the fitting accuracy of the B-spline will significantly decrease, such as... Figure 1 As shown in a. Furthermore, its definition based on nodes and control points is relatively complex, increasing the difficulty of model design and optimization, and potentially leading to the curse of dimensionality.

[0020] Step 2: Introduce wavelet transform; Wavelets possess finite spatial support, a property that makes them well-suited for fitting local signals and handling complex abrupt changes. This study primarily investigates the application of CWT in medical and related classification tasks. CWT is widely used for signal fitting, and its mathematical form can be expressed as follows: (6); in, This represents the signal to be fitted; The mother wavelet can be used as a template to apply translation and scaling to any local region of the signal. Indicates the scale factor; Indicates the translation factor; Indicates time; Used to measure wavelets and signals at scale factors Translation factor The relationship between them. By applying a positive scaling factor. Translation factor This allows for the approximation of different parts of a signal, thereby determining the region that needs to be emphasized.

[0021] Step 3: Build the MexiKAN framework; The Mexican hat wavelet is a typical type of CWT, characterized by a central positive peak flanked by negative troughs. This structure generates strong central and lateral responses to abrupt changes, making it highly sensitive to discontinuities in the signal, such as... Figure 1 As shown in b. Unlike most CWTs that are overly sensitive and susceptible to noise, the Mexican hat wavelet's spectral energy is mainly concentrated in the low-to-mid frequency region and decays rapidly in the high-frequency part, thus effectively suppressing noise and achieving a more balanced effect. Furthermore, this wavelet has a real-valued and even-symmetric form, avoiding complex calculations and improving computational efficiency. Its mathematical definition is as follows: (7); in, Used to control the width of the wavelet. This represents the input after translation and scale normalization. Since in practical applications only adjustments are needed... and To control the fitting accuracy, therefore As a standardization process to simplify the calculation, this setting does not affect the essence of wavelet transform. Therefore, formula (7) can be further simplified to: (8); The representational power of B-splines is mainly determined by the density of grid points and the order of the spline. Increasing these two factors can improve the expressive power, but may also lead to overfitting and increase computational cost. To mitigate this risk while improving the expressive power of the activation function, a weighted additive fusion with the Mexican hat wavelet is introduced while keeping the structure of equation (3) unchanged. Its definition is shown in equation (9). The proposed MexiKAN network is as follows: Figure 2 As shown, it consists of two layers: the first layer contains 116 activation functions, and the second layer contains 58.

[0022] (9); in, This is a trainable parameter used to control the influence of the Mexican cap wavelet on signal fitting, i.e., the fitting strength for abrupt changes; its initial value is set to 0. Normalization function. The definition is shown in Equation (10). This design is used to correct the deviation between different input sources and achieve feature alignment to ensure the effectiveness of weighted fusion. At the same time, this mechanism can be combined with external weights to jointly control the contribution ratio of each channel, thereby better fitting different frequency components in the signal.

[0023] (10); in, and These represent the mean and standard deviation of the input along the specified dimension, respectively. It is a minimal constant introduced for numerical stability.

[0024] In summary, the proposed MexiKAN extends the original KANs by introducing a weighted Mexican hat wavelet, combining the smoothness, efficient parameter utilization, and global feature extraction capabilities of KANs with the multi-level signal decomposition characteristics of the Mexican hat wavelet, thus enabling the simultaneous capture of both low-frequency and high-frequency components. Therefore, this method effectively removes irrelevant noise while preserving important information, ensuring the robustness and accuracy of signal approximation.

[0025] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. An improved Kolmogorov-Arnold neural network method, characterized in that, Includes the following steps: Step 1: Construct the basic neural network structure based on KANs, where the activation functions on the network edges are learnable B-spline functions; Step 2: Introduce continuous wavelet transform into the basic structure of neural networks; Step 3: Construct the MexiKAN framework, perform weighted additive fusion of the Mexican hat wavelet and the B-spline function to form a hybrid activation function, and apply normalization processing to the input before fusion.

2. The improved Kolmogorov-Arnold neural network method according to claim 1, characterized in that, In step 1, for a generalized L-layer KANs network Its output is obtained through the input. get: （1）； in, Indicates the KANs network's th Activation function matrix and operators in the layer Indicates matrix composition; No. A layer contains several nodes, the first... The first in the layer Each node is denoted as From node arrive Each edge is assigned an activation function. These functions act on the first... The output of the layer is aggregated to generate the first layer. layer activation value As shown below: （2）； In the formula, the activation function The details are as follows: （3）； in, and All of these are trainable coefficients, used to control the optimization process of the activation function over a wider range; equal Activation function: （4）； It consists of N B-spline basis functions The B-spline function, constructed from a linear combination of , has the following form: （5）； in, These are the trainable control point coefficients.

3. The improved Kolmogorov-Arnold neural network method according to claim 2, characterized in that, In step 2, the mathematical form of the continuous wavelet transform is as follows: （6）； in, This represents the signal to be fitted; The mother wavelet is used as a template to act on any local region of the signal through translation and scaling. Indicates the scale factor; Indicates the translation factor; Indicates time; Used to measure wavelets and signals at scale factors Translation factor The relationship between them; By applying a positive scaling factor Translation factor By approximating different parts of the signal, the region that needs to be emphasized can be determined.

4. The improved Kolmogorov-Arnold neural network method according to claim 3, characterized in that, In step 3, the mathematical definition of the Mexican hat wavelet is as follows: （7）； in, Used to control the width of the wavelet; This represents the input after translation and scale normalization; As a standardization process, formula (7) is further simplified to: （8）； While keeping the structure of equation (3) unchanged, a weighted additive fusion with the Mexican hat wavelet is introduced, the definition of which is shown in the following equation: （9）； in, is a trainable parameter used to control the influence of the Mexican hat wavelet in signal fitting, and its initial value is set to 0; This represents the normalization function.

5. The improved Kolmogorov-Arnold neural network method according to claim 4, characterized in that, Normalization function The definition is as follows: （10）； in, and These represent the mean and standard deviation of the input along the specified dimension, respectively. It is a minimal constant introduced for numerical stability.

6. The improved Kolmogorov-Arnold neural network method according to claim 1, characterized in that, The MexiKAN network consists of two layers, with the first layer containing 116 activation functions and the second layer containing 58 activation functions.

Images