A signal extension method based on Kolmogorov-Arnold network

Abstract

The present invention discloses a signal extension method based on a Kolmogorov‑Arnold network, comprising: step 1: selecting part of the data of a sampled signal as a training set, performing Fourier transform on it to obtain an odd function and an even function, and constructing a network model according to the Kolmogorov‑Arnold theorem; step 2: in the model, configuring functions with odd symmetry and even symmetry as activation functions respectively to form an odd network and an even network; step 3: the data will enter the odd network or the even network accordingly, and then learn and extend to obtain the complete signal form of the signal data. The present invention provides a signal extension method based on a Kolmogorov‑Arnold network, by optimizing the activation function configuration of the Kolmogorov‑Arnold network, improving the extension ability of the model when processing data with odd and even characteristics; through the decomposition learning and reorganization strategy for non-odd and non-even data, a wide range of data including random signals and irregular waveforms in fluid mechanics and aerospace are effectively extended.

Description

Technical Field

[0001] The present invention relates to a deep learning extension field. More specifically, the present invention relates to a signal extension method based on a Kolmogorov-Arnold network. Background Art

[0002] In the field of artificial intelligence and deep learning, neural networks have been widely used in various prediction and data analysis tasks, especially in image processing, speech recognition, and time series analysis. However, traditional neural network architectures, such as multi-layer perceptrons (MLPs) and convolutional neural networks (CNNs), usually use fixed, non-adaptive activation functions such as ReLU or Sigmoid. This fixedness limits their flexibility in dealing with problems with complex mathematical properties, especially in the ability to effectively extend data to unknown intervals.

[0003] Kolmogorov-Arnold network (KAN) was proposed as a new neural network architecture to overcome the shortcomings of traditional networks in dealing with highly mathematical and dynamic problems. The uniqueness of KAN is that it uses a learnable activation function on each edge, which allows the network to adaptively optimize its ability to handle various complex data patterns. This design enables KAN to show higher efficiency and accuracy than traditional neural networks when extending complex functions, dealing with highly nonlinear problems, and parsing complex mathematical structures.

[0004] In high-tech fields such as fluid mechanics and aerospace, the extension of a wide range of data including randomly distributed signals and irregular waveforms. The activation functions in traditional neural networks are usually fixed and cannot be flexibly adjusted to adapt to data with specific characteristics, which limits their performance in effectively extending such signals. Summary of the invention

[0005] An object of the present invention is to solve at least the above problems and / or disadvantages and to provide at least the advantages which will be described hereinafter.

[0006] In order to achieve these purposes and other advantages of the present invention, a signal extension method based on a Kolmogorov-Arnold network is provided, comprising:

[0007] Step 1: Select part of the sampled signal data as the training set, perform Fourier transform analysis on its frequency components, obtain the odd and even components of the signal, and then use inverse Fourier transform to form the odd function and even function parts, and build the network model according to the Kolmogorov-Arnold theorem;

[0008] Step 2: In the constructed network model, configure a function with odd symmetry and a function with even symmetry as activation functions to form an odd network and an even network respectively;

[0009] Step 3: For signals with odd or even symmetry, their data will enter the odd network or even network accordingly, and then be learned and extended to obtain the corresponding signal form;

[0010] For signals that satisfy neither odd symmetry nor even symmetry, the odd and even networks in the model will process the data separately, capture its odd and even parts and extend them to obtain the corresponding odd function part and even function part. Finally, the obtained odd function part and even function part are recombined to obtain the complete signal form of the signal data.

[0011] Preferably, when the network model is trained, the coefficients of the learnable activation function in the network model are updated by calculating the loss between the odd function part and the even function part obtained in step 1 and the odd function and the even function obtained in step 3.

[0012] Preferably, in step 2, the function with even symmetry is set to a cosine activation function:

[0013] or set to ;

[0014] In the above formula: is an adjustable parameter, Input values ​​to the neurons of the network model.

[0015] Preferably, in step 2, the function with odd symmetry is set to a sinusoidal activation function:

[0016]

[0017] or set to , where: is an adjustable parameter, satisfy .

[0018] Preferably, in step 1, for a set of discrete sampled signal data points , where N is the number of data points, The way to obtain odd and even components is:

[0019] S10, add a weight factor to the discrete Fourier transform definition in Fourier transform, Bring it in for calculation, as follows:

[0020]

[0021] In the above formula, k is the index in the frequency domain, is the kth component in the frequency domain, are discrete sampled signal data points, is an adjustable weight factor, is an imaginary unit, is a complex exponential;

[0022] S11. Inspection The real part of the signal is obtained to obtain the odd component, check The imaginary part of the signal is obtained to obtain the even component of the signal.

[0023] Preferably, the weight factor is obtained through an auxiliary neural network;

[0024] The mapping relationship of the auxiliary neural network is ,in are the parameters of the auxiliary network;

[0025] Will The eigenvector of As the input of the auxiliary neural network, is the dimension of the vector, Including the mean, variance, and spectrum characteristics of the signal, among which, ;

[0026] The weight factor As the output of the auxiliary neural network;

[0027] When training the auxiliary network, the odd and even components extracted by Fourier transform are subtracted from the actual odd and even components of the signal, and the difference is used as an evaluation indicator; when the difference is greater than the set threshold, the back propagation algorithm is used to calculate its parameter The gradient of the parameter is then adjusted by the gradient descent algorithm. Adjust; when the difference is less than or equal to the threshold, the model performance is considered acceptable and no parameters need to be adjusted .

[0028] Preferably, the losses between the odd function part, the even function part and the odd function and the even function are calculated and expressed by mean square error, that is:

[0029]

[0030] in is the mean square error, subscript is an index variable, It is The odd or even function obtained by Fourier transform corresponding to samples, It is The odd function part or the even function part predicted by the network model corresponding to the sample.

[0031] The present invention has at least the following beneficial effects:

[0032] By optimizing the activation function configuration of the Kolmogorov-Arnold network (KAN), the network model's ability to extend when processing data with specific mathematical properties is significantly improved. By accurately matching the odd-even symmetric activation function, not only is the model's learning effect in a known interval enhanced, but its effective extension in an unknown interval is also ensured, thereby maintaining high accuracy and consistency across the entire data range; through the decomposition learning and reorganization strategy for non-odd and non-even data, it can be used to process a wide range of data types, including randomly distributed signals and irregular waveforms in fluid mechanics and aerospace. For fields that require accurate processing of data with complex mathematical properties, multiple data types can be flexibly adjusted to correctly process the mathematical properties of the data for effective extension.

[0033] Other advantages, objectives and features of the present invention will be embodied in part through the following description, and in part will be understood by those skilled in the art through study and practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 This is a schematic diagram of the network structure of a 2-layer KAN in an embodiment of the present invention;

[0035] Figure 2 A flow chart of Fourier transform of a sampling signal in an embodiment of the present invention;

[0036] Figure 3 A basic flow chart of non-odd and non-even waveform processing in an embodiment of the present invention;

[0037] Figure 4 It is the result of extending the trapezoidal wave from the known interval (0, 1) to the unknown interval (-1, 0) when the activation function is taken to n=2 in Example 1 of the present invention;

[0038] Figure 5 It is the result of extending the trapezoidal wave from the known interval (0, 1) to the unknown interval (-1, 0) when the activation function is taken to n=3 in Example 1 of the present invention;

[0039] Figure 6 It is the result of extending the trapezoidal wave from the known interval (0, 1) to the unknown interval (-1, 0) when the activation function is taken to n=4 in Example 1 of the present invention;

[0040] Figure 7 It is the result of extending the sine wave from the known interval (0, 1) to the unknown interval (-1, 0) when the activation function is taken to n=4 in Example 1 of the present invention;

[0041] Figure 8 It is the result of extending the sine wave from the known interval (0, 1) to the unknown interval (-1, 0) when the activation function is taken to n=5 in Example 1 of the present invention;

[0042] Fig. 9 It is the result of extending the sine wave from the known interval (0, 1) to the unknown interval (-1, 0) when the activation function is taken to n=6 in Example 1 of the present invention;

[0043] Fig.10 It is the result of extending the complex wave from the known interval (0, 1) to the unknown interval (-1, 0) when the activation function is taken to n=3 in Example 1 of the present invention;

[0044] Fig.11 It is the result of extending the complex wave from the known interval (0, 1) to the unknown interval (-1, 0) when the activation function is set to n=4 in Example 1 of the present invention;

[0045] Fig.12 It is the result of extending the complex wave from the known interval (0, 1) to the unknown interval (-1, 0) when the activation function is taken to n=5 in Example 1 of the present invention. DETAILED DESCRIPTION

[0046] The present invention is further described in detail below in conjunction with the accompanying drawings so that those skilled in the art can implement the invention with reference to the description.

[0047] A signal extension method based on a Kolmogorov-Arnold network, comprising:

[0048] Step 1: Select part of the sampling signal data as the training set, perform Fourier transform on it to analyze its frequency components, obtain the odd and even components of the signal, and then use inverse Fourier transform to form odd and even functions. Construct the network model based on the Kolmogorov-Arnold theorem.

[0049] The KAN structure is designed according to the Kolmogorov-Arnold theorem. In theory, only two KAN layers are needed to fully represent various supervised learning tasks in the real number domain. The activation function in the network is placed on the edge instead of the node, which is a significant difference between KAN and traditional neural networks. In this structure, the nodes are mainly responsible for simple summation operations, while complex nonlinear transformations are handled by the activation functions on the edges. The network structure is as follows Figure 1 shown.

[0050] Figure 1 is a Kolmogorov-Arnold network with 2 inputs, 1 output, and 5 hidden neurons. Indicates Tier neurons, Indicates the connection Tier neurons and Tier The activation function of neurons in the hidden layer is 2h+1=5, where h refers to the number of input variables.

[0051] Step 2: In the constructed network model, configure the function that satisfies odd symmetry and the function that satisfies even symmetry as the activation function to form an odd network and an even network.

[0052] For target signals with specific mathematical properties, selecting an activation function that matches them can significantly improve the performance and accuracy of the model and ensure effective extension from known data intervals to unknown intervals.

[0053] Step 3: For signals with odd or even symmetry, their data will enter the odd network or even network accordingly, and then be learned and extended to obtain the corresponding signal form;

[0054] For signals that satisfy neither odd symmetry nor even symmetry, the odd and even networks in the model will process the data separately, capture its odd and even parts and extend them to obtain the corresponding odd function part and even function part. Finally, the obtained odd function part and even function part are recombined to obtain the complete signal form of the signal data.

[0055] For data with even symmetry, such as stress distribution encountered in some physical fields or mechanical problems, when the activation function satisfies even symmetry When , it can ensure that the network output is consistent with the input in the positive and negative directions, which helps the model maintain the even symmetry of the data during the learning and extension process and ensures accurate extension from the learning interval to the unknown interval.

[0056] A common activation function with even symmetry can be a cosine activation function, whose expression is ,in, and is an adjustable parameter, is the neuron input value of the network model; when When , the function has even symmetry, that is, For even-symmetric data such as stress distribution, the activation function can adapt to different data characteristics under different parameter settings, for example, adjusting the parameter The period and frequency of the function can be changed to better fit the changing pattern of the data. In addition, new even-symmetric activation functions can be designed, such as , which is characterized by the fact that when the input value is small, the function value grows faster, and when the input value is large, the function value gradually approaches 1.

[0057] For data with odd symmetry, such as those involving momentum or velocity fields, we choose an odd symmetry activation function to optimize the processing effect and ensure that the model can accurately capture the odd symmetry characteristics of the data. , improving the accuracy of predictions and the reliability of extensions. This is especially critical for modeling dynamic systems, where the directionality and flow nature of the data need to be accurately represented.

[0058] A typical odd-symmetric activation function can be a sinusoidal activation function, which is expressed as

[0059] , and is an adjustable parameter. When , the function has odd symmetry, that is, In the modeling of momentum or velocity fields, the sinusoidal activation function can be used based on the parameters The adjustment can be used to adapt to different data change speeds and amplitudes, so as to better capture the odd symmetry characteristics of the data. At the same time, new activation functions can also be designed to meet the requirements of odd symmetry. For example ,in satisfy This activation function can avoid the situation where the denominator is zero to a certain extent when processing odd-symmetric data, and can respond quickly to positive and negative changes in the input value.

[0060] For those complex data that are neither odd nor even, such as randomly distributed signals or irregular waveforms, we can use an effective method to process these data.

[0061] First, the sampled signal is decomposed by Fourier series or fast Fourier transform (FFT). If the signal is continuous, it is decomposed using Fourier series, which expresses a periodic function as an infinite sum of sine and cosine functions. For a continuous signal with a period of T , its Fourier series is expressed as:

[0062] in, is the average term or DC component, and are the Fourier coefficients, which are:

[0063] Here The term corresponds to an even function component, and The terms correspond to odd function components.

[0064] For discrete signals, the Fast Fourier Transform is an algorithm for the Fourier Transform that is used to quickly calculate the Discrete Fourier Transform (DFT) and its inverse transform. DFT converts discrete signals from the time domain to the frequency domain and analyzes their frequency components. For a set of discrete sampled signal data points , whose DFT is defined as:

[0065]

[0066] where k is the index in the frequency domain, is the kth component in the frequency domain, are discrete sampled signal data points, where N is the number of data points, is an imaginary unit, is a complex exponential that contains sine and cosine components (cosine for the real part and sine for the imaginary part), such that It also contains amplitude and phase information. In order to improve the accuracy of extracting odd and even components, a weight factor is further added to the DFT definition, namely:

[0067]

[0068] in is an adjustable weight factor. An intelligent optimization algorithm can be used to find the optimal weight value to achieve more accurate extraction of odd and even components. For example, in addition to the traditional genetic algorithm and particle swarm optimization algorithm, a weight optimization strategy based on deep learning is introduced here. A small auxiliary neural network is constructed, and the input of the network is the feature vector of the signal. ,in is the dimension of the vector, It can be the mean, variance, spectral characteristics of the signal, etc. The output of the network is the weight factor The predicted value of , assuming that the mapping relationship of the network is ,in are the parameters of the auxiliary network.

[0069] When training the auxiliary network, the difference between the extracted odd and even components and the real odd and even components is used as the evaluation index, and the threshold is set to , when the difference is greater than When , the back propagation algorithm is used to calculate its parameters The gradient of the parameter is then adjusted by the gradient descent algorithm. Adjustments are made. When the difference is less than or equal to When , the model performance is considered acceptable and no parameters need to be adjusted By constantly adjusting network parameters , so that the output weight factor can better adapt to different input signals, thereby improving the accuracy of parity component extraction.

[0070] After Fourier transformation, it can be checked by The real and imaginary parts of and odd ingredients .

[0071] Next, use the inverse Fourier transform (IFFT):

[0072]

[0073] Reconstruct the odd functions in the signal and even functions The process is as follows Figure 2 As shown, Loss is the loss and Model is the network model.

[0074] Then, for the input data , the odd and even networks in the network model will predict their corresponding odd function parts respectively and even function part , calculate the loss between the network prediction result and the parity function obtained after Fourier transform. The loss function can be the mean square error (MSE), that is:

[0075]

[0076] in is the mean square error, subscript is an index variable, It is The odd or even function obtained by Fourier transform corresponding to samples, It is The odd function part or even function part predicted by the network model corresponding to the sample. N is the number of samples. When calculating the loss, you can choose a regularization method to adjust the parameter value according to the specific task to balance the fitting and generalization capabilities.

[0077] Based on this, the coefficients of the learnable activation function in the network model are updated. When the set number of iterations is reached or the loss is reduced to a preset threshold, the parameter update is stopped to ensure that the network model can achieve the best performance in the signal extension task.

[0078] Finally, when the network model is trained to convergence, the even and odd components learned by the odd and even network are recombined to accurately restore the shape of the complete signal. Figure 3 shown.

[0079] Design experiments and verification. Experiments are designed to verify the performance of the network in the extension task, especially the extension ability when processing signals with obvious periodicity and symmetry. By comparing the performance with traditional neural network models, the advantages of the model in achieving signal extension are demonstrated.

[0080] Used in fields that require accurate processing of data with complex mathematical properties, such as fluid mechanics and aerospace, where correct processing of the mathematical properties of the data is critical for simulation and prediction.

[0081] This work optimizes the activation function configuration of the Kolmogorov-Arnold network (KAN), significantly improving the network model's ability to extend when processing data with specific mathematical properties. By precisely matching the odd-even symmetric activation function, not only is the model's learning effect in a known interval enhanced, but it also ensures its effective extension in an unknown interval, thereby maintaining high accuracy and consistency across the entire data range. In addition, the decomposition and reorganization strategy for non-odd and non-even data using Fourier series and fast Fourier transform (FFT) can handle a wide range of data types including randomly distributed signals and irregular waveforms, and is suitable for high-tech fields such as fluid mechanics and aerospace.

[0082] Example 1

[0083] In practical tasks, the integrity and continuity of complex data processing are particularly critical, especially in signal transmission and digital electronic clock signal processing, which are widely used in scientific and engineering applications. Therefore, this embodiment not only focuses on the effective extension ability of even-symmetric trapezoidal waves and odd-symmetric sine waves from the learning interval (0, 1) to the unknown interval (-1, 0), but also expands the research on non-odd and non-even waveforms. Such waveforms are neither completely symmetrical nor anti-symmetrical in characteristics. They combine the characteristics of multiple basic waveforms and play an important role in communication signal processing, control systems and many scientific and engineering fields.

[0084] Trapezoidal waves are often used to simulate impulse responses in analog signal processing due to their flat top and sharp corners. The mathematical expression of this waveform in the experiment is:

[0085]

[0086] In order to adapt to the even symmetry of the trapezoidal wave, a cosine activation function is selected or even function sequence As the activation function, Figure 4-6 The effect of using even function activation sequences to learn and extend trapezoidal waves is demonstrated.

[0087] Figure 4It shows the prediction when the activation function is set to n=2. 100 samples are used to train the network. The black straight line is the true value and the gray dotted line is the predicted value. Figure 5 , Figure 6 The predictions for n=3 and n=4 are as follows: Figure 4-6 Here x is the independent variable, i.e. the spatial position, y is the dependent variable, i.e. the amplitude value of the signal, the true value is the sampled signal, and the predicted value is the signal predicted by the network model).

[0088] Sine Wave It is used to evaluate the network's ability to handle periodic changes. Similarly, the sine wave is processed using a sinusoidal activation function. or odd function sequence As an activation function, it is extended by taking advantage of its natural odd symmetry characteristics. Figure 5 The effect of using odd function activation sequences to learn and extend sine waves is demonstrated.

[0089] Figure 7-9 It shows that the activation function is taken to Similarly, each model used 100 samples for network training ( Figure 7-9 Here x is the independent variable, i.e. the spatial position, y is the dependent variable, i.e. the amplitude value of the signal, the true value is the sampled signal, and the predicted value is the signal predicted by the network model).

[0090] In the above two embodiments, for signals with clear symmetry, the odd and even networks can accurately learn and extend them, ensuring the processing effect of the signals in the corresponding networks and laying the foundation for the perfect restoration of the final signals.

[0091] In order to fully evaluate the ability of the technical solution in processing complex waveforms, this embodiment introduces a complex waveform that is neither odd nor even. , the waveform has the characteristics of both sine wave and cosine wave, showing the unique property of being neither odd nor even. When processing such complex waveforms, the present embodiment carefully designs the activation function configuration to process the odd and even components of the waveform respectively.

[0092] Specifically, the odd network part uses an odd function sequence As an activation function, it aims to capture and extend the odd part of the waveform. Correspondingly, the even network part uses an even function sequence As an activation function, it simulates and expands the even part of the waveform. The odd and even components learned by the odd-even network are recombined to accurately reconstruct the complete waveform. Figure 6 The effect of using this mixed activation function sequence to learn and extend complex waves is demonstrated.

[0093] Figure 10-12 It shows that the activation function is taken to The prediction situation when , each model uses 100 samples for network training. It can be seen that for complex signals without specific symmetry, through the coordinated processing of odd and even networks and parameter adjustment, the effective extension of the signal is successfully achieved, which is highly consistent with the original signal ( Figure 10-12 Here x is the independent variable, i.e. the spatial position, y is the dependent variable, i.e. the amplitude value of the signal, the true value is the sampled signal, and the predicted value is the signal predicted by the network model).

[0094] By conducting a detailed study on these three waveforms with different parity characteristics, not only the applicability and effectiveness of KAN in accurately simulating and extending signals with specific mathematical characteristics are verified, but also the breadth and effectiveness of the technical solution in practical applications are further demonstrated through diversified activation function configurations.

[0095] The above solution is only an illustration of a preferred embodiment, but is not limited thereto. When implementing the present invention, appropriate replacement and / or modification can be performed according to user needs.

[0096] Although the embodiments of the present invention have been disclosed above, they are not limited to the applications listed in the specification and the embodiments. It can be fully applied to various fields suitable for the present invention. For those familiar with the art, additional modifications can be easily realized. Therefore, without departing from the general concept defined by the claims and equivalent scope, the present invention is not limited to the specific details and the illustrations shown and described here.

Claims

1. A signal extension method based on Kolmogorov-Arnold network, characterized in that: include: Step 1: Select part of the sampled signal data as the training set, perform Fourier transform analysis on its frequency components, obtain the odd and even components of the signal, and then use inverse Fourier transform to form the odd function and even function parts, and build the network model according to the Kolmogorov-Arnold theorem; Step 2: In the constructed network model, configure a function with odd symmetry and a function with even symmetry as activation functions to form an odd network and an even network respectively; Step 3: For signals with odd or even symmetry, their data will enter the odd network or even network accordingly, and then be learned and extended to obtain the corresponding signal form; For signals that do not satisfy either odd or even symmetry, the odd and even networks in the model will process the data separately, capture the odd and even parts and extend them to obtain the corresponding odd function part and even function part, and finally recombine the obtained odd function part and even function part to obtain the complete signal form of the signal data; In step 1, for a set of discrete sampled signal data points , where N is the number of data points, The way to obtain odd and even components is: S10, add a weight factor to the discrete Fourier transform definition in Fourier transform, Bring it in for calculation, as follows: In the above formula, k is the index in the frequency domain, is the kth component in the frequency domain, are discrete sampled signal data points, is an adjustable weight factor, is an imaginary unit, is a complex exponential; S11. Inspection The real part of the signal is obtained to obtain the odd component, check The even component of the signal is obtained by taking the imaginary part of 2. The signal extension method based on Kolmogorov-Arnold network according to claim 1, characterized in that: When training the network model, the coefficients of the learnable activation function in the network model are updated by calculating the loss between the odd function part and the even function part obtained in step 1 and the odd function and the even function obtained in step 3.

3. The signal extension method based on Kolmogorov-Arnold network according to claim 1, characterized in that: In step 2, the function with even symmetry is set to a cosine activation function: or set to ; In the above formula: is an adjustable parameter, Input values ​​to the neurons of the network model.

4. The signal extension method based on Kolmogorov-Arnold network according to claim 1, characterized in that: In step 2, the function with odd symmetry is set as a sinusoidal activation function: or set to , where: is an adjustable parameter, satisfy .

5. The signal extension method based on Kolmogorov-Arnold network according to claim 1, characterized in that: The weight factor is obtained through an auxiliary neural network; The mapping relationship of the auxiliary neural network is ,in are the parameters of the auxiliary network; Will The eigenvector of As the input of the auxiliary neural network, is the dimension of the vector, Including the mean, variance, and spectrum characteristics of the signal, among which, ; The weight factor As the output of the auxiliary neural network; When training the auxiliary network, the odd and even components extracted by Fourier transform are subtracted from the actual odd and even components of the signal, and the difference is used as an evaluation indicator; when the difference is greater than the set threshold, the back propagation algorithm is used to calculate its relationship with the parameter The gradient of the parameter is then adjusted by the gradient descent algorithm. Adjust; when the difference is less than or equal to the threshold, the model performance is considered acceptable and no parameters need to be adjusted .

6. The signal extension method based on Kolmogorov-Arnold network according to claim 2, characterized in that: The losses between the odd function part, the even function part, the odd function, and the even function are calculated and expressed by the mean square error, that is: in is the mean square error, subscript is an index variable, It is The odd or even function obtained by Fourier transform corresponding to samples, It is The odd function part or the even function part predicted by the network model corresponding to the sample.

Images