Kan adaptive function combination method and device based on multi-scale sparse branch

Abstract

The application discloses a KAN adaptive function combination method and device based on a multi-scale sparse branch, and belongs to the technical field of artificial intelligence and machine learning. The method comprises the following steps: starting a dynamic selection mechanism and initializing model parameters; performing standardization processing on input data, calculating function selection weights through multi-scale feature extraction and hierarchical attention mechanism, screening and activating relevant function modules based on the weights, forming a sparse calculation path, performing feature calculation of the activated modules and integrating the output by using a weighted fusion strategy, iteratively performing module selection and feature fusion in a deep network, constructing deep feature representation, finally generating a prediction result based on the fused features, and realizing model optimization through loss calculation and back propagation. The application not only significantly improves the reasoning speed and calculation efficiency, but also has good input adaptability, can reduce energy consumption and enhance model interpretability, and is suitable for application scenarios such as computer vision, natural language processing and industrial control.

Description

Technical Field

[0001] This invention relates to a method and apparatus for combining KAN adaptive functions based on multi-scale sparse branches, belonging to the field of artificial intelligence and machine learning technology. Background Technology

[0002] With the development of artificial intelligence technology, deep neural networks (DNNs) have made significant progress in many fields such as computer vision, natural language processing, and industrial control. However, traditional deep networks generally rely on a large number of parameters and stacked layer structures to improve expressive power, which leads to shortcomings in computational cost, generalization ability, and interpretability.

[0003] In recent years, the academic community has proposed a neural network structure based on the Kolmogorov-Arnold representation theorem, namely the Kolmogorov-Arnold Network (KAN). KAN is not a direct construction of the Kolmogorov-Arnold representation theorem, but rather a novel neural network inspired by its idea that "high-dimensional functions can be represented by combinations of univariate functions." Unlike traditional MLPs that use fixed activations at nodes and linear weights on edges, KAN replaces the linear weights of each edge with learnable univariate functions (often parameterized with B-splines), thus "moving" nonlinearity to the edges, with nodes only performing addition and aggregation. This design brings two key characteristics: first, function-level visualization and human-readable interaction enhance interpretability; second, in several data fitting and partial differential equation solving tasks, small-scale KANs can achieve accuracy comparable to or better than larger MLPs, and exhibit faster neural scaling laws.

[0004] In current research, typical KAN models employ a fixed set of functions, such as Gaussian radial basis functions (GRBF), B-spline functions, reflective switch activation functions, Chebyshev polynomials, Fourier transforms, wavelets, and other polynomial functions. Each input sample is computed through static, fixed function modules. While this design ensures the network's theoretical expressive power, it also introduces the following shortcomings: 1) Computational redundancy: Due to the lack of a dynamic selection mechanism, the model performs full computation on all function modules, even if some functions are invalid or contribute very little to the current input sample. This not only wastes computational resources but also affects the model's inference speed and energy consumption. 2) Lack of adaptability: In practical applications, different samples or tasks may depend on different feature patterns. For example, in industrial equipment fault prediction, the important feature functions differ under different operating states. However, existing KAN models use a uniform computational path and cannot dynamically adjust according to input characteristics, resulting in insufficient flexibility in model expression. 3) Lack of structural interpretability: Although KANs possess a certain degree of interpretability in theory, the lack of a mechanism to explicitly indicate "which function modules the current sample mainly depends on" makes it difficult to establish a mapping relationship between model decisions and physical mechanisms or business logic. This limits the practical application value of the model in high-risk fields such as healthcare, autonomous driving, and finance. The fundamental reason is that the design of existing KAN models remains a static structure, meaning that the selection and combination of function modules are fixed after training, lacking the ability to dynamically adjust for samples or tasks.

[0005] In computer vision, existing image classification, object detection, and image segmentation methods typically use fixed network structures and activation functions, failing to dynamically adjust computational paths based on the characteristics of the input image. This results in inefficiency when processing images of varying scales and complexities. Similarly, in natural language processing, traditional text classification models face similar challenges, unable to adaptively select the most suitable feature extraction method based on the characteristics of the text content. In industrial control, existing methods for equipment fault prediction, production process optimization, and quality inspection are mostly based on fixed model structures, making it difficult to adapt to the changing characteristics of different equipment and production environments. Therefore, there is an urgent need for an adaptive function combination method that can reduce computational redundancy while maintaining model expressiveness, improving model adaptability to different input scenarios, and enhancing the interpretability of the model structure. Summary of the Invention

[0006] To address the aforementioned issues, this invention proposes a KAN adaptive function combination method based on multi-scale sparse branches and its applications in multiple technical fields. This method can solve the problems of computational redundancy, lack of adaptability, and poor interpretability of existing KAN models, and achieve more efficient and accurate performance in various application fields.

[0007] The technical solution adopted by this invention to solve its technical problem is as follows:

[0008] In a first aspect, the present invention provides a KAN adaptive function combination method based on multi-scale sparse branches, comprising the following steps:

[0009] Step S1: Start the dynamic selection mechanism and initialize the neural network model parameters. The neural network model parameters include input and output layer parameters, dynamic selection controller parameters, and learnable function module parameters. The learnable function module includes a feature extraction function.

[0010] Step S2: Standardize the input data, obtain multi-level feature representation through multi-scale feature extraction method, and then dynamically calculate the selection weight of each learnable function module through hierarchical attention mechanism. The input data includes multi-source sensor data.

[0011] Step S3: Based on the selection weight filtering function module, only the function modules with selection weights greater than the preset activation threshold are activated to form an optimized sparse computation path.

[0012] Step S4: Drive the activated function modules to perform feature calculations, generate feature outputs for each module, and use a weighted fusion strategy to integrate the feature outputs of all activated modules into a unified feature representation.

[0013] Step S5: In the deep network structure, steps S3 and S4 are executed iteratively. Through layer-by-layer function module selection, feature calculation and feature stacking, a multi-level, high-dimensional deep feature representation is constructed, and finally a unified final fusion feature is formed.

[0014] Step S6: Generate task prediction results based on the final fused features, calculate the loss value between the prediction results and the real labels, and backpropagate the loss value back to each layer of the neural network model through the backpropagation algorithm to update all learnable parameters of the neural network model and realize dynamic optimization of the KAN adaptive function combination.

[0015] As one possible implementation of this embodiment, step S1 includes the following steps:

[0016] Step S11: Initialize the input and output layer parameters, establish the basic data path, and set the input-to-output weight matrix using the Kaiming Uniform initialization method;

[0017] Step S12: Construct a dynamic selection controller, which consists of multiple fully connected layers and performs nonlinear transformation through ReLU or SiLU activation functions to generate an initial selection score for each function module;

[0018] Step S13: Configure the parameters of the multi-scale learnable function module, which include the grid point parameters of the B-spline function, the center and width parameters of the radial basis function RBF, the scale and displacement parameters of the wavelet function, and the parameters of the Chebyshev polynomial.

[0019] Step S14: Set function weights and bias parameters, assign corresponding weight matrices and bias terms to the output features of each function module, and control the contribution of each module in the feature fusion process.

[0020] As one possible implementation of this embodiment, step S2 includes the following steps:

[0021] Step S21: Perform input data standardization processing. By calculating the mean μ and standard deviation σ of the input data, the original data is converted into standardized data with zero mean and unit variance to ensure numerical stability.

[0022] Step S22: Perform multi-scale feature extraction. Input the standardized data in parallel into multiple function modules of different scales. The multiple function modules of different scales include radial basis function (RBF), B-spline function, wavelet transform function and Chebyshev polynomial function. Each module extracts feature information from different dimensions based on its function characteristics.

[0023] Step S23: Implement the hierarchical attention mechanism calculation. Perform nonlinear transformation on the features at each scale through a multilayer perceptron (MLP) to generate the original selection score. Then, standardize the score into selection weights through a softmax function with a temperature coefficient.

[0024] Step S24: Perform computational path optimization decision-making. Based on the calculated selection weights and the preset activation threshold, determine the activation state of each function module to provide a basis for subsequent sparse selection.

[0025] As one possible implementation of this embodiment, step S3 includes the following steps:

[0026] Step S31: Receive selection weight input and obtain the selection weights of each function module. ;

[0027] Step S32: Perform threshold comparison and judgment, comparing the selection weight of each function module with the preset activation threshold. When comparing, The module is marked as active at this time.

[0028] Step S33: Perform sparse activation operation, allocate computing resources only to function modules marked as active, and form an optimized computing path;

[0029] Step S34: Generate an activation module set and record the identifiers of all activated function modules for use in the subsequent feature generation stage.

[0030] As one possible implementation of this embodiment, step S4 includes the following steps:

[0031] Step S41: Execute activation module feature generation. Input standardized data for each activated function module, perform the corresponding function transformation, and generate feature output.

[0032] Step S42: Collect the output features of each module, and store and organize the feature outputs generated by the activated module in a unified format;

[0033] Step S43: Implement weighted fusion strategy selection, select a fusion method according to preset fusion rules, the fusion rules are determined based on task type and data feature dimensions;

[0034] Step S44: Generate a unified feature representation by integrating the outputs of each module into a single feature vector through a selected weighted fusion strategy.

[0035] As one possible implementation of this embodiment, step S5 includes the following steps:

[0036] Step S51: Set the network depth parameters and determine the number of network layers L to be executed. Each layer includes a complete function selection and feature fusion process.

[0037] Step S52: Perform calculations iteratively layer by layer. For each layer... l ( l =1,2,...,L), repeatedly perform sparse branch selection and feature generation and fusion;

[0038] Step S53: Perform feature stacking operation, stacking the output features generated by each layer in the depth dimension to construct a hierarchical feature representation;

[0039] Step S54: Generate the final fused features by transforming and compressing the stacked features through a fully connected layer to generate the network's final output features.

[0040] As one possible implementation of this embodiment, step S6 includes the following steps:

[0041] Step S61: Based on the obtained final fusion features, generate predicted values ​​through the output layer corresponding to the task type to obtain the final prediction result;

[0042] Step S62: Calculate the loss function value, compare the final prediction result with the true label, and quantify the model prediction error;

[0043] Step S63: Perform backpropagation calculation, calculate the gradient layer by layer from the output layer to the input layer, and determine the contribution of each parameter to the loss.

[0044] Step S64: Update the neural network model parameters. Based on the calculated gradient, use an optimization algorithm to adjust all learnable parameters and minimize the loss function.

[0045] Secondly, embodiments of the present invention provide a KAN adaptive function combination device based on multi-scale sparse branches, comprising:

[0046] An initialization module is used to start the dynamic selection mechanism and initialize the neural network model parameters, which include input and output layer parameters, dynamic selection controller parameters, and learnable function module parameters, including feature extraction functions.

[0047] The weight calculation module is used to standardize the input data, obtain multi-level feature representation through multi-scale feature extraction method, and then dynamically calculate the selection weight of each learnable function module through hierarchical attention mechanism. The input data includes multi-source sensor data.

[0048] The sparse selection module is used to filter the function modules based on the selection weights, and only activate the function modules whose selection weights are greater than a preset activation threshold, thereby forming an optimized sparse computation path.

[0049] The feature fusion module is used to drive the activated function modules to perform feature calculations, generate feature outputs of each module, and integrate the feature outputs of all activated modules into a unified feature representation using a weighted fusion strategy.

[0050] The deep feature construction module is used to iteratively perform sparse selection and feature fusion operations in the deep network structure. Through layer-by-layer function module selection, feature calculation and feature stacking, it constructs multi-level, high-dimensional deep feature representations and finally forms a unified final fused feature.

[0051] The function optimization module is used to generate task prediction results based on the final fused features, calculate the loss value between the prediction results and the real labels, backpropagate the loss value to each layer of the neural network model through the backpropagation algorithm, update all learnable parameters of the neural network model, and realize dynamic optimization of the KAN adaptive function combination.

[0052] Thirdly, the present invention provides an application of a KAN adaptive function combination method based on multi-scale sparse branches in the field of computer vision, using the method described in any of the above to perform image classification, object detection or image segmentation tasks.

[0053] Fourthly, the present invention provides an application of a KAN adaptive function combination method based on multi-scale sparse branches in the fields of natural language processing and industrial control. The method described in any of the above embodiments is used for text classification tasks, as well as for equipment fault prediction, production process optimization, or quality inspection tasks.

[0054] The beneficial effects of the technical solutions of the embodiments of the present invention are as follows:

[0055] The technical solution of this invention is a KAN adaptive function combination method based on multi-scale sparse branches. The dynamic selection mechanism is advanced, and the function modules are dynamically selected during the calculation stage, avoiding invalid calculations and thus solving the computational redundancy problem in the traditional KAN model. Compared with the traditional full function calculation mode, this invention has significant advantages in the following aspects: (1) By selecting function modules in advance during the forward propagation stage, the calculation of irrelevant functions is reduced, and the inference speed and computational efficiency are improved; (2) The number and types of function modules are dynamically selected, which can be adapted according to the characteristics of different inputs, avoiding repeated calculations and unnecessary resource consumption in the traditional method; (3) Due to the dynamic selection of function modules, unnecessary computational resource consumption is reduced, thereby reducing energy consumption and making it suitable for running on resource-constrained devices; (4) The function modules selected each time are closely related to the input data, which can provide interpretability of model decisions, especially in complex tasks, which can effectively reveal which functions are more important to specific inputs. This invention not only solves the problems of computational redundancy, low inference efficiency, lack of adaptability and poor interpretability in the prior art, but also brings significant improvements in inference speed, saving of computing resources, model adaptability and energy efficiency. It is suitable for application scenarios that require high efficiency and real-time response and has a wide range of practical application value.

[0056] The technical solution of this invention, a KAN adaptive function combination method based on multi-scale sparse branches, has achieved excellent performance in various application fields. In computer vision, it improves the accuracy and efficiency of image classification, object detection, and image segmentation; in natural language processing, it improves the accuracy of text classification; and in industrial control, it improves the accuracy and efficiency of equipment fault prediction, production process optimization, and quality inspection. Simultaneously, this method significantly reduces computational redundancy, improves inference efficiency, reduces energy consumption, and enhances model interpretability, thus possessing broad practical application value. Attached Figure Description

[0057] Figure 1 This is a flowchart illustrating a KAN adaptive function combination method based on multi-scale sparse branches, according to an exemplary embodiment.

[0058] Figure 2This is a network architecture diagram of a neural network model according to an exemplary embodiment;

[0059] Figure 3 This is a schematic diagram illustrating the principle of a KAN adaptive function combination device based on multi-scale sparse branches, according to an exemplary embodiment. Detailed Implementation

[0060] To more clearly illustrate the technical features of the present invention, the present invention will be described in detail below through specific embodiments and in conjunction with the accompanying drawings.

[0061] Example 1

[0062] like Figure 1 and Figure 2 As shown in the figure, an embodiment of the present invention provides a KAN adaptive function combination method based on multi-scale sparse branches, which includes the following steps:

[0063] Step S1: Start the dynamic selection mechanism and initialize the neural network model parameters. The neural network model parameters include input and output layer parameters, dynamic selection controller parameters, and learnable function module parameters. The learnable function module contains at least two different types of feature extraction functions.

[0064] Step S2: Standardize the input data, obtain multi-level feature representation through multi-scale feature extraction method, and then dynamically calculate the selection weight of each learnable function module through hierarchical attention mechanism. The input data includes image data in the field of computer vision, text data in the field of natural language processing, or multi-source sensor data in application scenarios that require efficient real-time response, such as industrial control.

[0065] Step S3: Based on the selection weight filtering function module, only the function modules with selection weights greater than the preset activation threshold are activated to form an optimized sparse computation path.

[0066] Step S4: Drive the activated function modules to perform feature calculations, generate feature outputs for each module, and use a weighted fusion strategy to integrate the feature outputs of all activated modules into a unified feature representation.

[0067] Step S5: In the deep network structure, steps S3 and S4 are executed iteratively. Through layer-by-layer function module selection, feature calculation and feature stacking, a multi-level, high-dimensional deep feature representation is constructed, and finally a unified final fusion feature is formed.

[0068] Step S6: Generate task prediction results based on the final fused features, calculate the loss value between the prediction results and the real labels, and backpropagate the loss value back to each layer of the neural network model through the backpropagation algorithm to update all learnable parameters of the neural network model and realize dynamic optimization of the KAN adaptive function combination.

[0069] As one possible implementation of this embodiment, step S1 includes the following steps:

[0070] Step S11: Initialize the input and output layer parameters, establish the basic data path, and set the input-to-output weight matrix using the Kaiming Uniform initialization method;

[0071] Step S12: Construct a dynamic selection controller, which consists of multiple fully connected layers and performs nonlinear transformation through ReLU or SiLU activation functions to generate an initial selection score for each function module;

[0072] Step S13: Configure the parameters of the multi-scale learnable function module, which include the grid point parameters of the B-spline function, the center and width parameters of the radial basis function RBF, the scale and displacement parameters of the wavelet function, and the parameters of the Chebyshev polynomial.

[0073] Step S14: Set function weights and bias parameters, assign corresponding weight matrices and bias terms to the output features of each function module, and control the contribution of each module in the feature fusion process.

[0074] Step S15: Initialize Dropout layer parameters and configure regularization mechanism to randomly drop some neuron connections during training to prevent model overfitting and improve generalization ability.

[0075] As one possible implementation of this embodiment, the initialization of input / output layer parameters in step 11 specifically includes:

[0076] Based on the network's input dimension dim and output dimension dim The weight matrix is ​​set using the Kaiming Uniform initialization method. ;

[0077] Through mathematical formulas Implement weight initialization to ensure that the variance of activation values ​​in each layer remains consistent.

[0078] As one possible implementation of this embodiment, the process of constructing the dynamic selection controller in step 12 includes:

[0079] The controller network adopts a multilayer perceptron architecture, which includes an input layer, at least one hidden layer, and an output layer;

[0080] The hidden layer uses ReLU or SiLU activation functions for non-linear transformation, and the output layer uses linear activation functions to generate the original selection score.

[0081] The controller parameters are initialized using the Xavier method to ensure that information is effectively propagated at each layer of the network.

[0082] As one possible implementation of this embodiment, the specific implementation of configuring the parameters of the multi-scale learnable function module in step 13 includes:

[0083] For the B-spline function module, initialize the grid point parameter grid. points Based on a given grid range and resolution, the grid is uniformly distributed.

[0084] For the radial basis function (RBF) module, the initialization center point parameter centers is determined using the K-means clustering algorithm, and the width parameter σ is adaptively calculated based on the data distribution.

[0085] For the wavelet function module, initialize the scale parameter a and the displacement parameter b, and set a reasonable range of values ​​based on the feature distribution of the input data.

[0086] For Chebyshev polynomial modules, the coefficients of each order are initialized using the standardized initialization method of orthogonal polynomial bases.

[0087] As one possible implementation of this embodiment, setting the function weights and bias parameters in step 14 includes:

[0088] Assign an independent weight matrix W_fun and a bias vector b_fun to each function module;

[0089] The dimensions of the weight matrix are determined based on the output feature dimensions of the function module and the output dimensions of the network layer;

[0090] The bias vector is initialized with zero or small random numbers to avoid introducing excessive deviation in the initial stage.

[0091] As one possible implementation of this embodiment, the initialization of Dropout layer parameters in step 15 includes:

[0092] Set the Dropout rate p, with a value ranging from 0.1 to 0.5, and adjust it according to the network complexity and the amount of training data;

[0093] During the training phase, the neuron output is randomly set to zero with probability p during forward propagation.

[0094] During the inference phase, all neurons participate in the computation, but the output value is scaled by (1-p) to maintain the expected value.

[0095] As one possible implementation of this embodiment, step S2 includes the following steps:

[0096] Step S21: Perform input data standardization processing. By calculating the mean μ and standard deviation σ of the input data, the original data is converted into standardized data with zero mean and unit variance to ensure numerical stability.

[0097] Step S22: Perform multi-scale feature extraction. Input the standardized data in parallel into multiple function modules of different scales. The multiple function modules of different scales include radial basis function (RBF), B-spline function, wavelet transform function and Chebyshev polynomial function. Each module extracts feature information from different dimensions based on its function characteristics.

[0098] Step S23: Implement the hierarchical attention mechanism calculation. Perform nonlinear transformation on the features at each scale through a multilayer perceptron (MLP) to generate the original selection score. Then, standardize the score into selection weights through a softmax function with a temperature coefficient.

[0099] Step S24: Perform computational path optimization decision-making. Based on the calculated selection weights and the preset activation threshold, determine the activation state of each function module to provide a basis for subsequent sparse selection.

[0100] As one possible implementation of this embodiment, the data standardization process in step 21 specifically includes:

[0101] Calculate the mean μ and standard deviation σ of the input data X, and standardize it using the formula X_normalized=(X-μ) / σ;

[0102] For small batches of data, a batch normalization method is used to dynamically calculate the mean and variance during training.

[0103] As one possible implementation of this embodiment, the multi-scale feature extraction in step 22 includes:

[0104] The Radial Basis Function (RBF) module calculates the similarity between the input data and the center point using a Gaussian kernel function;

[0105] The B-spline function module performs piecewise polynomial interpolation based on preset grid points;

[0106] The wavelet transform function module performs time-frequency analysis by scaling and translating the mother wavelet;

[0107] Chebyshev polynomial modules use orthogonal polynomial bases for function approximation.

[0108] As one possible implementation of this embodiment, the hierarchical attention mechanism calculation in step 23 includes:

[0109] The MLP transformation is implemented through a two-layer fully connected network. The first layer uses the ReLU activation function, and the second layer uses the linear activation function.

[0110] The temperature coefficient τ controls the smoothness of the softmax output; the smaller the τ value, the more concentrated the weight distribution.

[0111] The formula for calculating the selection weight is: ,in, For the i-th function module, select the weights. The score for selecting the i-th module. It is the temperature coefficient.

[0112] As one possible implementation of this embodiment, step S3 includes the following steps:

[0113] Step S31: Receive selection weight input and obtain the selection weights of each function module. ;

[0114] Step S32: Perform threshold comparison and judgment, comparing the selection weight of each function module with the preset activation threshold. When comparing, The module is marked as active at this time.

[0115] Step S33: Perform sparse activation operation, allocate computing resources only to function modules marked as active, and form an optimized computing path;

[0116] Step S34: Generate an activation module set and record the identifiers of all activated function modules for use in the subsequent feature generation stage.

[0117] As one possible implementation of this embodiment, the specific implementation of the threshold comparison judgment in step 32 includes:

[0118] Activation threshold This is a preset fixed value, ranging from 0.1 to 0.3, which can be adjusted according to task complexity and computing resource requirements;

[0119] For each function module i, the activation condition is: ;

[0120] When the weights of all modules are not higher than the threshold At that time, the top k modules with the highest weights are forcibly activated to ensure that at least a certain number of modules participate in the calculation.

[0121] As one possible implementation of this embodiment, the optimization effect of the sparse activation operation in step 33 includes:

[0122] The computational complexity is reduced from O(N) to O(M), where N is the total number of modules, M is the number of activated modules, and M < 0. <N;

[0123] Memory usage is reduced, only the intermediate calculation results of the activated module need to be stored;

[0124] The inference speed is improved, avoiding the computational overhead of invalid modules.

[0125] As one possible implementation of this embodiment, the set of activated modules in step 34 is represented as follows:

[0126] The activation state of each module is represented by a binary mask vector, where 1 indicates activation and 0 indicates inactivation.

[0127] Alternatively, a module index list can be used to record the identifiers of all active modules;

[0128] The set of activated modules is passed as metadata to subsequent computation stages to guide the execution of specific functions.

[0129] As one possible implementation of this embodiment, step S4 includes the following steps:

[0130] Step S41: Execute activation module feature generation. Input standardized data for each activated function module, perform the corresponding function transformation, and generate feature output.

[0131] Step S42: Collect the output features of each module, and store and organize the feature outputs generated by the activated module in a unified format;

[0132] Step S43: Implement weighted fusion strategy selection, select a fusion method according to preset fusion rules, the fusion rules are determined based on task type and data feature dimensions;

[0133] Step S44: Generate a unified feature representation by integrating the outputs of each module into a single feature vector through a selected weighted fusion strategy.

[0134] As one possible implementation of this embodiment, the specific process of generating the activation module feature in step 41 includes:

[0135] The B-spline function module generates smooth feature outputs through a linear combination of basis functions;

[0136] The Radial Basis Function (RBF) module calculates local feature representations using Gaussian radial basis functions.

[0137] The wavelet function module extracts time-frequency domain features through wavelet transform;

[0138] Chebyshev polynomial modules approximate complex functions through orthogonal polynomial expansions.

[0139] As one possible implementation of this embodiment, the weighted fusion strategy in step 43 includes:

[0140] Weighted average fusion: ,in To select weights, Output for the module;

[0141] Max pooling fusion: ;

[0142] Attention-weighted fusion: The fusion weights of each module are dynamically adjusted through an additional attention network;

[0143] Serial fusion: The outputs of each module are spliced ​​together along the feature dimension.

[0144] As one possible implementation of this embodiment, the generation of the unified feature representation in step 44 ensures that:

[0145] Feature dimension consistency: the output feature dimension remains constant regardless of the number of activated modules.

[0146] Numerical stability is achieved by using normalization or standardization to avoid excessive differences in the range of eigenvalues.

[0147] Information integrity is ensured by retaining key feature information of each activated module.

[0148] As one possible implementation of this embodiment, step S5 includes the following steps:

[0149] Step S51: Set the network depth parameters and determine the number of network layers L to be executed. Each layer includes a complete function selection and feature fusion process.

[0150] Step S52: Perform calculations iteratively layer by layer. For each layer... l ( l =1,2,...,L), repeatedly perform sparse branch selection and feature generation and fusion;

[0151] Step S53: Perform feature stacking operation, stacking the output features generated by each layer in the depth dimension to construct a hierarchical feature representation;

[0152] Step S54: Generate the final fused features by transforming and compressing the stacked features through a fully connected layer to generate the network's final output features.

[0153] As one possible implementation of this embodiment, the specific characteristics of the layer-by-layer iterative execution in step 52 include:

[0154] Each layer recalculates the function selection weights based on the output features of the previous layer, achieving adaptive hierarchical feature selection;

[0155] Different layers can use different sets of function modules and activation thresholds to adapt to the feature abstraction needs of different levels;

[0156] Gradient flow is maintained between layers through residual connections or skip connections, which alleviates the training difficulties of deep networks.

[0157] As one possible implementation of this embodiment, the technical effects of the feature stacking operation in step 53 include:

[0158] Construct multi-level feature representations, with low-level features capturing local details and high-level features capturing global semantics;

[0159] Enhance feature representation capabilities by fully utilizing feature information from each layer through hierarchical combination;

[0160] Provide feature redundancy to improve the robustness and generalization ability of the model.

[0161] As one possible implementation of this embodiment, the process of generating the final fused feature in step 54 includes:

[0162] The stacked features are transformed by a fully connected layer, mapping the high-dimensional features to the output dimension required by the task.

[0163] Introduce nonlinear transformation capabilities using activation functions (such as ReLU, Sigmoid, etc.);

[0164] Optionally, a batch normalization layer can be added to accelerate the training process and improve stability.

[0165] As one possible implementation of this embodiment, step S6 includes the following steps:

[0166] Step S61: Based on the obtained final fusion features, generate predicted values ​​through the output layer corresponding to the task type to obtain the final prediction result;

[0167] Step S62: Calculate the loss function value, compare the final prediction result with the true label, and quantify the model prediction error;

[0168] Step S63: Perform backpropagation calculation, calculate the gradient layer by layer from the output layer to the input layer, and determine the contribution of each parameter to the loss.

[0169] Step S64: Update the neural network model parameters. Based on the calculated gradient, use an optimization algorithm to adjust all learnable parameters and minimize the loss function.

[0170] As one possible implementation of this embodiment, the generation of the final prediction result in step 61 includes, depending on the task type:

[0171] Classification task: Use the softmax output layer to generate class probability distributions;

[0172] Regression task: Generate continuous numerical predictions using a linear output layer;

[0173] Multi-task learning: Use multiple output heads to generate prediction results for different tasks.

[0174] As one possible implementation of this embodiment, the loss function calculation in step 62 includes:

[0175] Classification tasks use cross-entropy loss: ;

[0176] The regression task uses mean squared error loss: ;

[0177] Optionally, L1 / L2 regularization terms can be added to prevent overfitting;

[0178] Optionally, a sparsity constraint term can be added to encourage the model to use fewer activation modules.

[0179] As one possible implementation of this embodiment, the neural network model parameter update in step 64 adopts the following approach:

[0180] Stochastic gradient descent (SGD) and its variants (such as Adam, RMSprop, etc.) are optimization algorithms.

[0181] Learning rate scheduling strategies, such as exponential decay and cosine annealing;

[0182] Gradient clipping technique prevents gradient explosion;

[0183] Simultaneously update function module parameters, controller parameters, and output layer parameters to achieve end-to-end joint optimization.

[0184] The KAN adaptive function combination method based on multi-scale sparse branches in this embodiment advances the dynamic selection mechanism, dynamically selecting function modules during the computation phase to avoid invalid computation. Specifically, the neural network model parameters are first initialized, including input / output layer parameters, dynamic selection controller, and learnable function module parameters. Then, the input data is standardized to ensure the stability of subsequent computations. Next, multi-level features are extracted using a multi-scale feature extraction method, and function selection weights are calculated using a hierarchical attention mechanism. Based on these weights, sparse selection activates computation branches, activating only the function modules most relevant to the current input. The selected function modules are executed, generating feature outputs, and then the outputs of each function module are fused. This process is repeated layer by layer in the deep network, stacking features and performing fully connected operations. Finally, the prediction result is output, and loss calculation and backpropagation are performed.

[0185] The KAN adaptive function combination method based on multi-scale sparse branches in this embodiment achieves excellent performance in various application fields. In computer vision, it improves the accuracy and efficiency of image classification, object detection, and image segmentation; in natural language processing, it improves the accuracy of text classification; and in industrial control, it improves the accuracy and efficiency of equipment fault prediction, production process optimization, and quality inspection. Simultaneously, this method significantly reduces computational redundancy, improves inference efficiency, reduces energy consumption, and enhances model interpretability, thus possessing broad practical application value.

[0186] Example 2

[0187] like Figure 3 As shown, an embodiment of the present invention provides a KAN adaptive function combination device based on multi-scale sparse branches, comprising:

[0188] An initialization module is used to start the dynamic selection mechanism and initialize the neural network model parameters. The neural network model parameters include input and output layer parameters, dynamic selection controller parameters, and learnable function module parameters. The learnable function module contains at least two different types of feature extraction functions.

[0189] The weight calculation module is used to standardize the input data, obtain multi-level feature representation through multi-scale feature extraction method, and then dynamically calculate the selection weight of each learnable function module through hierarchical attention mechanism. The input data includes image data in the field of computer vision, text data in the field of natural language processing, or multi-source sensor data in application scenarios that require efficient real-time response, such as industrial control.

[0190] The sparse selection module is used to filter the function modules based on the selection weights, and only activate the function modules whose selection weights are greater than a preset activation threshold, thereby forming an optimized sparse computation path.

[0191] The feature fusion module is used to drive the activated function modules to perform feature calculations, generate feature outputs of each module, and integrate the feature outputs of all activated modules into a unified feature representation using a weighted fusion strategy.

[0192] The deep feature construction module is used to iteratively perform sparse selection and feature fusion operations in the deep network structure. Through layer-by-layer function module selection, feature calculation and feature stacking, it constructs multi-level, high-dimensional deep feature representations and finally forms a unified final fused feature.

[0193] The function optimization module is used to generate task prediction results based on the final fused features, calculate the loss value between the prediction results and the real labels, backpropagate the loss value to each layer of the neural network model through the backpropagation algorithm, update all learnable parameters of the neural network model, and realize dynamic optimization of the KAN adaptive function combination.

[0194] The specific process of performing KAN adaptive function combination using a KAN adaptive function combination device based on multi-scale sparse branches provided in this embodiment of the invention is as follows.

[0195] 1. Activate the dynamic selection mechanism and initialize model parameters:

[0196] By initializing the parameters of each learnable function module in the neural network and setting up the input feature processing flow, preparations are made for the subsequent dynamic selection mechanism.

[0197] (1) Initialize the parameters of the input and output layers:

[0198] First, the model's input and output layers need to be initialized according to the network's dimensions. The input dimension is... input_dim The output dimension is output_dim The system sets initial weights for the input and output layers through an initialization function (Kaiming Uniform initialization) to ensure stable network operation during subsequent computations.

[0199] For a fully connected layer from input to output:

[0200] ,

[0201] in, The weight matrix is ​​from input to output. and For the dimensions of input and output.

[0202] Initialize the parameters of the input and output layers; initialize the weight matrix using the Kaiming Uniform initialization method, thus enabling this function first.

[0203] (2) Initialize the dynamic selection controller:

[0204] The system initializes a dynamic selection controller, which calculates the selection weights for each function module based on the characteristics of the input data. The controller consists of multiple fully connected layers and undergoes nonlinear transformations using activation functions such as ReLU or SiLU. The controller's output is a selection score for each function module.

[0205] Assuming input features The controller network processes the input through linear transformations and activation functions:

[0206] ,

[0207] in, This is the weight matrix. For bias terms, It is an activation function.

[0208] Initialize the controller network (including weights and biases); perform nonlinear transformations using ReLU or SiLU activation functions to generate selection weights for each function module, thus enabling the function to function first.

[0209] (3) Initialize the parameters of the learnable function module:

[0210] This step involves initializing the parameters of all function modules, such as B-spline functions, radial basis functions (RBF), Chebyshev polynomials, etc. The parameters of these function modules (such as grid point locations, scale factors, etc.) are set using a standardized initialization method to ensure that each function module functions correctly in computation.

[0211] Taking B-spline functions as an example, the system initializes the parameters of the grid points as follows:

[0212] ,

[0213] in, It represents the value of the grid points, initialized based on the given grid range and resolution.

[0214] Initialize each function module (such as B-spline, RBF, Chebyshev); initialize the learnable parameters of the function module, such as grid points, scale factor, etc., so as to enable the function.

[0215] (4) Initialize function weights and biases:

[0216] Each function module generates an output feature, which needs to be weighted using a weight matrix. The system initializes the weights and biases for each module to control the influence of each module's output.

[0217] For the weight initialization of the B-spline function module:

[0218] ,

[0219] in, This is the weight matrix of the B-spline module.

[0220] (5) Initialize the Dropout layer:

[0221] If Dropout is enabled (to prevent overfitting), the system will initialize the corresponding Dropout layer.

[0222] II. Input data preprocessing and weight selection using a hierarchical attention gating computation function based on multi-scale sparse branches:

[0223] After receiving the input data, it is standardized to a format suitable for the model. Then, using feature extraction methods at multiple different scales, an attention model is used to assign an importance weight to each feature at each scale, and the selection weights are calculated based on the hierarchical attention mechanism.

[0224] (1) Data standardization and feature preparation:

[0225] The purpose of this step is to standardize the input data to ensure that the model's input data has zero mean and unit variance, preventing numerical instability during data propagation in the network. Standardized data X This will be used as input for subsequent calculations.

[0226] Let the input data be The standardization process is as follows:

[0227] ,

[0228] in, and These are the mean and standard deviation of the input data, respectively. The input data is standardized.

[0229] The input data is standardized to ensure the stability and efficiency of subsequent calculations; the input data is made to meet the distribution of zero mean and unit variance, thereby realizing this function.

[0230] (2) Multi-scale feature extraction:

[0231] After standardization, the system utilizes multiple feature extraction methods at different scales to extract multi-level feature information from the input data. Each feature extraction method at each scale can be a different function module (e.g., radial basis function (RBF), B-spline, wavelet transform, etc.), which serves to understand the features of the data from different perspectives.

[0232] make For the standardized input data, feature extraction is performed using different function modules for each scale iii to obtain the output for each scale:

[0233] ,

[0234] in, Indicates the quantity of scale, For the first i Feature representation at each scale, This represents an adaptive parameter related to the data distribution, used to determine the size and applicability of each scale.

[0235] The data is analyzed using clustering algorithms (such as K-means), principal component analysis (PCA), Gaussian mixture models (GMM), or other statistical methods. These methods enable the system to identify local density regions or important features in the data, and then assign an appropriate scale to each region or feature. Based on K-means clustering, the input dataset... Classified as K There are several clusters, and each cluster can choose a scale parameter. And determine the feature extraction method at this scale:

[0236] ,

[0237] in, Indicates the first i The cluster center of the class is used as an adaptive parameter for this scale to determine the size of the feature extraction at this scale.

[0238] Different function modules (such as RBF, B-spline, Chebyshev, etc.) are used to extract multi-scale features from the input data; each scale extracts feature information at different levels from the input data, enhancing the model's ability to represent the data, thereby achieving this function.

[0239] (3) Hierarchical attention mechanism calculation function selects weights:

[0240] To enable dynamic selection of function modules, this step introduces a hierarchical attention mechanism, assigning an importance weight to features at each scale through an attention model. In this way, the model can adaptively determine which scales of features are most important to the current task based on the characteristics of the input data.

[0241] The attention mechanism computes the selection score for each scale feature using a multilayer perceptron (MLP). Scores for each scale The weights are then standardized using the softmax function. This is used to represent the importance of each scale feature:

[0242] ,

[0243] in, It's a temperature coefficient that controls the smoothness of the selected parameters. A smaller one... The value makes attention more focused, meaning only the scale with the highest score is selected.

[0244] In the hierarchical attention mechanism, selection weights are calculated using an attention model. The calculation of selection weights utilizes a multilayer perceptron (MLP), and then the scores are standardized using a softmax function to derive the importance weights for each scale feature. (Temperature coefficient in the attention mechanism) t The smoothness of the selection was controlled, and the smaller the selection... t Values ​​that enhance focus. Aside from the temperature coefficient, the weighting mechanism is dynamic, adaptively adjusting based on input features.

[0245] The score for each scale feature is calculated using a multilayer perceptron (MLP); the selection weight for each scale is calculated using a softmax function. This provides a basis for subsequent function selection, thereby enabling the implementation of this function.

[0246] (4) Sparse branching and computation path optimization:

[0247] Selection weights calculated based on hierarchical attention mechanism The system determines which function modules need to be computed and which can be skipped. Through this sparse selection mechanism, only the function modules most relevant to the current input are activated, while other function modules do not participate in the computation, thereby avoiding redundant calculations and improving computational efficiency.

[0248] For each scale i If its selection weight is greater than the set threshold If the condition is met, the function module will be activated and participate in the calculation; otherwise, the function module will be skipped.

[0249] Finally, the system fuses the weighted output features to generate the final feature representation:

[0250] ,

[0251] in, The weighted fusion features For the first i The output of each function module.

[0252] The important features are selected for computation based on the weights output by the attention mechanism; the computation efficiency is significantly improved by reducing the computation of irrelevant features through a sparse branching strategy, thereby achieving this function.

[0253] III. Sparse selection activation computational branch:

[0254] Based on the attention weights output in step two, activate the most relevant function branch, and keep unselected branches closed to avoid redundant calculations.

[0255] (1) Filtering important function modules based on attention weights:

[0256] Based on the attention weights of each function module calculated in step two The system will determine which function modules are most important to the current input and activate those function modules for computation.

[0257] (2) Activate and select the most relevant function module:

[0258] Based on the output weights, the system will select the more important function modules for computation at each layer. To do this, the system first determines the weights... Set the preset activation threshold to That is, choose those The function modules are used for calculation. Function modules with weights less than a threshold are skipped from the calculation.

[0259] The mathematical formula is expressed as follows:

[0260] ,

[0261] in, For the set of activated function modules, It is the first i The selection weights of each function module It is the activation threshold.

[0262] The preset activation threshold for a function module is a fixed value. During the computation process, if the attention weight of a certain function module exceeds this threshold, the module will be activated to determine which function modules need to be computed.

[0263] Based on the calculated attention weights Select the most important function module for calculation, ignoring irrelevant modules; based on the set threshold... This function is achieved by activating only the function modules with higher weights and reducing redundant calculations.

[0264] (3) Optimization of computational path in the sparse selection process:

[0265] Through the above filtering steps, the system ensures that only the most relevant function modules are calculated, significantly reducing the amount of computation. Unselected modules remain closed and do not consume computing resources, thus optimizing the computation path.

[0266] This sparse selection mechanism avoids redundant computations that must be performed on all function modules in traditional methods by pre-determining which function modules need to be computed. This makes the inference process more efficient and allows for faster response to different inputs.

[0267] By selectively calculating the most relevant function modules, redundant calculations of irrelevant modules are avoided; the amount of computation is significantly reduced, the inference speed is improved, and thus the computational efficiency is increased.

[0268] Description of all included function branches:

[0269] 1. LiuKAN function (third-order B-spline):

[0270] formula: ,

[0271] in: ;

[0272] 2. Radial Basis Function (RBF):

[0273] formula: ,

[0274] in Represents the input vector With the center The distance between them It is the width control parameter of the Gaussian function;

[0275] The expression for the RBF network: ;

[0276] 3. Wav-KAN wavelet function:

[0277] The DoG function formula for Wav-KAN:

[0278] ,

[0279] in, Indicates to Differentiate, It is a Gaussian function centered at 0;

[0280] 4. Chebyshev KAN function:

[0281] formula: ,

[0282] in, It is a normalized input tensor. It is the first k --order Chebyshev polynomial n It is the order of the Chebyshev polynomial. These are the learnable coefficients of Chebyshev interpolation;

[0283] This formula approximates the objective function using a weighted sum of Chebyshev polynomials. It utilizes the Kolmogorov-Arnold theorem, which guarantees the existence of an expression that represents any continuous multivariable function by a superposition of univariable functions.

[0284] IV. Execute the selected function module and generate feature output:

[0285] For each selected function module, forward propagation computation is performed. Each function module independently processes the input data and generates an output feature. This step ensures that only functions valid for the current input are executed, thus saving computational resources.

[0286] (1) Feature processing after activation function module:

[0287] Once the activated function module receives the input data, it will perform the corresponding computational task (e.g., wavelet transform, radial basis function (RBF), etc.) and generate feature output. The computation process of each module is independent and will not be affected by other inactive modules.

[0288] Each activation function module Upon receiving input Then, its corresponding output features are generated:

[0289] ,

[0290] in, It is the first i The output features of each activation function module It corresponds to the function module's response to the input data. The mapping.

[0291] Each activated function module Based on the input data Generate corresponding output features Inactive modules do not perform calculations, thus saving computing resources and achieving this function.

[0292] (2) Generation and storage of output features:

[0293] Each function module After the forward propagation is completed, the corresponding output features will be generated. These features will be stored and passed to subsequent computational steps for use in further fusion, splicing, or fully connected operations.

[0294] For the output of each function module, the system ensures that its features can be saved in a certain format (such as tensor or matrix form) for use in feature fusion and weighting in subsequent steps.

[0295] Mathematically represented as:

[0296] ,

[0297] in, This represents the set of all calculated output features. It is the output of each function module.

[0298] Store the output characteristics of each selected function module; ensure that these characteristics are stored in a uniform format for later use, thereby achieving this functionality.

[0299] (3) Quality assurance of output features:

[0300] In this step, the system also checks the quality of each output feature to ensure the accuracy of the calculation results. If the output feature of a function module is of low quality (e.g., vanishing gradients or abnormal activations), certain strategies will be used to correct or ignore it to ensure the stability of the output.

[0301] V. Merge the outputs of each function module:

[0302] All calculated feature results are fused. The output results are weighted and fused according to previously calculated weights to generate the final output features. This weighted fusion strategy can employ simple weighted averaging, max pooling, or other weighting mechanisms.

[0303] (1) Collect the output characteristics of each function module:

[0304] In step four, the selected function module has been executed and the output features have been generated. In this step, the system collects all calculated feature results. These feature results come from each activated function module, and each module calculates based on the input data. The corresponding output features were generated.

[0305] Collect the output characteristics of all activated function modules ,in It is the first i The output features of each activation function module.

[0306] (2) Select a weighted fusion strategy:

[0307] Depending on the task requirements and the characteristics of the input data, the system can employ different weighted fusion strategies to merge output features. Common weighted fusion strategies include:

[0308] Weighted average: The output features of each function module are summed according to their weights.

[0309] ;

[0310] Max pooling: Takes the maximum value at each position to generate the final fused feature.

[0311] ;

[0312] Other weighting mechanisms, such as minimum pooling and mean pooling, should be selected based on the specific task requirements.

[0313] When choosing a weighted fusion strategy, the most suitable strategy should be determined based on the characteristics of the input data and the task objective. Weighted average fusion is often used in scenarios where the influence of different features needs to be balanced, while max pooling is typically used to capture the maximum value or strongest signal among the features.

[0314] Using attention weights For output features Perform weighted fusion; select an appropriate weighted fusion strategy, such as weighted average or max pooling, based on task requirements to achieve this function.

[0315] (4) Generate the final fusion features:

[0316] After weighted fusion, the output features of all function modules are combined into a single final output feature. This feature will be used as the output of the current layer and passed to the next layer for further calculation.

[0317] Final fusion features It will be used as input to the network and fed into subsequent layers for computation until the final prediction result is produced.

[0318] By fusing features from different function modules, the model can combine the strengths of each module to improve feature representation capabilities; using dynamic weights to adjust the influence of different module outputs helps the model adapt to different input features, thereby improving inference accuracy.

[0319] VI. Perform function selection and computation layer by layer, stack features, and perform fully connected operations:

[0320] In deep networks, steps three through five are executed layer by layer, concatenating the output features of each layer and performing final feature integration through fully connected layers. In this way, the model can effectively handle the dynamic selection and combination of multiple function modules.

[0321] (1) Perform steps three through five layer by layer:

[0322] In deep networks, the model executes steps three through five layer by layer. Each layer selects relevant function modules for computation based on the output of the previous layer, generates feature outputs, and then performs weighted fusion of these features. This ensures that each layer can adaptively select the most relevant computation path and feature fusion strategy.

[0323] (2) Output characteristics of stacked layers:

[0324] Output features generated at each layer These features are stacked together to form a larger feature vector, which serves as the input for subsequent layers. In this way, the model progressively builds higher-dimensional feature representations, allowing features from different layers to effectively interact and enhance each other.

[0325] (3) Fully connected operation:

[0326] After each layer of feature stacking, the system integrates these features through a fully connected layer to generate the final output of that layer. The fully connected layer performs a linear transformation on the input features to obtain the final feature representation. .

[0327] VII. Output the final prediction results and perform loss calculation and backpropagation:

[0328] Task prediction, such as classification or regression, is performed based on the final fused features (the output features after final fusion). The loss is calculated based on the prediction results, and backpropagation is performed to update the controller parameters and the weights of the function modules to optimize model performance.

[0329] (1) Output prediction results:

[0330] By performing calculations across all layers, the features are ultimately fused. This will be passed to the task output layer, such as a classifier or regressor, to generate the final prediction result based on the task type.

[0331] (2) Loss calculation:

[0332] Based on the difference between the final output and the true label, a loss function (such as cross-entropy loss, mean squared error, etc.) is calculated. The loss function measures the model's prediction error and serves as the basis for model optimization.

[0333] (3) Backpropagation and parameter update:

[0334] The loss is propagated from the output layer back to all layers using the backpropagation algorithm, and the gradient of the parameters in each layer is calculated. Based on these gradients, all learnable parameters in the model are updated, including the parameters of the function modules, the weights of the controller, and the weights of the output layer, thus optimizing model performance.

[0335] Example 3

[0336] This invention provides an application of a KAN adaptive function combination method based on multi-scale sparse branches in the field of computer vision, using the method described in any of the preceding claims for image classification, object detection, or image segmentation tasks.

[0337] In image classification tasks, the input image is first preprocessed, including resizing and normalization. Then, multi-scale feature extraction methods are used, employing different function modules (such as RBF, B-spline, and wavelet transform) to extract multi-level features from the image. A hierarchical attention mechanism calculates the selection weight of each function module based on image characteristics, dynamically selecting the most relevant function module for computation. Finally, feature fusion and fully connected operations are used to output the image classification result. Experiments show that this method achieves superior classification accuracy compared to traditional methods on standard datasets such as ImageNet, while significantly reducing computational cost.

[0338] In object detection tasks, this method processes target features at different scales through multi-scale sparse branches. For large targets, a function module capable of extracting global features is selected; for small targets, a function module capable of extracting detailed features is selected. Through a dynamic selection mechanism, the model can adaptively adjust the feature extraction strategy to accurately identify and locate target objects in images. Experiments on standard datasets such as COCO show that this method significantly improves detection speed while maintaining high detection accuracy.

[0339] In image segmentation tasks, this method extracts pixel-level features through a multi-scale function module. It optimizes the computation path using a sparse selection mechanism, performing detailed calculations only on important image regions to reduce redundant computation. Ultimately, it achieves accurate classification of each pixel in the image. Experiments on standard datasets such as Cityscapes demonstrate that this method outperforms traditional methods in both segmentation accuracy and computational efficiency.

[0340] Example 4

[0341] This invention provides an application of a KAN adaptive function combination method based on multi-scale sparse branches in the field of natural language processing, using the method described in any of the preceding embodiments for text classification tasks.

[0342] In the text classification task of natural language processing, the input text is first preprocessed, including word segmentation and vectorization. Then, multi-scale feature extraction methods are used to extract word-level, phrase-level, and sentence-level features from the text. A hierarchical attention mechanism calculates the selection weight of each function module based on the characteristics of the text content, dynamically selecting the most relevant function module for computation. Finally, feature fusion and fully connected operations are used to output the text classification result. Experiments on multiple standard text classification datasets show that this method outperforms traditional methods in both classification accuracy and computational efficiency.

[0343] Example 5

[0344] This invention provides an application of a KAN adaptive function combination method based on multi-scale sparse branches in the field of industrial control, using the method described in any of the preceding embodiments to perform equipment fault prediction, production process optimization, or quality inspection tasks.

[0345] In equipment fault prediction tasks within the industrial control field, this method analyzes equipment operating data (such as vibration signals, temperature, and pressure) and extracts fault features using multi-scale sparse branches. For periodic faults, function modules capable of extracting periodic features are selected; for sudden faults, function modules capable of extracting abrupt change features are selected. Through a dynamic selection mechanism, the model can predict the type and timing of potential equipment faults in advance. Experiments on real-world industrial datasets demonstrate that this method outperforms traditional methods in both fault prediction accuracy and timeliness.

[0346] In production process optimization tasks, this method analyzes production parameters (such as temperature, pressure, and flow rate) and quality data, and optimizes the feature extraction process using a dynamic selection mechanism. Through multi-scale feature extraction, it considers both short-term production fluctuations and long-term production trends, providing accurate production process optimization suggestions. Experiments in a real-world production environment demonstrate that this method can significantly improve production efficiency and product quality.

[0347] In quality inspection tasks, this method extracts product quality features through a multi-scale function module. It utilizes a sparse selection mechanism to reduce redundant computation, performing detailed analysis only on key quality features, thus achieving rapid and accurate product quality inspection. Experiments on real-world quality inspection datasets demonstrate that this method outperforms traditional methods in both accuracy and speed.

[0348] This invention employs a dynamic selection mechanism to pre-select function modules during the forward propagation stage, effectively avoiding computational redundancy issues in traditional KAN models and significantly improving inference speed and computational efficiency. This method exhibits good input adaptability, dynamically adjusting the computation path based on different features, reducing energy consumption and enhancing model interpretability. It is suitable for applications requiring efficient real-time responses, such as computer vision, natural language processing, and industrial control.

[0349] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the specific implementation of the present invention. Any modifications or equivalent substitutions that do not depart from the spirit and scope of the present invention should be covered within the protection scope of the claims of the present invention.

Claims

1. A KAN adaptive function combination method based on multi-scale sparse branches, used to construct efficient and highly interpretable neural network models for image classification, image segmentation, or text classification tasks, characterized in that... Includes the following steps: Step S1: Start the dynamic selection mechanism and initialize the neural network model parameters. The neural network model parameters include input and output layer parameters, dynamic selection controller parameters, and learnable function module parameters. The learnable function module includes a feature extraction function. Step S2: Standardize the input data, obtain multi-level feature representation through multi-scale feature extraction method, and then dynamically calculate the selection weight of each learnable function module through hierarchical attention mechanism. The input data includes multi-source sensor data. Step S3: Based on the selection weight filtering function module, only the function modules with selection weights greater than the preset activation threshold are activated to form an optimized sparse computation path. Step S4: Drive the activated function modules to perform feature calculations, generate feature outputs for each module, and use a weighted fusion strategy to integrate the feature outputs of all activated modules into a unified feature representation. Step S5: In the deep network structure, steps S3 and S4 are executed iteratively. Through layer-by-layer function module selection, feature calculation and feature stacking, a multi-level, high-dimensional deep feature representation is constructed, and finally a unified final fusion feature is formed. Step S6: Generate task prediction results based on the final fused features, calculate the loss value between the prediction results and the real labels, and backpropagate the loss value back to each layer of the neural network model through the backpropagation algorithm to update all learnable parameters of the neural network model and realize dynamic optimization of the KAN adaptive function combination.

2. The KAN adaptive function combination method based on multi-scale sparse branches according to claim 1, characterized in that, Step S1 includes the following steps: Step S11: Initialize the input and output layer parameters, establish the basic data path, and set the input-to-output weight matrix using the Kaiming Uniform initialization method; Step S12: Construct a dynamic selection controller, which consists of multiple fully connected layers and performs nonlinear transformation through ReLU or SiLU activation functions to generate an initial selection score for each function module; Step S13: Configure the parameters of the multi-scale learnable function module, which include the grid point parameters of the B-spline function, the center and width parameters of the radial basis function RBF, the scale and displacement parameters of the wavelet function, and the parameters of the Chebyshev polynomial. Step S14: Set function weights and bias parameters, assign corresponding weight matrices and bias terms to the output features of each function module, and control the contribution of each module in the feature fusion process. Step S15: Initialize Dropout layer parameters and configure regularization mechanism to randomly drop some neuron connections during training to prevent model overfitting and improve generalization ability.

3. The KAN adaptive function combination method based on multi-scale sparse branches according to claim 1, characterized in that, Step S2 includes the following steps: Step S21: Perform input data standardization processing. By calculating the mean μ and standard deviation σ of the input data, the original data is converted into standardized data with zero mean and unit variance to ensure numerical stability. Step S22: Perform multi-scale feature extraction. Input the standardized data in parallel into multiple function modules of different scales. The multiple function modules of different scales include radial basis function (RBF), B-spline function, wavelet transform function and Chebyshev polynomial function. Each module extracts feature information from different dimensions based on its function characteristics. Step S23: Implement the hierarchical attention mechanism calculation. Perform nonlinear transformation on the features at each scale through a multilayer perceptron (MLP) to generate the original selection score. Then, standardize the score into selection weights through a softmax function with a temperature coefficient. Step S24: Perform computational path optimization decision-making. Based on the calculated selection weights and combined with the preset activation threshold, determine the activation status of each function module.

4. The KAN adaptive function combination method based on multi-scale sparse branches according to claim 1, characterized in that, Step S3 includes the following steps: Step S31: Receive selection weight input and obtain the selection weights of each function module. ; Step S32: Perform threshold comparison and judgment, comparing the selection weight of each function module with the preset activation threshold. When comparing, The module is marked as active at this time. Step S33: Perform sparse activation operation, allocate computing resources only to function modules marked as active, and form an optimized computing path; Step S34: Generate an active module set and record the identifiers of all activated function modules.

5. The KAN adaptive function combination method based on multi-scale sparse branches according to claim 1, characterized in that, Step S4 includes the following steps: Step S41: Execute activation module feature generation. Input standardized data for each activated function module, perform the corresponding function transformation, and generate feature output. Step S42: Collect the output features of each module, and store and organize the feature outputs generated by the activated module in a unified format; Step S43: Implement weighted fusion strategy selection, select a fusion method according to preset fusion rules, the fusion rules are determined based on task type and data feature dimensions; Step S44: Generate a unified feature representation by integrating the outputs of each module into a single feature vector through a selected weighted fusion strategy.

6. The KAN adaptive function combination method based on multi-scale sparse branches according to claim 1, characterized in that, Step S5 includes the following steps: Step S51: Set the network depth parameters and determine the number of network layers L to be executed. Each layer includes a complete function selection and feature fusion process. Step S52: Perform calculations iteratively layer by layer. For each layer... l , l =1,2,...,L, repeat sparse branch selection and feature generation and fusion; Step S53: Perform feature stacking operation, stacking the output features generated by each layer in the depth dimension to construct a hierarchical feature representation; Step S54: Generate the final fused features by transforming and compressing the stacked features through a fully connected layer to generate the network's final output features.

7. The KAN adaptive function combination method based on multi-scale sparse branches according to any one of claims 1-6, characterized in that, Step S6 includes the following steps: Step S61: Based on the obtained final fusion features, generate predicted values ​​through the output layer corresponding to the task type to obtain the final prediction result; Step S62: Calculate the loss function value, compare the final prediction result with the true label, and quantify the model prediction error; Step S63: Perform backpropagation calculation, calculate the gradient layer by layer from the output layer to the input layer, and determine the contribution of each parameter to the loss. Step S64: Update the neural network model parameters. Based on the calculated gradient, use an optimization algorithm to adjust all learnable parameters and minimize the loss function.

8. A KAN adaptive function combination device based on multi-scale sparse branches, used to implement the KAN adaptive function combination method based on multi-scale sparse branches as described in any one of claims 1-7, characterized in that, include: An initialization module is used to start the dynamic selection mechanism and initialize the neural network model parameters, which include input and output layer parameters, dynamic selection controller parameters, and learnable function module parameters, including feature extraction functions. The weight calculation module is used to standardize the input data, obtain multi-level feature representation through multi-scale feature extraction method, and then dynamically calculate the selection weight of each learnable function module through hierarchical attention mechanism. The input data includes multi-source sensor data. The sparse selection module is used to filter the function modules based on the selection weights, and only activate the function modules whose selection weights are greater than a preset activation threshold, thereby forming an optimized sparse computation path. The feature fusion module is used to drive the activated function modules to perform feature calculations, generate feature outputs of each module, and integrate the feature outputs of all activated modules into a unified feature representation using a weighted fusion strategy. The deep feature construction module is used to iteratively perform sparse selection and feature fusion operations in the deep network structure. Through layer-by-layer function module selection, feature calculation and feature stacking, it constructs multi-level, high-dimensional deep feature representations and finally forms a unified final fused feature. The function optimization module is used to generate task prediction results based on the final fused features, calculate the loss value between the prediction results and the real labels, backpropagate the loss value to each layer of the neural network model through the backpropagation algorithm, update all learnable parameters of the neural network model, and realize dynamic optimization of the KAN adaptive function combination.

Images