Residual adaptive network architecture design method and device based on MultiKAN

Abstract

The invention discloses a residual adaptive network architecture design method and device based on a MultiKAN, and the method comprises the steps: building a residual MultiKAN architecture through adding jump connection to each KAN layer of the MultiKAN architecture; and on the basis of the residual MultiKAN architecture, constructing a loss-driven adaptive grid to obtain a residual adaptive network architecture. According to the method, the jump connection is added to each KAN layer of the MultiKAN architecture to establish an RMultiKAN trunk, so that the ill-conditioned gradient behavior is reduced, and the training stability is ensured; by constructing a loss-driven adaptive grid, the grid is intelligently refined in an area with high errors by using loss information, so that the accuracy is improved under the condition that the calculation cost is controllable; in addition, the final loss calculation is always carried out on an original and unmodified data set, so that the robustness of the architecture is improved.

Description

Technical Field

[0001] This invention relates to the fields of artificial intelligence and neural network technology, and in particular to a residual adaptive network architecture design method and apparatus based on MultKAN. Background Technology

[0002] Kolmogorov-Arnold Network (KAN) and its multiplicative variant (MultKAN) introduce a paradigm shift from traditional MLPs by replacing the fixed activation function in a traditional Multi-Layer Perception (MLP) with a learnable univariate spline function. The flexibility of this architecture facilitates its rapid application in various fields, including time series analysis, computer vision, and medical imaging. However, existing technologies suffer from two inherent limitations:

[0003] Training instability: The coefficients of spline functions are unconstrained during training, which may lead to ill-shaped activation functions, resulting in gradient explosion or vanishing problems and disrupting the training process.

[0004] Suboptimal resource allocation: Existing grid-adaptive strategies are entirely driven by the density of the input data, which leads to inefficient allocation of computational resources and an inability to achieve higher accuracy in high-error regions, resulting in a suboptimal trade-off between accuracy and computational cost. Summary of the Invention

[0005] This invention provides a residual adaptive network architecture design method and apparatus based on MultKAN, which solves the defects of unstable training and low grid efficiency in the existing neural network architecture, and realizes a highly stable and efficient neural network architecture with intelligent resource allocation.

[0006] This invention provides a residual adaptive network architecture design method based on MultKAN, comprising:

[0007] A residual MultKAN architecture is established by adding skip connections to each KAN layer of the MultKAN architecture;

[0008] Based on the residual MultKAN architecture, a loss-driven adaptive mesh is constructed to obtain the residual adaptive network architecture.

[0009] According to the residual adaptive network architecture design method based on MultKAN provided by the present invention, the establishment of the residual MultKAN architecture includes:

[0010] The residual multiplicative layers are constructed by combining multiple residual multiplicative layers, wherein each residual multiplicative layer is a combination of a residual KAN layer and a multiplicative layer in the residual MultKAN architecture.

[0011] According to the residual adaptive network architecture design method based on MultKAN provided by the present invention, the output of the residual KAN layer Determined by the following formula:

[0012]

[0013] in, Indicates the sequence number of the residual KAN layer; The activation function matrix of a standard KAN layer; and These are the weight matrix and bias vector of the learnable linear transformation, respectively.

[0014] According to the residual adaptive network architecture design method based on MultKAN provided by the present invention, the construction of the loss-driven adaptive mesh specifically includes the following steps:

[0015] The input space of the residual MultKAN architecture is divided into multiple subdomains;

[0016] Calculate the loss distribution over each of the subdomains;

[0017] The training dataset in the input space is resampled based on the loss distribution to construct a resampled dataset;

[0018] Based on the input distribution of the resampled dataset in each layer of the forward propagation, an adaptive grid is generated for each learnable activation function.

[0019] According to the residual adaptive network architecture design method based on MultKAN provided by the present invention, the method for generating an adaptive grid for each learnable activation function further includes:

[0020] Adaptive computational grid It is constructed through iteration using the following formula:

[0021] ;

[0022] Among them, the dynamic mixing coefficient Update using the following formula:

[0023]

[0024] Adaptive Mesh Update using the following formula:

[0025] ;

[0026] Uniform grid Update using the following formula:

[0027] ;

[0028] in, Indicates iteration Total loss at that time; Indicates the threshold for loss variation; Indicates adaptive step size; Indicates the first The sample until the 1st The layer's output vector; Specifically refers to the first The output vector of the layer Dimensional components; G represents a set of quantiles. As the index i changes from 0 to G, it takes values ​​from the 0-quantile (minimum) to the 1-quantile (maximum); G is the grid size parameter, which represents the number of intervals divided for each input dimension. This represents a resampled dataset; , They represent the first The first layer of the input vector The minimum and maximum values ​​of the dimensional components.

[0029] According to the residual adaptive network architecture design method based on MultKAN provided by the present invention, after the adaptive mesh is constructed, the method further includes:

[0030] During the backpropagation process of the residual adaptive network architecture, the coefficients of the B-spline basis functions are updated on the adaptive grid through gradient descent.

[0031] According to the residual adaptive network architecture design method based on MultKAN provided by the present invention, the resampling of the training dataset of the input space based on the loss distribution specifically includes:

[0032] Will Each of the input dimensions is divided into These intervals form Subdomains; the training dataset Classified as Disjoint subsets each of the subsets It includes data points that fall into a unique subdomain;

[0033] Resampling probability for each subdomain Determined by the following formula:

[0034] ;

[0035] Resampled dataset It is constructed by drawing a corresponding proportion of samples from each of the aforementioned subsets:

[0036] ;

[0037] in, This represents the loss for each of the subdomains. This represents the total loss of the residual adaptive network architecture; Indicates from the subset Extraction One sample.

[0038] The present invention also provides a residual adaptive network architecture design device based on MultKAN, comprising:

[0039] Add a module to establish a residual MultKAN architecture by adding skip connections to each KAN layer of the MultKAN architecture;

[0040] The building module is used to construct a loss-driven adaptive mesh based on the residual MultKAN architecture, thereby obtaining a residual adaptive network architecture.

[0041] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the residual adaptive network architecture design method based on MultKAN as described above.

[0042] The present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the residual adaptive network architecture design method based on MultKAN as described above.

[0043] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the residual adaptive network architecture design method based on MultKAN as described above.

[0044] Compared with the prior art, the beneficial effects of the present invention are:

[0045] This invention mitigates ill-conditioned gradient behavior and ensures training stability by adding skip connections to each KAN layer of the MultKAN architecture to establish an RMultKAN backbone. By constructing a loss-driven adaptive grid, the grid is intelligently refined in regions with high error using loss information, thereby improving accuracy while keeping computational costs under control. In addition, the final loss calculation is always performed on the original, unmodified dataset, increasing the robustness of the architecture. Attached Figure Description

[0046] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0047] Figure 1 This is a schematic diagram of the ill-conditioned learning dynamics in the spline activation function according to an embodiment of the present invention, where (a) and (b) are respectively ill-conditioned activation functions. Unstable gradient .

[0048] Figure 2 This is a structural comparison diagram of MultKAN and RMultKAN according to an embodiment of the present invention, wherein (a) and (b) are the MultKAN architecture and the RMultKAN architecture, respectively.

[0049] Figure 3 This is a flowchart illustrating the residual adaptive network architecture design method based on MultKAN according to an embodiment of the present invention.

[0050] Figure 4 This is a schematic diagram of a loss-driven adaptive mesh update process according to an embodiment of the present invention, wherein (a) and (b) are the initial uniform mesh and the updated adaptive mesh, respectively.

[0051] Figure 5 This is the multiplication layer according to an embodiment of the present invention. A diagram illustrating the recursive decomposition.

[0052] Figure 6 This is a comparative diagram of training stability analysis according to an embodiment of the present invention, wherein (a) and (b) are comparisons of the gradients of the B-spline coefficients of the first and second layers in MultKAN and RMultKAN, respectively.

[0053] Figure 7This is a schematic diagram comparing the convergence analysis according to an embodiment of the present invention, wherein (a) and (b) are the parameter norms of the B spline coefficients of MultKAN and RMultKAN, respectively; and (c) and (d) are the parameter norms of the gradients of MultKAN and RMultKAN, respectively.

[0054] Figure 8 This is a schematic diagram comparing reference values ​​and network approximation values ​​according to an embodiment of the present invention, wherein (a) RMultKAN-I (the method described herein), (b) RMultKAN-II (the method described herein), (c) MultKAN-I, (d) MultKAN-II, and (e) MLP.

[0055] Figure 9 This is a schematic diagram comparing the reference spectrum and the network approximation spectrum according to an embodiment of the present invention, wherein (a) RMultKAN-I (the method described in this paper), (b) RMultKAN-II (the method described in this paper), (c) MultKAN-I, (d) MultKAN-II, and (e) MLP.

[0056] Figure 10 The diagram shows a comparison of five methods according to embodiments of the present invention, where (a) and (b) are comparisons of relative L2 error trajectories and loss function trajectories, respectively.

[0057] Figure 11 This is a schematic diagram of the structure of a residual adaptive network architecture design device based on MultKAN according to the present invention.

[0058] Figure 12 This is a schematic diagram of the structure of an electronic device according to an embodiment of the present invention. Detailed Implementation

[0059] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0060] For existing technologies, the current KAN architecture faces the following challenges:

[0061] 1) Taming the Ill-conditioned Learning Dynamics of Activation Functions: KAN is built upon the powerful Kolmogorov-Arnold Representation Theorem (KART). These theoretical univariate functions possess "ill-conditioned" properties; they are not guaranteed to be smooth and may even be fractal. KAN overcomes this obstacle by implementing these abstract univariate functions as learnable, B-spline-based activation functions. However, the success of this approach relies on an implicit assumption: that a sufficiently large network size (width or depth) is sufficient to ensure the emergence of good, smooth function representations. The key issue is that while using B-splines guarantees the mathematical smoothness of the activation function, it does not prevent unconstrained learning of its coefficients (learnable parameters) from producing ill-conditioned shapes. As shown in (a), a set of inherent smooth B-spline basis functions and a fixed, well-behaved basis function They can be combined into an irregular and poorly behaving composite activation function. This, in turn, disrupts the training process by generating unstable gradients. For example... As shown in (b), the resulting gradient function Regions with maximum gradient values ​​(gradient explosion) and gradient values ​​close to zero (gradient vanishing).

[0062] 2) The Inefficiency of Static Mesh and the Pitfalls of Data-Centered Adaptation: A fundamental efficiency challenge exists in the B-spline meshes of existing KANs. Achieving high accuracy requires a globally dense mesh, which is computationally infeasible, while a coarse mesh, though efficient, sacrifices the ability to capture complex, localized features. Furthermore, in machine learning, a common approach to introducing adaptation is to modify the training process itself, for example, by reweighting or resampling data points to focus on high-error regions. However, this data-centric adaptation, by altering the data distribution upon which the loss is calculated, can distort the loss landscape and potentially mislead the optimization process, as the model no longer learns from a true, unbiased data distribution.

[0063] To overcome the shortcomings of existing technologies, this invention improves the MultKAN architecture by decoupling the unstable dynamics of the main gradient path and the learnable activation function, thereby directly forcing training stability. By introducing a performance-driven adaptive grid update mechanism, the grid is intelligently refined in regions of high error, thus improving accuracy without incurring significant computational costs. Simultaneously, the final loss calculation can always be performed on the original, unmodified dataset. In this way, the mechanism resolves the accuracy-cost trade-off without creating a biased training objective, representing a more robust and principled form of adaptation.

[0064] Before delving into the specific implementation of the embodiments of the present invention, some technical contents related to the various embodiments of the present invention will be introduced first, so as to better understand their working principles and innovations.

[0065] like Figure 2 As shown in (a), the multiplicative Kolmogorov-Arnold network MultKAN differs architecturally from traditional MLPs. MLPs use a fixed activation function for each node, while a single MultKAN layer... Then, a standard KAN layer activation function matrix is ​​used. With an additional multiplication layer Combined, its mathematical expression is: Among them, the multiplication layer The output nodes are divided into two categories: addition nodes and multiplication nodes. The latter perform element-wise multiplication on selected child nodes, while the former are directly copied from the corresponding child nodes of the previous layer. The standard KAN layer... Each edge in the graph corresponds to a learnable univariate activation function. Its output node (child node) As input to the multiplication layer. For those with Each input dimension and Standard KAN layer activation function matrix in each output dimension It can be represented as:

[0066]

[0067] in This represents a univariate activation function. It is defined as a basis function. and B-spline function Weighted combination:

[0068]

[0069] in and These are the weights of the balanced basis functions and the B-spline functions, respectively. Typically, ,and Due to its polynomial order and grid size Characterization. It is a trainable parameter used to control the overall magnitude of the activation function.

[0070] Therefore, the complete ( A MultKAN layer can be represented as a composite of multiple layers:

[0071]

[0072] By stacking these layers, MultKAN can handle complex functions involving addition and multiplication interactions. This structure enables MultKAN to address problems that require modeling multiplicative interactions between variables, thereby enhancing the interpretability and capacity of KANs without introducing additional parameters.

[0073] The following is combined Figures 2-3 The residual adaptive network architecture design method based on MultKAN disclosed in the embodiments of the present invention will be described in detail.

[0074] like Figure 3 As shown, the method includes the following steps:

[0075] Step 300: Establish the residual MultKAN architecture by adding skip connections to each KAN layer of the MultKAN architecture.

[0076] Specifically, in step 300 above, after adding skip connections to each KAN layer of the MultKAN architecture, as follows: Figure 2 As shown in (b), the residual multiplicative Kolmogorov-Arnold network (RMultKAN), i.e., the residual MultKAN architecture, is an architecture that enhances MultKAN by introducing skip connections. By comparison... (a) and (b) It can be seen that RMultKAN introduces a direct linear path (e.g., X0 to Z0, X1 to Z1) within each network layer.

[0077] RMultKAN is achieved by stacking multiple residual multiplicative layers. Constructed. Each layer is a residual KAN layer. and multiplication layer The composite:

[0078]

[0079] Key components The standard KAN layer is improved by adding a parallel linear path. It consists of a standard univariate activation matrix (structure and formula (1)). (same) and a learnable linear transformation Composition. The weight matrix of this transformation is... The bias vector is It can adapt to inputs and outputs of different dimensions. Therefore, the output of the residual KAN layer... Defined as their sum:

[0080]

[0081] Where, if the input dimension and output dimensions same, It is an identity matrix. It is a zero vector. This design makes This allows for a focus on learning the complex nonlinear residuals in the mapping, significantly reducing the optimization burden. The output of this layer... It was then passed up to the multiplication layer. To produce a complete layer Final output:

[0082]

[0083] By stacking For such layers, the complete RMultKAN can be represented as a deep composite of these residual multiplicative blocks:

[0084]

[0085] The architecture of a network is represented by its width configuration, for example... ,in and These represent the number of addition and multiplication nodes, respectively. If the... If the layer outputs only have addition nodes and no multiplication nodes, then the RMultKAN architecture can be simplified as follows: .

[0086] This design creates a powerful "division of labor": a learnable univariate activation function is responsible for learning the complex nonlinear residuals of the function, while the linear part is efficiently passed through skip connections. This approach improves the training stability of the model, thereby accelerating convergence.

[0087] Step 301: Based on the residual MultKAN architecture, construct a loss-driven adaptive mesh to obtain the residual adaptive network architecture.

[0088] Specifically, higher grid resolution can capture finer function details but increases the number of parameters and computational overhead, while coarser grids are more efficient but may sacrifice accuracy in subdomains exhibiting complex behavior. To achieve a balance between accuracy and resource overhead, after obtaining RMultKAN through skip connections, a loss-driven design is implemented for the grid, allowing it to dynamically adjust according to the behavior of the subdomain, thus achieving an adaptive grid and resulting in the residual adaptive network architecture, namely Adaptive Grid-based Residual MultKAN (ARMultKAN).

[0089] For example, such as Figure 4 As shown, activation function (defined in formula (2)) is a set of B-spline basis functions The shape, support domain, and position of these basis functions are directly determined by the adaptive mesh. Furthermore, in the ARMultKAN architecture, this activation function does not directly learn the target function. Instead, it approximates the residual components. Therefore, the network expressive power determined by the density of B-spline basis functions must be focused on In subdomains exhibiting significant changes, i.e., subdomains with large losses, such as regions with high-frequency oscillations or steep gradients, this design provides a smarter meshing strategy than using a simple uniform spacing approach.

[0090] like Figure 4 As shown in (a), an initial, simple, uniform grid may inefficiently distribute basis functions and fail to accurately model residuals in challenging subdomains. However, by dynamically adjusting the grid based on loss, making it denser in high-error regions, the grid for high-error subdomains is refined, as shown in [example missing]. Figure 4 In (b), 𝒟6 can effectively reconfigure the basis functions to match the complexity of the residual functions.

[0091] Through steps 300-301 above, this invention mitigates ill-conditioned gradient behavior and ensures training stability by adding skip connections to each KAN layer of the MultKAN architecture to establish an RMultKAN architecture; by constructing a loss-driven adaptive grid, the grid is intelligently refined in regions with high error using loss information, thereby improving accuracy while keeping computational costs under control; in addition, the final loss calculation is always performed on the original, unmodified dataset, increasing the robustness of the architecture.

[0092] Optionally, the construction of the loss-driven adaptive mesh specifically includes the following steps:

[0093] Step A1: Divide the input space of the residual MultKAN architecture into multiple subdomains;

[0094] Step A2: Calculate the loss distribution over each of the subdomains;

[0095] Step A3: Resample the training dataset of the input space based on the loss distribution to construct a resampled dataset;

[0096] Step A4: Generate an adaptive grid for each learnable activation function based on the input distribution of the resampled dataset in each layer of the forward propagation.

[0097] Specifically, the core of the architecture design method provided in this embodiment of the invention is to enable adaptive meshes. The approximation error of the directly perceived model.

[0098] In step A1, to locally evaluate the model's performance, the residual MultKAN architecture is first evaluated. The input space is partitioned. This partitioning is based on the grid size parameter. It defines the number of intervals into which each dimension is divided. By... Each of the input dimensions is divided into The intervals formed a total of [number] intervals. A superrectangular subdomain. Therefore, the training dataset... Classified as Disjoint subsets 𝒟1, 𝒟2, ..., each subset It contains data points that fall into a unique subdomain.

[0099] In step A2, after dividing the domains into subdomains, the loss distribution across each subdomain is calculated. For example, such as... Figure 3 (a) shows the one-dimensional case (where) =1, the subfield is a simple interval 𝒟1,…, ), some subdomains (such as This subdomain exhibits a much larger approximation error than the other subdomains. The loss for each subdomain is calculated. Total model loss :

[0100]

[0101]

[0102] in These are the true values. The resampling probability for each of the subdomains. It is determined by its relative contribution to the total error:

[0103]

[0104] like Figure 4 As shown by the central arrow, this loss-driven resampling step ensures that more samples are drawn from the high-error subdomain.

[0105] In step A3 above, the training dataset in the RMultKAN input space is resampled using the loss distribution obtained in step A2 to construct a resampled dataset. For example, the resampled dataset... It is constructed by drawing a corresponding proportion of samples from each subset:

[0106]

[0107] in, Indicates from subset Extraction This is a resampled dataset. It now includes a higher concentration of data points from areas where the architecture model performed poorly.

[0108] In step A4 above, an adaptive grid is generated for each learnable activation function based on the input distribution of the resampled dataset obtained in step A3 across each layer during the forward propagation. For example, the adaptive grid is constructed based on the specific data distribution actually received by each B-spline activation function during the forward propagation. This ensures that the grid for each layer is customized for its unique, transformed input values.

[0109] This process is performed sequentially within the network architecture. To build any given layer... The grid, first by resampling the entire dataset The spread is through the front The layers determine their true input distribution. Let... yes One of the samples. This sample to the... The input of the layer is That is, the network until the 1st The layer's output vector. This set of propagated input values ​​is computed for the entire dataset. Then, the mesh can be constructed. For the... Layer An activation function associated with each input, whose adaptive grid By its specific input value The nodes are placed on the quantiles to construct the structure.

[0110]

[0111] in This represents a set of quantiles. As the index i changes from 0 to G, it takes values ​​from the 0-quantile (minimum) to the 1-quantile (maximum). Specifically refers to the first The first layer output vector Dimensional components.

[0112] Initial resampled dataset The increased proportion of medium-to-high error samples affects the data distribution of each subsequent layer. By constructing the grid based on the layer-specific input distribution after propagation at each layer, it is ensured that the increased grid density is accurately allocated to the transformed data regions corresponding to the initially challenging input. This makes adaptation effective across the entire depth of the network.

[0113] Optionally, by generating an adaptive grid for each learnable activation function, a dynamic grid blend with adaptive blending coefficients can also be constructed, thereby integrating the Adaptive Grid Update (AGU) mechanism into... In the RMultKAN architecture shown in (b).

[0114] Final adaptive computational grid It is constructed through iteration using the following formula.

[0115]

[0116] This dynamic blending mechanism will be guided by a loss-driven adaptive mesh (e.g. Figure 4 (b) ) and standard uniform grid (e.g. Figure 4 (a) Combined, by adjusting the dynamic mixing coefficient ,control and The balance between these two factors. This coefficient is not static; it can evolve according to the progress of training, thus enabling a strategic shift from exploration to exploitation. In the early stages of training, when the loss... When the change is significant, This will increase. This prioritizes a uniform grid, which facilitates a stable and broad exploration of the functional landscape. Conversely, as training progresses and the loss stabilizes, it indicates that the architectural model has captured the overall structure, therefore... This will reduce [the impact of the error]. This shifts the focus to the adaptive grid, concentrating computational resources on refining the details of challenging, high-error subdomains.

[0117] This adaptive behavior is achieved through the following update rules:

[0118]

[0119] in Indicates iteration Total loss at that time A threshold for loss variation was defined. The adaptive step size was controlled.

[0120] In this architecture, the two component meshes play different but complementary roles. For example... Figure 4 As shown, This provides a stable, uniformly spaced basis within the input domain of each activation function. For the th The first layer Given one input, the precise operating range is first determined by propagating the entire resampled dataset through the preceding layers to identify the actual minimum and maximum values ​​encountered:

[0121]

[0122] in Indicates the first The input sample up to the nth... The output vector of the layer, and Indicates its first Each component. Using these empirically determined boundaries, a uniform grid is formed. It is constructed using equidistant nodes:

[0123]

[0124] This data-driven approach ensures that the uniform grid for each activation function perfectly covers its actual operating range, thus establishing a robust baseline resolution. In contrast, loss-driven adaptive grids... Strategically concentrate grid points in subdomains where the model exhibits higher approximation errors. With this, such as... Figure 4 As shown in (b), grid points cluster in previously identified high-loss regions to provide enhanced local resolution.

[0125] The hybrid method provided in this embodiment of the invention strategically utilizes the complementary advantages of two mesh types: the stable and comprehensive coverage provided by a uniform structure, and the targeted accuracy provided by an adaptive configuration.

[0126] Optionally, after the adaptive mesh is constructed, the method further includes:

[0127] During the backpropagation process of the residual adaptive network architecture, the coefficients of the B-spline basis functions are updated on the adaptive grid through gradient descent.

[0128] Specifically, after the adaptive grid is constructed, the B-spline basis functions are reconstructed on the new adaptive grid, and their coefficients are optimized by gradient descent.

[0129] It should be noted that the complete loss-driven AGU mechanism is integrated into the training process and runs iteratively, as described in Algorithm 1. The training process consists of three stages:

[0130] 1) Initialization: The model starts with a uniform grid. ).

[0131] 2) Adaptive Update: After the warm-up period, the AGU process is periodically activated. During this phase, the model performs its core adaptive update: as detailed in Algorithm 2, it uses loss-driven resampling to build... Updated based on the total loss trend It is important to note that to ensure the final blended mesh is correctly aligned, It is also rebuilt at each step. This is because, as the network weights are optimized, the range of the operational inputs of each activation function may change. It must be updated to cover this new scope. Finally, from the newly created... And the newly rebuilt Generate a new hybrid adaptive computational grid .

[0132] 3) Optimization phase: B-spline basis functions In the new adaptive computation grid The coefficients are reconstructed and optimized using gradient descent.

[0133] This iterative loop allows ARMultKAN to progressively refine its internal mesh structure based on its approximate performance.

[0134]

[0135]

[0136] This invention employs a loss-driven mechanism that uses the loss function as a direct signal to refine the computational grid, thereby directly reshaping the underlying B-spline basis functions. Since the network's learnable activation function is a linear combination of these basis functions, this process constitutes true architectural adaptation, where the model's functional form evolves during training. Crucially, because the architecture is adaptive, the loss function used for parameter optimization can always be computed on the original, unbiased training dataset.

[0137] The following provides a theoretical completeness analysis of the RMultKAN architecture, which is the residual adaptive network architecture design.

[0138] (1) Approximation ability

[0139] The ability of neural networks to approximate arbitrary functions is the foundation of their theoretical basis and the key to their practical success. This section explores the approximation capability of RMultKAN, demonstrates its general approximation properties, and establishes a quantization error bound, which lays the theoretical foundation for the aforementioned adaptive network architecture design method.

[0140] The results of Theorem 1 establish two key aspects of RMultKAN's capabilities: 1) It possesses universal approximation capabilities, enabling it to represent any continuous multivariate function; 2) When using B-spline approximation, it provides precise quantitative limits to the approximation error, which decreases as the grid size (number of grid intervals) increases. These theoretical guarantees not only underpin the practical effectiveness of RMultKAN in modeling complex real-world relationships but also inspire the development of adaptive grid strategies.

[0141] Theorem 1 (Approximation Theorem): Consider a - layer RMultKAN. For any and any multivariable continuous function The following conclusion holds true:

[0142] 1) In the RMultKAN structure, there exists an ideal single-variable function. , making

[0143]

[0144] 2) Assume an ideal function yes It is continuously differentiable for a given grid size. , can be used The B-order spline construction approximation is given. Let... For this approximate network, each ideal activation function... All are represented by their B-splines Approaching. Therefore, for any condition satisfying... integers There exists a constant (depends on) ), making

[0145]

[0146] in, - Norm is used to measure the highest to The magnitude of the first derivative:

[0147]

[0148] Proof: 1) According to the Kolmogorov-Arnold representation theorem, any definition in a finite field Multivariate continuous functions on All of these can be represented as a finite combination of single-variable continuous functions using addition. This means that... Multiplication in the equation can be rewritten as a compound addition of single-variable functions.

[0149] However, if these multiplication operations are explicitly preserved, rather than represented as additive composition, single-variable functions can be introduced. To capture nonlinear terms in univariate functions. Simultaneously, coefficients... and Used for building The linear terms in the equation, while multiplication is retained in the equation. Therefore, It can be represented as:

[0150] because Therefore, the statement in 1) is true.

[0151] 2) Based on classical B-spline theory and continuous functions The fact that a function can be uniformly bounded over a bounded domain indicates the existence of B-spline functions with finite grids. , such that for any The following equation holds true:

[0152] where constant With grid size Irrelevant. By fixing these B-spline approximations, we can deduce:

[0153] ,

[0154] where constant With grid size Irrelevant.

[0155] Therefore, the layer-by-layer residuals defined by the following formula :

[0156]

[0157] satisfy

[0158]

[0159] where constant and Irrelevant. Note:

[0160]

[0161] So

[0162]

[0163] Finally, let and Therefore, it can be deduced that the assertion in 2) is true. Q.E.D.

[0164] (2) Multiplicative layer decomposition

[0165] A major challenge in studying the RMultKAN training process is calculating the gradients of the multiplicative layers. Performing element-wise multiplication on selected child nodes results in complex gradient expressions that are difficult to derive using traditional methods due to the inherent complexity of multivariate multiplication operations.

[0166] The method provided in this invention solves this challenge through mathematical reconstruction. Multiplication in MultKAN is decomposed into a series of addition operations combined with easily differentiable univariate spline functions. Existing MultKAN acknowledges that multiplication can theoretically be represented by combinations of addition and univariate functions (typically logarithmic and exponential functions). However, their analysis is limited to the simple case of multiplying only two child nodes. This is extended by establishing a comprehensive mathematical framework that generalizes to arbitrary complexity, handles multiple multiplication nodes, and addresses the case of multiplying more than two child nodes. This theoretical framework provides mathematical rigor far exceeding current MultKAN implementations, offering a principled approach to gradient computation in complex multiplication scenarios.

[0167] Theorem 2 (Multiplicative Level Decomposition): Let... It is a multiplication layer in an RMultKAN, with The nth multiplication output node. Assume the nth... Each multiplication node calculates its The product of the input child nodes. Then, It can be represented as layer The composite, in which each Both are layers composed of non-spline univariate functions, with the same structure as the standard KAN layer defined in formula (1). The mathematical form is as follows:

[0168]

[0169] in , It is an identity mapping.

[0170] Proof: Binary multiplication: two inputs and Multiplication can be expressed by the following identity:

[0171] like Figure 5 As shown in the highlighted area, this operation requires two types. Layer: First layer ( ) Calculation and ( ) and difference ( ), second layer ( Calculate the scaled squared terms and their final sum. Therefore, one binary multiplication requires two layers.

[0172] Recursive steps and levels: a set of recursive steps and levels: Multiplication of inputs can be performed by executing The decomposition is performed using consecutive binary multiplications. This recursive process... For a three-input multiplication ( The explanation is clear. As shown in the highlighted area in the figure, the first binary multiplication ( ) From the first two layers and Execution. Subsequent layers. and Then the second binary multiplication is performed: they receive the intermediate results from the previous step ( ) and combine it with the next input ( Multiply them to obtain the final product.

[0173] This example illustrates a general principle: a person with Multiplication of one input requires There are 1 recursive step, each step requires 2 types. The number of layers is . Therefore, the total number of layers required is .

[0174] Generalization: For the entire multiplication layer The total number of decomposition layers required It must be sufficient to handle the most complex multiplication nodes (i.e., nodes with the largest number of inputs). Therefore, we set:

[0175] Since all multiplication nodes can be decomposed in parallel, and addition nodes pass directly as identity functions, the proof is complete.

[0176] This theorem provides a crucial insight into the RMultKAN architecture: seemingly disparate multiplication layers can be reformulated as deep compositions of addition and univariate functions. It proves the theoretical completeness of the architecture, showing that all components can be constructed from univariate function matrices and linear transformations. Furthermore, this decomposition provides a principled approach to gradient computation, offering a formalized and constructive path that avoids black-box methods and naturally extends to complex multi-input scenarios. A direct consequence of this decomposition is that the multiplication layer… derivative It can be represented by applying the chain rule to the composite function:

[0177]

[0178] Each of them This represents the derivative of the corresponding univariate function matrix. This well-defined derivative is crucial for subsequent gradient analysis.

[0179] (3) Gradient decomposition

[0180] Learnable activation functions in KAN-based models can exhibit unstable learning dynamics, which in turn can lead to unstable gradients. This section introduces a formal analysis of how skip connections in the RMultKAN architecture address this issue. Embodiments of the invention demonstrate that these connections create a direct, stable path for gradient flow that runs parallel to the unstable path through the learnable activation function. Through mathematical decomposition, it is formally proven that this dual-pathway system effectively mitigates ill-conditioned gradient behaviors, namely vanishing and exploding gradients, that can affect training stability.

[0181] To conduct the analysis, consider a... -Layer RMultKAN, training instances are ,in It is input. These are the corresponding real labels. The model's forward propagation predictions... It is given by the following formula:

[0182]

[0183] in , and These are trainable parameters. This indicates fixed hyperparameters, and .

[0184] The loss function is defined as the mean squared error between the prediction and the label:

[0185]

[0186] The main theoretical results accurately characterize the gradient flow in RMultKAN.

[0187] Theorem 3 (Gradient Decomposition): In a given... In the - layer of RMultKAN, the loss relative to the input of any layer The gradient can be decomposed into:

[0188]

[0189] The direct path component is defined as a matrix product:

[0190]

[0191] The remainder term consists of all terms in the complete gradient expansion that contain at least one B-spline activation derivative. It is the derivative of the multiplication layer, as defined by formula (18).

[0192] Proof: According to the chain rule, gradient It is given by the following formula:

[0193]

[0194] Based on formulas (20), (5) and (4), for The derivatives of its components are as follows:

[0195]

[0196]

[0197]

[0198] Substituting these terms, we get:

[0199] Q.E.D.

[0200] This decomposition provides the theoretical basis for the enhanced stability claimed in the embodiments of the invention, formally demonstrating how the architecture mitigates the vanishing and exploding gradient problems. Direct Path It acts as a "gradient highway," ensuring a stable training signal is maintained throughout the network. This mechanism offers dual benefits, addressing two major ill-conditioned gradient problems:

[0201] 1) Mitigating the vanishing gradient problem: If the path is nonlinear... It becomes pathologically small, the total gradient Will because from The continuous and stable signal prevents it from disappearing completely.

[0202] 2) Mitigating the gradient explosion problem: By simplifying the learning task required for the B-spline activation function, the nonlinear path ( Only a simpler residual function needs to be learned (by subtracting the linear term). This architectural design principle leverages the fact that it is much easier for a network to learn a near-zero correction than to learn a large mapping from scratch. Therefore, B-spline activation functions are unlikely to produce large gradients that characterize gradient explosion scenarios.

[0203] This dual protection mechanism, guaranteed by skip connection paths, offers a significant advantage over standard KAN architectures that lack such resilient gradient flow. The theoretical superiority of this approach is directly validated by the numerical experiments presented later. By reducing the optimization burden on learnable components, the residual architecture facilitates an inherently more stable training process. This enhanced stability, in turn, promotes faster convergence and greater robustness across diverse learning scenarios.

[0204] Next, the complexity of ARMultKAN is analyzed, demonstrating that it does not introduce significant computational overhead. The high efficiency and scalability of this architecture are established from two perspectives: 1) Parameter complexity: The AGU mechanism is "parameterless." It does not introduce any additional learnable parameters, ensuring that ARMultKAN maintains the exact same model size as the benchmark RMultKAN. 2) Training complexity: The periodic overhead introduced by AGU is negligible. Its cost is proven to be lower than the standard training cost. ,in It is the update interval. Therefore, for The typical setup results in an overhead of less than 0.1%, confirming the high efficiency of the AGU.

[0205] First, the key structural variables of ARMultKAN: Represents the number of floors. Indicates the first Layer width (number of nodes) It is the number of grid intervals. This is the order of the B-spline. The average network width of each layer is expressed as... .

[0206] (1) General analysis: Complexity calculation

[0207] 1) Parameter Complexity: The total number of learnable parameters in ARMultKAN is determined by RMultKAN. Importantly, the AGU mechanism, which refines the grid point locations during training, does not introduce additional parameters. The total number of parameters P consists of two parts: B-spline coefficients from the nonlinear activation layers (…). ) and parameters from the linear skip connection ( The latter is used to handle potential dimension mismatch issues:

[0208]

[0209] in and Assuming each layer The total parameter complexity can be approximated as:

[0210]

[0211] This approximation captures the main contributions of both, among which This reflects the combined effect of spline and skip connection parameters.

[0212] 2) Training complexity: The total computational cost of training ARMultKAN It is the standard training complexity The periodic overhead introduced by the AGU mechanism The sum:

[0213]

[0214] Now let's analyze each component separately.

[0215] a) Standard training complexity ( A standard training iteration consists of one forward propagation and one backward propagation. The computational cost of one iteration is related to the dataset size. and the number of model parameters (Formula (23)) is directly proportional, resulting in a complexity of . For a containing The complete training plan for this iteration has a total baseline complexity of:

[0216]

[0217] b) AGU overhead complexity ( The AGU mechanism (Algorithm 2) introduces additional costs. This occurs periodically. A single call to AGU involves three main steps:

[0218] Subdomain loss calculation: This step involves traversing the original dataset. Each sample is assigned to a subdomain and the corresponding loss is calculated. The cost is linearly related to the size of the original dataset, i.e. .

[0219] Dataset resampling: Construct a new dataset using the computed subdomain loss. This process is related to the size of the new dataset ( The cost is directly proportional to the cost of the product. .

[0220] right Perform forward propagation: on the resampled dataset A forward propagation is performed to collect activation statistics. The cost is... .

[0221] Sorting and Quantile Calculation: The collected activation values ​​are sorted to determine the new grid boundaries. The complexity of this step is O(n log n). .

[0222] The total cost of a single AGU call is the sum of the costs of all the components mentioned above. Because And subdomain loss calculation ( ) and dataset resampling ( The initial costs of both preprocessing and forward propagation are linearly related to the dataset size. In all real-world scenarios, these two preprocessing costs are asymptotically outpaced by the more expensive forward propagation. Sorting Therefore, complexity analysis will focus on these latter two dominant terms.

[0223] AGU per Triggered once per iteration. Throughout the entire... In the training plan of the next iteration, by substituting into formula (23) Its overall computational cost can be expressed as:

[0224]

[0225] (2) Typical Case Analysis

[0226] In typical deep learning scenarios, the overhead of AGU can be further simplified. The key is to compare the two main terms in the AGU cost expression: from... Network architecture items and from Dataset items In most applications, architectural items Significantly larger than the number of data items For example, for a medium-sized network ( ) and a large dataset ( ,so ), architectural factors ( The dataset factor (23) clearly dominates. This directly means that the total sorting cost is asymptotically less than the forward propagation cost, i.e.

[0227]

[0228] Therefore, forward propagation cost Becoming the dominant item in AGU overhead simplifies to:

[0229]

[0230] To quantify relative overhead, AGU cost Compared to standard training costs Comparison. Because The ratio becomes:

[0231]

[0232] this The constant factor in the notation represents the ratio of the cost of a single forward propagation (for AGU) to the cost of a complete forward and backward propagation (for standard training). Since backward propagation is typically more expensive than forward propagation, this ratio is usually less than one. Therefore, the upper limit of amortized AGU overhead is... Under typical settings ( =1000), which is less than 0.1% of the total training cost.

[0233] Therefore, the total computational cost of training ARMultKAN satisfies:

[0234]

[0235] After the theoretical analysis was completed, the embodiments of the present invention verified the effectiveness of the above theory through experiments.

[0236] This invention validates the computational performance of RMultKAN and ARMultKAN through embodiments. The validation consists of two phases: first, the enhanced training stability brought about by skip connections in RMultKAN is analyzed; second, its superior function approximation ability is demonstrated on a challenging benchmark test; model performance is evaluated based on relative L2 error and training time measured in milliseconds per iteration (ms / iter). Relative L2 error is defined as... ,in It is a predicted solution. This is a reference solution. This represents the L2 norm. For optimization, all models are trained using the Adam optimizer, with a maximum of [number of iterations]. Unless otherwise stated, all computational experiments were conducted on a single NVIDIA V100-16GB GPU.

[0237] (1) Enhanced training stability and convergence analysis

[0238] To experimentally verify the enhanced stability claimed in the theoretical analysis, the training dynamics of RMultKAN and MultKAN were compared. Both models were required to run under the same configuration (width [2, 2, 1], grid size...). order Solve the Poisson equation (detailed PDE formulas can be found in the supplementary materials). The analysis focuses particularly on learnable B-spline coefficients. (Equation (2)), because these parameters control the nonlinear activation function, which is the main source of potential ill-conditioned gradient problems. The parameters of the linear skip connection ( They are excluded from this analysis because their good, convex optimized landscapes do not lead to instabilities intended to be mitigated.

[0239] To quantify the magnitude of these parameters and the gradient tensor, the Frobenius norm is used. The B-spline coefficient tensor... The norm is defined as:

[0240]

[0241] Similarly, the gradient norm of the loss with respect to these coefficients is given by the following equation:

[0242]

[0243] The validation results provide compelling evidence that residual connections are crucial for ensuring a robust and efficient training process. An examination of the gradient distribution provides the first evidence. As shown in the histogram in Figure 6, the gradients of RMultKAN (blue) are sharply concentrated in a narrow distribution near zero. This indicates that most gradient values ​​are small and behave well. In contrast, the gradients of MultKAN (red) are widely distributed and have heavy tails, suggesting much larger variance and the presence of many large updates that could destabilize the system.

[0244] Figure 7 The gradient norm trajectories in (c) and (d) further confirm this time-varying stability. While the norm of MultKAN exhibits significant volatility, especially in the early stages of training, the norm of RMultKAN rapidly and smoothly decays to a stable and lower order of magnitude. This stable gradient flow directly leads to more controlled parameter evolution. Figure 7 As shown in (a), the parameter norms of MultKAN exhibit significant and unbalanced growth; the norm of the first layer remains small, while the norm of the second layer grows out of control, indicating that the learning burden is pathologically shifted to deeper layers. In stark contrast, Figure 7 (b) shows that RMultKAN facilitates more asymptotic and balanced parameter evolution. While the norm of the first layer in RMultKAN is relatively large (reflecting its ability to learn effectively from the outset), the explosive growth of parameters that occurs in the deeper layers of MultKAN is completely mitigated. In RMultKAN, the norms of both layers converge to stable and comparable orders of magnitude, demonstrating how the residual architecture ensures a more controlled and distributed learning process.

[0245] (2) Approximation of a discontinuous and oscillatory function

[0246] To evaluate the approximation capability of the architecture proposed in this embodiment, its performance was tested on a challenging benchmark function designed to probe spectral biases. This function incorporates both discontinuities and a mixture of high- and low-frequency modes, providing an ideal testbed for comparing RMultKAN with MultKAN and MLP.

[0247]

[0248] To ensure fair comparison, two experimental configurations were designed, with hyperparameters detailed in Table I. Type I models (RMultKAN-I and MultKAN-I) shared the same architecture (2 hidden layers and 40 neurons) as the MLP to compare performance under similar structural complexity. Type II models (RMultKAN-II and MultKAN-II) were configured with a similar number of trainable parameters to the MLP to compare expressive power under similar parameter budgets. All models were trained using the Adam optimizer with the same learning rate schedule and KAN-specific parameters (…). and ).

[0249] Experimental results consistently demonstrate the superior performance of the RMultKAN architecture. These findings are analyzed from two perspectives: accuracy and convergence stability.

[0250] Table I: Comparison of various networks approximating formula (26)

[0251]

[0252] Approximation accuracy: Figure 8 The final approximation results presented in Table I highlight the superior expressive power of RMultKAN. In comparisons with similar architectures, RMultKAN-I achieves a relative L2 error of only 0.0628%, significantly outperforming MultKAN-I (0.4024%) and MLP (0.3294%). Even under parameter budget constraints, RMultKAN-II (0.0823%) remains more accurate than MultKAN-II (0.1087%) and MLP.

[0253] These accuracy results were further validated in the frequency domain. To analyze the ability of each model to capture the multi-scale characteristics of the function, Fourier analysis was performed, decomposing the function into its component frequencies. For example... Figure 9 The resulting spectrum, as shown, provides a powerful diagnostic tool. The close agreement between the spectrum of the RMultKAN-I approximation and the spectrum of the reference solution confirms that the residual design not only effectively captures low-frequency trends but also captures challenging high-frequency oscillations and discontinuities in the objective function, thereby mitigating the spectral bias that typically affects standard networks.

[0254] Convergence and training stability: In addition to final accuracy, Figure 10The training dynamics revealed in the analysis highlight another key advantage of the architecture. Not only does RMultKAN-I exhibit the fastest convergence speed, but more importantly, it overcomes a key training ill-condition observed in MultKAN-I. For MultKAN-I, a significant divergence in performance on training and test data emerges: while the training loss computed on training data points consistently decreases, its relative L2 error (an effective measure of model performance on unseen test sets) remains high. After several iterations, the model stagnates. This divergence is a typical sign of overfitting, where the model memorizes training data at the expense of its ability to generalize to unseen data. In contrast, RMultKAN-I exhibits strong generalization capabilities: training loss and relative L2 error decrease simultaneously, indicating that the model captures the true underlying functions rather than overfitting the training data.

[0255] The batch data integrity verification apparatus provided by the present invention is described below. The batch data integrity verification apparatus described below and the batch data integrity verification method described above can be referred to in correspondence.

[0256] like Figure 11 As shown, the present invention also provides a residual adaptive network architecture design device based on MultKAN, the device comprising:

[0257] Add module 1110 to establish a residual MultKAN architecture by adding skip connections to each KAN layer of the MultKAN architecture;

[0258] The construction module 1120 is used to construct a loss-driven adaptive mesh based on the residual MultKAN architecture to obtain a residual adaptive network architecture.

[0259] According to the residual adaptive network architecture design apparatus based on MultKAN provided by the present invention, the establishment of the residual MultKAN architecture includes:

[0260] A composite module is used to construct multiple residual multiplicative layers by combining them, wherein each of the residual multiplicative layers is a combination of a residual KAN layer and a multiplication layer in the residual MultKAN architecture.

[0261] According to the residual adaptive network architecture design device based on MultKAN provided by the present invention, the output of the residual KAN layer Determined by the following formula:

[0262]

[0263] in, Indicates the sequence number of the residual KAN layer; The activation function matrix of a standard KAN layer; and These are the weight matrix and bias vector for the linear transformation, respectively.

[0264] According to the residual adaptive network architecture design apparatus based on MultKAN provided by the present invention, the construction module 1120 further includes the following units:

[0265] A spatial partitioning unit is used to divide the input space of the residual MultKAN architecture into multiple subdomains;

[0266] A loss calculation unit is used to calculate the loss distribution over each of the subdomains;

[0267] A resampling unit is used to resample the training dataset of the input space based on the loss distribution to construct a resampled dataset;

[0268] The generation unit is used to generate an adaptive grid for each learnable activation function based on the input distribution of the resampled dataset in each layer of the forward propagation.

[0269] According to the residual adaptive network architecture design apparatus based on MultKAN provided by the present invention, the step of generating an adaptive mesh for each learnable activation function further includes:

[0270] Computational grid It is constructed through iteration using the following formula:

[0271] ;

[0272] Among them, the dynamic mixing coefficient Update using the following formula:

[0273]

[0274] Adaptive Mesh Update using the following formula:

[0275] ;

[0276] Uniform grid Update using the following formula:

[0277] ;

[0278] in, Indicates iteration Total loss at that time; Indicates the threshold for loss variation; Indicates adaptive step size; Indicates the first The sample until the 1st The layer's output vector; Specifically refers to the first The output vector of the layer Dimensional components; G represents a set of quantiles. As the index i changes from 0 to G, it takes values ​​from the 0-quantile (minimum) to the 1-quantile (maximum); G is the grid size parameter, which represents the number of intervals divided for each input dimension. This represents a resampled dataset; , They represent the first The first layer of the input vector The minimum and maximum values ​​of the dimensional components.

[0279] The residual adaptive network architecture design apparatus based on MultKAN provided by the present invention further includes, after the adaptive mesh is constructed:

[0280] The basis function update unit is used to update the coefficients of the B-spline basis functions on the adaptive mesh through gradient descent during the backpropagation process of the residual adaptive network architecture.

[0281] According to the residual adaptive network architecture design apparatus based on MultKAN provided by the present invention, the resampling of the training dataset of the input space based on the loss distribution specifically includes:

[0282] Will Each of the input dimensions is divided into These intervals form Subdomains; the training dataset Classified as Disjoint subsets each of the subsets It includes data points that fall into a unique subdomain;

[0283] Resampling probability for each subdomain Determined by the following formula:

[0284] ;

[0285] Resampled dataset It is constructed by drawing a corresponding proportion of samples from each of the aforementioned subsets:

[0286] ;

[0287] in, This represents the loss for each of the subdomains. This represents the total loss of the residual adaptive network architecture; Indicates from the subset Extraction One sample.

[0288] Figure 12 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 12 As shown, the electronic device may include a processor 1210, a communications interface 1220, a memory 1230, and a communication bus 1240. The processor 1210, communications interface 1220, and memory 1230 communicate with each other via the communication bus 1240. The processor 1210 can call logical instructions from the memory 1230 to execute a residual adaptive network architecture design method based on MultKAN.

[0289] Furthermore, the logical instructions in the aforementioned memory 1230 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, external hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0290] On the other hand, the present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to perform the residual adaptive network architecture design method based on MultKAN provided by the above methods.

[0291] In another aspect, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer is able to execute the residual adaptive network architecture design method based on MultKAN provided by the above methods.

[0292] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0293] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0294] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A residual adaptive network architecture design method based on MultKAN, characterized in that, The method comprises the following steps: A residual MultKAN architecture is established by adding skip connections to each KAN layer of the MultKAN architecture; Based on the residual MultKAN architecture, a loss-driven adaptive grid is constructed to obtain a residual adaptive network architecture.

2. The MultKAN-based residual adaptive network architecture design method of claim 1, wherein, The residual MultKAN architecture is established by: A plurality of residual multiplicative layers are combined, wherein each residual multiplicative layer is a combination of a residual KAN layer and a multiplication layer in the residual MultKAN architecture.

3. The MultKAN-based residual adaptive network architecture design method of claim 2, wherein, the output of the residual KAN layer is determined by the following equation: ， wherein, denotes the index of the residual KAN layer; is a standard KAN layer activation function matrix; and are the weight matrix and bias vector of the linear transformation, respectively.

4. The MultKAN-based residual adaptive network architecture design method of claim 2, wherein, The loss-driven adaptive grid is constructed by the following steps: The input space of the residual MultKAN architecture is divided into a plurality of sub-domains; The loss distribution on each sub-domain is calculated; Based on the loss distribution, the training data set of the input space is resampled to construct a resampled data set; According to the input distribution of each layer in the forward propagation of the resampled data set, an adaptive grid is generated for each learnable activation function.

5. The MultKAN-based residual adaptive network architecture design method of claim 4, wherein, The method for generating an adaptive grid for each learnable activation function further comprises: Adaptive computational grid is constructed after iteration by the following equation: ； where the dynamic mixing coefficients are updated by the following equation: ， Adaptive grid Update by the following equation: ； Uniform grid Update by the equation: ； wherein, denotes the total loss at iteration ; denotes the loss change threshold; denotes the adaptive step size; denotes the output vector of the th sample until the th layer; denotes the th dimensional component of the output vector of the th layer; denotes a set of quantiles, which takes values from 0-quantile (smallest value) to 1-quantile (largest value) as the index i varies from 0 to G; G is the grid size parameter, which denotes the number of intervals each input dimension is divided into; denotes the resampled dataset , denotes the minimum and maximum value of the th dimensional component of the input vector of the th layer, respectively.

6. The MultKAN-based residual adaptive network architecture design method of claim 5, wherein, After the adaptive grid is constructed, the method further comprises: During the backpropagation process of the residual adaptive network architecture, the coefficients of the B-spline basis function are updated on the adaptive grid through gradient descent.

7. The MultKAN-based residual adaptive network architecture design method of claim 4, wherein, The resampling of the training data set of the input space based on the loss distribution comprises: Each of the input dimensions is divided into intervals, forming sub-domains; the training dataset is divided into disjoint subsets , where each of the subsets contains data points falling into a unique sub-domain; a resampling probability for each of the subfields is determined by the equation: ； Resampled dataset is constructed by drawing a respective proportion of samples from each of the sub-sets: ， wherein, represents a loss for each of the sub-domains, represents a total loss for the residual adaptive network architecture; represents extracting samples from the subset of the training dataset.

8. A MultKAN-based residual adaptive network architecture design apparatus, characterized by, The method comprises the following steps: An adding module is configured to establish a residual MultKAN architecture by adding skip connections to each KAN layer of the MultKAN architecture; A constructing module is configured to construct a loss-driven adaptive grid based on the residual MultKAN architecture to obtain a residual adaptive network architecture.

9. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor executes the program to implement the residual adaptive network architecture design method based on MultKAN according to any one of claims 1 to 7.

10. A computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the residual adaptive network architecture design method based on MultKAN according to any one of claims 1 to 7.

Images