Explainable recognition method and device based on kan network and adaptive unified trajectory optimization

Abstract

The application discloses a kind of based on KAN network and adaptive unified trajectory optimization explainable identification method and equipment, belong to artificial intelligence technical field.The method includes: using the adaptive unified trajectory optimization method of combining adaptive first-order and second-order moment and gradient amplification mechanism to carry out parameter optimization to improved Kolmogorov-Arnold deep neural network;Optimization, pseudo-transient continuation and quotient gradient system modify traditional gradient flow by a specific way, so as to amplify smaller gradient, and accelerate the trajectory away from flat area or saddle point area, so as to improve the stability of training and reduce the possibility of falling into local minimum;The method of introducing pseudo-time continuation is used to quickly determine the steady-state solution of differential equation during network training;Adaptive updating technology is used to update pseudo-time step.The application converts KAN training problem into finding stable equilibrium point by constructing quotient gradient system, and realizes fast and robust convergence.

Description

Technical Field

[0001] This invention belongs to the field of artificial intelligence technology, specifically relating to an interpretable recognition method and device based on KAN network and adaptive unified trajectory optimization. Background Technology

[0002] Lung and colorectal cancer are among the most common types of cancer worldwide, placing a heavy burden on people, especially patients and their families. According to global cancer statistics, lung cancer is the leading cause of cancer-related deaths, while the incidence of colorectal cancer continues to rise. Therefore, early and accurate diagnosis, along with effective treatment, is crucial. Pathology is fundamental to cancer diagnosis and treatment. Pathologists examine tissue sections under a microscope, analyzing changes in cell morphology and structure to determine the type of cancer; this information is essential for selecting treatment options and assessing prognosis.

[0003] Colorectal cancer (CRC) is one of the leading causes of cancer-related deaths worldwide, and accurate cancer assessment and diagnosis are crucial for improving patient survival rates. In recent years, advancements in artificial intelligence (AI) and deep learning technologies have prompted researchers to explore their applications in cancer prognosis prediction. Compared to traditional deep learning neural networks, interpretable neural networks overcome the "black box" characteristic and have been widely applied in multiple fields. The Kolmogorov-Arnold network (KAN), based on the Kolmogorov-Arnold representation theorem, has achieved a significant breakthrough in the theoretical research of interpretable neural networks, achieving high prediction accuracy while significantly reducing the number of parameters. This provides a theoretically sound and practical example for the application of deep learning in resource-constrained scenarios. Summary of the Invention

[0004] The present invention aims to at least partially solve one of the technical problems in the aforementioned related technologies.

[0005] Therefore, the purpose of this invention is to provide an interpretable recognition method and device based on KAN network and adaptive unified trajectory optimization. By constructing a quotient gradient system, the KAN training problem is transformed into a problem of finding a stable equilibrium point, thereby achieving fast and robust convergence.

[0006] To solve the above-mentioned technical problems, the present invention is implemented as follows:

[0007] This invention provides an interpretable recognition method based on KAN network and adaptive unified trajectory optimization. The method includes: using an adaptive unified trajectory optimization (ADTJU) method that combines adaptive first and second moments with a gradient amplification mechanism to optimize the parameters of an improved Kolmogorov-Arnold deep neural network; during optimization, the pseudo-transient continuation and quotient gradient system modify the traditional gradient flow in a specific way, thereby amplifying smaller gradients and accelerating trajectories away from flat regions or saddle point regions, thereby improving the stability of training and reducing the possibility of getting trapped in local minima;

[0008] A pseudo-time continuation method is introduced during deep neural network training to quickly determine the steady-state solution of the differential equation; an adaptive update technique with switching evolutionary relaxation is also used to update the pseudo-time step, and the pseudo-time step increases rapidly as the system trajectory approaches the stable equilibrium point.

[0009] In addition, the KAN-based and adaptive unified trajectory optimization interpretable recognition method according to the present invention may also have the following additional technical features:

[0010] In some embodiments, the stages of the method include:

[0011] S1. Initialization phase, which includes initializing the parameters of the deep neural network and the momentum estimation parameters;

[0012] S2, Adaptive Sparse Training Phase: This phase includes dynamically adjusting the sparse connections at each layer of the improved KAN network, including calculating the remaining network connections and randomly adding connections at each layer.

[0013] Calculate the remaining number of network connections. ,

[0014] Add connections randomly in each layer. ,

[0015] in, Represents a function matrix, This indicates the number of layers in the KAN network. Indicates sparsity. This indicates a newly added connection number. Represents a sparse weight matrix;

[0016] S3. Preparation phase: Data processing and system construction for the dataset, including:

[0017] Divide the dataset x into B batches of equal size. For each batch, compute the nonlinear system and its gradient.

[0018] Establish a dynamic system and determine the focusing function;

[0019] S4. Trajectory and Global Convergence Stage: Based on the results and settings of the previous steps, iterative calculations are performed, including:

[0020] a. After completing one integration step, the model weights are obtained using the pseudo-time continuation method, and the weights are updated.

[0021] b. When the convergence condition is met, return to step a; when the maximum number of iterations is reached, proceed to the next stage.

[0022] S5, Local Solution Stage, includes: using the global convergence iteration result of S4 as the initial guess value, and using the local solver strategy to solve for the local optimal solution.

[0023] In some implementations, the parameters of the deep neural network initialized in step S1 include the number of model layers, function matrix, sparsity, pruning rate, and number of training rounds.

[0024] The momentum estimation parameters include the initial model weights and initial step size of the improved KAN network. The initial step size is calculated as follows:

[0025] .

[0026] In some of these implementations, for a given input vector ,Depend on The output of the improved Kolmogorov-Arnold deep neural network, composed of layers, is as follows:

[0027] (9)

[0028] in, For spline functions, This is the weight matrix;

[0029] Its weight matrix is ​​characterized by, It is A random graph, whose probability function is given below:

[0030]

[0031] in, The number of connections between neurons is given by an adaptive sparsity parameter. Control; regarding the number of neurons in the sparse layer,

[0032]

[0033] if and , then represents a linear number of connections (i.e., non-zero elements); the number of connections in a fully connected layer is quadratic, that is... ;

[0034] More precisely, the function matrix is ​​initialized in the following way:

[0035]

[0036] in Represented as a sparse function matrix, at the end of each training epoch, the least important connections in each layer (whose proportion) The specific ratio is The remaining connections will be pruned;

[0037]

[0038] in, Non-zero elements are A subset of non-zero elements, which correspond to The largest negative weight and the smallest positive weight in the middle, while Is with Binary matrices of the same size , The weight is a constant; next, an equal number of weights are randomly added to each layer in the following manner: Connection:

[0039]

[0040] in, This indicates a newly added connection number. The non-zero value is exactly The positions of non-zero elements are selected from a random uniform distribution, and their values ​​are set using small Gaussian noise. Finally, the following is the modified version. :

[0041]

[0042] In general, natural selection manifests as the elimination of adaptive sparse connections, while the variation phase in natural evolution manifests as the formation of new connections.

[0043] In some implementations, in step S4, a mini-batch of samples is drawn from the training dataset in each iteration, and the predicted value for colorectal cancer identification is calculated through forward propagation; then the loss is calculated and the QGS coupling mechanism is applied to adjust the gradient flow based on the current loss value;

[0044] The optimization and update principle is based on the first and second-order estimates of momentum, ensuring that the learning process can adapt to the ever-changing terrain of the loss function.

[0045] In some of these implementations, the convergence condition for the iteration is: or ,

[0046] in, It is a positive minimum value. This indicates the maximum number of iterations set for colorectal cancer identification. Indicates the current iteration number;

[0047] In some implementations, the nonlinear system of the KAN network is as follows:

[0048] (10)

[0049] Where N is the mini-batch size, z is the input data, and x is the weight of ASKAN(z).

[0050] In some of these implementations, the dynamic optimization method is as follows:

[0051] (11)

[0052] in, The Jacobian matrix is ​​denoted as , The focusing function is used; this nonlinear dynamic system is a nonlinear and non-hyperbolic dynamic system.

[0053] In some implementations, step S3 combines adaptive momentum with the enhanced saddle point repulsion of QGS to establish a continuous dynamic system, expressed as follows:

[0054] (15)

[0055] (16)

[0056] (17)

[0057] In the formula, and These represent continuous-time simulations of the accumulation of first and second moments, respectively. This represents the base learning rate as it changes over time. Representative coefficient, This represents a coupling term.

[0058] This invention also provides a computer device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the content of the KAN network-based and adaptive unified trajectory optimization interpretable recognition method as described in any of the preceding embodiments.

[0059] Compared with the prior art, the present invention has at least the following beneficial effects:

[0060] In this embodiment of the invention, the unified trajectory method for colorectal cancer identification based on KAN network is provided. The ADTJU method is used for model optimization and training. Through its two-stage training strategy and the introduction of influence balance fine-tuning, it not only enhances the model's ability to learn from imbalanced datasets, but also ensures the robustness and convergence of the training process.

[0061] In this embodiment of the invention, the unified trajectory method for colorectal cancer identification based on KAN network, and the AdTJU training method, provide a powerful framework for optimizing deep neural networks by combining adaptive momentum with continuous dynamical system methods. This method improves the training process by allowing dynamic adjustments based on the influence of each sample, thereby enhancing model performance at the decision boundary.

[0062] In this embodiment of the invention, the provided unified trajectory method for colorectal cancer identification based on KAN networks, the AdTJU method, combines adaptive momentum with the QGS principle, enabling more refined parameter updates. By employing bias-corrected momentum estimation, this method effectively mitigates initialization biases that may affect convergence, especially in the early stages of training. The adaptive learning rate calculated based on the corrected momentum ensures appropriate scaling of updates, thereby promoting the stability and efficiency of the optimization process. This combination of techniques not only accelerates convergence but also improves the model's ability to generalize from training data, making it particularly effective in applications dealing with imbalanced datasets.

[0063] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0064] Figure 1 This is a flowchart illustrating the training of a KAN model using the AdTJU method according to an embodiment of the present invention;

[0065] Figure 2 The test accuracy and training loss of different optimization methods disclosed in one embodiment of the present invention on CIFAR-10;

[0066] Figure 3 The test accuracy and training loss of different optimization methods disclosed in one embodiment of the present invention on the CIFAR-100 dataset;

[0067] Figure 4 The following are feature visualization results (obtained via t-SNE) of different methods disclosed in one embodiment of the present invention on the CIFAR-10 dataset.

[0068] Figure 5 The dataset categories and seven random images in each category are disclosed in one embodiment of the present invention;

[0069] Figure 6 The test accuracy and training loss of different optimization methods disclosed in one embodiment of the present invention on LC25000;

[0070] Figure 7 Visualization results of different optimization methods disclosed in one embodiment of the present invention;

[0071] Figure 8 A heatmap visualization of the lung cancer and colon cancer histopathological image dataset (Adam) disclosed in one embodiment of the present invention;

[0072] Figure 9 A heatmap visualization of the lung cancer and colon cancer histopathological image dataset (SGD) disclosed in one embodiment of the present invention;

[0073] Figure 10 A heatmap visualization of the lung cancer and colon cancer histopathological image dataset (TJU) disclosed in one embodiment of the present invention;

[0074] Figure 11 A heatmap visualization of the lung cancer and colon cancer histopathological image dataset (AdTJU) disclosed in one embodiment of the present invention. Detailed Implementation

[0075] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0076] The embodiments of the present invention will be described in detail below with reference to the accompanying drawings and specific examples and application scenarios.

[0077] In some embodiments of this invention, a unified trajectory method for colorectal cancer identification based on KAN networks is provided. This method aims to improve the diagnostic capability for cancer patients by deeply analyzing cancer tissue pathological images and utilizing an improved Kolmogorov-Arnold network (KAN) deep learning framework—Adaptive Momentum (AdTJU). Extensive experiments were conducted in multiple patient cohorts, using a certain number of image slices for model training, validation, and testing. Results show that the deep learning model trained based on the KAN network deep learning framework can accurately diagnose and assess cancer patients, while significantly reducing the number of parameters without compromising prediction accuracy. Furthermore, the AdTJU training method optimizes the model training process, significantly improving convergence speed and robustness, especially in escaping saddle points and converging to local minima. This invention demonstrates that the decisions made by KAN networks are based on interpretable anti-tumor immune principles, providing clinicians with an effective decision-making tool.

[0078] Unlike purely discrete iterative schemes, deep neural network (DNN) training can be viewed as a continuous dynamical system evolving over time. From this perspective, pseudo-transient continuation (PTC) and quotient gradient systems (QGS) modify the traditional gradient flow by, for example, multiplying the descent direction by a function of the current loss. This modification amplifies smaller gradients and accelerates trajectories away from flat or saddle-point regions, thereby improving training stability and reducing the likelihood of getting trapped in local minima. By studying the differential equations governing these systems, existing analytical tools in dynamical system theory can be used to explore ideal convergence and stability properties. Notably, the ability to incorporate various coupling terms, including those designed to reshape the gradient flow as training progresses, makes these continuous-time methods highly flexible in dealing with various loss surface geometries. Furthermore, combining such frameworks with well-established adaptive methods offers a promising direction for balancing stable, context-sensitive gradient updates with aggressive, theoretically sound saddle-point escapes.

[0079] The AdTJU training method is introduced below. This method transforms the KAN training problem into a problem of finding a stable equilibrium point by constructing a quotient gradient system, thereby achieving fast and robust convergence.

[0080] 1. Adaptive sparse training.

[0081] For a given input vector The output of a typical KAN network consisting of L layers is as follows:

[0082] (1)

[0083] Among them, the The function matrix of layer KAN is used Let z represent the input data. Formally, define a sparsely connected ( As The function matrix for this layer is:

[0084] (2)

[0085] For spline functions ,in, This indicates the number of layers in the KAN network. and Represented as ( , The position coordinates of this layer are compiled into a sparse weight matrix. .first, It is A random graph, whose probability function is given below:

[0086] (3)

[0087] in, The number of connections between neurons is given by an adaptive sparsity parameter. Control. Regarding the number of neurons in the sparse layer,

[0088] (4)

[0089] if and This indicates a linear number of connections (i.e., non-zero elements). The number of connections in a fully connected layer is quadratic, i.e., ... .

[0090] More precisely, the function matrix is ​​initialized in the following way:

[0091] (5)

[0092] in Represented as a sparse function matrix, at the end of each training epoch, the least important connections in each layer (whose proportion) The specific ratio is (This part will be pruned.) The remaining connections are...

[0093] (6)

[0094] in, Non-zero elements are A subset of non-zero elements, which correspond to The largest negative weight and the smallest positive weight in the middle, while Is with Binary matrices of the same size , This is a constant. Next, in each layer, an equal number of weights are randomly added in the following manner: Connection:

[0095] (7)

[0096] in, This indicates a newly added connection number. The non-zero value is exactly The positions of non-zero elements are selected from a random uniform distribution, and their values ​​are set using small Gaussian noise. Finally, the following is the modified version. :

[0097] (8)

[0098] In general, natural selection manifests as the elimination of adaptive sparse connections, while the variation phase in natural evolution manifests as the formation of new connections.

[0099] For a given input vector ,Depend on The output of a typical adaptive sparse KAN network composed of layers is as follows:

[0100] (9)

[0101] in, For spline functions, This is the weight matrix.

[0102] Second: Construct the KAN network as a dynamic system.

[0103] Due to its global convergence, the TJU technique has a higher probability of identifying high-quality local optima and can avoid them. The PTC method improves training efficiency and accelerates numerical integration. Large-scale KAN model training is one application of this technique.

[0104] The nonlinear system of the KAN network is as follows:

[0105] (10)

[0106] Where N is the mini-batch size, z is the input data, and x is the weight of ASKAN(z).

[0107] To solve the local optimal solution of the system, a nonlinear dynamic system was constructed, and the relationship between its steady state and local optimal solution was explored.

[0108] (11)

[0109] in, The Jacobian matrix is ​​denoted as , The focusing function is denoted as . This nonlinear dynamic system is a nonlinear and non-hyperbolic dynamic system.

[0110] Definition 1: (Regular stable equilibrium point and degenerate stable equilibrium point) When the system's stable equilibrium point... satisfy When this condition is met, it is called a canonical stable equilibrium point (RSEP); if it satisfies , If the value is such that it is called the Degenerate Stable Equilibrium Point (DSEP), then it is called the De

[0111] Definition 2: (Stability Region) Assumption It is the equilibrium point of a dynamic system, from the state space. The curve from which the system originates is called the trajectory of the dynamic system. The stability region of the equilibrium point is defined as follows:

[0112] (12)

[0113] stable region It is a connected, open, invariant set. Every trajectory within the stable region is completely contained within it, and over time, every trajectory converges to a stable equilibrium point within the stable region. The local optima of a nonlinear system correspond to the local optima of a dynamic system. Therefore, the stable region of a dynamic system is equivalent to the convergence region of a nonlinear system.

[0114] Theorem 1: (Global Convergence) Dynamic systems built on the KAN model are globally stable.

[0115] This means that every trajectory of the system converges to a corresponding stable equilibrium point. It can be proven that a local minimum of the subsequent energy function corresponds to a stable equilibrium point of the system:

[0116] (13)

[0117] in, For the focusing function, Indicates the weight value. This represents the input data. Therefore, by integrating the trajectories of the dynamic system until it converges to a stable equilibrium point, the loss function of ASKAN can be found.

[0118] 3. Adaptive Unified Trajectory (AdTJU) Optimization Method.

[0119] In the Quotient Gradient System (QGS) or Pseudo-Transient Continuation (PTC) framework, a key mechanism for escaping saddle points is introduced: by multiplying the negative gradient flow by the loss function L(θ). Instead of...

[0120] ,

[0121] QGS adopts

[0122] (14)

[0123] This makes the behavior of neutral or weakly unstable saddle points extremely unstable, thus driving the trajectory away from the undesirable stable region.

[0124] To combine the advantages of adaptive momentum with the enhanced saddle point repulsion of QGS, "AdTJU" unifies the two into a continuous powertrain:

[0125] (15)

[0126] (16)

[0127] (17)

[0128] here, and These represent continuous-time simulations of the accumulation of first and second moments, respectively. This represents the base learning rate (which changes over time). Representative coefficient, term Represents the coupling term. This is determined based on the parameters. The current loss value is used to amplify or suppress gradient flow. The introduction of "second-order coupling" in QGS is typically chosen as...

[0129] or

[0130] in, It is a hyperparameter used to control the degree of gain amplification.

[0131] Local minimum: if And since the Hessian matrix is ​​positive definite, then the coupling term between the adaptive mechanism and QGS ( The combined effect of these factors ensures asymptotic stability. Small perturbations dissipate, thus preventing unstable drift.

[0132] Saddle point: for but Hessian matrices that have both positive and negative eigenvalues ​​are typically .therefore, This can lead to strong local instability, causing a rapid escape from the saddle point region.

[0133] Mini-batch noise: For stochastic gradients, equations (1)-(3) can be regarded as a stochastic differential system, where moderate noise helps to escape saddle points while maintaining robust convergence to local minima.

[0134] Through this continuous-time perspective, AdTJU combines adaptive momentum with QGS-based "second-order coupling," achieving flexible saddle point rejection and momentum adaptation in a unified approach. Practical deployments can be customized for Equation (3) based on the loss function, network size, decay factors β1 and β2, and learning rate scheduling strategy. Theoretical linearization around the equilibrium point shows that multiplying by L(θ) in (14) improves convergence efficiency in high-dimensional non-convex settings and remains easily extendable to distributed parallelization, regularization, and mixed-precision scenarios.

[0135] 4. AdTJU training method for KAN network optimization.

[0136] This invention presents an AdTJU training method for KAN networks. Training KAN models is typically time-consuming, even with optimization techniques such as pruning that significantly reduce the number of model parameters. To address these challenges and improve training efficiency, a method called PTC (Pseudo-Time Continuation) is introduced.

[0137] The PTC method aims to quickly determine the steady-state solution of differential equations, although this comes at the cost of not being able to accurately calculate the system trajectory. In dynamic systems, the iterative process of the PTC method can be represented by the following equation:

[0138] (18)

[0139] Here, Representing a nonlinear system, For a nonlinear system, the Jacobian matrix is ​​given by [reference needed]. The pseudo-time step is [reference needed]. This is crucial for accelerating the integration process. An adaptive update technique, called switching evolution relaxation, was also implemented.

[0140] (19)

[0141] This mechanism shows that the pseudo-time step can be rapidly increased as the system trajectory approaches a stable equilibrium point.

[0142] This work highlights a training scheme based on a mini-batch focusing dynamic system, where the introduction of a mini-batch approach aims to improve the training performance of the KAN model. Please refer to... Figure 1As shown, the specific steps for training a KAN model using the AdTJU method are summarized below:

[0143] 1) Initialization:

[0144] Data. A KAN network with L layers, KAN(·), and function matrix. Sparseness S, pruning rate And the number of training rounds, n.

[0145] Input. Initialize the weights of the KAN model. ,set up Simultaneously calculate the initial step size. .

[0146] 2) Adaptive sparse training:

[0147] Calculate the remaining number of network connections: .

[0148] Add connections randomly at each level: .

[0149] 3) Preparation stage

[0150] Divide the dataset x into B batches of equal size. For each batch Calculate nonlinear systems and gradient .

[0151] Establish by Define the dynamic system and determine the focusing function. .

[0152] 4) Trajectory and Global Convergence

[0153] a. After completing one integration step, the PTC technique is used to obtain... Update the weights using the following equation:

[0154] .

[0155] b. If the convergence condition is met or (in It is a small positive constant, for example If ), then return to step 4.a. After reaching the maximum number of iterations, proceed to the next stage.

[0156] 5) Local solver

[0157] To extract high-quality local optima, a local solver strategy is applied to the training problem, using the final iteration result of the PTC method established in step 4.b as the initial guess value.

[0158] When combined with stochastic gradient descent in the third stage, the AdTJU method becomes the AdTJU-SGD method, which in turn gives rise to various variants such as AdTJU-Adam.

[0159] The AdTJU (Adaptive Unified Trajectory) method has made significant progress in the optimization of deep neural networks by combining adaptive momentum techniques with a continuous dynamical system framework. This innovative approach addresses the challenges faced by traditional optimization methods when dealing with imbalanced datasets. By combining the advantages of adaptive learning rates and quotient gradient systems, the AdTJU method improves the training process, thereby increasing convergence speed and model performance. The following outlines the training process of the one-stage AdTJU optimization method.

[0160] This method first initializes the model parameters and momentum estimates, laying the foundation for effective learning. In each iteration, a minibatch of samples is drawn from the training dataset, and predictions are computed through forward propagation. The loss is then calculated, and a QGS coupling mechanism is applied to adjust the gradient flow based on the current loss value. This coupling mechanism is crucial because it amplifies gradients in regions of significant loss, thus contributing to more robust escape from saddle points and optimizing the overall optimization trajectory. The first and second-order momentum estimates are updated according to the Adam optimization principle, ensuring the learning process adapts to the evolving terrain of the loss function.

[0161] The AdTJU method combines adaptive momentum with the QGS principle, enabling more nuanced parameter updates. By employing bias-corrected moment estimation, it effectively mitigates initialization biases that can affect convergence, especially in the early stages of training. The adaptive learning rate, calculated based on the corrected momentum, ensures appropriate scaling of updates, thus promoting the stability and efficiency of the optimization process. This combination of techniques not only accelerates convergence but also improves the model's generalization ability from the training data, making it particularly effective in applications dealing with imbalanced datasets.

[0162] The AdTJU training method of this invention provides a powerful framework for optimizing deep neural networks by combining adaptive momentum with continuous dynamical systems methods. This method improves the training process by allowing dynamic adjustments based on the impact of each sample. Ultimately, it enhances model performance at the decision boundary. The algorithm's structured parameter update approach, coupled with its ability to navigate complex loss surfaces, makes the AdTJU method a valuable tool in deep learning, especially under the significant challenges posed by imbalanced data.

[0163] To address the challenge of training deep neural networks on imbalanced datasets, this invention proposes an adaptive dynamic unified trajectory optimization method. This method transforms the optimization problem into a dynamic system framework, enabling the invention to leverage the stability and convergence of dynamic systems to improve the training process. The theoretical basis of this approach is explained below, along with a comprehensive exposition of its conceptual framework and mathematical derivation.

[0164] A. Stable equilibrium point

[0165] Definition 1 (Trajectory): The dynamic system solution corresponding to the AdTJU method from Moment Starting from This trajectory describes the evolution of the system state over time under the influence of optimized dynamics.

[0166] Definition 2 (Equilibrium Point): The equilibrium point of a dynamic system A point is defined as having a gradient of zero for the loss function, satisfying the following condition:

[0167]

[0168] in, This represents the loss function associated with deep neural networks. The focus function is denoted by . This condition indicates that at the equilibrium point, the system has no net change, making it a candidate point for the optimal solution.

[0169] Next, this invention introduces the concepts of stable and unstable manifolds related to equilibrium points:

[0170] Definition 3 (Stable manifold): Equilibrium point The stable manifold is represented as:

[0171] (20)

[0172] in, yes A neighborhood of . This manifold consists of all initial conditions that converge to an equilibrium point over time.

[0173] Definition 4 (Unstable manifold): Equilibrium point The unstable manifold is represented as:

[0174] (twenty one)

[0175] This manifold contains all the initial conditions that deviate from the equilibrium point as time progresses:

[0176] If an equilibrium point (EP) is of type 0, then its unstable manifold is empty, and this equilibrium point is called a stable equilibrium point. The existence of stable equilibrium points is crucial for ensuring the stability of the training process.

[0177] B. Stable Regions and Stable Boundaries

[0178] The stable region and stable boundary of the ADTJU method are defined as follows:

[0179] Definition 5 (Stable Region): Let This is a balance point for the AdTJU method. If for any... All exist , making This means for all have If established, it is called It is stable. Stable region. Defined as:

[0180] (twenty two)

[0181] This region represents the attraction domain of a stable equilibrium point, which is the set of all initial states that will eventually converge to the equilibrium point under the influence of a dynamic system.

[0182] Definition 6 (Stability Boundary): The stability boundary of a stable equilibrium point It is the boundary of its stability region, separating the stable and unstable regions of the dynamical system. Formally, it is defined as:

[0183] (twenty three)

[0184] The stability boundary plays a crucial role in understanding the transition between stable and unstable dynamics. The stability boundary of the AdTJU method can be fully described under the following conditions:

[0185] a1) The equilibrium point on the stability boundary is hyperbolic.

[0186] a2) Stable and unstable manifolds at equilibrium points on the stability boundary satisfy the cross-section condition.

[0187] a3) Every trajectory on the stability boundary converges to an equilibrium point as t approaches infinity.

[0188] The stability boundary characteristics are considered for the AdTJU method satisfying assumptions a1) and a3). Let... , The stability boundary of a stable equilibrium point The extreme point on.

[0189] Proof: To prove this theorem, we first consider the nature of singularities on stable boundaries. By definition, if the Jacobian matrix of the system at a singularity is... A singularity is a point that has no eigenvalues ​​with a real part of zero. It is hyperbolic. This condition ensures that the trajectory near the singularity exhibits predictable behavior.

[0190] 1. Hyperbolicity: due to Since the system is hyperbolic, the Hartman-Grobman theorem can be applied. This theorem states that near the hyperbolic equilibrium point, the dynamics of the system can be analyzed through linearization. Therefore, the Jacobian matrix can be used. The eigenvalues ​​are used to represent the dynamics of the system:

[0191]

[0192] matrix The eigenvalues ​​determine the stability of the equilibrium point. If the real parts of all eigenvalues ​​are negative, the equilibrium point is stable; if there exists any eigenvalue with a positive real part, the equilibrium point is unstable.

[0193] 2. Transverse Intersection Condition: The transverse intersection condition states that stable and unstable manifolds intersect transversely at their equilibrium points. This means that the sum of the dimensions of the stable and unstable manifolds equals the dimension of the phase space.

[0194]

[0195] This condition ensures that the trajectory can smoothly transition between stable and unstable regions.

[0196] 3. Convergence of the trajectory: According to hypothesis a3, each trajectory on the stability boundary... The time intervals tend towards an equilibrium point. This means that the stability boundary consists of trajectories attracted by the stable equilibrium point, thus confirming that:

[0197]

[0198] Therefore, it can be concluded that a stable boundary can be completely described as the union of stable manifolds with singularities on the boundary.

[0199] C. Complete stability and equivalence relation

[0200] This paper discusses the convergence of the AdTJU method and the relationship between its stable equilibrium point and the local optimum of the original optimization problem. Theorem 2 shows that the AdTJU method is completely stable, meaning that every trajectory of the system converges to a stable equilibrium point. Specifically, for almost all initial conditions... trajectory along with It converges to a stable equilibrium point.

[0201] Complete stability: The AdTJU method is completely stable, ensuring that each trajectory converges to a stable equilibrium point.

[0202] To demonstrate complete stability, this invention analyzes the behavior of trajectories near the equilibrium point.

[0203] 1. Existence of a trajectory: for stable regions Any initial conditions within ,have:

[0204]

[0205] This indicates that trajectories originating from the stable region will converge to the equilibrium point. .

[0206] 2. Attractiveness of a stable equilibrium point: The stability region characterizes the attractive domain of a stable equilibrium point. According to the definition of the stability region, for any... There exists a Make:

[0207]

[0208] This property ensures that a trajectory starting from a position sufficiently close to the equilibrium point will remain nearby and eventually converge to that equilibrium point.

[0209] 3. Global Convergence: Since the AdTJU method aims to optimize the training process, it inherently guides the trajectory towards a stable extreme point. The combination of influencing balance fine-tuning and dynamic adjustment of sample weights ensures that the optimized terrain is favorable, thereby achieving convergence.

[0210] 4. Conclusion: Therefore, it is concluded that the AdTJU method exhibits complete stability because each trajectory converges to a stable equilibrium point, which confirms the robustness of the training process.

[0211] The equivalence relation is considered in relation to the optimization problem corresponding to the AdTJU method. If we assume a1) and a2) hold, then the following properties can be derived: 1) If If the solution is a local optimum of the optimization problem, then a corresponding stable expansion path exists in the AdTJU method.

[0212] Proof: To establish the equivalence relationship between the local optimum of the optimization problem and the stable equivalent point of the AdTJU method, the following steps are taken:

[0213] 1. Definition of Local Optimum: If there exists a neighborhood U such that: then the point is called a local optimum. It is a local optimum.

[0214]

[0215] This means that in The gradient vanishes at that point, i.e. .

[0216] 2. Relationship with stable equilibrium points: According to the definition of an equilibrium point, if If it is a local optimum, then it satisfies:

[0217]

[0218] therefore, It is also the equilibrium point of a dynamic system defined by the AdTJU method.

[0219] 3. Stability of local optima: Given It is a locally optimal solution, and it can be asserted that it is stable under the AdTJU method. The hyperbolic condition ensures that the locally optimal solution corresponds to a stable equilibrium point, thus confirming:

[0220] such that and is a SEP

[0221] 4. Conclusion: Therefore, it can be concluded that the local optimal solution of the optimization problem corresponds to the stable equivalent point of the AdTJU method, and there is a one-to-one correspondence between the two.

[0222] In summary, theoretical analysis provides a new perspective for understanding the optimization process of deep neural networks. By transforming the optimization problem into a dynamical system, stability and convergence analysis from dynamical system theory can be used to guide the training of neural networks, thereby obtaining more stable and reliable models. This has significant theoretical and practical implications for solving key challenges in deep learning, such as training instability and overfitting.

[0223] The AdTJU method, through its two-stage training strategy, not only enhances the model's ability to learn from imbalanced datasets but also ensures the robustness and convergence of the training process. The established equivalence relationship between stable equilibrium points and local optima further solidifies the effectiveness of the method, laying a solid foundation for future research and applications in the field of deep learning optimization.

[0224] To verify the effectiveness of the proposed improved Kolmogorov-Arnold Network (KAN) deep learning framework, comprehensive experiments were conducted on multiple public datasets, including CIFAR-10 and CIFAR-100. Furthermore, to evaluate the framework's performance in cancer applications, particularly colorectal cancer, training and testing were performed on the lung and colon cancer histopathological images (LC25000) dataset. These experiments aimed to comprehensively evaluate the performance of the proposed framework and compare it with other common optimization methods. Testing was conducted on a single machine: an Intel CPU core (2.67 GHz) and a GeForce RTX 2080 Ti graphics card. Experiments were performed using Python and its GPU plugin. The performance of the proposed improved Kolmogorov-Arnold Network (KAN) deep learning framework in the aforementioned tests will be presented below. The experimental results of the proposed deep learning framework are analyzed in the following section.

[0225] 1) Performance on the standard CIFAR-10 and CIFAR-100 datasets

[0226] To demonstrate the high performance of the proposed improved Kolmogorov-Arnold network (KAN) deep learning framework, it was compared with widely used optimization methods such as Adam, SGD, and TJU. All optimization methods were trained and tested on the standard CIFAR-10 and CIFAR-100 datasets using KAN networks with a batch size of 128. Results are as follows: Figure 2 , Figure 3 As shown.

[0227] Figure 2 and Figure 3 The loss and accuracy curves of different optimization methods under the KAN network are presented. The results show that, compared with other common optimization methods, the improved Kolmogorov-Arnold Network (KAN) deep learning framework achieves faster training speed and significantly reduces the error rate on the CIFAR-10 dataset. To visualize the feature extraction process of different methods in the KAN network, t-distributed random neighborhood embedding (t-SNE) is used to reduce the feature dimension of the model's global average pooling layer. t-SNE is a non-linear dimensionality reduction algorithm that maps high-dimensional features to a low-dimensional space, allowing observation of the distribution of different categories and the shape of the classification boundary.

[0228] Figure 4The results shown visualize the Adam, SGD, TJU, and AdTJU methods on the CIFAR-10 dataset. It is evident that the Adam and SGD optimization methods exhibit significant overlap between different categories, indicating that these optimization techniques struggle to effectively learn features across different categories within the KAN network framework. The TJU method improves upon clustering, but significant overlap between categories persists. In contrast, AdTJU demonstrates the best feature clustering and separation capabilities among all optimization methods, exhibiting superior clustering ability and classification performance. The visualization results for AdTJU demonstrate that the improved Kolmogorov-Arnold Network (KAN) deep learning framework is more efficient than the Adam, SGD, and TJU optimization methods in extracting features from different categories. By incorporating the loss function into the gradient flow, the framework of this invention significantly improves convergence speed and quality, facilitating more efficient convergence to higher-quality local optima.

[0229] 2) Performance on the Lung and Colon Cancer Histopathological Images (LC25000) dataset

[0230] Traditional pathological diagnosis relies on the subjective judgment of specialist physicians, which is often influenced by their experience and training. With advancements in medical imaging technology, digitization has made the storage, transmission, and sharing of pathological images much easier. The digitization of tissue sections (digital pathology) is becoming increasingly prevalent, which will drive the development of computational pathology, including large-scale whole-slice image (WSI) datasets. The development and evaluation of computer-aided diagnostic (CAD) systems play a crucial role in facilitating the construction and use of large-scale datasets. Therefore, researchers urgently need reliable and well-annotated tumor tissue image datasets. This need has led to the creation of the Lung and Colon Cancer Histopathological Images (LC25000) dataset, which not only provides a valuable resource for lung and colorectal cancer research but also lays the foundation for advancing the application of medical image analysis and artificial intelligence technologies in clinical practice.

[0231] Figure 5 The dataset displays the categories and seven random images from each category. The dataset contains 25,000 histopathological images, divided into five categories, with 5,000 images in each category. The training set contains 20,000 images, and the test set contains 5,000 images, including benign lung tissue, lung adenocarcinoma, lung squamous cell carcinoma, colon adenocarcinoma, and benign colon tissue. To increase the training difficulty, the number of images in each category of the original dataset was modified, as shown in Table 1. This table reveals a significant difference in the number of images between different categories, with the largest category accounting for 42% of the total data, while the smallest category accounts for only 7%. This imbalance in data distribution presents additional challenges to the training process.

[0232] Table 1 Data Types and Proportions

[0233]

[0234] The modified LC25000 dataset was trained and tested using the KAN network with Adam, SGD, TJU, and AdTJU methods. Figure 6 As can be seen, when tested on the KAN network, AdTJU converges faster and achieves higher accuracy than other common optimization methods. Therefore, AdTJU performs better and is more suitable for model training on the KAN network, exhibiting a lower error rate and faster training speed. Furthermore, this invention includes visualization experiments of the classification results. All optimization methods employ the t-SNE algorithm for dimensionality reduction, and the visualization results obtained from different optimization methods are shown below. Figure 7 As shown.

[0235] Compared with other optimization methods, Figure 7 The results show that models trained using the AdTJU method can more accurately capture features from different categories, thus reducing information loss. This advantage not only enhances the model's predictive ability but also deepens the understanding of the data's intrinsic structure. Furthermore, for models generated using different optimization methods, the Grad-CAM method was used to generate corresponding heatmaps on the test set, with results as follows: Figure 8 , Figure 9 , Figure 10 and Figure 11 As shown.

[0236] The heatmap generated by the AdTJU method shows that the highlighted areas are concentrated in cancerous tissue, while the highlighted areas in the heatmap generated by the TJU method are more dispersed. In contrast, the highlighted areas in the heatmaps generated by the Adam and SGD methods are less obvious and more scattered. Experimental results show that the heatmap generated by the AdTJU method exhibits stronger and clearer feature saliency, and the AdTJU method can accurately locate the target position.

[0237] Any part of this invention not described in detail can be referred to in the prior art or in the art known to those skilled in the art. This embodiment does not limit such part and will not describe it in detail here.

[0238] The embodiments of the present invention have been described above with reference to the accompanying drawings. However, the present invention is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of the present invention without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of the present invention.

Claims

1. A method for interpretable recognition based on KAN network and adaptive uniform trajectory optimization, characterized in that, The method includes: using an adaptive unified trajectory optimization method that combines adaptive first and second moments with a gradient amplification mechanism to optimize the parameters of an improved Kolmogorov-Arnold deep neural network; during optimization, the pseudo-transient continuation and quotient gradient system modify the traditional gradient flow in a specific way, thereby amplifying smaller gradients and accelerating trajectories away from flat regions or saddle point regions, so as to improve the stability of training and reduce the possibility of getting trapped in local minima. The gradient flow formula is modified as follows: （14）； wherein, is a loss function; A pseudo-time continuation method is introduced during deep neural network training to quickly determine the steady-state solution of the differential equation; an adaptive update technique with switching evolutionary relaxation is also used to update the pseudo-time step, and the pseudo-time step increases rapidly as the system trajectory approaches the stable equilibrium point. The pseudo-time continuation method is represented as follows: （18）； wherein denotes a pseudo-time step, is the identity matrix, denotes a weight, represents a nonlinear system, represents the Jacobian matrix of the nonlinear system; The expression for the adaptive update technique of switching evolutionary relaxation is: （19）。 2.The KAN network and adaptive unified trajectory optimization interpretable identification method based on KAN network and adaptive unified trajectory optimization according to claim 1, wherein, The method includes the following stages: S1. Initialization phase, which includes initializing the parameters of the deep neural network and the momentum estimation parameters; S2, Adaptive Sparse Training Phase: This phase includes dynamically adjusting the sparse connections at each layer of the improved KAN network, including calculating the remaining network connections and randomly adding connections at each layer. S3. Preparation phase: Data processing and system construction for the dataset, including: Divide the dataset x into B batches of equal size. For each batch, compute the nonlinear system and its gradient. Establish a dynamic system and determine the focusing function; S4. Trajectory and Global Convergence Stage: Based on the results and settings of the previous steps, iterative calculations are performed, including: a. After completing one integration step, the model weights are obtained using the pseudo-time continuation method, and the weights are updated. b. When the convergence condition is met, return to step a; when the maximum number of iterations is reached, proceed to the next stage. S5, Local Solution Stage, includes: using the global convergence iteration result of S4 as the initial guess value, and using the local solver strategy to solve for the local optimal solution. 3.The KAN network and adaptive unified trajectory optimization interpretable identification method based on claim 2, characterized in that, The parameters of the deep neural network initialized in step S1 include the number of model layers, function matrix, sparsity, pruning rate, and number of training rounds; The momentum estimation parameters include the initial model weights and initial step size of the improved KAN network. The initial step size is calculated as follows: 。 4. The interpretable recognition method based on KAN network and adaptive unified trajectory optimization according to claim 3, characterized in that, For a given input vector The output of the improved Kolmogorov-Arnold deep neural network consisting of layers is as follows: （9） wherein is a spline function, is a weight matrix; The weight matrix of which is characterized by is a random graph whose probability function is given by ； wherein, is the number of connections of the neuron, the sparsity degree is controlled by the adaptive sparsity parameter ; regarding the number of neurons in the sparsity layer, ； if and , then represents a linear number of connections; the number of connections in a fully connected layer is quadratic, that is... ; The function matrix is ​​initialized in the following way: ； in Represented as a sparse function matrix, at the end of each training epoch, unimportant connections in each layer are pruned; the remaining connections are... ； in, Non-zero elements are A subset of non-zero elements, which correspond to The largest negative weight and the smallest positive weight in the middle, while Is with Binary matrices of the same size , The weight is a constant; next, an equal number of weights are randomly added to each layer in the following manner: Connection: ； Where r represents the newly added connection number, The non-zero value is exactly The positions of non-zero elements are selected from a random uniform distribution, and their values ​​are set using small Gaussian noise; the following is the modified version. : ； In general, natural selection manifests as the elimination of adaptive sparse connections, while the variation phase in natural evolution manifests as the formation of new connections.

5. The interpretable recognition method based on KAN network and adaptive unified trajectory optimization according to claim 2, characterized in that, In step S4, a mini-batch of samples is extracted from the training dataset in each iteration, and the predicted value for colorectal cancer identification is calculated through forward propagation; then the loss is calculated and the QGS coupling mechanism is applied to adjust the gradient flow according to the current loss value. The optimization and update principle is based on the first and second-order estimates of momentum, ensuring that the learning process can adapt to the ever-changing terrain of the loss function.

6. The interpretable recognition method based on KAN network and adaptive unified trajectory optimization according to claim 2, characterized in that, The convergence condition of the iteration is or , in, It is a positive minimum value. This indicates the maximum number of iterations set for colorectal cancer identification. This indicates the current iteration number.

7. The interpretable recognition method based on KAN network and adaptive unified trajectory optimization according to claim 2, characterized in that, The nonlinear system of the KAN network is as follows: （10）； Where N is the mini-batch size, z is the input data, and x is the weight of ASKAN(z).

8. The interpretable recognition method based on KAN network and adaptive unified trajectory optimization according to claim 2, characterized in that, The dynamic optimization method is as follows: （11）； in, The Jacobian matrix is ​​denoted as , The focusing function is used; this nonlinear dynamic system is a nonlinear and non-hyperbolic dynamic system.

9. The interpretable recognition method based on KAN network and adaptive unified trajectory optimization according to claim 2, characterized in that, In step S3, the adaptive momentum is combined with the enhanced saddle point repulsion force of QGS to establish a continuous dynamic system, as expressed below: （15）； （16）； （17）； In the formula, and These represent continuous-time simulations of the accumulation of first and second moments, respectively. This represents the base learning rate as it changes over time. Representative coefficient, This represents a coupling term.

10. A computer device, comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that the processor executes the computer program to implement the content of the KAN network-based and adaptive unified trajectory optimization interpretable recognition method according to any one of claims 1-9.

Images