Fixed-time stable learning framework of Shenchang differential equation

Abstract

The invention relates to the technical field of big data modeling, and discloses a fixed-time stable learning framework of a Shenchang differential equation, the framework designs loss (FxTS-loss) based on a fixed-time stable Lyapunov function, and encourages dynamic reasoning to converge to a correctly predicted state within fixed time defined by a user. The invention provides a method for constructing a Lyapunov function by utilizing supervision information in a training process. The method can meet the requirements of different tasks and network structures. In order to optimize the FxTS-Loss, the invention also provides a learning algorithm, which comprises capturing sample points in a key area to approximate the FxTS-Loss, which is beneficial to the local-global structure of the Lyapunov condition. According to the method, it is proved that the minimum FxTS-loss not only ensures the fixed-time stable behavior of dynamics, but also ensures the robustness of input disturbance. Experimental results show that the FxTS-Net has better prediction performance, higher reasoning dynamic convergence speed and better adversarial robustness.

Description

Technical Field

[0001] The present invention relates to the technical field of big data modeling, and in particular to a fixed-time stable learning framework for neural ordinary differential equations. Background Art

[0002] Neural ordinary differential equations (Neural ODEs) have become a very promising research field, making significant progress in modeling complex large-scale datasets through continuous dynamical systems. Originating from the reinterpretation of residual networks as continuous-time dynamical systems, Neural ODEs extend the ability of neural networks to model continuous dynamics and address the limitations of discrete architectures. By combining techniques from dynamical systems theory, Neural ODEs are able to model data continuously and show wide applicability in various fields, including irregular time series modeling, generative modeling, and financial analysis.

[0003] However, neural ODEs still face several limitations, especially in ensuring the stability of the learned system. Standard learning methods rely on differentiating through ODE solutions using techniques such as adjoint methods, striving to guarantee stable behavior (informally, the tendency of the system to remain in some invariant bounded set). Such instabilities lead to slow convergence and inaccurate predictions. More worrisome, dynamical systems that lack stability guarantees can lead to significant output distortions when subjected to small input perturbations.

[0004] In recent years, some studies have applied Lyapunov stability theory to enhance the stability of neural ordinary differential equations. However, these methods exacerbate the trade-off between stability and accuracy, hindering their applicability in many real-world scenarios. Constructing suitable Lyapunov functions remains a long-standing challenge in the field of dynamical systems and control. Therefore, it is necessary to further study and enrich the techniques for constructing Lyapunov functions to meet the requirements of complex network structures and diverse tasks in practical applications.

[0005] Despite recent progress in enhancing the stability of neural ordinary differential equations, most existing methods focus on exponential convergence and ignore the critical aspect of fixed-time stability. Fixed-time stability is essential in many real-world applications, where the system must reliably reach a desired state within a user-defined evolution time. However, without fixed-time guarantees, it is difficult for these methods to ensure reliable performance within a predefined time frame, which limits their practical applicability.

[0006] Therefore, it is necessary to provide a fixed-time stable learning framework for neural ordinary differential equations to solve the above technical problems. Summary of the invention

[0007] The present invention provides a fixed-time stable learning framework for neural ordinary differential equations, aiming to improve prediction performance, accelerate the speed of dynamic convergence of reasoning and enhance adversarial robustness.

[0008] The fixed-time stable learning framework for the neural ordinary differential equation provided by the present invention includes:

[0009] A method for constructing Lyapunov functions using supervised information during training, which can be adapted to the requirements of different tasks and network structures;

[0010] A loss designed based on a fixed-time stable Lyapunov function, namely FxTS-loss, is used to encourage dynamic inference to converge to the correctly predicted state within a user-defined fixed time;

[0011] A robust learning algorithm for optimizing FxTS-loss, including capturing sample points in key regions to approximate FxTS-loss, to benefit from the local-global structure of Lyapunov conditions;

[0012] Provable adversarial robustness is obtained by minimizing FxTS-Loss, which utilizes robust control based on Lyapunov stability theory to fundamentally ensure that the system remains stable when perturbed.

[0013] Preferably, the Lyapunov function is based on finding the parameterization of the model by minimizing the supervised loss on the training data.

[0014] Preferably, the learning algorithm includes a simulated perturbation sampling method, which simulates input perturbations by perturbing extracted features, generates perturbed integral trajectories, and captures sampling points in key areas on the perturbed integral trajectories to approximate FxTS-loss.

[0015] Preferably, by optimizing FxTS-Loss to make the loss zero, the learned neural ordinary differential equation stabilizes to the optimal state point within a predefined fixed time, where the supervision loss is minimized.

[0016] Preferably, the method for constructing a Lyapunov function includes the following steps: defining the form of the Lyapunov function; determining the ideal convergence point of the dynamical system by minimizing the supervised loss; constructing the Lyapunov function based on the supervised loss and the ideal convergence point; selecting an initial point; using a numerical method (such as a multi-step gradient projection method) to solve the Lyapunov function starting from the initial point; and ensuring that the constructed Lyapunov function is continuously differentiable to meet theoretical requirements.

[0017] Compared with the related art, the fixed-time stable learning framework for neural ordinary differential equations provided by the present invention has the following beneficial effects:

[0018] FxTS-Net achieves the correct prediction state within a predefined fixed time, thus effectively improving the prediction accuracy;

[0019] FxTS-Net is more robust to small perturbations of the input and improves its robustness to random noise image perturbations;

[0020] The fixed-time stable evolutionary behavior improves the decision boundary of the model and alleviates the boundary fuzziness problem;

[0021] By generalizing the FxTS Lyapunov conditions to bounded non-decreasing perturbation systems, we prove that minimizing the FxTS loss guarantees the FxTS behavior of the dynamics and achieve new robustness guarantees. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 Comparison chart of the inference time and prediction loss between FxTS-Net and the standard Neural ODE in the present invention: (a) Normal Samples, (b) Adversarial Samples;

[0023] Figure 2 A method architecture diagram of a fixed-time stable learning framework for neural ordinary differential equations proposed by the present invention;

[0024] Figure 3 TSNE visualization results of the neural ordinary differential equation and FxTS-Net classification features using CIFAR10 in the present invention (400 images are randomly selected for each category): (a) FGSM attack perturbation, (b) Gaussian noise perturbation, and (c) normal data. DETAILED DESCRIPTION

[0025] Next, the present invention will explore the fixed-time stability of the Neural Ordinary Differential Equation from both theoretical and experimental aspects.

[0026] 1. Research motivation behind FxTS-Net

[0027] To illustrate the research motivation behind FxTS-Net, the present invention Figure 1 FxTS-Net is compared to a standard Neural ODE in Figure 2. As can be seen from the orange line and region, the dynamics of the Neural ODE in state space do not guarantee stable convergence to the correct prediction within the maximum time set for training. This instability leads to slower convergence and incorrect predictions under input perturbations.

[0028] like Figure 1: We compare learning dynamics on 1000 test examples of CIFAR-100 by plotting inference time versus prediction loss. The orange line represents the standard Neural ODE, the green line represents FxTS-Net, and the corresponding light-colored areas represent the fluctuating areas in the samples. (a): Experiments on Normal Samples show that FxTS-Net has faster convergence speed and more stable evolutionary behavior than Neural ODEs. (b): Experiments on Adversarial Samples show that Neural ODEs without fixed-time stability can cause severe distortions even under small input perturbations.

[0029] In this work, we study how to learn fixed-time stable dynamical systems using Lyapunov stability theory, which can reliably and quickly converge to correct predictions within a user-defined fixed time, even under input perturbations. The contributions of this paper are as follows:

[0030] (1) In order to ensure that the learned neural ordinary differential equations reach the correct prediction state within a fixed time defined by the user, the present invention proposes FxTS-Net, which introduces FxTS-Loss to enable the learning of dynamic systems to have stable dynamic reasoning in a fixed time.

[0031] (2) For various tasks and network structures, the present invention proposes a method for constructing a Lyapunov function by embedding a sub-optimization problem in training to construct it using supervised information.

[0032] (3) In order to optimize FxTS-Loss, the present invention proposes a practical algorithm, including a simulated perturbation sampling method, that is, simulating the input perturbation by perturbation extracted features to generate a perturbed integral trajectory, which captures the sampling points in the key area to approximate FxTS-Loss.

[0033] (4) Theoretically, the present invention proves that minimizing FxTS-loss will ensure that the dynamical system satisfies the FxTS Lyapunov condition, and achieves new adversarial robustness guarantees by extending the FxTS Lyapunov condition to the case of bounded non-decreasing perturbations.

[0034] (5) Experimental results show that this method can effectively control the dynamic stability of the system at a fixed time, has good prediction accuracy, and improves the adversarial robustness of the system, which is consistent with the theoretical analysis.

[0035] 2. The research background and related work of the present invention are described below:

[0036] 2.1. Supervised Learning of Neural Ordinary Differential Equations

[0037] Neural ODEs: Based on previous research work, this paper will consider a class of data-controlled Neural ODEs. Let (x, y) represent the input data pair, where x belongs to Its mathematical model is:

[0038] x c =φ(x;θ φ ), (2-1)

[0039]

[0040] in The parameter is θ ψ The output function of φ is similarly the input function. The idea of ​​Neural ODEs is to use a neural network to parameterize a differential equation that governs the hidden state with respect to time t.

[0041] In practice, the present invention sets equation (2-2) to evolve in the time interval [0,1], that is, the upper limit of the integral t = 1, although in theory any time can be selected to predict t. In addition, the present invention assumes that the state space H is bounded and path-connected.

[0042] Assumptions of the model: The present invention imposes the Lipschitz continuity assumption on the neural network used in the neural ordinary differential equations as shown below.

[0043] Assumption 2.1: For any neural network The parameter is θ F , there exists a positive constant L F > 0, so that for all t ≥ 0 and

[0044] |F(t,x;θ F )-F(t,x';θ F )|≤L F |x-x'|. (2-4)

[0045] This assumption is not overly onerous, since neural networks with activation functions such as tanh, ReLU, and sigmoid usually satisfy the Lipschitz continuity condition in Assumption 2.1.

[0046] Supervised Learning: Consider a standard supervised learning setting, where the training set of input / output pairs is denoted as (x, y) ~ D. The goal is to find the parameterization of the proposed model by minimizing the supervised loss on the training data:

[0047]

[0048] in, is a shorthand for equations (2-1) to (2-3). The learning goal is to find the parameterization of the model by minimizing the supervised loss on the training data.

[0049] As is typical in deep learning, the standard way to train Neural ODEs is to back-propagate via Equation (2-5). In addition, the end-to-end training optimization problem can be viewed as the following optimal control problem (for simplicity, only a single (x,y) is used), given by:

[0050]

[0051] Equation (2-6) can be optimized and can be derived from the dynamics using solver backpropagation or adjoint methods. However, in Equation (2-6), there is no explicit penalty or regularization to ensure that the dynamic system reaches a steady state within a user-defined fixed time. In fact, the present invention can be derived from Figure 1 Such a problem is observed in

[15] , where the dynamics of the Neural ODEs are not guaranteed to be stable in fixed time, leading to Neural ODEs learning a brittle solution. In this work, we propose the FxTS-Net method to address these limitations via a fixed-time control-theoretic learning objective.

[0052] 2.2. Lyapunov conditions for timing stability

[0053] Intuitively, a time-stable dynamical system means that all solutions in the region around an equilibrium point flow to that point at a prescribed fixed time. The field of Lyapunov theory has established various connections between stability and descent based on a specific type of function called a Lyapunov function. In this work, the present inventors assume that the equilibrium point is h * .

[0054] Definition 2.1 (Lyapunov function). A continuously differentiable function V:H→R is a Lyapunov function if V is a positive definite function, that is, when h≠h * and V(H * )=0, V(H)>0.

[0055] The present invention can now define fixed-time stability with respect to V. The concept of fixed-time stability (FxTS), defined as follows, allows the settling time to remain consistently bounded under all initial conditions.

[0056] Definition 2.2 (FxTS). If there exists a Lyapunov function V and a constant T max >0, then the ordinary differential equations defined by equations (2-1) to (2-3) are fixed-time stable, so that all solution trajectories h(t) of the ordinary differential equations satisfy:

[0057] sup{T≥0:V(h(t))=0 for all t≥T}≤T max (2-7)

[0058] FxTS means that the dynamics converges quickly to a state with minimum V within a fixed time of control. Later, the present invention will use supervision information to instantiate V. This is desirable because: 1) it guarantees fast convergence to the desired state (defined by V) after a fixed time of integration (e.g., for t∈[0,1]); 2) it has adversarial robustness implications, which will be discussed later.

[0059] In order to ensure the stability of fixed time, the present invention introduces the FxTS control Lyapunov function, which is used to encode the system trajectory to converge to the equilibrium point within a user-defined fixed time, as shown in the following theorem.

[0060] Theorem 2.1 (FxTS-CLFs). For the ordinary differential equations in formulas (2-1), (2-2), and (2-3), if α1, α2>0, When μ>1, a continuously differentiable Lyapunov function V is a FxTS-CLF, for example:

[0061]

[0062] This holds for all h∈H,t∈[0,1]. The existence of FxTS-CLF implies that there is a θ∈Θ that can achieve:

[0063]

[0064] And the convergence time T satisfies

[0065] In other words, the present invention requires that the ODEs satisfy the additional structure specified by equations (2-8) and (2-9) in order for the ODEs to be fixed-time stable with respect to V. In Section 3, the present invention will develop a learning framework that encourages finding parameters θ that satisfy equations (2-9).

[0066] 2.3. Learning stable dynamics

[0067] For learning dynamical systems, most of the previous work focuses on using adjoint methods to infer the dynamics, but they cannot guarantee that the learned dynamical system has stable properties. How to avoid fragile solutions and obtain strong stability guarantees is a challenge in training neural networks.

[0068] In recent years, many researchers have conducted in-depth studies on how to enhance the stability of neural ordinary differential equations through different formulations. Yan et al. proposed TisODE based on time-invariant and steady-state conditions to regularize the flow of perturbed data, which is more robust than ordinary neural ordinary differential equations. Zakwan et al. improved the robustness of neural ordinary differential equations to Gaussian noise and impulse noise on the MNIST dataset by introducing contraction theory, and provided exponential convergence of solutions. In addition, Kang et al. imposed Lyapunov stability guarantees around the equilibrium point to eliminate the effects of perturbations in the input.

[0069] We observe that most of the above works focus on learning dynamical systems with stability guarantees, but they are limited to exponential stability. Results show that by significantly increasing the evolution time, robustness to adversarial examples is improved. Therefore, it is crucial to guarantee the stability of the dynamical system solution within a user-defined fixed time. In this work, we propose a fixed-time stable learning framework that allows the learned dynamical system to be stable within a predefined fixed time.

[0070] 3. The fixed-time stable learning framework proposed by the present invention is described below:

[0071] The intent of the proposed method is straightforward: as described in Section 2.2, the goal of the present invention is to find the parameters (θ ) of the Neural ODE that satisfy the fixed-time stable Lyapunov stability condition in Lemma 2.1. φ ,θ f ,θ ψ ), which is used to construct the Lyapunov function V.

[0072] The present invention develops the formula in two steps:

[0073] 1. In Section 3.1, for any given supervised loss, we construct an appropriate Lyapunov function from the supervised information.

[0074] 2. In Section 3.2, based on V, the present invention defines FxTS-Loss, which describes the degree of violation of the FxTS contraction condition in equation (2-9). Then by optimizing FxTS-Loss, it forces the model to learn stable evolutionary behavior within a predefined fixed time.

[0075] Theoretically, the present invention proves that by optimizing FxTS-Loss, if the loss is zero, then it means that the learned Neural ODE stabilizes to the optimal state point in a predefined fixed time, where the supervision loss is minimal (Theorem 3.1). At the same time, this also enables the present invention to obtain a new adversarial robustness guarantee (Theorem 3.2). The method architecture is as follows Figure 2 As shown, the corresponding learning algorithm is described in Section 3.3.

[0076] 3.1. Constructing Lyapunov functions using supervised information

[0077] As we all know, the construction of Lyapunov functions has always been a challenge in the field of dynamical systems and control. In this section, we use supervised information to construct Lyapunov functions for practical applications with arbitrary output network structures and learning tasks.

[0078] As mentioned before, V needs to be positive definite, but in addition, V needs to be * Otherwise, if it has a local optimum far away from h * The local optimum of , then theoretically the dynamics may be stuck at that position. Therefore, the key to constructing the Lyapunov function is to find a suitable h * , then V can be engraved into h * is defined by the distance.

[0079] The training optimization problem of Neural ODEs is reviewed, which implies that the desired dynamical system converges to the optimal state h * , and the supervision loss is zero. This shows that the present invention can directly optimize the supervision loss to find the ideal convergence point h of the dynamic system. * As an example of a classification task, its supervision loss For a given input-output pair (x, y), the present invention defines h * as follows:

[0080]

[0081] Where ψ is the output layer of equation (2-3). An additional technical requirement is the continuous differentiability of ψ, which is satisfied by typical network structures. According to h * , define the Lyapunov function V y for:

[0082]

[0083] like Figure 2 As shown, for the solution h(·) of formula (2-1), (2-2), (2-3), the present invention sets h(T) as the initial point and uses the multi-step gradient projection method to calculate h * It is worth noting that the present invention has experimentally found that choosing h(T) as the initial point is crucial. The reason is that the supervised loss may exist in multiple local optima, so choosing h(T) can find the h with the best correlation with the dynamic system. * .

[0084] 3.2 Fixed-time stable Lyapunov loss

[0085] In this section, the present invention will use the Lyapunov function V in Section 3 to impose stability on the Neural ODE, i.e., define the fixed-time stable Lyapunov loss (FxTS-Loss) using the stability condition in formula (2-9). For clarity, the present invention starts with the point-wise version.

[0086] Definition 3.1 (Point-wise FxTS-Loss). For a single input-output pair (x, y), the Lyapunov function Vy:H→R≥0 in equation (3-2). The point-wise FxTS-Loss is:

[0087]

[0088] where h∈H,α1>0,

[0089] Essentially, Definition 3.1 describes the extent to which the FxTS contraction condition is violated, which depends only on the local state h, not the entire trajectory. This can be interpreted as a local invariance. Lemma 2.1 exploits the local-to-global structure, so establishing an invariance based on local contraction implies global stability. That is, for all H and t∈[0,1], if V is zero, this means that the inference dynamics will converge to a loss-minimizing prediction in a user-defined fixed time.

[0090] Definition 3.2 (FxTS-Loss). For the dynamic solution h(·) defined in Equation 1-3, that is, the data set of input-output pairs (x, y) ~ D and V in Equation 12, FxTS-Loss is:

[0091]

[0092] Theorem 3.1. Consider the setting in Definition 3.2. If there exists a parameter θ of the dynamical system * ∈Θ makes Then for each (x,y)~D,:

[0093] 1. For t∈[0,1], the inference dynamics system satisfies V(x,y,h(t),t)=0.

[0094] 2. All solution trajectories h of the dynamic system defined in equations 1 to 3 x (t)Satisfy:

[0095]

[0096] where h * Defined in formula (3-1).

[0097] Definition 3.2 constructs FxTS-Loss only on the coupled distribution {(h(t), t)|t∈[0,1]}, instead of constructing FxTS-Loss on all h and t∈[0,1]. Theorem 3.1 proves that the learned neural ordinary differential equation is time-stability by minimizing FxTS-Loss.

[0098] 3.3 Robust Learning Algorithm

[0099] As discussed in Section 3.2, for the FxTS stability condition, Definition 3.1 only emphasizes its local invariance. However, it is necessary to use Theorem 3.1 to establish an invariance based on local contraction, which implies global stability, which requires choosing a suitable H and ensuring that V is zero on H×[0,1]. In Theorem 3.1, the present invention achieves convergence to the state h in constant time by minimizing the FxTS-Loss constructed only on {(h(t),t)|t∈[0,1]}. * However, this stability relies on the integrated trajectory, resulting in a lack of robustness to input perturbations.

[0100] Therefore, the difficulty in designing a robust learning algorithm lies in the calculation of the integral in formula (3-4). It involves two challenges:

[0101] |How to choose a suitable H to make the learned neural ordinary differential equations robust.

[0102] How to collect sample points in H×[0,1] to approximate FxTS-Loss when computing resources are limited.

[0103] In order to overcome the above difficulties, the present invention designs a simulated perturbation sampling method. Figure 1 As shown in the state space visualization, the present invention simulates the input disturbance through the disturbance extraction feature, generates the integral trajectory after the disturbance (the orange curve in the figure), and samples the integral trajectory after the disturbance. Algorithm 2.1 gives the pseudo code of the robust learning algorithm. This method can effectively collect sample points in the key area of ​​ordinary differential of the neural network, improving the robustness and accuracy.

[0104]

[0105]

[0106] 3.4 Adversarial Robustness

[0107] It is well known that small perturbations at the input of an unstable dynamical system can lead to large distortions in the system output. In this section, we explore the provable adversarial robustness obtained by minimizing FxTS-Loss, which exploits robust control based on Lyapunov stability theory. Robust control fundamentally ensures that the system remains stable when perturbed, a concept that is closely related to the study of adversarial robustness in machine learning.

[0108] First, in order to obtain adversarial robustness guarantees, the present invention develops a more accurate time upper bound estimate for non-vanishing perturbation systems. Specifically, considering the dynamical system in formula (2-1)(2-2)(2-3), for input perturbations, the present invention can use the Lyapunov continuity in Assumption 2.1 to transform the perturbed dynamics into the original dynamics with bounded non-vanishing perturbations.

[0109] Next, the present invention further relaxes the FxTS Lyapunov condition in formula (2-9) by allowing a constant term to appear in the upper bound of the time derivative of the Lyapunov function. Consider the FxTS Lyapunov conditions as follows:

[0110]

[0111] Where α1, α2>0, δ>0, Note that if δ = 0, then Lyapunov condition (3-6) is simplified to equation (2-9). Then the present invention gives the following proposition, which provides an expression for estimating the time upper bound of equation (3-6).

[0112] Proposition 3.1. Let V0,α1,α2,δ,v>0,γ>1, such that set up

[0113]

[0114] Then for all The following conclusions are drawn:

[0115] 1. For 1<μ<2, we have

[0116]

[0117] 2. For 2≤μ<3, we have

[0118]

[0119] in

[0120] 3. For μ ≥ 3, we have

[0121]

[0122] in

[0123] It is worth mentioning that Proposition 3.1 has an upper bound and boundary on time I Both provide more accurate estimates. In fact, some people scale Equation (3-6) by multiplying V by δ to solve the integration failure caused by the introduction of the constant term. This means that the convergence bounds also need to satisfy In addition, multiplying by V is not the best choice for scaling. In Proposition 3.1, the present invention multiplies To obtain a more precise estimate, and by introducing v 1-μ To eliminate the condition of V>1. Figure 3 In , the present invention visualizes different scaling techniques, and it can be seen that the scaling technique of the present invention has a tighter effect. It is worth noting that when μ is selected as 2,3, the estimation result of the present invention is accurate and error-free.

[0124] Secondly, the present invention will use Proposition 3.1 to further prove that optimizing FxTS-Loss can obtain robustness guarantee. Let (x, y) be the input-output pair, h x (·) is the solution of the dynamic system defined in equations (2-1), (2-2), and (2-3). Consider is a perturbed version of the input data, such that

[0125]

[0126] Where ρ>0 indicates the degree of disturbance. Assume that the present invention considers the supervised learning task mentioned in Section 2 and uses h x (·) to represent h x A perturbed version of (T).

[0127] Theorem 3.2 Let (x, y) be an input data pair, x be the perturbation version of each, and x and Satisfying formula (3-7), L V, L f, L φ ,L ψ is the Lipschitz constant in Assumption 2.1, And v satisfies α1v μ+1 +α2v μ-1 =L ρ , where L = L V L f L φ . Then for all t≥T we have where T has the following form:

[0128] 1. For 1≤μ≤2, we have

[0129]

[0130] 2. For 2≤μ≤3, we have

[0131]

[0132] 3. For μ ≥ 3, we have

[0133]

[0134] Basically, Theorem 3.2 provides a stabilization of the neighborhood of the origin for the difference between the perturbed output and the label y at a fixed time (the radius of the neighborhood is proportional to the perturbation amplitude). It is worth mentioning that by setting α1 and α2 appropriately in learning, the radius of the neighborhood can be effectively reduced. In contrast, other neural ordinary differential equations that lack this stabilization may produce drastically different solutions from small changes in the input data, resulting in severe performance degradation.

[0135] 4. The following will demonstrate the effectiveness of Jupiter by setting up an experiment based on the experimental structure:

[0136] Experimental setup

[0137] Datasets: The method of the present invention is evaluated on three classification datasets: Fashion-MNIST, CIFAR-10, and CIFAR-100. Fashion-MNIST includes 70,000 grayscale images (28×28) of 10 categories, of which 60,000 are used for training and 10,000 are used for testing. CIFAR-10 and CIFAR-100 consist of 60,000 color images (32×32), 50,000 training images and 10,000 test images, divided into 10 and 100 classes respectively.

[0138] Training details: The implementation of the present invention is based on open source code. In the forward propagation process, the Runge-Kutta method is still used as the numerical solution method. Afterwards, in order to simplify the tuning, the present invention uses Nero to train the model of the present invention. The learning rate of the model trained with LyaNet and the present invention is 0.01, the batch size is 64, and the model trained with conventional back propagation is 128. All models were trained for a total of 120 rounds. In the model of the present invention, the present invention sets the internal learning rate η2=1, the internal iteration N2=3, and the sample n δ =100, time discrete resolution Γ=5.

[0139] Evaluation details: The classification error of the model was evaluated on the Fashion-MNIST, CIFAR-10, and CIFAR-100 test datasets. Next, the robustness was evaluated by two perturbation methods: random noise images and adversarial attacks, and the model was trained on the original non-perturbed dataset. For random noise images, the present invention directly uses four types of noise (Gaussian noise, shot noise, impulse noise, speckle noise, and motion noise). For adversarial attacks, the fast gradient sign method (FGSM), the basic iterative method (BIM), the projected gradient descent method (PGD), and the automatic projected gradient descent method (APGD) can be considered for adversarial robust experiments.

[0140] Traditional convolutional neural network: ResNet-18, whose residual connection can be regarded as an Euler integrated dynamical system.

[0141] Standard Neural ODE: Data-controlled Neural ODE, where ResNet-18 is used as feature extractor.

[0142] Stable Neural ODEs: LyaNet, which guarantees Neural ODEs by learning exponential stability via Lyapunov exponential stability conditions.

[0143] 4.2. Accuracy and robustness against random noise

[0144] In this section, the present invention focuses on the classification error of standard and random noise images within a predetermined evolution time (the maximum evolution time is set to 1). The specific results are shown in Table 1. In terms of standard or nominal test error, the present invention can see that the proposed model (FxTS-Net) improves by 0.04%, 0.63% and 1.41% on Fashion-MNIST, CIFAR-10 and CIFAR-100 datasets, respectively. Compared with the three benchmark models, it shows excellent learning ability. The results show that the fixed-time stable learning framework proposed in the present invention can enable the learned neural ordinary differential equation to reach a state of correct prediction within a predefined fixed time, thereby effectively improving the prediction accuracy.

[0145] In addition, FxTS-Net also achieved overall improvements in random noise data, only 0.04% worse than LyaNet in motion noise of CIFAR10, 0.07% worse in speckle noise of Fashion-MNIST, and 0.88% worse in shooting noise. This fully demonstrates the applicability of FxTS-Net, that is, the model of the present invention is more robust to small perturbations of the input. This also shows that FxTS-Net plays a positive role in helping the neural ordinary differential equations regulate dynamic evolutionary behavior to enhance stability, which is implicitly consistent with the motivation of the present invention. Therefore, the model of the present invention improves the robustness to random noise image perturbations.

[0146] 4.3. Robustness to Adversarial Attacks

[0147] Just like the robustness to random noise images, we also explore the robustness of FxTS-Net and the three models to adversarial attacks within a predefined evolution time. We test each model on Fashion-MNIST, CIFAR-10, and CIFAR-100 images, under the perturbation of FGSM, BIM, PGD, and APGD attacks, using attack radius ∈=8 / 255 and ∈=16 / 255, see Table 2.

[0148] Tahle 2: Robiutness against acversarial attacks with diiferent,attackradii.The best results are shown in bold.Unit: %.

[0149]

[0150] Specifically, on Fashion-MNIST and CIFAR-10, the model of the present invention achieved good results, and was better than the other three types of models. On the CIFAR-100 dataset, FxTS-Net outperformed other models in all perturbations of FGSM, PGD and APGD, and was only slightly lower than LyaNet by 0.85% and 1.21% on BIM. The improvement in robustness shows that the robust learning algorithm designed by the present invention helps to capture sample points in the critical area to approximate the calculation of FxTS-Loss, and is beneficial to the local global structure of the Lyapunov condition. It also proves the importance of stability guarantee of the neural ordinary differential equation within a predefined fixed time, especially the effective approximation of FxTS-Loss leads to greater robustness, which is consistent with the theoretical analysis of the present invention.

[0151] 4.4. Examining the decision boundary

[0152] In this part, the present invention further explores the impact of introducing timing stability guarantee on the decision bound of the learned Neural ODE. The present invention uses FGSM adversarial attack perturbation, Gaussian noise and CIFAR10 standard dataset to perform TSNE clustering analysis on the classification features of Neural ODE and FxTS-Net.

[0153] Table 3: Classification error on original data, motion blur, and FGSM (∈=16 / 255). “Original” denotes the classification error on original data. The best result is in bold.Unit:%.

[0154]

[0155] The results are as follows Figure 3 As shown in Figure 2. On the standard data, compared with the neural ODE, the proposed FxTS-Net has a more obvious separation effect between cluster points of different categories. Especially in the case of Gaussian noise, the decision boundary of the neural ODE is greatly blurred, while FxTSNet still maintains a clear decision boundary.

[0156] The results show that the fixed-time stable evolutionary behavior improves the decision boundary of the model. In addition, FxTS-Net is significantly robust to input noise perturbations, i.e., it alleviates the boundary fuzziness problem.

[0157] In addition, unlike Gaussian noise, FGSM that resists attack perturbations does not blur the decision boundary, but directly confuses the classification results. Compared with the Neural ODE, FxTS-Net still maintains better classification results under adversarial attack perturbations, which greatly supports the method of the present invention. In summary, FxTS-Net exhibits excellent classification ability and decision boundary stability under different data perturbations, proving the effectiveness of introducing timing stability to improve the robustness and classification performance of Neural ODEs.

[0158] 4.5. Parameter Ablation Experiment

[0159] In this section, we designed two ablation experiments, using CIFAR10 as the dataset, to explore the impact of different parameters on FxTS-Net. First, we conducted experiments on the parameters μ, α1, and α2 in the FxTS-Loss function to evaluate their impact on robustness under different conditions. Secondly, we conducted an ablation study on the maximum perturbation radius of the algorithm. These experiments aim to reveal the key role of each parameter in the stability and noise resistance of FxTS-Net and guide the selection of parameters.

[0160] Table 4: Classification error on original data, motion blur, and FGSM (∈=16 / 255). “Original” denotes the classification error on original data. The best result is in bold. “Avg” denotes the average error. Unit: %.

[0161]

[0162] In the first part of the ablation study, the present invention explores the impact of different combinations of μ, α1, and α2 on the model performance. As shown in Table 3, the parameter combination μ = 2.0, α1 = 10.0, α2 = 1.00 has the best effect on the original data and the attack data (including FGSM and motion blur), especially the highest robustness under FGSM attack (38.32%). Compared with α2, the larger the value of α1, the better the error and robustness of the model, while the change of μ between 2.0 and 3.0 has little effect on the final result.

[0163] The second part of the experiment examined the perturbation radius The results in Table 4 show that the smaller (e.g., 0.4) can lead to better performance on clean data, but at the expense of robustness under adversarial conditions. Increasing to 1.2 achieves the best overall performance on both original and adversarial data, with a maximum average accuracy of 26.56%. This suggests that moderate perturbations during training encourage the model to generalize well across different types of inputs, thereby enhancing its robustness. However, for larger maximum values, FxTS-Net exhibits diminishing returns, indicating an optimal range of perturbations for balancing accuracy and robustness.

[0164] In summary, the ablation study highlights the importance of tuning the loss function parameters and the perturbation radius to balance accuracy and robustness. The results show that α1 = 10.0, α2 = 1.00, and The combination of provides the best overall performance, especially in the adversarial setting.

[0165] 5. Through the elaboration of the above contents, the present invention can draw the following conclusions:

[0166] In this work, the present invention explores Neural ODEs with Fixed-Time Stability Guarantee (FxTS-Net), FxTS-Net uses FxTS Lyapunov conditions and FxTS Lyapunov loss to force the inference dynamics to converge to the correctly predicted state within a predetermined fixed time, and the present invention's method is compatible with existing general Neural ODEs without further architectural changes;

[0167] In addition, the present invention also proposes a method for constructing a Lyapunov function using supervisory information during training, which can adapt to the requirements of different tasks and network structures;

[0168] The present invention also provides practical algorithms, including capturing sample points in critical regions to approximate FxTS-Loss, which promotes the local global structure of Lyapunov conditions. In theory, the present invention proves that minimizing FxTS loss guarantees the FxTS behavior of dynamics and achieves new robustness guarantees by generalizing FxTS Lyapunov conditions to bounded non-decreasing perturbation systems. Experiments show that FxtS-Net has good robustness and good classification accuracy, verifying its effectiveness.

[0169] The above descriptions are merely embodiments of the present invention and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A fixed-time stable learning framework for neural ordinary differential equations, characterized in that include: A method for constructing Lyapunov functions using supervised information during training, which can be adapted to the requirements of different tasks and network structures; A loss designed based on a fixed-time stable Lyapunov function, namely FxTS-loss, is used to encourage dynamic inference to converge to the correctly predicted state within a user-defined fixed time; A robust learning algorithm for optimizing FxTS-loss, including capturing sample points in key regions to approximate FxTS-loss, to benefit from the local-global structure of Lyapunov conditions; Provable adversarial robustness is obtained by minimizing FxTS-Loss, which utilizes robust control based on Lyapunov stability theory to fundamentally ensure that the system remains stable when perturbed.

2. The fixed-time stable learning framework for Neural Ordinary Differential Equations as claimed in claim 1, characterized in that The Lyapunov function is based on minimizing the supervised loss on the training data to find the parameterization of the model.

3. The fixed-time stable learning framework for Neural Ordinary Differential Equations as claimed in claim 1, characterized in that The learning algorithm includes a simulated perturbation sampling method, which simulates input perturbations by perturbing extracted features, generates perturbed integral trajectories, and captures sampling points in key areas on the perturbed integral trajectories to approximate FxTS-loss.

4. The fixed-time stable learning framework for Neural Ordinary Differential Equations as claimed in claim 1, characterized in that By optimizing FxTS-Loss to make the loss zero, the learned neural ordinary differential equation stabilizes to the optimal state point in a predefined fixed time, where the supervision loss is minimized.

5. The fixed-time stable learning framework for Neural Ordinary Differential Equations as claimed in claim 1, characterized in that: The method for constructing a Lyapunov function includes the following steps: defining the form of the Lyapunov function; determining the ideal convergence point of the dynamic system by minimizing the supervision loss; constructing the Lyapunov function based on the supervision loss and the ideal convergence point; selecting an initial point; Use numerical methods to solve the Lyapunov function starting from the initial point; ensure that the constructed Lyapunov function is continuously differentiable to meet theoretical requirements.

Images