Zubov equation-based Sheng differential stable learning framework (Zubov-Net)

Abstract

The invention provides a Zubov equation-based Shenchang differential stable learning framework (Zubov-Net for short), and the problem of inconsistency between a predetermined attraction domain (PROA) and an actual attraction domain (ROA) is solved by integrating a Zubov equation theory into Shenchang differential equation training for the first time. According to the framework, a convex Lyapunov function structure (IACNN) based on input attention and condition Softmax is adopted, triple collaborative loss-Zubov equation consistency loss is designed to force boundary alignment, classification loss ensures track stability, basin intervals between loss improvement categories are separated, meanwhile, boundary points are efficiently sampled through a parallel boundary sampler, and the accuracy of the classification loss is improved. And optimization is carried out through end-to-end joint training. Theoretical analysis proves the dynamic alignment of PROA-ROA, the track authenticability, the non-overlapping category basin and the probabilistic convex separability in the high-dimensional space. PGD attack experiments show that the accuracy of the model can be improved by 1.2-3.8%, the robustness can be improved by 9.5-15.3%, and the adversarial robustness and the prediction performance can be effectively balanced.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of neural ordinary differential equation (Neural ODE) robust learning, and in particular to a stable learning framework fusing Zubov equation theory for efficient balancing of adversarial robustness and prediction accuracy in deep neural networks. BACKGROUND

[0002] In recent years, deep neural networks (DNNs) have achieved breakthrough development in the fields of vision, language processing, and robotics. However, such networks are extremely vulnerable to adversarial attacks, i.e., a small but deliberately designed input perturbation can mislead the model to produce incorrect predictions, which not only seriously affects the accuracy of the model, but also exposes great risks in safety-critical scenarios such as autonomous driving and medical diagnosis. This vulnerability has become an important driving force for the study of adversarial robustness.

[0003] As a continuous dynamical system architecture, Neural ODE utilizes differential equations to simulate data flow, exhibiting stronger robustness than traditional discrete networks. Experiments have shown that Neural ODEs still maintain performance advantages under adversarial attacks (such as PGD or CW) or random noise perturbations, with reasons including non-intersecting continuous paths, flow stability, and gradient suppression effects of numerical solvers.

[0004] To provide formal stability guarantees, multiple studies have applied Lyapunov theory to the training and structure design of neural ODEs, achieving adversarial robustness enhancement by constructing carefully designed Lyapunov functions and imposing stability constraints. However, these methods usually rely on pre-defined positively invariant sets (PROAs) and ignore the possible boundary misalignment between PROAs and the real attractor domain (ROA) of the system, resulting in an inability to balance model accuracy and robustness.

[0005] On the other hand, the data-driven paradigm with each "clean input" as an equilibrium point avoids the explicit definition of PROA by directly aligning the convergence paths of samples and their perturbations. However, this method ignores the attractor structure at the class level, leading to possible overlap between attractor regions of different classes, affecting classification performance and adversarial protection.

[0006] Zubov's stability theory can obtain precise ROA boundaries for nonlinear dynamical systems by solving specific partial differential equations. Existing research has applied it to ROA verification and Lyapunov function learning for neural network systems and proven its scalability in large-scale high-dimensional systems. However, there is currently no research applying Zubov theory to the robust training of Neural ODEs to dynamically coordinate the structural differences between PROA and actual ROA.

[0007] In summary, the existing Neural ODE robust learning method has the following shortcomings: first, the stability constraint depends on the accuracy of the preset attractor domain and the definition region; second, there is a lack of PROA and ROA alignment mechanism; third, the attractor structure between classes is not fully separated; fourth, the formal constraints on high-dimensional complex systems are difficult to balance precision and robustness. Therefore, there is an urgent need for a method that takes both theory and practice into account to achieve dynamic consistency of PROA and ROA and improve the overall adversarial performance of the system. SUMMARY

[0008] In order to achieve the above-mentioned purpose of the application, the application proposes a Neural ODE stable learning framework based on Zubov equation (Zubov-Net), which takes the consistency of the pre-defined attractor domain (PROA) defined by the Lyapunov sub-horizon set and the real attractor domain (ROA) of the system as the core, and optimizes the Neural ODE dynamic parameters θ f and the learnable Lyapunov function parameters θ W , and introduces a triple synergistic loss structure and a parallel boundary sampler to dynamically align the boundaries of PROA and ROA, achieving double improvement of robustness and prediction accuracy. BRIEF DESCRIPTION OF DRAWINGS

[0009] Figure 1 is a model structure diagram of the Neural Ordinary Differential Equation stable learning framework Zubov-Net based on Zubov equation of the application. DETAILED DESCRIPTION

[0010] 1. Design of learnable convex Lyapunov function (IACNN)

[0011] Define the Lyapunov function V(x, c; θ V ) constructed by the input attention condition Softmax network IACNN, aiming at the class anchor point set Let

[0012]

[0013] where σ is a positive convex non-decreasing function; η>0 is a constant; g is a k-layer IACNN forward mapping defined by formulas (2)-(4), which guarantees strong convexity and unique minimum for each c.

[0014]

[0015] g(x, c)=z k , u0=c, z0=x. (4)

[0016] 2. Zubov consistency loss L con

[0017] Sampling point at the boundary between PROA and ROA Above, calculate the local consistency deviation:

[0018]

[0019] The global consistency loss is defined as:

[0020]

[0021] This loss constrains the PROA bound to align with the ROA bound defined by the Zubov equation.

[0022] 3. Classification loss L cla

[0023] Given an input sample The final state obtained by neural ODE dynamics evolution after time TT is denoted as h x (T), then the output function is defined as:

[0024] W(h x (T))=[W1(h x (T)),…,W L (h x (T))] T (7)

[0025] Where W i (·) represents the Lyapunov function value of the i-th category.

[0026] Define a classification function based on this vector

[0027]

[0028] Among them, W(h x (T))° -1 Represents the vector W(h x (T)) takes the inverse of each element, ɑ∈[0,1] is the stability boundary parameter, is a vector of all 1s.

[0029] Finally, the classification loss is defined as the cross entropy:

[0030]

[0031] here Represents the probability component of the predicted label y.

[0032] 4. Boundary separation loss L sep

[0033] Boundary sampling points set of each class PROA Above, separation penalty is performed on all point pairs between two classes:

[0034]

[0035] where W i β (h) = 1 - exp(-βW i (h)) is the transformed Lyapunov function, 0 < β < 1, to enhance the separability of class boundaries.

[0036] 5. Joint training objective function

[0037] The above loss definitions are integrated to form the joint optimization objective:

[0038] L(θ f ,θ W ) = λ1L cla + λ2L con + λ3L sep (11)

[0039] The gradient descent optimization solution is:

[0040]

[0041] 6. Effect and innovation point

[0042] Boundary alignment: L con = 0 to achieve accurate fitting of PROA and ROA and ensure no intersection between class attraction basins;

[0043] Trajectory authentication: L cla = 0 to ensure that the trajectory of the data-driven system is always located in the authenticable attraction domain;

[0044] Non-overlapping between classes: Minimize L sep to ensure the original separation between class attraction basins;

[0045] High-dimensional convex separable: IACNN provides a class convex separable attraction structure with probability guarantee in high-dimensional space.

Claims

1. A neural ordinary differential stable learning framework based on the Zubov equation (Zubov-Net), characterized by: The following steps are involved: S1. Construct a trainable neural Lyapunov function network to adaptively learn PROA (predetermined region of attraction); S2. Construct the consistency loss function L derived based on the Zubov equation con , used to dynamically align PROA with the true region of attraction (ROA) obtained from Neural ODE; S3. Use the boundary parallel sampler to sample training points from the PROA boundary simultaneously; S4. Joint use of consistency loss L con , stability classification loss L cla and boundary separation loss L sep , the Neural ODE model and Lyapunov function network are trained end-to-end to achieve PROA-ROA alignment, trajectory stability and category basin separation.

2. The frame according to claim 1, characterized in that Consistency loss L con The definition includes: ●Use this deviation to achieve dynamic consistency between PROA and ROA boundaries. ●Calculate the local consistency variance based on the Zubov equation: Calculate global consistency loss based on robust optimization:

3. The frame according to claim 1, characterized in that The parallel boundary sampler includes sampling boundary samples from Lyapunov sublevel sets (PROA).

4. The frame according to claim 1, characterized in that The classification loss is used to ensure that: The Neural ODE trajectory falls within the certified ROA. The basin of attraction defined by the Lyapunov function is consistent with the training data labels.

5. The frame according to any one of claims 1 to 5, characterized in that The described Lyapunov function adopts the input attention based convex neural network (IACNN) architecture with conditional softmax and strong convexity to construct differentiable basins of attraction.

6. The frame according to any one of claims 1 to 6, characterized in that The joint training uses an end-to-end approach to synchronously optimize Neural ODE parameters and Lyapunov function parameters.

7. The frame according to any one of claims 1 to 7, characterized in that By proving the convex separability of Lyapunov functions in high-dimensional space, we provide the probabilistic separability guarantee for the classification basin of attraction of neural Lyapunov functions for the first time.

8. The frame according to any one of claims 1 to 8, characterized in that The consistency loss, consistency loss, classification loss, and separation loss constitute the final training objective in the form of a weighted sum, and training consists of minimizing this weighted loss combination.

9. A stable learning method based on the framework according to any one of claims 1 to 9, characterized in that: include: S1. Initialize the Neural ODE model and Lyapunov function (IACNN); S2. Use the parallel boundary sampler to sample the boundary points of PROA; S3. Calculate the consistency loss, classification loss, and separation loss of the Zubov equation; S4. Synthesize the final loss function and use the gradient descent algorithm to update the Neural ODE and Lyapunov function parameters; S5. Repeat S1 to S4 until the training converges.

10. The method according to any one of claims 10, characterized in that During the training process, the Lyapunov function parameters automatically focus on important feature dimensions related to various equilibrium points through the conditional Softmax input attention mechanism.

Images