A volterra integral equation solving method based on attention neural network

Abstract

The application discloses a Volterra integral equation solving method based on an attention neural network, relates to the field of scientific calculation and engineering analysis, and comprises the following steps: discretizing a solving interval of a second type Volterra integral equation to be solved into a plurality of configuration points to obtain a discrete configuration point set; a deep neural network integrating a multi-head self-attention mechanism and a residual connection is constructed; the second type Volterra integral equation is substituted into the deep neural network, the equation is forced to be established on the discrete configuration point set, a numerical integral algorithm is used to calculate integral terms in the equation, and a discrete residual equation is obtained; a smoothing loss function is constructed as a training target; an L-BFGS optimization algorithm is used to iteratively update network parameters, and optimal network parameters are obtained; and the final numerical solution of the second type Volterra integral equation is obtained by substituting the deep neural network. The method can realize continuous and high-precision numerical solution of the second type Volterra integral equation containing a high-oscillation kernel function, and can better adapt to the high-precision numerical solution demand in engineering and scientific calculation.

Description

Technical Field

[0001] This invention relates to the fields of scientific computing and engineering analysis, and more specifically to a method for solving Volterra integral equations based on attention neural networks. Background Technology

[0002] In the fields of scientific computing and engineering analysis, the second type of Volterra integral equation is a core mathematical model for describing nonlocal physical processes such as electromagnetic scattering, heat conduction and fluid dynamics. However, its analytical solution is usually difficult to obtain and must be solved approximatingly by numerical methods.

[0003] Existing methods for solving integral equations based on deep neural networks often employ simple multilayer perceptrons or stacked layers with a few fully connected layers, lacking specific designs for the nonlocal characteristics of integral operators. For example, the two-layer neural network used by Effati et al. could only handle one-dimensional Fredholm equations, while the deep learning method proposed by Guan et al., although scalable to multidimensional problems, only achieved numerical accuracy at the order of magnitude, far from meeting the practical engineering requirements for high-precision solutions. Furthermore, when the kernel function of the integral equation contains high-frequency oscillatory components, traditional neural networks, limited by their limited feature representation capabilities and gradient propagation characteristics, often struggle to capture drastic local changes in the oscillatory function, leading to convergence difficulties or approximate solutions that deviate significantly from the true solution.

[0004] Regarding training strategies, existing methods mostly use mean squared error (MSE) as the loss function. This loss function is overly sensitive to large residuals. When the solution of the integral equation exhibits local peak errors near the boundary or singular points, MSE assigns excessively high penalty weights to outliers, causing the gradient update direction to be dominated by a few outliers, thus compromising the overall convergence stability. Furthermore, common first-order optimizers, when dealing with ill-conditioned loss surfaces caused by integral operators, exhibit slow convergence and are prone to getting trapped in local optima. They fail to fully utilize the second-order curvature information of the loss function to accelerate convergence, further limiting solution accuracy and training efficiency.

[0005] For calculating integral terms, some neural network methods directly employ simple rectangular or trapezoidal methods for discrete approximation. This low-order numerical integration lacks accuracy, especially when the grid points are sparse or the kernel function oscillates violently, introducing significant discrete errors that ultimately contaminate the neural network's training residuals. Furthermore, existing methods typically only output approximate solutions at discrete grid points, failing to directly obtain the function expression defined over the entire continuous interval. When predictions are needed at non-grid points, interpolation must be relied upon, which not only increases computational overhead but may also introduce additional interpolation errors.

[0006] Therefore, how to design a Volterra integral equation solution method based on attention neural networks that can obtain continuously differentiable numerical solutions over the entire solution interval and maintain a stable absolute error magnitude under different oscillation parameters and various kernel function forms is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0007] In view of this, the present invention provides a method for solving Volterra integral equations based on attention neural networks, which aims to solve the problems of low solution accuracy, poor adaptability to oscillating kernel functions, and insufficient optimization convergence efficiency of existing deep learning methods for solving Volterra integral equations of the second kind. It can achieve high-precision, robust, and continuously numerical solutions for Volterra integral equations of the second kind that can be evaluated at arbitrary points, so as to meet the actual needs of high-precision numerical solutions in engineering and scientific computing.

[0008] To achieve the above objectives, the present invention adopts the following technical solution:

[0009] A method for solving Volterra integral equations based on attention neural networks includes the following steps: S1. Discretize the solution interval of the second type of Volterra integral equation to be solved into multiple collocation points to obtain a set of discretized collocation points; S2. Construct a deep neural network integrating multi-head self-attention mechanism and residual connection to approximate the unknown solution of the second type of Volterra integral equation; S3. Substitute the deep neural network into the second type of Volterra integral equation, force the equation to hold on the discretized configuration point set, and use a numerical integration algorithm to calculate the integral term in the equation to obtain the discrete residual equation of the relevant network parameters. S4. Based on the discrete residual equation, construct a smoothing loss function as the training objective; S5. The L-BFGS optimization algorithm is adopted to minimize the smooth loss function, the network parameters are iteratively updated, and the training is terminated by the early stopping strategy to obtain the optimal network parameters. S6. Substitute the optimal network parameters into the deep neural network to obtain the final numerical solution of the second type of Volterra integral equation.

[0010] Preferably, S1 includes: solving the interval Discretize into a uniform grid ,in N+1 represents the total number of configuration points. Let be the current configuration point in the discretized configuration point set, and let be the step size between adjacent configuration points. , where i is the configuration point index.

[0011] Preferably, in S2, the deep neural network sequentially includes: an input layer, a first fully connected layer, a second fully connected layer, a multi-head self-attention layer, a first residual normalization layer, a third fully connected layer, a second residual normalization layer, and an output layer.

[0012] Preferably, the activation functions of the first fully connected layer, the second fully connected layer, and the third fully connected layer are all Tanh functions; the multi-head self-attention layer contains multiple parallel attention heads; the number of hidden layer neurons in the first fully connected layer, the second fully connected layer, and the third fully connected layer is 20; and the network parameters of the deep neural network are initialized using the Xavier uniform initialization method.

[0013] Preferably, in step S3, the numerical integration algorithm is the Gaussian quadrature formula, used to calculate each subinterval. Integral terms on:

[0014] in, A known kernel function for the second kind of Volterra integral equation, used to describe the variables. The interaction relationship between the integral variable t and the integral variable t; Let represent the output function of a deep neural network with input t and parameters θ. For each Gaussian quadrature node mapped onto a subinterval, For the current configuration point in the discretized configuration point set, Find the number of Gaussian product points. To find the product weight.

[0015] Preferably, the Gaussian quadrature node From the reference interval [ The coordinate transformation of the root node of the Legendre polynomial on [1,1] yields: in, For the pre-computed Legendre polynomial root node.

[0016] Preferably, in step S4, the smoothing loss function is the smoothing L1 loss function, expressed as:

[0017]

[0018] Where N is the total number of configuration points in the discretized configuration point set. For the configuration point The discrete residuals obtained at the given point, where β is the smoothing parameter.

[0019] Preferably, in step S5, the termination condition of the early stopping strategy is expressed as follows:

[0020] in, Let t be the termination round of the early stopping strategy, and t be the training round index. For the first The loss value calculated by the round on the test configuration point set. The historical best test loss is represented by Δ, which is a preset improvement threshold. P is the training round index, and P is the preset patience window parameter.

[0021] As can be seen from the above technical solution, compared with the prior art, the technical solution of the present invention has the following beneficial effects: 1. By integrating a deep neural network with a multi-head self-attention mechanism and residual connections, and using the Gaussian quadrature formula to perform accurate numerical calculations of integral terms, this method can effectively capture the high-frequency oscillation characteristics of kernel functions. In engineering problems involving high-oscillation kernel functions, such as electromagnetic field analysis and optical scattering simulation, this method can obtain stable and smooth numerical solutions throughout the entire solution domain, significantly improving the numerical divergence or sharp drop in accuracy that traditional numerical methods are prone to when dealing with oscillating components.

[0022] 2. Using a smooth L1 loss function as the training objective, it has a linear penalty characteristic for local large residuals, such as peak errors that appear near the boundary or singular points of integral equation solutions, avoiding the drawback of outliers dominating gradient updates under quadratic loss functions; in practical problems with multi-scale or non-smooth solutions such as mathematical biology and thermodynamics, it can maintain a stable convergence process and obtain approximate solutions with uniform overall deviation.

[0023] 3. Combining the L-BFGS second-order optimization algorithm with an early stopping strategy based on test loss, it can still quickly approximate the optimal network parameters even under conditions of flat or ill-conditioned loss surfaces. At the same time, by constraining the forced equations on the discretized configuration point set, the numerical solution of the trained neural network is a function defined on a continuous interval, which can be directly evaluated at any point. This feature makes it suitable for engineering scenarios that require frequent updates to the computational domain or real-time evaluation of the solution field without the need for re-discretization or interpolation calculations. Attached Figure Description

[0024] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0025] Figure 1A schematic diagram of a method for solving the Volterra integral equation based on an attention neural network provided in an embodiment of the present invention; Figure 2 A deep neural network structure framework diagram provided for embodiments of the present invention; Figure 3 This is a comparison chart of the actual solution and the neural network solution for the exponential-cosine kernel k=100 provided in an embodiment of the present invention; Figure 4 This is a diagram showing the absolute error distribution of test points with an exponential-cosine kernel of k=100, provided in an embodiment of the present invention. Figure 5 This is a comparison diagram of the actual solution and the neural network solution of the sine-cosine composite kernel k=10 provided in the embodiments of the present invention; Figure 6 This is a diagram showing the absolute error distribution of test points for a sine-cosine composite kernel with k=10, provided in an embodiment of the present invention. Figure 7 The waveform diagram of the kernel function of the sine-cosine composite kernel k=400 provided in the embodiment of the present invention; Figure 8 A comparison diagram of the actual solution and the neural network solution of the sine-cosine composite kernel k=400 provided in the embodiments of the present invention; Figure 9 The absolute error distribution diagram of the test points for a sine-cosine composite kernel k=400 provided in the embodiment of the present invention; Figure 10 A comparison diagram of the actual solution and the neural network solution for cosine kernel k=5 provided in an embodiment of the present invention; Figure 11 The absolute error distribution diagram of the test points with cosine kernel k=5 provided in the embodiment of the present invention. Detailed Implementation

[0026] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0027] like Figure 1 As shown, this embodiment provides a method for solving the Volterra integral equation based on an attention neural network, including: S1. Discretize the solution interval of the second type of Volterra integral equation to be solved into multiple collocation points to obtain a set of discretized collocation points; S2. Construct a deep neural network integrating multi-head self-attention mechanism and residual connection to approximate the unknown solution of the second type of Volterra integral equation; S3. Substitute the deep neural network into the second type of Volterra integral equation, force the equation to hold on the discretized configuration point set, and use a numerical integration algorithm to calculate the integral term in the equation to obtain the discrete residual equation of the relevant network parameters. S4. Based on the discrete residual equation, construct a smoothing loss function as the training objective; S5. The L-BFGS optimization algorithm is adopted to minimize the smooth loss function, the network parameters are iteratively updated, and the training is terminated by the early stopping strategy to obtain the optimal network parameters. S6. Substitute the optimal network parameters into the deep neural network to obtain the final numerical solution of the second type of Volterra integral equation.

[0028] By constructing a deep neural network integrating multi-head self-attention and residual connections, using the Gaussian quadrature formula to accurately calculate the integral term, enhancing robustness to residual inconsistencies with a smoothed L1 loss function, and utilizing L-BFGS second-order optimization and early stopping strategies to achieve efficient training, it obtains a continuous numerical solution that is stable and has a consistent absolute error magnitude across the entire solution interval in solving the second type of Volterra integral equation. In particular, it maintains good accuracy and generalization ability for complex problems where the kernel function contains high oscillatory components.

[0029] The following provides further explanation of each step and related technical features in the above scheme; In this embodiment, S1, the solution interval of the second type of Volterra integral equation to be solved is discretized into multiple collocation points to obtain a set of discretized collocation points; The standard form of the second kind of linear Volterra integral equation to be solved is:

[0030] Where K∈C(D) is a known kernel function, C(D) is the set of all continuous functions defined on region D, f∈C(I) is a known right-hand side function, and C(I) is the set of all continuous functions defined on region I. Let be the unknown function to be solved, s be the independent variable, and t be the integration variable; The solution interval I is discretized into a uniform grid. ,in N+1 represents the total number of configuration points. Let be the placement point, and the step size between adjacent placement points is . , where i is the configuration point index.

[0031] In practical implementation, the solution interval is taken as [0,1]. Uniform discretization is used to generate configuration points. In this embodiment, the number of training configuration points is fixed at 1000, and the number of test configuration points is 200, which balances computational efficiency and approximate accuracy. For regions with severe kernel function oscillations (such as the high-frequency fluctuations of the kernel function near t=0 when k=400, as discussed later), uniform discretization can ensure sufficient local sampling density and avoid introducing additional errors due to mesh distortion. Step size The kernel function can be adaptively adjusted according to its local rate of change. However, the equidistant grid used in this embodiment has achieved stable accuracy under various oscillation parameters, indicating that uniform discretization has sufficient robustness for common engineering oscillation kernel functions.

[0032] In this embodiment S2, a deep neural network integrating multi-head self-attention mechanism and residual connection is constructed to approximate the unknown solution of the second type of Volterra integral equation; like Figure 2 As shown, the deep neural network consists of the following layers in sequence: input layer, first fully connected layer, second fully connected layer, multi-head self-attention layer, first residual normalization layer, third fully connected layer, second residual normalization layer, and output layer.

[0033] The input layer is used to receive scalar input; Both the first and second fully connected layers employ the Tanh activation function to progressively transform the input into a high-dimensional latent representation. The multi-head self-attention layer contains multiple parallel attention heads, each of which is used to learn different association patterns of the input features; The first residual normalization layer is used to perform layer normalization by adding the input and output of the multi-head self-attention layer; The third fully connected layer uses the Tanh activation function to perform a nonlinear transformation on the output of the first residual normalization layer; The second residual normalization layer is used to add the input and output of the third fully connected layer and then perform layer normalization; The output layer is a fully connected layer used to output approximate values ​​of the solution to the equation.

[0034] In the deep neural network, the number of hidden neurons in the first, second, and third fully connected layers is set to 20, and the number of multi-head attention heads is set to 4. This hyperparameter combination was determined through a small number of trials and did not require retuning in multiple test cases with oscillation parameters ranging from 5 to 400, demonstrating the generalization ability of the network structure. It is necessary to clearly define the residual connection path: in the first residual normalization layer, the objects of the addition operation are the outputs of the second fully connected layer and the multi-head self-attention layer; in the second residual normalization layer, the objects of the addition operation are the outputs of the first residual normalization layer and the outputs of the third fully connected layer. This explicit design ensures that the gradient flows smoothly back to the shallow layers, avoiding gradient vanishing or degradation.

[0035] In this embodiment S3, the deep neural network is substituted into the second type of Volterra integral equation, the equation is forced to hold on the discretized configuration point set, and the integral term in the equation is calculated by a numerical integration algorithm to obtain the discrete residual equation of the relevant network parameters. Let the deep neural network constructed in step S2 be denoted as Where s is the network input and θ is the network parameter, substituting them into the second kind of Volterra integral equation, and obtaining the discrete locus of points in S1. When the forced equation holds, the discretized equation is obtained:

[0036] Furthermore, for each subinterval integration in the discretized equation, the numerical integration algorithm uses the Gaussian quadrature formula to calculate each subinterval. Integral terms on:

[0037] in, A known kernel function for the second kind of Volterra integral equation, used to describe the variables. The interaction relationship between the integral variable t and the integral variable t; Let represent the output function of a deep neural network with input t and parameters θ. For each Gaussian quadrature node mapped onto a subinterval, For the current configuration point in the discretized configuration point set, Find the number of Gaussian product points. To find the product weight.

[0038] Furthermore, Gaussian quadrature nodes From the reference interval [ The coordinate transformation of the root node of the Legendre polynomial on [1,1] yields: in, For the pre-computed Legendre polynomial root node.

[0039] Gaussian method for calculating the number of integral points In this embodiment, the value is set to 8. For cases where the kernel function oscillation frequency is extremely high, it can be... The number was increased to 16, but the 8-point Gauss-Legendre quadrature combined with 1000 uniformly distributed points was sufficient to control the integral discretization error within a certain range. The following values ​​are significantly lower than the network approximation error; in implementation, Legendre nodes... and weight Pre-compute and store as constant tensors, then quickly generate them over each subinterval using affine transformations. To avoid redundant calculations, due to the configuration points With Gaussian nodes Generally, they do not overlap; neural networks need to... The value is to be evaluated at a point, and since the network itself is defined on a continuous interval, it can be obtained directly through forward propagation. No interpolation is required.

[0040] In this embodiment, S4, a smoothing loss function is constructed as the training target based on the discrete residual equation. The smoothing loss function is the smoothing L1 loss function, expressed as:

[0041]

[0042] in, For the configuration point The discrete residuals obtained at:

[0043] N is the total number of configuration points in the discretized configuration point set, and β is the smoothing parameter.

[0044] In this step, the smoothing parameter β in the smoothed L1 loss function is set to 1.0 by default, so that the error is within ( Within the range of 1,1), a quadratic penalty is applied; outside this range, a linear penalty is applied. In this embodiment, the absolute error of some test points is close to... The isolated peaks of magnitude did not disrupt the overall convergence, thanks to the robustness of the linear penalty. The denominator N+1 in the loss function ensures that the loss magnitude is consistent under different numbers of configuration points, which facilitates cross-model comparison. A larger β can be used in the early stage of training to make the loss surface smoother, and then decay to 1.0.

[0045] In this embodiment, S5, the L-BFGS optimization algorithm is used to iteratively update the network parameters with the goal of minimizing the smoothing loss function, and the training is terminated using an early stopping strategy to obtain the optimal network parameters. The termination condition for the early stopping strategy is expressed as follows:

[0046] in, Let t be the termination round of the early stopping strategy, and t be the training round index. For the first The loss value calculated by the round on the test configuration point set. The historical best test loss is represented by Δ, which is a preset improvement threshold. P is the training round index, and P is the preset patience window parameter.

[0047] The historical gradient storage size of the L-BFGS optimization algorithm is set to 30, which effectively approximates the Hessian matrix inverse under limited memory. The patience window parameter P of the early stopping strategy is set to 100 rounds, and the improvement threshold Δ is set to... Especially for extremely high frequency oscillating kernel functions, the loss surface has a large number of local minima. L-BFGS can still converge to a globally better solution by using second-order curvature information, while first-order optimizers such as Adam are prone to getting stuck in local flat regions under the same conditions.

[0048] In this embodiment, S6, the optimal network parameters are substituted into the deep neural network to obtain the final numerical solution of the second type of Volterra integral equation; With optimal parameters Corresponding network function As a numerical solution to an integral equation, it can be found in the interval Fast evaluation at any point within the interior; the final output is the numerical solution. It is an explicit function defined on a continuous interval. For any s∈[0,1], it can be calculated by one forward propagation. This characteristic is particularly suitable for engineering scenarios that require real-time querying or dynamic updating of the solution domain, such as calculating the field strength at any point in an electromagnetic field, or evaluating the state at unequal time intervals in a control system.

[0049] In electromagnetic field simulations, engineers often need to solve the second kind of Volterra integral equations containing oscillating kernel functions to calculate the field strength distribution at any location in space. For example, when an antenna radiation source generates high-frequency electromagnetic waves, traditional numerical methods require dividing the solution domain into a dense grid and outputting results only at the grid points. Using the method in this embodiment, engineers only need to input the kernel function K(s,t) of the integral equation to be solved and the right-hand side f(s), and set the solution interval [a,b]. The system automatically completes the discretization of the configuration points and the training of the neural network, ultimately obtaining a continuously differentiable numerical solution function. After that, engineers can directly input any coordinate values, such as 100 test points with non-equidistant spacing, to obtain the corresponding field strength without the need for re-discretization or interpolation, which greatly improves the efficiency of post-processing analysis.

[0050] Furthermore, in the aircraft attitude control system, the controller needs to update control commands in real time based on continuous state changes. However, sensor sampling times are often not equidistant. The continuous numerical solution function obtained by the method in this embodiment can directly output the state estimate at any sampling time. In specific implementation, the engineer transforms the dynamic equation of the control system into the form of a second-type Volterra integral equation and trains AttentionNet using offline collected historical data. After training, the model is deployed to the embedded controller. When the sensor returns a measurement value at any non-uniform time point, the controller can directly call... The system calculates the current state and generates control commands immediately, avoiding interpolation errors or control delays caused by timing mismatches.

[0051] Furthermore, this embodiment verifies the solution performance of the above method under different oscillation types and different oscillation parameters through several representative examples of the second type of Volterra integral equations; like Figure 3 As shown, for the second kind of Volterra integral equation containing an exponential-cosine oscillatory kernel function (k=100), the neural network solution obtained in this embodiment highly overlaps with the exact solution y=cos(s) on the interval [0,1]; furthermore, as Figure 4 As shown, on 200 untrained, equally spaced test points, the overall absolute error is less than [value missing]. Only a few isolated points showed slight peaks, and the overall error magnitude remained stable, verifying that the method in this embodiment has good solution accuracy and generalization ability for mid-to-high frequency oscillation kernel functions.

[0052] like Figure 5 As shown, for the case containing a sine-cosine composite oscillation kernel function and an oscillation parameter k=10, the neural network solution and the exact solution... The fitted curves are also highly consistent; such as Figure 6 As shown, the absolute error distribution across 200 test points is illustrated, with the overall absolute error being lower than [value missing]. This further proves that the method in this embodiment can still converge stably under the condition of composite oscillating kernel function, and has a consistent approximation effect on points that have not participated in the training.

[0053] like Figure 7 As shown, when the oscillation parameter is increased to an extremely high frequency k=400, the kernel function sin(x)cos(400exp( x) exhibits drastic fluctuations near 0, posing a severe challenge to numerical methods; however, as Figure 8 As shown, the neural network solution and the exact solution Still highly consistent; such as Figure 9 As shown, the absolute error at the 200 test points is generally lower than This indicates that the method in this embodiment can still maintain a stable and high-precision solution capability for extremely high-frequency oscillating kernel functions.

[0054] Furthermore, for the second kind of Volterra integral equation containing a cosine kernel function (k=5), the exact solution is y=s+cos(5s) / 25; as Figure 10 As shown, the neural network solution and the exact solution almost completely overlap over the entire interval; as Figure 11 As shown, the absolute error at 200 test points is presented, and it stabilizes at... Magnitude.

[0055] This embodiment verifies the solution performance of the proposed method under different oscillation types and parameters. All embodiments employ the same network architecture and training strategy. Using only 1000 uniformly distributed points, the neural network solution and each exact solution show high agreement across the entire range, and the absolute error at 200 test points is stably controlled within a certain range. Even when faced with the severe challenge of extremely high-frequency oscillations in the kernel function, the Volterra integral equation solution method based on attention neural network in this implementation still maintains stable convergence and consistent accuracy, fully demonstrating its high accuracy, strong generalization, and excellent adaptability to oscillating kernel functions.

[0056] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the systems disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the descriptions are relatively simple; relevant parts can be referred to the method section.

[0057] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for solving Volterra integral equations based on attention neural networks, characterized in that, Includes the following steps: S1. Discretize the solution interval of the second type of Volterra integral equation to be solved into multiple collocation points to obtain a set of discretized collocation points; S2. Construct a deep neural network integrating multi-head self-attention mechanism and residual connection to approximate the unknown solution of the second type of Volterra integral equation; S3. Substitute the deep neural network into the second type of Volterra integral equation, force the equation to hold on the discretized configuration point set, and use a numerical integration algorithm to calculate the integral term in the equation to obtain the discrete residual equation of the relevant network parameters. S4. Based on the discrete residual equation, construct a smoothing loss function as the training objective; S5. The L-BFGS optimization algorithm is adopted to minimize the smooth loss function, the network parameters are iteratively updated, and the training is terminated by the early stopping strategy to obtain the optimal network parameters. S6. Substitute the optimal network parameters into the deep neural network to obtain the final numerical solution of the second type of Volterra integral equation.

2. The method for solving the Volterra integral equation based on an attention neural network according to claim 1, characterized in that, S1 includes: dividing the solution interval Discretize into a uniform grid ,in N+1 represents the total number of configuration points. Let be the current configuration point in the discretized configuration point set, and let be the step size between adjacent configuration points. , where i is the configuration point index.

3. The method for solving the Volterra integral equation based on an attention neural network according to claim 1, characterized in that, In S2, the deep neural network sequentially includes: an input layer, a first fully connected layer, a second fully connected layer, a multi-head self-attention layer, a first residual normalization layer, a third fully connected layer, a second residual normalization layer, and an output layer.

4. The method for solving the Volterra integral equation based on an attention neural network according to claim 3, characterized in that, The activation functions of the first, second, and third fully connected layers are all Tanh functions; the multi-head self-attention layer contains multiple parallel attention heads; the number of hidden neurons in the first, second, and third fully connected layers is 20; and the network parameters of the deep neural network are initialized using the Xavier uniform initialization method.

5. The method for solving the Volterra integral equation based on an attention neural network according to claim 1, characterized in that, In S3, the numerical integration algorithm is the Gaussian quadrature formula, used to calculate each subinterval. Integral terms on: in, A known kernel function for the second kind of Volterra integral equation, used to describe the variables. The interaction relationship between the integral variable t and the integral variable t; Let represent the output function of a deep neural network with input t and parameters θ. For each Gaussian quadrature node mapped onto a subinterval, For the current configuration point in the discretized configuration point set, Find the number of Gaussian product points. To find the product weight.

6. The method for solving the Volterra integral equation based on an attention neural network according to claim 5, characterized in that, Gaussian quadrature node From the reference interval [ The coordinate transformation of the root node of the Legendre polynomial on [1,1] yields: in, For the pre-computed Legendre polynomial root node.

7. The method for solving the Volterra integral equation based on an attention neural network according to claim 1, characterized in that, In S4, the smoothing loss function is the smoothing L1 loss function, expressed as: Where N is the total number of configuration points in the discretized configuration point set. For the configuration point The discrete residuals obtained at the given point, where β is the smoothing parameter.

8. The method for solving the Volterra integral equation based on an attention neural network according to claim 1, characterized in that, In S5, the termination condition of the early stopping strategy is expressed as follows: in, Let t be the termination round of the early stopping strategy, and t be the training round index. For the first The loss value calculated by the round on the test configuration point set. The historical best test loss is represented by Δ, which is a preset improvement threshold. P is the training round index, and P is the preset patience window parameter.

Images