Privacy protection method for multivariate time series data based on negative database and deep learning

Abstract

The present invention discloses a privacy protection method for multivariate time series data based on negative database and deep learning. First, the data in the original data set is preprocessed, the original data is converted into a fixed-length binary string, the activation function estimation problem based on the negative database is analyzed by Bayesian theorem and probability statistics method, and then a more fine-grained negative database generation algorithm QK‑hidden is used to perform privacy protection operations on it. According to the long-term and short-term periodic characteristics of multivariate time series data, a deep learning model composed of a convolutional layer and a recurrent neural network is selected to capture the short-term and long-term dependencies of the time series. At the same time, an autoregressive model is used as a linear layer to model it, and a model with privacy protection effect is trained to be obtained, and prediction is performed on it. The present invention uses a negative database to protect the privacy of the original multivariate time series data, and predicts it through a neural network model, and achieves good prediction performance under the premise of meeting certain data security.

Description

Technical Field

[0001] The present invention belongs to the field of information security technology and relates to a multivariate time series data privacy protection method, and specifically to a multivariate time series data privacy protection method based on a negative database and deep learning. Background Art

[0002] In the era of big data, mining and analyzing massive amounts of data and extracting the potentially valuable information hidden in them have greatly improved social productivity and facilitated people's lives. Time series data is a sequence of data arranged in chronological order. Through the analysis and prediction of time series data, forward-looking opinions and guiding strategies can be provided to decision makers to cope with possible changes in the future; at the same time, the potential laws contained in these time series data can also be discovered, so as to better understand various scientific theories and social phenomena. However, in real applications, most of the data used by data mining algorithms are in plain text, which poses a great risk of privacy leakage. Therefore, while people have achieved certain results in the research on time series data mining, its privacy security issues have also received more and more attention.

[0003] The data security technology currently used in privacy-preserving data mining has been proven to have some problems. The most typical method based on data perturbation is differential privacy technology, which achieves the purpose of privacy protection by adding numerical or non-numerical noise to sensitive data. The query request publicly visible information will not leak the individual's private information. Although this type of algorithm is highly efficient and simple to implement, data perturbation adds noise to the original data, which will reduce the availability of the data. The most mainstream method based on data encryption is homomorphic encryption, which is characterized by being able to directly calculate the ciphertext and obtain the same result as the same calculation performed directly in the plaintext state. However, the privacy-preserving data mining algorithm based on homomorphic encryption also has the disadvantages of complex implementation, large amount of calculation and low efficiency. Summary of the invention

[0004] In order to overcome the defects of the above-mentioned background technology, the present invention provides a multivariate time series data privacy protection method based on negative database and deep learning, which adopts a method between the perturbation and encryption schemes, which can not only meet the requirements of privacy protection but also ensure good performance.

[0005] The technical solution adopted by the present invention is: a multivariate time series data privacy protection method based on negative database and deep learning, comprising the following steps:

[0006] Step 1: Substitute the multivariate time series data Y = {y 1 y 2 … y T}Preprocess and convert into binary string in, n is the variable dimension, is the set of real numbers;

[0007] Step 2: Select the negative database generation algorithm QK-hidden, for the data converted in step 1 Generate the corresponding negative database NDB = {NDB 1 NDB 2 … NDB T};

[0008] Step 3: Extract the summary S of the negative database from step 2 = {S 1 S 2 … S T}, where S i NDB i synopsis of;

[0009] Step 4: Based on the summary S, complete the activation function estimation based on the negative database;

[0010] Step 5: Input S into the multivariate time series data prediction network to obtain the prediction results of the integration of the nonlinear partial neural network and the linear partial autoregressive model, and select S that meets the preset conditions as the final privacy-preserving data;

[0011] The multivariate time series data prediction network includes a deep learning model that combines the convolutional layer of a convolutional neural network CNN with a controlled recurrent unit network GRU. The features extracted by the convolutional layer are input into the controlled recurrent unit network GRU in chronological order to capture the short-term and long-term dependencies of the time series and obtain the state vector at the next moment. At the same time, the autoregressive model AR is combined as the linear layer of the multivariate time series data prediction network to build a multivariate time series data prediction network. Through the multivariate time series data prediction network, a prediction result integrating the nonlinear part and the linear part is finally obtained.

[0012] The beneficial effects of the present invention are:

[0013] (1) According to the long-term and short-term periodic characteristics of multivariate time series data, a deep learning model combining a convolutional layer of a convolutional neural network (CNN) and a controlled recurrent unit network (GRU) is selected to capture the short-term and long-term dependencies of the time series, and an autoregressive model is used as a linear layer to model it;

[0014] (2) The present invention adopts Bayesian theorem and probability statistics methods, combined with the activation function of the deep learning model, to analyze the activation function estimation problem based on negative databases (NDB), and uses a more fine-grained negative database generation algorithm QK-hidden to perform privacy protection operations on it;

[0015] (3) The present invention can ensure good data availability while meeting the requirements of data privacy and security. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 The figure is a flowchart of an embodiment of the present invention. DETAILED DESCRIPTION

[0017] In order to facilitate ordinary technicians in the field to understand and implement the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the implementation examples described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention.

[0018] Please see Figure 1 The present invention first preprocesses the data in the original multivariate time series data set and converts it into a fixed-length binary string with the same number of bits as the maximum value; according to the long-term and short-term periodic characteristics of the multivariate time series data, a deep learning model combining the convolutional layer of a convolutional neural network (CNN) and a controlled recurrent unit network GRU is selected to capture the short-term and long-term dependencies of the time series, and an autoregressive model is combined as a linear layer to model it; for the binary string, the Bayesian theorem and probability statistics methods are used to analyze the activation function estimation problem based on the negative database (NDB), and a more fine-grained negative database generation algorithm QK-hidden is used to perform privacy protection operations on it; finally, training is performed to obtain a model with privacy protection effect, and prediction is performed on it to output the prediction results.

[0019] The present invention is described in detail below with reference to an example. The present invention provides a multivariate time series data privacy protection method based on a negative database and deep learning, comprising the following steps:

[0020] Step 1: Substitute the multivariate time series data Y = {y 1 y 2 … y T}Preprocess and convert into binary string in, n is the variable dimension, is the set of real numbers;

[0021] In this embodiment, the multivariate time series data is preprocessed and converted into a fixed-length binary string with the same number of bits as the maximum value.

[0022] Step 2: Select the negative database generation algorithm QK-hidden, for the data converted in step 1 Generate the corresponding negative database NDB = {NDB 1 NDB 2 … NDB T};

[0023] In this embodiment, the negative database generation algorithm QK-hidden stores the compressed form of the content in the complement of a data set consisting of n-bit binary strings; for m hidden strings s with a maximum attribute bit number of L, the probability p of the QK-hidden algorithm generating different types of records in the negative database is 1 , p 2 ,…,p K While controlling, a set of inverted parameters q is introduced 1 ,q 2 , …, q L The probability of negation of each attribute bit is controlled; where the parameter K is the number of confirmed bits of each record in the negative database, r is the parameter that controls the size of the negative database, N = m × r, N is the total number of entries in the negative database; the negative database is based on the probability parameter p i Generate the i-th type of record. The method of generating the i-th type of record is to select i inverted bits according to the inverted parameter, then randomly select Ki positive bits, and finally add them to the negative database.

[0024] The pseudo code of the QK-hidden negative database generation algorithm is as follows:

[0025]

[0026]

[0027] Step 3: Extract the summary S of the negative database from step 2 = {S 1 S 2 …S T}, where S i NDB i synopsis of;

[0028] Step 4: Input S into the multivariate time series data prediction network, complete the activation function estimation based on the negative database, obtain the prediction results of the integration of the nonlinear partial neural network and the linear partial autoregressive model, and select S that meets the preset conditions as the final privacy protection data;

[0029] The multivariate time series data prediction network includes a deep learning model that combines the convolutional layer (Convolutional Layer) of a convolutional neural network (CNN) (Convolutional Neural Network) with a controlled recurrent unit network (GRU). The features extracted by the convolutional layer are input into the controlled recurrent unit network GRU in chronological order to capture the short-term and long-term dependencies of the time series and obtain the state vector at the next moment; at the same time, the autoregressive model AR (Autoregressive) is combined as the linear layer of the multivariate time series data prediction network to build a multivariate time series data prediction network; through the multivariate time series data prediction network, a prediction result integrating the nonlinear part and the linear part is finally obtained.

[0030] In this embodiment, a convolution layer is set to extract short-term features of the time series and capture short-term patterns between multivariate variables, that is, local dependencies in the time dimension. The convolution layer of this embodiment is composed of several filters with a width of w and a height of n, and the height is set to be the same as the number of variables; the kth filter scans the input matrix Y and generates:

[0031] h k =RELU(W K *Y+b k ) (1)

[0032] Among them, * represents the convolution operation, and the output value h k is a vector, k is the kth convolution kernel, RELU is the RELU function, RELU(x)=max(0,x); W K is the weight matrix, b k is bias.

[0033] The extracted features are then input into the recurrent neural network in chronological order to obtain the state vector of the next moment. Since the recurrent neural network will have gradient vanishing and gradient explosion phenomena, it is impossible to grasp the nonlinear relationship of long spans. In order to solve the long-term dependency problem, a large number of optimization theories have been introduced and many improved algorithms have been derived, including neural history compressors (Neural History Compressor), long short-term memory networks (Long Short-Term Memory networks, LSTM), gated recurrent unit networks (Gated Recurrent Unit networks, GRU), independent recurrent neural networks (Independent RNN), etc., which are all variants of ordinary recurrent neural networks, and the most common are long short-term memory networks LSTM and gated recurrent unit networks GRU. The performance of GRU and LSTM is comparable in many tasks that require "long-term memory". However, LSTM also introduces many selective mechanisms, resulting in more parameters and a lot of training difficulty. Therefore, the present invention selects a GRU network structure that is comparable to LSTM in effect, but has fewer parameters and is easier to converge, capturing the time dependency and long-term pattern of time series data.

[0034] The GRU network of this embodiment uses the RELU function as the hidden update activation function;

[0035] The hidden state of the recurrent unit at time t is:

[0036] r t =σ(y t W xr +h t-1 W hr +b r ) (2)

[0037] u t =σ(y t W xu +h t-1 W hu +b u ) (3)

[0038] c t =RELU(y t W xc +r t ⊙(h t-1 W hc )+b c ) (4)

[0039] h t =(1-u t )⊙h t-1 +ut ⊙c t (5)

[0040] Where ⊙ is the product of the elements, σ ​​is the sigmoid function, and y t is the input at time t, r t is the hidden state of the reset gate component in GRU at time t, u t is the hidden state of the component update gate in GRU at time t, c t is the hidden state of the candidate memory unit in GRU at time t; W xr , W hr , W xu , W hu , W xc , W hc are the reset gate, update gate, and weight matrix trained in the candidate memory unit, respectively. r 、b u 、b c are the reset gate, update gate, and bias of candidate memory unit training respectively; h t-1 is the state transmitted at the previous time t-1, h t is the output state at the current time t.

[0041] Due to the nonlinear nature of the convolutional and recursive components, a major drawback of this neural network model is that the scale of the output is not sensitive to the scale of the input. Therefore, this embodiment uses the classic autoregressive model (AR) as the linear component.

[0042] The autoregressive model AR of this embodiment is:

[0043]

[0044] Where i = 1, 2, 3, ..., n, is the coefficient of the AR model, bias q ar Indicates the number of single steps in multi-step prediction; y t-k，i It means to use the state of the previous k moments to predict the input value at time t, It means using the state of the previous k moments to predict the prediction result at time t;

[0045] The prediction result of AR component is recorded as:

[0046]

[0047] Integration of the nonlinear part and the linear part of the final prediction result of the multivariate time series data prediction network;

[0048]

[0049] in, is the prediction result of the GRU (nonlinear part) network. It is the prediction result of the autoregressive model AR (linear part).

[0050] In this embodiment, based on the outline S, the activation function estimation based on the negative database is completed, and the multivariate time series data prediction network is trained until the multivariate time series data prediction network converges to obtain a trained multivariate time series data prediction network;

[0051] For the sigmoid function and RELU function of the deep learning model, the activation function is estimated based on the negative database; where:

[0052]

[0053]

[0054] Among them, z represents the linear calculation result in the neuron and is expressed as [x 1 … x m ] represents the original private data, m represents the number of input x attributes, [w 1 … w m ] T is the weight matrix;

[0055] The negative database of hidden string s is NDB s , the probability P that the i-th bit of the attribute in the record is different from the corresponding bit of s diff [i] is:

[0056]

[0057] In the above formula, K represents K types of negative database records, among which the negative database record of the i-th type has i definite bits, p j represents the probability of generating the jth type of negative database record, which has j determined bits opposite to the corresponding positions of the hidden string, and the remaining Kj determined bits are the same as the hidden string, q i represents the probability that the i-th bit of the selected attribute is different from the corresponding position of the hidden string, and L represents the attribute bit length;

[0058] The probability that the jth bit of the i-th attribute of the hidden string s is 0 for:

[0059]

[0060] Among them, P same [j] is the probability that the jth bit of the attribute is the same as the corresponding bit of s, n0 NDB s The total number of records whose jth bit of the i-th attribute is 0, n 1 NDB s The total number of records in which the jth bit of the i-th attribute is 1; if s = S k , then n 0 =S k [i×L+j][0],n 1 =S k [i×L+j][1],

[0061] The probability that the jth bit of the i-th attribute of the hidden string s is 1 for:

[0062]

[0063] The probability P(s) that the i-th attribute value of the hidden string s is d i =d) is:

[0064]

[0065] Where 0≤d≤2 L -1, Yes i The binary representation of d is d bin =b 1 ... b L ;

[0066] Therefore, the estimated value of z after negative database privacy protection is:

[0067]

[0068] The estimation formula for the activation function sigmoid function and RELU function is:

[0069]

[0070]

[0071] Substituting formula (15) into formula (16) and (17) respectively, the activation function estimation based on the negative database is completed.

[0072] This embodiment trains a multivariate time series data prediction network, and its optimization goal is:

[0073]

[0074] Among them, ΩTrain is the set of time series used for training, ||·|| Fis the Frobenius norm, h is the ideal limit at the current moment, and the h value in the prediction task is a hyperparameter determined by the requirements of the environment setting;

[0075] This embodiment is to perform a prediction task on multivariate time series data after privacy protection. Given a set of known time series data Y = {y 1 ,y 2 ,…,y T}, predict the future time series signal in the form of rolling (sliding) prediction; the input matrix at timestamp T is represented as When starting to train the multivariate time series data prediction network, the weights are first randomly initialized; the objective function evaluates the multivariate time series data prediction network and returns a value indicating the usefulness of the multivariate time series data prediction network, and at the same time, the weights of the multivariate time series data prediction network are modified in each iteration, thereby changing the value returned by the objective function;

[0076] The stochastic gradient descent method is used to train the multivariate time series data prediction network. The error output is calculated from the input to the output, while the weights and thresholds are adjusted from the output to the input. During forward propagation, the input signal acts on the output node through the hidden layer, and after nonlinear transformation, an output signal is generated. If the actual output does not match the expected output, the error is turned to the reverse propagation process. Error backpropagation is to propagate the output error through the hidden layer to the input layer layer by layer, and distribute the error to all units in each layer, and use the error signal obtained from each layer as the basis for adjusting the weights of each unit. By adjusting the connection strength between the input node and the hidden layer node, the connection strength between the hidden layer node and the output node, and the threshold, the error is reduced along the gradient direction. After repeated learning and iteration, after a certain stage of learning, the loss change curve tends to be stable, and the network parameters corresponding to the minimum error, including weights and thresholds, are determined, and then training is stopped.

[0077] This embodiment selects two evaluation indicators to evaluate the prediction performance of the model, Root Relative Squared Error (RSE) and Empirical Correlation Coefficient (CORR). The smaller the RSE, the better the model prediction results; the larger the CORR, the better the model prediction results.

[0078]

[0079]

[0080] The feasibility of the present invention has been verified on the public UCI database (dataset for machine learning proposed by the University of California, Irvine) electricity and traffic. The experimental results show that the prediction accuracy is slightly reduced compared with the model without privacy protection. Therefore, the present invention can achieve different security and prediction accuracy by adjusting parameters, and avoid the loss caused by privacy leakage to a certain extent.

[0081] The above is a simple analysis of a multivariate time series data privacy protection solution based on a negative database and deep learning. The present invention is suitable for time series data privacy protection, and can ensure good data availability for time series data prediction.

[0082] It should be understood that the above description of the preferred embodiment is relatively detailed and cannot be regarded as limiting the scope of patent protection of the present invention. Under the enlightenment of the present invention, ordinary technicians in this field can also make substitutions or modifications without departing from the scope of protection of the claims of the present invention, which all fall within the scope of protection of the present invention. The scope of protection requested for the present invention shall be based on the attached claims.

Claims

1. A privacy protection method for multivariate time series data based on negative database and deep learning, characterized in that: The following steps are involved: Step 1: Convert the multivariate time series data Y = {y1y2…y T }Preprocess and convert into binary string in, n is the variable dimension, is the set of real numbers; Step 2: Select the negative database generation algorithm QK-hidden, for the data converted in step 1 Generate the corresponding negative database NDB = {NDB1NDB2…NDB T }; Step 3: Extract the summary of negative database S from step 2 = {S1S2…S T }, where S i NDB i synopsis of; Step 4: Input S into the multivariate time series data prediction network, complete the activation function estimation based on the negative database, obtain the prediction results of the integration of the nonlinear partial neural network and the linear partial autoregressive model, and select S that meets the preset conditions as the final privacy protection data; The multivariate time series data prediction network includes a deep learning model that combines a convolutional layer of a convolutional neural network CNN with a gated recurrent unit network GRU. The features extracted by the convolutional layer are input into the gated recurrent unit network GRU in chronological order to capture the short-term and long-term dependencies of the time series and obtain the state vector at the next moment. At the same time, the autoregressive model AR is combined as the linear layer of the multivariate time series data prediction network to build a multivariate time series data prediction network. Through the multivariate time series data prediction network, a prediction result integrated with the nonlinear part and the linear part is finally obtained. Wherein, based on the outline S, the activation function estimation based on the negative database is completed, and the multivariate time series data prediction network is trained until the multivariate time series data prediction network converges to obtain a trained multivariate time series data prediction network; For the sigmoid function and RELU function of the deep learning model, the activation function is estimated based on the negative database; where: Among them, z represents the linear calculation result in the neuron and is expressed as [x1…x m ] represents the original private data, m represents the number of input x attributes, [w1…w m ] T is the weight matrix; The negative database of hidden string s is NDB s , the probability P that the i-th bit of the attribute in the record is different from the corresponding bit of s diff [i] is: In the above formula, K represents K types of negative database records, among which the negative database record of the i-th type has i definite bits, p j represents the probability of generating the jth type of negative database record, which has j determined bits opposite to the corresponding positions of the hidden string, and the remaining Kj determined bits are the same as the hidden string, q i represents the probability that the i-th bit of the selected attribute is different from the corresponding position of the hidden string, and L represents the attribute bit length; The probability that the jth bit of the i-th attribute of the hidden string s is 0 for: Among them, P same [j] is the probability that the jth bit of the attribute is the same as the corresponding bit of s, and n0 is NDB s The total number of records in which the jth bit of the i-th attribute is 0, and n1 is NDB s The total number of records in which the jth bit of the i-th attribute is 1; if s = S k , then n0=S k [i×L+j][0],n1=S k [i×L+j][1], The probability that the jth bit of the i-th attribute of the hidden string s is 1 for: The probability P(s) that the i-th attribute value of the hidden string s is d i =d) is: Where 0≤d≤2 L -1, Yes i The binary representation of d is d bin =b1...b L ; Therefore, the estimated value of z after negative database privacy protection is: The estimation formula for the activation function sigmoid function and RELU function is: Substituting formula (15) into formula (16) and (17) respectively, the activation function estimation based on the negative database is completed.

2. The multivariate time series data privacy protection method based on negative database and deep learning according to claim 1 is characterized by: In step 1, the multivariate time series data is preprocessed and converted into a fixed-length binary string with the same number of bits as the maximum value.

3. The multivariate time series data privacy protection method based on negative database and deep learning according to claim 1 is characterized by: In step 2, the negative database generation algorithm QK-hidden stores the compressed form of the content in the complement of the data set composed of n-bit binary strings; for m hidden strings s with a maximum attribute bit number of L, the QK-hidden algorithm generates the probability p1, p2, ..., p of different types of records in the negative database K While controlling, a set of inverted parameters q1, q2, ..., q are introduced L The probability of negation of each attribute bit is controlled; where the parameter K is the number of confirmed bits of each record in the negative database, r is the parameter that controls the size of the negative database, N = m × r, N is the total number of entries in the negative database; the negative database is based on the probability parameter p i Generate the i-th type of record. The method of generating the i-th type of record is to select i inverted bits according to the inverted parameter, then randomly select Ki positive bits, and finally add them to the negative database.

4. The multivariate time series data privacy protection method based on negative database and deep learning according to claim 1, characterized in that: In step 4, the convolutional layer consists of several filters with width w and height n, where the height is set to be the same as the number of variables; the kth filter sweeps over the input matrix Y and produces: h k =RELU(IN K *Y+b k ) (1) Among them, * represents the convolution operation, and the output value h k is a vector, k is the kth convolution kernel, RELU is the RELU function, RELU(x)=max(0,x); W K is the weight matrix, b k is bias.

5. The multivariate time series data privacy protection method based on negative database and deep learning according to claim 1 is characterized by: In step 4, the gated recurrent unit network GRU uses the RELU function as a hidden update activation function; The hidden state of the recurrent unit at time t is: r t =σ(y t W xr +h t-1 W hr +b r ) (2) u t =σ(y t W xu +h t-1 W hu +b u ) (3) c t =RELU(y t IN xc +r t ⊙(h t-1 IN hc )+b c ) (4) h t =(1-u t )⊙h t-1 +u t ⊙c t (5) Where ⊙ is the product of the elements, σ ​​is the sigmoid function, and y t is the input at time t, r t is the hidden state of the component reset gate in GRU at time t, u t is the hidden state of the component update gate in GRU at time t, c t is the hidden state of the candidate memory unit in GRU at time t; W xr , W hr , W xu , W hu , W xc , W hc are the reset gate, update gate, and weight matrix trained in the candidate memory unit, respectively. r , b u , b c are the reset gate, update gate, and bias of candidate memory unit training respectively; h t-1 is the state transmitted at the previous time t-1, h t is the output state at the current time t.

6. The multivariate time series data privacy protection method based on negative database and deep learning according to claim 1 is characterized in that: In step 4, the autoregressive model AR is: Where i = 1, 2, 3, ..., n, is the coefficient of the AR model, bias q ar Indicates the number of single steps in multi-step prediction; y t-k,i It means to use the state of the previous k moments to predict the input value at time t, It means using the state of the previous k moments to predict the prediction result at time t; The prediction result of AR component is recorded as: Integration of the nonlinear part and the linear part of the final prediction result of the multivariate time series data prediction network; in, is the prediction result of the gated recurrent unit network GRU, is the prediction result of the autoregressive model AR.

7. The multivariate time series data privacy protection method based on negative database and deep learning according to claim 1 is characterized in that: The optimization goal of the training multivariate time series data prediction network is: Among them, ΩTrain is the set of time series used for training, ||·|| F is the Frobenius norm, h is the ideal limit at the current moment, and the h value in the prediction task is a hyperparameter determined by the requirements of the environment setting; Given a set of known time series data Y = {y1, y2, ..., y T }, n is the variable dimension; predict the future time series signal in the form of rolling forecast; the input matrix at timestamp T is represented as When starting to train the multivariate time series data prediction network, the weights are first randomly initialized; the objective function evaluates the multivariate time series data prediction network and returns a value indicating the usefulness of the multivariate time series data prediction network, and at the same time, the weights of the multivariate time series data prediction network are modified in each iteration, thereby changing the value returned by the objective function; The stochastic gradient descent method is used to train the multivariate time series data prediction network. The error output is calculated from the input to the output, while the weights and thresholds are adjusted from the output to the input. During forward propagation, the input signal acts on the output node through the hidden layer, and after nonlinear transformation, an output signal is generated. If the actual output does not match the expected output, the error is turned to the reverse propagation process. Error backpropagation is to propagate the output error through the hidden layer to the input layer layer by layer, and distribute the error to all units in each layer, and use the error signal obtained from each layer as the basis for adjusting the weights of each unit. By adjusting the connection strength between the input node and the hidden layer node, the connection strength between the hidden layer node and the output node, and the threshold, the error is reduced along the gradient direction. After repeated learning and iteration, after a certain stage of learning, the loss change curve tends to be stable, and the network parameters corresponding to the minimum error, including weights and thresholds, are determined, and then training is stopped.

Images