A section basin tracing method based on deep bayesian neural network

Abstract

The application provides a section basin tracing method based on a deep Bayesian neural network, and comprises the following steps: acquiring real-time monitoring data of different sections; performing data analysis and data synthesis on the real-time monitoring data to obtain an initial data set and generate an input data set; designing a deep Bayesian neural network based on expert knowledge and experience; inputting the input data set into the Bayesian neural network and training and reasoning the Bayesian network; selecting a pollution source contribution rate as an evaluation index according to a tracing target, outputting the evaluation index as an evaluation result of the deep Bayesian neural network, obtaining a pollution source contribution degree of different monitoring factors in different pollution sources, and realizing the tracing of the pollution source.

Description

Technical Field

[0001] This invention relates to computer systems with specific computational models and to water management science and technology. Specifically, it relates to a cross-sectional watershed source tracing method based on a deep Bayesian neural network. Background Technology

[0002] Accurate source tracing requires scientific estimation of pollution source emissions. However, the current establishment of pollution source inventories mainly relies on extensive and time-consuming field investigations. Apart from a few industrial enterprises with hourly emission data, the pollution emissions of most point and area sources are estimated on an annual basis, ignoring the intermediate process of pollutant emissions. However, water quality issues vary from month to month, making it impossible to make dynamic estimates based on pollution sources and failing to meet the needs of scientific management.

[0003] Based on the data resource system for watershed water environment management, an intelligent analysis model for accurate identification of pollution source types entering rivers at the cross-sectional scale is established. This model forms intelligent management functional modules such as watershed-scale water environment problem analysis, pollution source type identification, and dynamic estimation of emissions. It provides technical support for precise pollution control, rapid source tracing, and spatiotemporal integrated monitoring of the water environment, and is an urgent technical problem that needs to be solved in current water environment management. Summary of the Invention

[0004] This invention addresses the shortcomings of existing technologies by proposing a cross-sectional watershed source tracing method based on a deep Bayesian neural network. Specifically, the method includes: acquiring real-time monitoring data from different cross-sections; performing data analysis and synthesis on the real-time monitoring data to obtain an initial dataset; preprocessing and cleaning the initial dataset to obtain the contribution of various pollutants to different pollution sources, generating an input dataset; designing a deep Bayesian neural network based on expert knowledge and experience, the deep Bayesian neural network comprising a Bayesian network, a deep generative model, and a variational autoencoder; inputting the input dataset into the Bayesian neural network and training and inferring the Bayesian network; the deep generative model and the variational autoencoder collaboratively completing the learning of the Bayesian network; selecting the pollution source contribution rate as an evaluation index according to the source tracing target, and outputting the evaluation index as the evaluation result of the deep Bayesian neural network to obtain the pollution source contribution of different monitoring factors to different pollution sources, thereby achieving source tracing.

[0005] Preferably, the deep generative model includes a DNN module and a multi-layer memory module, wherein the deep generative model in the DNN module is based on the input dataset h. in Calculate low-level generated data h g Next, a linear combination strategy is used to extract knowledge from the multi-layer memory module, and then the element-wise multilayer perceptron is defined as a combination function h. out .

[0006] Preferably, the data analysis process includes abstracting the sources of pollutants, namely industrial pollution, urban pollution, non-point source pollution, and unknown sources; combining the sources of pollutants and the monitoring data, statistically analyzing 10 monitoring factors, namely: total phosphorus, ammonia nitrogen, chemical oxygen demand, pH value, fluoride, permanganate index, volatile phenols, dissolved oxygen, petroleum hydrocarbons, and five-day biochemical oxygen demand.

[0007] Preferably, the Bayesian network can be represented by a directed acyclic graph G=(V,E), where V is the set of nodes, each node representing an observable variable or a latent variable, and E is the set of edges, each edge representing a directed relation. The joint probability distribution can be factorized as follows:

[0008]

[0009] Among them, pa G (v) represents the set of parent nodes of v in G; where parent nodes represent online monitoring indicators and child nodes represent the proportion of different pollution types; directed edges between nodes represent their causal relationships, and conditional probabilities between nodes represent the strength of the relationship.

[0010] Preferably, the input dataset is constructed as follows: Its distribution is P D (X) The deep generative model is used to approximate the joint distribution of the input dataset. P θ (X) ,in θ For model parameters, θ It can be parameterized as a deep neural network model (DNN), achieving [the desired result] by minimizing the statistical divergence between two probability distributions. P θ (X) Approximation P D (X) .

[0011] Preferably, the deep neural network model in the deep generation model is used to simulate the nonlinear correlation between monitoring factors such as total phosphorus, ammonia nitrogen, chemical oxygen demand, pH value, fluoride, permanganate index, volatile phenols, dissolved oxygen, petroleum hydrocarbons, and five-day biochemical oxygen demand during source tracing. The training process of the deep neural network model is as follows:

[0012] (a) Define the loss function

[0013]

[0014] in:o i For the output node in the sample i Output during operation y i For the sample i Target value, N The total number of samples;

[0015] (b) Network initialization: Determine the number of nodes in the input layer (i.e., the dimension of the samples), the number of nodes in the hidden layer, and the number of nodes in the output layer. The output layer represents the contribution of various contaminants, and all indicators are 1-dimensional.

[0016] (c) Calculation of hidden layer output

[0017]

[0018] in, w ij For the hidden layer j Node to the parent node i The weights, n The number of nodes in the previous layer. x i Let a be the variable of the i-th node in the previous layer. i Threshold for hidden layer nodes;

[0019] (d) Output layer computation

[0020]

[0021] Where, w j 'The hidden layer node corresponding to the output layer node' j The weights, l The number of hidden layer nodes, b The threshold value for the output layer nodes;

[0022] (e) Weight update

[0023]

[0024]

[0025] Where η is the learning rate;

[0026] (f) Weight update

[0027]

[0028] (g) Determine if the iteration has ended. If it has not ended, proceed to step (c).

[0029] Preferably, the variational autoencoder consists of a Q-network and a P-network, with the P-network containing multiple memory modules, so the Q-network and P-network are not symmetrical; in the P-network, θ is the parameter set in the deep generative model, the input is x, and z is a latent variable; the Q-network extracts features h using a nonlinear function. （i） .

[0030] Preferably, the evaluation index is determined using the mean absolute error (MAE). The MAE refers to the average absolute value of the errors between corresponding points in the predicted data and the original data; a smaller value indicates a better prediction effect. The mathematical expression is:

[0031]

[0032] Preferably, the training set, validation set, and test set of the deep Bayesian neural network model change with the number of iterations (epcoch). The deep Bayesian neural network model achieves its optimal performance at epoch=15. Before epoch=15, the training set, validation set, and test set all follow a monotonically decreasing trend. When epcoch is greater than 15, the loss of the training set continues to decrease, while the loss of the test set and validation set increases. At this point, overfitting begins to occur. Attached Figure Description

[0033] Various embodiments or examples (“Examples”) of this disclosure are disclosed in the following detailed description and accompanying drawings. It is not necessary to draw the drawings to scale. Generally, unless otherwise specified in the claims, the operations of the methods disclosed in this invention can be performed in any order. In the drawings:

[0034] Figure 1 The flowchart of the pollution source tracing model based on deep Bayesian network learning according to the present invention is shown below.

[0035] Figure 2 The deep neural network (DNN) model used in the deep Bayesian network of this invention is shown.

[0036] Figure 3 Diagram of a deep Bayesian network learning framework;

[0037] Figure 4 Diagram of the inference structure of a variational autoencoder;

[0038] Figure 5 The variation patterns of the training set, severe set, and test set with Epcoch;

[0039] Figure 6 Contribution rate of different pollution sources in the Binan River Basin in April 2015 (dashed line represents contribution rate);

[0040] Figure 7The contribution rate of the pollution sources in April 2015, obtained by the method according to the present invention, is shown in the figure (dashed line represents the contribution rate). Detailed Implementation

[0041] Before explaining one or more embodiments of this disclosure in detail, it should be understood that the embodiments are not limited to the construction details in their specific applications, and the steps or methods presented in the following embodiments or drawings.

[0042] Figure 1 This is a flowchart of the pollution source tracing model based on deep Bayesian network learning according to the present invention. Figure 1 As shown, the pollution source tracing process based on deep Bayesian network learning is as follows:

[0043] (1) Obtain real-time monitoring data from different cross-sections;

[0044] (2) Perform data analysis and data synthesis on the real-time monitoring data to obtain an initial dataset; perform data preprocessing and data cleaning on the initial dataset to obtain the contribution of various pollutants under different pollution sources, generate an input dataset, and select the pollution source contribution rate as an evaluation index according to the source tracing target;

[0045] (3) Design Bayesian networks based on expert knowledge and experience;

[0046] (4) Input the dataset into the Bayesian network, use a deep generative model to learn the probability of related events and train and infer the Bayesian network. The deep generative model is a deep neural network with a hierarchical memory module, and the inference uses a variational autoencoder. The two work together to complete the learning of the Bayesian network.

[0047] (5) Combine the set evaluation indicators to output the evaluation results, obtain the pollution source contribution of different monitoring factors in different pollution sources, and realize the source tracing of pollution sources.

[0048] The data analysis process is illustrated using pollution source emission data from the Binan River Basin as an example.

[0049] The invention "A Method for Estimating the Emissions of Pollutants into Rivers at the Cross-sectional Scale" reveals that the sources of pollutants in the studied water body (e.g., the Binan River) can be abstracted into four different sources: industry, urban areas, aquaculture, and non-point sources. Since many non-point source (non-point source) pollutants are indistinguishable, adjustments were made to the above-mentioned pollution sources. Industrial and aquaculture pollution sources remained unchanged. Due to the construction of new rural areas and the increase in rural urbanization, 50% of rural pollution was attributed to urban pollution. The remaining 50% of rural pollution load, along with farmland / soil and water pollution and urban surface runoff pollution, was integrated into a single non-point source pollution. When using the CMB method for source apportionment of the four pollution sources, the contribution of aquaculture pollution was consistently negative, indicating that the collinearity between aquaculture pollution sources and other sources, especially non-point sources, was too strong. Therefore, aquaculture pollution sources are not suitable for source apportionment. Thus, the pollution sources were ultimately divided into industrial pollution, urban pollution, non-point source pollution, and unknown sources. Research has revealed that current monitoring data at the Binan River monitoring sections includes data on various pollutants such as permanganate index (mercury), 5-day biochemical oxygen demand (BOD5), chemical oxygen demand (COD), selenium, fluoride, arsenic, sulfide, pH (dimensionless), volatile phenols, petroleum hydrocarbons, ammonia nitrogen, total phosphorus, dissolved oxygen, copper, cadmium, zinc, lead, chromium, and cyanide. However, statistical analysis shows that data for the following 10 factors are relatively complete and highly correlated with water quality: total phosphorus, ammonia nitrogen, COD, pH, fluoride, permanganate index, volatile phenols, dissolved oxygen, petroleum hydrocarbons, and 5-day BOD5. Therefore, the core objective of the intelligent algorithm for accurate identification of river pollution source types at the cross-section scale is to trace the proportion of different monitoring factors among the four pollution types based on these 10 monitoring factors at a specific cross-section.

[0050] The following is in conjunction with the appendix Figure 1 This invention provides a detailed description of the model and application method used in the cross-sectional watershed source tracing method.

[0051] Bayesian networks (BNs) are directed probabilistic graphical models that clearly and intuitively display causal relationships between variables. They possess a solid mathematical foundation and can handle various environmental monitoring factors and uncertainties in variables. The information stored in Bayesian networks is largely specified by experts. The rapid development of artificial intelligence, especially deep learning, has provided new ideas for improving BNs and enhancing causal reasoning capabilities. Combining BNs with deep learning improves uncertainty modeling capabilities, adapts to small-sample learning, and enhances the expressive power of generative models through neural networks with memory modules and attention mechanisms. Using deep neural networks (DNNs) to implement the learning capabilities of BNs yields better results than Markov chain Monte Carlo and variational autoencoders. Deep generative models learned by deep Bayesian networks approximate nonlinear functions by preserving a deep hierarchical structure and learn the joint probability distribution of data from the low-dimensional latent variable z in a top-down manner, thus enhancing uncertainty modeling capabilities. This makes them particularly suitable for learning from small samples and data with uncertainty, whereas other machine learning methods require large amounts of deterministic data.

[0052] This invention addresses the problem of insufficient and inaccurate representation of pollutant sources due to long monitoring intervals and uncertainties in estimated variables. It proposes a deep Bayesian network model based on memory modules and variational autoencoders to enhance the uncertainty modeling capabilities of Bayesian models and improve the accuracy of source tracing.

[0053] I. Bayesian Networks

[0054] A Bayesian network can be represented by a directed acyclic graph G=(V,E), where V is the set of nodes, each node representing an observable or latent variable, and E is the set of edges, each edge representing a directed relation. Their joint probability distribution can be factorized as follows:

[0055]

[0056] Where pa G (v) represents the set of parent nodes of v in G.

[0057] In source tracing analysis, causal Bayesian networks are often used, where parent nodes represent observed factors, i.e., online monitoring indicators, and child nodes represent results, i.e., the proportion of different pollution types. Directed edges between nodes represent causal relationships, and conditional probabilities between nodes represent the strength of the relationship.

[0058] Assuming the training data is Its distribution is P D (X) Deep generative models are mainly used to approximate the joint distribution of input data. P θ (X),in θ For model parameters, θ It can be parameterized as a deep neural network model (DNN) and achieves its function by minimizing the statistical divergence (such as KL divergence) between two probability distributions. P θ (X) Approximation P D (X) .

[0059] II. Deep Neural Network Regression Model (DNN)

[0060] Figure 2 This is the deep neural network regression model used in the deep Yeats neural network of this invention. In the source tracing, there is a certain mechanistic correlation between the monitoring factors among total phosphorus, ammonia nitrogen, chemical oxygen demand, pH value, fluoride, permanganate index, volatile phenols, dissolved oxygen, petroleum hydrocarbons, and five-day biochemical oxygen demand. This is a nonlinear problem. Deep network technology can be used to simulate nonlinear systems. Deep neural networks have a simple structure and are often used for data prediction and analysis. They can better reflect the steady-state characteristics of nonlinear systems. Training a good deep network requires a large amount of data; otherwise, overfitting can easily occur, leading to reduced reliability of the prediction results. Deep neural networks, such as... Figure 2 As shown, a network is trained by continuously adjusting the parameters of each node using the residual backpropagation algorithm with a certain number of samples. The main steps in the deep neural network training process are:

[0061] (a) Define the loss function

[0062] (1)

[0063] in: o i For the output node in the sample i Output during operation y i For the sample i Target value, N The total number of samples;

[0064] (b) Network initialization. Determine the number of nodes in the input layer (i.e., the dimension of the samples), the number of nodes in the hidden layer, and the number of nodes in the output layer. The output layer represents the contribution of various contaminants, and all indicators are one-dimensional.

[0065] (c) Calculation of hidden layer output

[0066]

[0067] in, w ij For the hidden layer j Node to the parent nodei The weights, n The number of nodes in the previous layer. x i Let a be the variable of the i-th node in the previous layer. i Threshold for hidden layer nodes;

[0068] (d) Output layer computation

[0069]

[0070] Where, w j 'The hidden layer node corresponding to the output layer node' j The weights, l The number of hidden layer nodes, b The threshold value for the output layer nodes;

[0071] (e) Weight update

[0072]

[0073]

[0074] Where η is the learning rate;

[0075] (f) Weight update

[0076]

[0077] (g) Determine if the iteration has ended. If it has not ended, proceed to step (3).

[0078] III. Deep Generative Model with Integrated Memory Module

[0079] The deep generative model of this invention refers to the fusion of multiple models, such as DNN models and generative models, to achieve better results and enhance the generalization ability of the integrated model. Simpler methods include averaging or voting on the results of weak learners, but these may result in large learning errors. Therefore, learning-based methods have emerged. A representative method for learning-based methods is stacking. When using a stacking strategy, instead of simply processing the results of the weak learners, we add another layer of learners. That is, we use the learning results of the weak learners in the training set as input and the output of the training set as output to retrain a high-level learner to obtain the final result. The generative model with a memory module used in this invention is as follows: Figure 3 As shown.

[0080] like Figure 3 As shown, the deep generative model in the DNN module is based on the input dataset h in Calculate low-level generated data h gNext, a linear combination strategy is used to extract knowledge from the multi-layer memory module, and then the element-wise multilayer perceptron is defined as a combination function h. out .

[0081] from Figure 2 and Figure 3 As can be seen, deep generative models learn the joint probability distribution from top to bottom, mainly using generated data provided by a large input dataset as labeled data, and performing inference based on variational autoencoders. Simultaneously, hierarchical memory modules are used to store a large number of local features extracted by the deep neural network as unlabeled data.

[0082] IV. Variational Autoencoder (VAE)

[0083] The learning and reasoning in this invention employs a variational autoencoder (VAE), the structure of which is as follows: Figure 4 As shown, the Variational Autoencoder (VAE) consists of a Q-network and a P-network. The P-network contains multiple layers of memory modules, so the Q-network and P-network are not symmetrical. In the P-network, θ is the parameter set in the deep generative model, and the input is x, with z being latent variables. The Q-network extracts features h using a nonlinear function. (i) .

[0084] V. Evaluation Indicators for Regression Prediction Performance

[0085] Commonly used indicators for evaluating the predictive performance of a method include mean squared error (MSE), mean absolute error (MAE), and R-squared (coefficient of determination).

[0086] The mean squared error (MSE) is the sum of the squared errors between corresponding points in the predicted data and the original data. Clearly, the smaller the value, the better the regression performance. The mathematical expression is:

[0087]

[0088] Where y = {yi} (i = 1, 2, ..., n) is the true value. These are predicted values.

[0089] Mean absolute error (MAE) refers to the mean of the absolute values ​​of the errors between corresponding points in the predicted data and the original data. A smaller value indicates better prediction performance. The mathematical expression is:

[0090]

[0091] The coefficient of determination is often used to evaluate the effectiveness of regression analysis; a larger value indicates a better fit and stronger predictive ability of the model. The mathematical expression is:

[0092]

[0093] VI. Validation of the Deep Bayesian Network Learning Model

[0094] This invention uses mean absolute error as the evaluation index.

[0095] The evaluation results of the trained model are shown in Table 1, where epoch represents the number of iterations of the model simulation, Train_Loss represents the training set loss, Valid_loss represents the validation set loss, and Test_loss represents the test set loss.

[0096] Table 1 Simulation Iteration Results of the Training Model

[0097] Epoch 1 2 3 4 5 6 7 8 9 Train_Loss 71.34 42.11 34.20 21.91 20.55 15.23 12.32 10.10 10.09 Vaild_loss 69.57 51.99 46.95 33.87 23.39 20.12 15.48 11.59 11.33 Test_loss 99.83 65.08 56.11 49.89 34.40 22.16 19.00 11.80 11.30 epoch 10 11 12 13 14 15 16 17 18 Train_Loss 8.76 6.17 3.04 1.04 0.32 0.15 0.11 0.09 0.10 Vaild_loss 10.20 9.18 6.01 2.93 0.50 0.49 0.75 0.80 1.65 Test_loss 10.40 8.20 5.12 3.01 0.60 0.50 1.39 1.41 1.83

[0098] Based on the results in Table 1, the variation patterns of the training set, validation set, and test set with epcoch are obtained, such as... Figure 5 As shown.

[0099] As shown in the figure above, the model achieves its optimal performance at epoch=15. Before epoch=15, the loss of the training set, validation set, and test set all exhibit a monotonically decreasing trend. When epoch=15, the loss of the training set continues to decrease, while the loss of the test set and validation set both increase, indicating that the model begins to overfit.

[0100] VII. Results Output

[0101] Based on the pollution source concentration data for different months in 2015 from the invention "A Method for Estimating Pollution Source Emissions into Rivers at the Cross-sectional Scale," the source tracing results are as follows: Figure 6 and Figure 7 As shown. Figure 6 Contribution rate of different pollution sources in the Binan River Basin in April 2015 (dashed line represents contribution rate); Figure 7 The contribution rate of pollution sources in April 2015 obtained by the method of the present invention (dashed line represents the contribution rate) is obviously able to trace pollution sources well and reflect the contribution of different monitoring factors to pollution sources.

Claims

1. A cross-sectional watershed source tracing method based on deep Bayesian neural networks, comprising: Obtain real-time monitoring data from different cross-sections; The real-time monitoring data is analyzed and synthesized to obtain an initial dataset; The initial dataset is preprocessed and cleaned to obtain the contribution of various monitoring factors under different pollution sources, and an input dataset is generated. Based on expert knowledge and experience, a deep Bayesian neural network is designed, which includes: a Bayesian network, a deep generative model, and a variational autoencoder; The input dataset is fed into the Bayesian neural network, and the Bayesian network is trained and inferred; the deep generative model and the variational autoencoder work together to complete the learning of the Bayesian network; Based on the source tracing target, the pollution source contribution rate is selected as the evaluation index. The evaluation index is output as the evaluation result of the deep Bayesian neural network to obtain the pollution source contribution of different monitoring factors in different pollution sources, thereby realizing the source tracing of pollution sources. in, The deep generative model includes a DNN module and a multi-layer memory module. The deep generative model, in the DNN module, calculates the input dataset h... in Calculate low-level generated data h g Next, a linear combination strategy is used to extract knowledge from the multi-layer memory module, and then the element-wise multilayer perceptron is defined as a combination function h. out ; The Bayesian network can be represented by a directed acyclic graph G=(V,E), where V is the set of nodes, each node representing an observable variable or a latent variable, and E is the set of edges, each edge representing a directed relation. The joint probability distribution can be factorized as follows: but G (in) Let G represent the set of parent nodes of v; let parent nodes represent online monitoring indicators, and let child nodes represent the proportion of different pollution types; let directed edges between nodes represent their causal relationships, and let conditional probabilities between nodes represent the strength of the relationship. The input dataset is constructed as follows Its distribution is P D (X) The deep generative model is used to approximate the joint distribution of the input dataset. P θ (X) Where θ is the model parameter, θ can be parameterized as a deep neural network (DNN) model. By minimizing the statistical divergence between two probability distributions, the desired result can be achieved using... P θ (X) Approximation P D (X) ; The variational autoencoder consists of a Q-network and a P-network. The P-network contains multiple memory modules, so the Q-network and P-network are not symmetrical. In the P-network, θ is the parameter set in the deep generative model, and the input is x, with z being latent variables. The Q-network extracts features using a nonlinear function. h (i) .

2. The method for tracing the source of a cross-section watershed based on a deep Bayesian neural network as described in claim 1, characterized in that... The data analysis process includes abstracting the sources of pollutants, namely industrial pollution, urban pollution, non-point source pollution, and unknown sources; combining the sources of the pollutants and the monitoring data, 10 monitoring factors are statistically analyzed, namely: total phosphorus, ammonia nitrogen, chemical oxygen demand, pH value, fluoride, permanganate index, volatile phenols, dissolved oxygen, petroleum hydrocarbons, and five-day biochemical oxygen demand.

3. The cross-sectional watershed source tracing method based on deep Bayesian neural networks as described in claim 1, characterized in that... The deep neural network model in the deep generative model is used to simulate the nonlinear correlation between monitoring factors such as total phosphorus, ammonia nitrogen, chemical oxygen demand, pH value, fluoride, permanganate index, volatile phenols, dissolved oxygen, petroleum hydrocarbons, and five-day biochemical oxygen demand during source tracing. The training process of the deep neural network model is as follows: (a) Define the loss function in: o i The output of node i is the output when it is applied to sample i. y i For the sample i Target value, N The total number of samples; (b) Network initialization: Determine the number of nodes in the input layer (i.e., the dimension of the samples), the number of nodes in the hidden layer, and the number of nodes in the output layer. The output layer represents the contribution of various contaminants, and all indicators are 1-dimensional. (c) Calculation of hidden layer output in, w ij For the hidden layer j Node to the parent node i The weights, n The number of nodes in the previous layer. x i Let a be the variable of the i-th node in the previous layer. i Threshold for hidden layer nodes; (d) Output layer computation Where, w j 'The hidden layer node corresponding to the output layer node' j The weights, l The number of hidden layer nodes, b The threshold value for the output layer nodes; (e) Weight update Where η is the learning rate; (f) Weight update Among them, a i Threshold for hidden layer nodes; b The threshold value for the output layer nodes; o The output calculated for the output layer; y The actual value; (g) Determine if the iteration has ended. If it has not ended, proceed to step (c).

4. The method for tracing the source of a cross-section watershed based on a deep Bayesian neural network as described in claim 1, characterized in that... The evaluation index uses mean absolute error (MAE), which refers to the average absolute value of the errors between corresponding points in the predicted data and the original data. A smaller MAE value indicates better prediction performance. The mathematical expression is:

5. The method for tracing the source of a cross-section watershed based on a deep Bayesian neural network as described in claim 4, characterized in that... The training set, validation set, and test set of the deep Bayesian neural network model change with the number of iterations (epcoch). The deep Bayesian neural network model achieves its optimal state at epoch=15. Before epoch=15, the training set, validation set, and test set all follow a monotonically decreasing pattern. When epcoch is greater than 15, the loss of the training set continues to decrease, while the loss of the test set and validation set increases. At this point, overfitting begins to occur.

Images