Sewage treatment total nitrogen prediction method based on convolution time sequence self-attention mechanism

Abstract

The invention discloses a sewage treatment total nitrogen prediction method based on a convolutional time sequence self-attention mechanism, and relates to a sewage treatment total nitrogen prediction model based on a convolutional neural network, a long-short term memory network, a multi-head self-attention mechanism and a PID search optimization algorithm. According to the model, the advantages of the convolutional neural network in feature extraction, the powerful capability of the long and short term memory network in time series data modeling and the unique advantages of the self-attention mechanism in inter-feature relevance modeling are comprehensively utilized, and global optimization is performed on the model through a PID search optimization algorithm, so that the prediction precision and stability are remarkably improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of data processing, and specifically is a method for predicting total nitrogen in sewage treatment based on a convolutional temporal self-attention mechanism. Background Art

[0002] As a key facility in environmental protection, sewage treatment plants play a vital role in the sewage treatment process. With the acceleration of industrialization and the increase in domestic sewage discharge, total nitrogen emissions have become one of the main problems in the operation of sewage treatment plants. Excessive total nitrogen emissions not only pollute the water environment, but may also cause eutrophication of water bodies, which in turn seriously affects the ecological balance. Therefore, accurately predicting the total nitrogen emissions in the sewage treatment process has become the key to improving sewage treatment efficiency, optimizing the treatment process, and protecting the ecological environment.

[0003] Most of the existing total nitrogen prediction methods for sewage treatment are based on traditional statistical methods, machine learning or basic neural network technology (single architecture models such as MLP, RNN, LSTM, etc.). However, these methods usually have the following shortcomings: First, they cannot effectively process complex time series data, resulting in low prediction accuracy on a long time scale; second, it is difficult to capture the complex nonlinear relationship between different input features, which affects the prediction accuracy of the model; third, most existing methods ignore the adaptability of different sewage treatment plants under different working conditions, have poor generalization capabilities, and are difficult to cope with diverse actual needs. Because traditional models cannot fully utilize these high-dimensional and nonlinear data features, the accuracy and reliability of the prediction results are affected. When the total nitrogen prediction output error of a sewage treatment plant is large, it may lead to a series of serious problems in actual production and environmental management, including the following aspects:

[0004] Environmental compliance issues: During operation, wastewater treatment plants must comply with various emission standards, especially restrictions on nitrogen pollution. Excessive discharge of total nitrogen concentration will lead to eutrophication of water bodies, seriously affecting the aquatic ecosystem and causing water pollution. Therefore, large prediction errors may cause managers to be unable to adjust the treatment process in a timely manner, resulting in excessive nitrogen pollution discharge.

[0005] Waste of resources: If the total nitrogen forecast is inaccurate, the sewage treatment plant may invest too many resources (such as chemicals, energy, etc.) to ensure the treatment effect. However, over-treatment not only increases operating costs, but also causes waste of resources. On the contrary, if the forecast is too optimistic and the treatment effect is not up to standard, more repairs and supplementary treatments may be required afterwards, further increasing additional expenses and labor costs.

[0006] Unstable water quality: Errors in total nitrogen prediction will lead to untimely adjustments during the treatment process, further affecting the stability of water quality. Water quality fluctuations not only affect the water quality of downstream water bodies, but may also affect the stable operation of subsequent processes and equipment in the treatment plant. Large errors may cause water quality to fail to meet drinking water standards or discharge standards, and even cause related quality accidents.

[0007] Inefficient treatment: Inaccurate total nitrogen predictions will directly affect the adjustment of wastewater treatment processes, especially in terms of nitrogen removal. For example, some biological treatment systems (such as nitrification and denitrification processes) are extremely sensitive to nitrogen removal efficiency. If the treatment parameters are not adapted to the actual nitrogen load changes, it may lead to overload or underload of the system, resulting in inability to fully utilize the treatment capacity, thereby extending the treatment cycle and reducing the overall operating efficiency.

[0008] Decision lag: If the total nitrogen prediction model has too large an error, managers will not be able to make scientific decisions in a timely manner, thus affecting the operation and long-term planning of the sewage treatment plant. Accurate total nitrogen prediction can provide early warning and help managers take effective adjustment measures quickly when sewage quality changes. However, large prediction errors may cause decision lags and increase the difficulty of treatment.

[0009] Ecological and environmental impact: Excessive nitrogen discharge from sewage treatment plants, especially sewage containing excessive total nitrogen discharged into rivers and lakes, will cause eutrophication of water bodies, deterioration of water quality, growth of aquatic plants, and even the emergence of "dead water areas". The imbalance of the ecosystem will affect local fisheries, agriculture and even biodiversity. Therefore, accurate total nitrogen prediction is not only related to the operational benefits of sewage treatment plants, but also to the protection and sustainable development of the ecological environment.

[0010] Difficulties in operation and maintenance monitoring: With the real-time collection and continuous updating of monitoring data, sewage treatment plants are increasingly dependent on data. If the prediction error is large, it may cause feedback lag or inaccurate information from the monitoring system, affecting the response of the automatic control system. Without accurate prediction information, operators cannot grasp the actual situation of the treatment plant in a timely manner, which increases the complexity of human intervention and real-time monitoring.

[0011] The impact of aging technology and equipment is increasing: The equipment used in many sewage treatment plants is approaching the end of its service life and its performance is gradually declining. When the total nitrogen prediction accuracy is poor, it may lead to the inability to identify and respond to equipment aging in a timely manner. Misjudgment of equipment performance will further aggravate the fluctuation and instability of sewage treatment, affecting the normal operation and maintenance scheduling of equipment.

[0012] Inaccurate feature capture of sewage data: Data in the sewage treatment process usually has strong time series, mutual influence between multiple variables, periodic fluctuations, noise and missing values. Traditional statistical methods and machine learning models often ignore these complex features when processing such complex data, resulting in insufficient feature extraction capabilities. Compared with time series data in the fields of finance and meteorology, the characteristics of sewage treatment data are more complex, so more sophisticated and multi-level feature extraction methods are required to effectively capture the time series dependencies and cross-variable relationships in the data. However, existing technologies often ignore these factors when processing sewage data, resulting in the inability of the model to accurately extract key information from the data, thereby affecting the prediction accuracy.

[0013] Insufficient feature capture capabilities: Many current prediction models, especially those based on traditional machine learning, have difficulty capturing the complex nonlinear relationships between input features. Changes in total nitrogen during sewage treatment are affected by a variety of factors, including water quality parameters, process conditions, and seasonal changes, and the interactions between these factors are very complex. Most existing models rely on simple linear or single nonlinear relationships and are unable to fully capture the complex dynamic relationships between the features, resulting in instability and large errors in prediction results.

[0014] Poor model generalization ability: Existing deep learning models are prone to gradient vanishing or gradient exploding problems during training, especially in deep networks, resulting in poor model training convergence and weak generalization ability to new data. Many models show large fluctuations in different sewage treatment plants or under different operating conditions, lacking the robustness to cope with variable working conditions and different types of sewage data.

[0015] Weak ability to process time series data: Total nitrogen data in sewage treatment has significant time series and periodicity. Although traditional regression models and some deep learning models (such as RNN) can capture time dependence, they often cannot effectively utilize the information of each time step in the case of long time series, resulting in insufficient prediction accuracy. Especially when processing long-term and high-frequency monitoring data, the existing models have weak modeling capabilities for time series information and cannot capture long-term trends and complex fluctuations.

[0016] Insufficient flexibility and adaptability: The network structure and hyperparameter settings of existing prediction models are relatively fixed, making it difficult to adapt to the complex changes in characteristics during sewage treatment. Many models do not take into account the differences in different seasons and operating conditions, and cannot flexibly adapt to the needs of different sewage treatment plants. In addition, the existing models have poor processing capabilities for noise and missing values, which can easily lead to performance degradation and lack sufficient robustness.

[0017] Lack of global optimization capabilities: Although many existing models perform well in predictive capabilities, they are often not optimized properly due to insufficient hyperparameter optimization or inability to perform global optimization. Traditional hyperparameter optimization methods, such as grid search and random search, can improve model performance, but due to large computational complexity and low efficiency, it is difficult to achieve the optimal solution in large-scale data sets or complex models. In addition, existing optimization methods lack adaptability to complex dynamic changes in the sewage treatment process, resulting in the model performing less than expected in actual applications. Summary of the invention

[0018] In view of the shortcomings of the prior art, the present invention proposes a method for predicting total nitrogen in sewage treatment based on a convolutional temporal self-attention mechanism, comprising the following steps: Step S1: Collecting total nitrogen output data for preprocessing, obtaining total nitrogen input characteristics, and constructing a data set based on them; Step S2: construct a total nitrogen output prediction model, which includes an input module, a data processing module and an output module. The total nitrogen output features of the data set in step S1 are imported into the input module to obtain coding features in a unified format. Step S3: importing the encoded features in step S2 into the data processing module to obtain the temporal dependency features; the data processing module includes a convolutional layer, a long short-term memory network layer and a multi-head self-attention layer connected in series; Step S4: importing the timing dependency features of step S3 into the output module to obtain the predicted total ammonia output; Step S5: training the model. During the training process, the parameters of the model are optimized based on the improved particle swarm algorithm. The parameters include: the input dimension of the encoding feature, the hidden layer dimension of the convolutional layer, the number of hidden units of the long short-term memory network layer, the number of attention heads of the multi-head self-attention layer, and the regularization ratio of the multi-head self-attention layer; Step S6: Import the real-time total nitrogen output characteristics into the optimized model to obtain the real-time total nitrogen prediction output.

[0019] Furthermore, step S1 is specifically as follows: Step S11: Collecting total nitrogen output data, including influent ammonia nitrogen, chemical oxygen demand, influent total nitrogen, oxidation ditch aerobic section and oxidation ditch anoxic section; Step S12: Clean and standardize the total nitrogen output data in turn, obtain the total nitrogen input features, and construct a data set based on them.

[0020] Further, the input module in step S2 includes a linear layer and a position encoding layer connected in series in sequence; Step S2 is specifically as follows: Step S21: The linear layer receives the total nitrogen input feature and maps the total nitrogen input feature to the hidden layer dimension through linear transformation, and then performs layer normalization to obtain linear features in a unified format; it is expressed as: ; ; in, represents the total ammonia input characteristics; Indicates the batch size; Indicates the length of the sequence; Indicates the input dimension; represents the hidden layer input features, represents the linear layer weight matrix, represents the bias term of the linear layer; Represents linear features in a unified format; Representation layer normalization function; and Represent the mean and standard deviation of the current layer respectively, and All are learnable parameters; Step S22: The position coding layer obtains the coding feature by marking the position information of each element of the linear feature in the sequence format through the position coding. ; expressed as: ; ; ; in, represents the position code, Indicates the position of the word in the sequence, i is the index of the encoding dimension, represents the dimension of the encoding, and An even number.

[0021] Furthermore, the data processing module in step S3 includes an embedding layer, a convolutional layer, a multi-head long short-term memory network layer, and a multi-head self-attention mechanism layer connected in series in sequence; Step S3 is specifically as follows: Step S31: The embedding layer converts the encoded features into continuous vector representations to obtain embedded features; expressed as: ; in, represents the embedded features, Represents feature transformation; Step S32: The convolution layer performs convolution pooling processing on the embedded features to capture local features and obtain convolution features; Step S33: The long short-term memory network layer processes the convolutional features, captures the long-term dependency between the total ammonia data through the gating mechanism, and obtains the temporal dependency features; Step S34: The multi-head self-attention mechanism processes the temporal dependency features to obtain attention features.

[0022] Further, the output module in step S4 includes a fully connected layer; Step S4 is specifically as follows: The fully connected layer linearly transforms the attention features to obtain the final output, which is the predicted total ammonia output; it is expressed as: ; in, represents the predicted total ammonia output; Indicates attention characteristics; Represents the preset weight matrix; Represents the bias term.

[0023] Furthermore, the convolutional layer in step S32 adopts a CNN convolutional network, specifically: Step S321: The convolution kernel of the convolution layer is embedded in the input feature Slide up and obtain the output feature map through dot multiplication and weighted summation , expressed as: ; in, represents the convolution kernel, Represents the convolution operation; Step S322: Increase the output feature map through the activation function Nonlinear ability to obtain nonlinear characteristics , expressed as: ; in, Represents the RELU activation function; Step S323: Obtain convolutional features by pooling the size of nonlinear features and retaining important features , expressed as: ; in, Represents the maximum pooling function.

[0024] Further, step S33 is specifically as follows: Step S331: Convolution feature Transpose and adjust its format to obtain time series input features ; Step S332: The long short-term memory network layer passes through the input gate , Forget Gate and output gate Control the flow of time-series input features and obtain time-series dependent features by updating cell states, specifically: ; ; ; ; ; ; in, represents the candidate cell state, Indicates updating the cell status. Indicates the hidden state, , , and Both represent the preset weight matrix, , , and Both represent preset weight parameters. represents the SigMoid activation function, represents the tanh activation function, represents the hidden state, that is, the time-series dependent feature, Indicates the hidden state at the previous moment.

[0025] Further, step S34 is specifically as follows: Step S341: Calculate query vectors based on the input time-dependent features , value vector and key vector ; expressed as: ; ; ; in, and All are learnable weight matrices; Step S342: Calculate self-attention weight ; expressed as: ; in, Represents the dimension of the key vector, which is used to scale the dot product to prevent excessive gradients; Step S343: Calculate self-attention according to self-attention weight ; expressed as: ; in, represents the concatenation function; represents a linear transformation matrix; represents the i-th self-attention weight; represents the nth self-attention weight; Step S344: Perform residual connection on the input time-dependent features and self-attention; obtain combined features ; expressed as: ; Step S345: perform layer normalization on the result after residual connection; obtain conversion features , expressed as: ; in, Representation layer normalization function; Step S346: Convert the features Input into the feedforward network for dimension conversion and regularization to obtain attention features; The feedforward network includes the first fully connected layer and the second fully connected layer connected in series; it is expressed as: ; ; ; in, represents the activation function; represents the regularization function; represents the output of the first fully connected layer; represents the dimension conversion function, represents the output of the second fully connected layer; Indicates attention characteristics; represents the regularization function.

[0026] Furthermore, in step S6, the parameters of the model are optimized based on the improved particle swarm algorithm, specifically: Step S61: Initialize the particle swarm population; expressed as: ; ; in, represents the individual position of the i-th particle in the j-th dimension; and are the upper and lower bounds of the j-th variable respectively; is a random number between 0 and 1, Indicates the total number of variables; Step S62: Determine the fitness function ; Specifically: ; in, represents the predicted value of the model based on the individual position of the i-th particle; represents the true value of the model based on the i-th particle; And update the population according to the fitness function, in this process, obtain the deviation between the current solution and the optimal solution; Specifically: If the current particle individual position The fitness value is better than the historical optimal fitness value, then the individual optimal solution is updated , expressed as: , ; in, represents the fitness value of the individual particle position, Represents the fitness value of the individual optimal solution;

[0027] For the global optimal solution , if the fitness value of the current particle individual position is better than the global optimal fitness value , then update the global optimal solution : ; deviation It is expressed as:

[0028] or ; Step S63: Search and adjust the deviation in step S62 based on the PID controller, so as to update the population position; specifically: ; in, It represents the deviation at the current moment; represents the proportionality coefficient, represents the integral coefficient; represents the differential coefficient; Represents the output of the PID controller, which is used to adjust the particle search strategy; represents the integral term; represents the differential term; According to the output of the PID controller, the search position of each particle in the population is updated; it is expressed as: ; in, Indicates the individual position of the particle before updating; Represents the updated individual particle position; Step S64: Optimize the search process based on the Levy flight function search, enhance the global search capability, and further update the population; specifically: Generate Levy step , expressed as: ; in, Represents the standard normal distribution Randomly generated variables in Represents the extended parameter, which is used to adjust the step size. is the parameter of the Levy distribution, which is a random number between (1,2); is the step length index; The position update formula is: ; in, Indicates the individual position of the particle after updating again; Step S65: Evaluate the fitness of each individual particle position according to the fitness function to obtain the individual particle position corresponding to the optimal fitness value; Repeat S61 to S65 until the optimal individual particle position is obtained. As output;

[0029] Step S66: Format adjustment of the optimal individual particle position; obtaining the individual particle position after format adjustment , expressed as: ; in, Represents a formatting function.

[0030] Compared with the prior art, the present invention has the following beneficial effects:

[0031] 1) The present invention combines convolutional layers, long short-term memory networks and multi-head self-attention mechanisms to provide strong support for modeling time series data in the sewage treatment process. Specifically, the convolutional layer focuses on the extraction of local features and can quickly capture the key features in the data, while the long short-term memory network layer effectively handles long-term temporal dependencies and learns the impact of historical data on future predictions; the multi-head self-attention mechanism can intelligently focus on the time steps most relevant to the change in total nitrogen emissions by weighting the importance of each moment in the input data, thereby enhancing the model's memory capacity and accuracy. Compared with traditional models, this structure can more accurately predict future total nitrogen emissions in the context of long-term spans and high-frequency monitoring data. It effectively improves the model's ability to handle long-period, complex fluctuating data, especially when dealing with sewage data with strong seasonality and periodicity, and can make more stable and accurate predictions. By introducing activation functions and regularization techniques, the present invention enhances the model's nonlinear fitting ability, enabling it to adapt to various changes in the sewage treatment process, such as different pollutant concentrations, different types of sewage sources, different seasons and climatic conditions. Especially when faced with nonlinear and non-stationary sewage data, the model can automatically adjust parameters and structures to maintain a high prediction accuracy. In addition, when faced with abnormal data (such as noise and missing values), the model has strong robustness and can effectively deal with the incompleteness and uncertainty of the data, maintain the stability of the model performance, and avoid the significant impact of data anomalies on the prediction results.

[0032] 2) The PID search optimization algorithm introduced in the present invention can accurately adjust the hyperparameters in the model, and further improve the performance of the model by optimizing the model structure and weights. The PID algorithm has strong adaptive capabilities and can adjust the model parameters according to real-time feedback. It can effectively balance the convergence speed and accuracy of the model during the training process, ensuring that the model can achieve optimal performance in different data sets and practical applications. Compared with traditional optimization methods, the PID algorithm can better explore and optimize the hyperparameter space, avoid overfitting, improve the generalization ability of the model under various working conditions, and improve the reliability and accuracy of the prediction results. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 This is a flowchart of the steps of a total nitrogen prediction method for sewage treatment based on convolutional temporal self-attention mechanism. DETAILED DESCRIPTION

[0034] In order to make the technical means, creative features, objectives and effects achieved by the present invention easy to understand, in the description of the present invention, it should be noted that the terms "center", "up", "down", "left", "right", "vertical", "horizontal", "inside", "outside" and the like indicate the orientation or position relationship based on the orientation or position relationship shown in the accompanying drawings, which is only for the convenience of describing the present invention and simplifying the description, and does not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present invention. In addition, the terms "No. 1", "No. 2" and "No. 3" are only used for descriptive purposes and cannot be understood as indicating or implying relative importance. The present invention is further explained below in conjunction with specific embodiments.

[0035] Reference Figure 1 , a total nitrogen prediction method for sewage treatment based on convolutional temporal self-attention mechanism, comprising the following steps: Step S1: Collecting total nitrogen output data for preprocessing, obtaining total nitrogen input characteristics, and constructing a data set based on them; Step S2: construct a total nitrogen output prediction model, which includes an input module, a data processing module and an output module. The total nitrogen output features of the data set in step S1 are imported into the input module to obtain coding features in a unified format. Step S3: importing the encoded features in step S2 into the data processing module to obtain the temporal dependency features; the data processing module includes a convolutional layer, a long short-term memory network layer and a multi-head self-attention layer connected in series; Step S4: importing the timing dependency features of step S3 into the output module to obtain the predicted total ammonia output; Step S5: training the model. During the training process, the parameters of the model are optimized based on the improved particle swarm algorithm. The parameters include: the input dimension of the encoding feature, the hidden layer dimension of the convolutional layer, the number of hidden units of the long short-term memory network layer, the number of attention heads of the multi-head self-attention layer, and the regularization ratio of the multi-head self-attention layer; Step S6: Import the real-time total nitrogen output characteristics into the optimized model to obtain the real-time total nitrogen prediction output.

[0036] Furthermore, step S1 is specifically as follows: Step S11: Collecting total nitrogen output data, including influent ammonia nitrogen, chemical oxygen demand, influent total nitrogen, oxidation ditch aerobic section and oxidation ditch anoxic section; Step S12: Clean and standardize the total nitrogen output data in turn, obtain the total nitrogen input features, and construct a data set based on them.

[0037] Further, the input module in step S2 includes a linear layer and a position encoding layer connected in series in sequence; Step S2 is specifically as follows: Step S21: The linear layer receives the total nitrogen input feature and maps the total nitrogen input feature to the hidden layer dimension through linear transformation, and then performs layer normalization to obtain linear features in a unified format; it is expressed as: ; ; in, represents the total ammonia input characteristics; Indicates the batch size; Indicates the length of the sequence; Indicates the input dimension; represents the hidden layer input features, represents the linear layer weight matrix, represents the bias term of the linear layer; Represents linear features in a unified format; Representation layer normalization function; and Represent the mean and standard deviation of the current layer respectively, and All are learnable parameters; Step S22: The position coding layer obtains the coding feature by marking the position information of each element of the linear feature in the sequence format through the position coding. ; expressed as: ; ; ; in, represents the position code, Indicates the position of the word in the sequence, i is the index of the encoding dimension, represents the dimension of the encoding, and An even number.

[0038] Furthermore, the data processing module in step S3 includes an embedding layer, a convolutional layer, a multi-head long short-term memory network layer, and a multi-head self-attention mechanism layer connected in series in sequence; Step S3 is specifically as follows: Step S31: The embedding layer converts the encoded features into continuous vector representations to obtain embedded features; expressed as: ; in, represents the embedded features, Represents feature transformation; Step S32: The convolution layer performs convolution pooling processing on the embedded features to capture local features and obtain convolution features; Step S33: The long short-term memory network layer processes the convolutional features, captures the long-term dependency between the total ammonia data through the gating mechanism, and obtains the temporal dependency features; Step S34: The multi-head self-attention mechanism processes the temporal dependency features to obtain attention features.

[0039] Further, the output module in step S4 includes a fully connected layer; Step S4 is specifically as follows: The fully connected layer linearly transforms the attention features to obtain the final output, which is the predicted total ammonia output; it is expressed as: ; in, represents the predicted total ammonia output; Indicates attention characteristics; Represents the preset weight matrix; Represents the bias term.

[0040] Furthermore, the convolutional layer in step S32 adopts a CNN convolutional network, specifically: Step S321: The convolution kernel of the convolution layer is embedded in the input feature Slide up and obtain the output feature map through dot multiplication and weighted summation , expressed as: ; in, represents the convolution kernel, Represents the convolution operation; Step S322: Increase the output feature map through the activation function Nonlinear ability to obtain nonlinear characteristics , expressed as: ; in, Represents the RELU activation function; Step S323: Obtain convolutional features by pooling the size of nonlinear features and retaining important features , expressed as: ; in, Represents the maximum pooling function.

[0041] Further, step S33 is specifically as follows: Step S331: Convolution feature Transpose and adjust its format to obtain time series input features ; Step S332: The long short-term memory network layer passes through the input gate , Forget Gate and output gate Control the flow of time-series input features and obtain time-series dependent features by updating cell states, specifically: ; ; ; ; ; ; in, represents the candidate cell state, Indicates updating the cell status. Indicates the hidden state, , , and Both represent the preset weight matrix, , , and Both represent preset weight parameters. represents the SigMoid activation function, represents the tanh activation function, represents the hidden state, that is, the time-series dependent feature, Indicates the hidden state at the previous moment.

[0042] Further, step S34 is specifically as follows: Step S341: Calculate query vectors based on the input time-dependent features , value vector and key vector ; expressed as: ; ; ; in, and All are learnable weight matrices; Step S342: Calculate self-attention weight ; expressed as: ; in, Represents the dimension of the key vector, which is used to scale the dot product to prevent excessive gradients; Step S343: Calculate self-attention according to self-attention weight ; expressed as: ; in, represents the concatenation function; represents a linear transformation matrix; represents the i-th self-attention weight; represents the nth self-attention weight; Step S344: Perform residual connection on the input time-dependent features and self-attention; obtain combined features ; expressed as: ; Step S345: perform layer normalization on the result after residual connection; obtain conversion features , expressed as: ; in, Representation layer normalization function; Step S346: Convert the features Input into the feedforward network for dimension conversion and regularization to obtain attention features; The feedforward network includes the first fully connected layer and the second fully connected layer connected in series; it is expressed as: ; ; ; in, represents the activation function; represents the regularization function; represents the output of the first fully connected layer; represents the dimension conversion function, represents the output of the second fully connected layer; Indicates attention characteristics; represents the regularization function.

[0043] Furthermore, in step S6, the parameters of the model are optimized based on the improved particle swarm algorithm, specifically: Step S61: Initialize the particle swarm population; expressed as: ; ; in, represents the individual position of the i-th particle in the j-th dimension; and are the upper and lower bounds of the j-th variable respectively; is a random number between 0 and 1, Indicates the total number of variables; Step S62: Determine the fitness function ; Specifically: ; in, represents the predicted value of the model based on the individual position of the i-th particle; represents the true value of the model based on the i-th particle; And update the population according to the fitness function, in this process, obtain the deviation between the current solution and the optimal solution; Specifically: If the current particle individual position The fitness value is better than the historical optimal fitness value, then the individual optimal solution is updated , expressed as: , ; in, represents the fitness value of the individual particle position, Represents the fitness value of the individual optimal solution;

[0044] For the global optimal solution , if the fitness value of the current particle individual position is better than the global optimal fitness value , then update the global optimal solution : ; deviation It is expressed as:

[0045] or ; Step S63: Search and adjust the deviation in step S62 based on the PID controller, so as to update the population position; specifically: ; in, It represents the deviation at the current moment; represents the proportionality coefficient, represents the integral coefficient; represents the differential coefficient; Represents the output of the PID controller, which is used to adjust the particle search strategy; represents the integral term; represents the differential term; According to the output of the PID controller, the search position of each particle in the population is updated; it is expressed as: ; in, Indicates the individual position of the particle before updating; Represents the updated individual particle position; Step S64: Optimize the search process based on the Levy flight function search, enhance the global search capability, and further update the population; specifically: Generate Levy step , expressed as: ; in, Represents the standard normal distribution Randomly generated variables in Represents the extended parameter, which is used to adjust the step size. is the parameter of the Levy distribution, which is a random number between (1,2); is the step length index; The position update formula is: ; in, Indicates the individual position of the particle after updating again; Step S65: Evaluate the fitness of each individual particle position according to the fitness function to obtain the individual particle position corresponding to the optimal fitness value; Repeat S61 to S65 until the optimal individual particle position is obtained. As output;

[0046] Step S66: Format adjustment of the optimal individual particle position; obtaining the individual particle position after format adjustment , expressed as: ; in, Represents a formatting function.

[0047] The specific implementation modes as described above further describe the purpose, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific implementation mode of the present invention and is not intended to limit the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for predicting total nitrogen in sewage treatment based on convolutional temporal self-attention mechanism, characterized in that: The following steps are involved: Step S1: Collecting total nitrogen output data for preprocessing, obtaining total nitrogen input characteristics, and constructing a data set based on them; Step S2: construct a total nitrogen output prediction model, which includes an input module, a data processing module and an output module. The total nitrogen output features of the data set in step S1 are imported into the input module to obtain coding features in a unified format. Step S3: importing the encoded features in step S2 into the data processing module to obtain the temporal dependency features; the data processing module includes a convolutional layer, a long short-term memory network layer and a multi-head self-attention layer connected in series; Step S4: importing the timing dependency features of step S3 into the output module to obtain the predicted total ammonia output; Step S5: training the model. During the training process, the parameters of the model are optimized based on the improved particle swarm algorithm. The parameters include: the input dimension of the encoding feature, the hidden layer dimension of the convolutional layer, the number of hidden units of the long short-term memory network layer, the number of attention heads of the multi-head self-attention layer, and the regularization ratio of the multi-head self-attention layer; Step S6: Import the real-time total nitrogen output characteristics into the optimized model to obtain the real-time total nitrogen prediction output.

2. According to claim 1, a method for predicting total nitrogen in sewage treatment based on convolutional temporal self-attention mechanism is characterized in that: Step S1 is specifically as follows: Step S11: Collecting total nitrogen output data, including influent ammonia nitrogen, chemical oxygen demand, influent total nitrogen, oxidation ditch aerobic section and oxidation ditch anoxic section; Step S12: Clean and standardize the total nitrogen output data in turn, obtain the total nitrogen input features, and construct a data set based on them.

3. According to claim 1, a method for predicting total nitrogen in sewage treatment based on convolutional temporal self-attention mechanism is characterized in that: The input module in step S2 includes a linear layer and a position encoding layer connected in series; Step S2 is specifically as follows: Step S21: The linear layer receives the total nitrogen input feature and maps the total nitrogen input feature to the hidden layer dimension through linear transformation, and then performs layer normalization to obtain linear features in a unified format; it is expressed as: ； ； in, represents the total ammonia input characteristics; Indicates the batch size; Indicates the length of the sequence; Indicates the input dimension; represents the hidden layer input features, represents the linear layer weight matrix, represents the bias term of the linear layer; Represents linear features in a unified format; Representation layer normalization function; and Represent the mean and standard deviation of the current layer respectively, and All are learnable parameters; Step S22: The position coding layer obtains the coding feature by marking the position information of each element of the linear feature in the sequence format through the position coding. ; expressed as: ； ； ； in, represents the position code, Indicates the position of the word in the sequence, i is the index of the encoding dimension, represents the dimension of the encoding, and Is an even number.

4. According to claim 3, a method for predicting total nitrogen in sewage treatment based on convolutional temporal self-attention mechanism is characterized in that: The data processing module in step S3 includes an embedding layer, a convolutional layer, a multi-head long short-term memory network layer, and a multi-head self-attention mechanism layer connected in series in sequence; Step S3 is specifically as follows: Step S31: The embedding layer converts the encoded features into continuous vector representations to obtain embedded features; expressed as: ； in, represents the embedded features, Represents feature transformation; Step S32: The convolution layer performs convolution pooling processing on the embedded features to capture local features and obtain convolution features; Step S33: The long short-term memory network layer processes the convolutional features, captures the long-term dependency between the total ammonia data through the gating mechanism, and obtains the temporal dependency features; Step S34: The multi-head self-attention mechanism processes the temporal dependency features to obtain attention features.

5. According to claim 4, a method for predicting total nitrogen in sewage treatment based on convolutional temporal self-attention mechanism is characterized in that: The output module in step S4 includes a fully connected layer; Step S4 is specifically as follows: The fully connected layer linearly transforms the attention features to obtain the final output, which is the predicted total ammonia output; it is expressed as: ； in, represents the predicted total ammonia output; Indicates attention characteristics; Represents the preset weight matrix; Represents the bias term.

6. According to claim 4, a method for predicting total nitrogen in sewage treatment based on convolutional temporal self-attention mechanism is characterized in that: The convolution layer in step S32 uses a CNN convolutional network, specifically: Step S321: The convolution kernel of the convolution layer is embedded in the input feature Slide up and obtain the output feature map through dot multiplication and weighted summation , expressed as: ； in, represents the convolution kernel, Represents the convolution operation; Step S322: Increase the output feature map through the activation function Nonlinear ability to obtain nonlinear characteristics , expressed as: ； in, Represents the RELU activation function; Step S323: Obtain convolutional features by pooling the size of nonlinear features and retaining important features , expressed as: ； in, Represents the maximum pooling function.

7. A method for predicting total nitrogen in sewage treatment based on convolutional temporal self-attention mechanism according to claim 6, characterized in that: Step S33 is specifically as follows: Step S331: Convolution feature Transpose and adjust its format to obtain time series input features ; Step S332: The long short-term memory network layer passes through the input gate , Forget Gate and output gate Control the flow of time-series input features and obtain time-series dependent features by updating cell states, specifically: ； ； ； ； ； ； in, represents the candidate cell state, Indicates updating the cell status. Indicates the hidden state, , , and Both represent the preset weight matrix, , , and Both represent preset weight parameters. represents the SigMoid activation function, represents the tanh activation function, represents the hidden state, that is, the time-series dependent feature, Represents the hidden state at the previous moment.

8. According to claim 7, a method for predicting total nitrogen in sewage treatment based on convolutional temporal self-attention mechanism is characterized in that: Step S34 is specifically as follows: Step S341: Calculate query vectors based on the input time-dependent features , value vector and key vector ; expressed as: ； ； ； in, and All are learnable weight matrices; Step S342: Calculate self-attention weight ; expressed as: ； in, Represents the dimension of the key vector, which is used to scale the dot product to prevent excessive gradients. represents the transpose of a value vector; Step S343: Calculate self-attention according to self-attention weight ; expressed as: ； in, represents the concatenation function; represents a linear transformation matrix; represents the i-th self-attention weight; represents the nth self-attention weight; Step S344: Perform residual connection on the input time-dependent features and self-attention; obtain combined features ; expressed as: ； Step S345: perform layer normalization on the result after residual connection; obtain conversion features , expressed as: ； in, Representation layer normalization function; Step S346: Convert the features Input into the feedforward network for dimension conversion and regularization to obtain attention features; The feedforward network includes the first fully connected layer and the second fully connected layer connected in series; it is expressed as: ； ； ； in, represents the activation function; represents the regularization function; represents the output of the first fully connected layer; represents the dimension conversion function, represents the output of the second fully connected layer; Indicates attention characteristics; represents the regularization function.

9. According to claim 1, a method for predicting total nitrogen in sewage treatment based on convolutional temporal self-attention mechanism is characterized in that: In step S6, the parameters of the model are optimized based on the improved particle swarm algorithm, specifically: Step S61: Initialize the particle swarm population; expressed as: ； ； in, represents the individual position of the i-th particle in the j-th dimension; and are the upper and lower bounds of the j-th variable respectively; is a random number between 0 and 1, Indicates the total number of variables; Step S62: Determine the fitness function ; Specifically: ； in, represents the predicted value of the model based on the individual position of the i-th particle; represents the true value of the model based on the i-th particle; And update the population according to the fitness function, in this process, obtain the deviation between the current solution and the optimal solution; Specifically: If the current individual particle position The fitness value is better than the historical optimal fitness value, then the individual optimal solution is updated , expressed as: ， ； in, represents the fitness value of the individual particle position, Represents the fitness value of the individual optimal solution; For the global optimal solution , if the fitness value of the current particle individual position is better than the global optimal fitness value , then update the global optimal solution : ； deviation It is expressed as: or ; Step S63: Search and adjust the deviation in step S62 based on the PID controller, so as to update the population position; specifically: ； in, It represents the deviation at the current moment; represents the proportionality coefficient, represents the integral coefficient; represents the differential coefficient; Represents the output of the PID controller, which is used to adjust the particle search strategy; represents the integral term; represents the differential term; According to the output of the PID controller, the search position of each particle in the population is updated; it is expressed as: ； in, Indicates the individual position of the particle before updating; Represents the updated individual particle position; Step S64: Optimize the search process based on the Levy flight function search, enhance the global search capability, and further update the population; specifically: Generate Levy step , expressed as: ； in, Represents the standard normal distribution Randomly generated variables in Represents the extended parameter, which is used to adjust the step size. is the parameter of the Levy distribution, which is a random number between (1,2); is the step length index; The position update formula is: ； in, Indicates the individual position of the particle after updating again; Step S65: Evaluate the fitness of each individual particle position according to the fitness function to obtain the individual particle position corresponding to the optimal fitness value; Repeat S61 to S65 until the optimal individual particle position is obtained. As output; Step S66: Format adjustment of the optimal individual particle position; obtaining the individual particle position after format adjustment , expressed as: ； in, Represents a formatting function.

Images