Water quality multivariable time sequence prediction AI model parameter automatic calibration method

Abstract

The invention discloses a water quality multivariable time series prediction AI model parameter automatic calibration method, and relates to the technical field of water quality model parameter calibration, and the method comprises the following steps: defining a hyper-parameter space of a Transform model, setting the value range of each to-be-optimized hyper-parameter in the hyper-parameter space, and obtaining an initial hyper-parameter combination and an initial loss function value; performing optimization processing on the initial hyper-parameter combination to obtain an optimal hyper-parameter combination, and training a Transform model by using the optimal hyper-parameter combination to obtain a trained model; and performing prediction performance analysis on the trained model by using the test set and the verification set to obtain a prediction performance value, analyzing the prediction performance value by using a preset performance threshold, and if the prediction performance value is smaller than the preset performance threshold, taking the trained model as a water quality multivariable time sequence prediction AI model. According to the method and the device, the performance of the Transform model in the multivariable time sequence prediction task can be improved.

Description

Technical Field

[0001] This application relates to the technical field of water quality model parameter calibration, and particularly to an automatic calibration method for AI model parameters of multi-variable time series prediction of water quality. Background Art

[0002] With the rapid development of artificial intelligence technology, deep learning models based on the Transformer architecture have shown excellent performance in time series prediction tasks, especially in complex multi-variable time series prediction scenarios such as water quality monitoring. However, the performance of the Transformer model highly depends on the selection of its hyperparameters, such as learning rate, number of layers, number of attention heads, and hidden layer dimension, etc. But the existing optimization methods have the following problems: (1) Using the traditional manual tuning method to select hyperparameters has the problems of time-consuming and laborious, and it is difficult to find the global optimal solution.

[0003] (2) Water quality prediction tasks usually involve multiple conflicting optimization goals, such as prediction accuracy, model complexity, and computational efficiency, etc. But the existing parameter optimization methods are usually single-objective optimization methods, that is: only one goal can be optimized at a time, and it is difficult to meet the actual needs.

[0004] (3) The existing parameter optimization methods (such as: grid search, random search, and genetic algorithm, etc.) have problems such as low efficiency and being easily trapped in local optima when dealing with high-dimensional parameter spaces and multi-objective optimization problems.

[0005] Therefore, there is an urgent need for an efficient and automatic multi-objective parameter calibration method to improve the performance of the Transformer model in multi-variable time series prediction tasks. Summary of the Invention

[0006] The purpose of this application is to provide an automatic calibration method for AI model parameters of multi-variable time series prediction of water quality, which can improve the performance of the Transformer model in multi-variable time series prediction tasks.

[0007] To achieve the above object, the present application provides a method for automatically calibrating AI model parameters for multi-variable time series prediction of water quality, including the following steps: S1: Obtain water quality monitoring data, and use the water quality monitoring data to construct a multi-variable time series data set, where the multi-variable time series data set includes: a training set, a validation set, and a test set; S2: Define the hyperparameter space of the Transformer model, set the value range of each hyperparameter to be optimized in the hyperparameter space, and obtain an initial hyperparameter combination and an initial loss function value; S3: Optimize the initial hyperparameter combination to obtain an optimal hyperparameter combination, and use the optimal hyperparameter combination to train the Transformer model to obtain a trained model; S4: Use the test set and the validation set to analyze the prediction performance of the trained model to obtain a prediction performance value, and use a preset performance threshold to analyze the prediction performance value. If the prediction performance value is less than the preset performance threshold, the trained model is used as the AI model for multi-variable time series prediction of water quality; if the prediction performance value is greater than or equal to the preset performance threshold, S2 is executed again.

[0008] As above, among them, the sub-steps of obtaining water quality monitoring data and using the water quality monitoring data to construct a multi-variable time series data set are as follows: S11: Determine the water quality indicators to be monitored, form the water quality monitoring data to be obtained from the water quality indicators to be monitored, and obtain the water quality monitoring data from each monitoring site according to the water quality monitoring data to be obtained; S12: Preprocess the water quality monitoring data to obtain preprocessed data; S13: Extract features from the preprocessed data to obtain time features and derivative features; S14: Integrate the preprocessed data, time features, and derivative features into a data set to form an initial multi-variable time series data set, where each column in the initial multi-variable time series data set represents a variable, and each row in the initial multi-variable time series data set represents the observed values of multiple variables at a time point; among them, the variables include: multiple water quality indicators and derivative features; the variable observed values include: the monitored values of each water quality indicator and the calculated values of each derivative feature; S15: Divide the initial multi-variable time series data set to obtain a training set, a validation set, and a test set, and use the training set, the validation set, and the test set as the multi-variable time series data set and store it.

[0009] As described above, the sub - steps for pre - processing water quality monitoring data to obtain pre - processed data are as follows: S121: Perform anomaly analysis on the water quality monitoring data. If there are missing values and / or outliers in the water quality monitoring data, then execute S122; if there are no missing values and outliers in the water quality monitoring data, then use the water quality monitoring data as the data to be processed and execute S123; S122: Process the missing values and / or outliers in the water quality monitoring data to obtain the data to be processed, and execute S123; S123: Perform standardization processing on the data to be processed, map the data to be processed to a preset interval or make the data to be processed have zero mean and unit variance, so as to obtain the pre - processed data.

[0010] As described above, the sub - steps for processing the missing values and / or outliers in the water quality monitoring data to obtain the data to be processed are as follows: S1221: If there are outliers in the water quality monitoring data, then correct the outliers greater than the preset upper limit to the upper limit value, and correct the values less than the preset lower limit to the lower limit value. After completion of the correction, obtain the corrected data and execute S1222; if there are no outliers in the water quality monitoring data, then execute S1222; S1222: If there are missing values in the water quality monitoring data or the corrected data, then use the interpolation method to interpolate the missing values in the water quality monitoring data. After completion of the interpolation, obtain the data to be processed; if there are no missing values in the corrected data, then directly use the corrected data as the data to be processed.

[0011] As described above, the hyperparameters to be optimized at least include: input time step, output time step, number of layers, number of attention heads, hidden layer dimension, batch size, and Dropout rate.

[0012] As described above, use the Sobol sequence or Latin hypercube sampling method to generate the initial hyperparameter combination.

[0013] As described above, determine the time order of the initial multivariate time - series dataset according to the time characteristics of the initial multivariate time - series dataset. The time order is from front to back. Sort the initial multivariate time - series dataset according to the time order. After completion of the sorting, determine the division point according to the preset division ratio. Divide the initial multivariate time - series dataset into a training set, a validation set, and a test set according to the division point, and use the training set, the validation set, and the test set as the multivariate time - series dataset and store them.

[0014] As described above, in which, the initial hyperparameter combination is optimized to obtain the optimal hyperparameter combination, and the Transformer model is trained using the optimal hyperparameter combination. The sub-steps for obtaining the trained model are as follows: S31: Define multiple optimization objectives, and obtain a comprehensive optimization objective according to the multiple optimization objectives. Among them, the multiple optimization objectives at least include: prediction accuracy, model complexity, and computational efficiency; S32: Use the initial hyperparameter combination as the input and the initial loss function value as the output to form fitting data; Select a kernel function, and fit the simulator according to the kernel function and the fitting data to obtain a fitted simulator; S33: Use the fitted simulator to calculate the candidate point prediction mean and candidate point standard deviation of each candidate point in the hyperparameter space, and calculate the selection boundary value according to the candidate point prediction mean and candidate point standard deviation; S34: Obtain the iterative change value of the selection boundary value. If the iterative change value is less than the preset iteration threshold, the convergence condition is satisfied, the iteration is stopped, the optimal hyperparameter combination and the corresponding loss function value are output, and S35 is executed; If the iterative change value is greater than or equal to the preset iteration threshold, the convergence condition is not satisfied, and the point with the largest selection boundary value is selected as the new hyperparameter combination, and S33 is executed; S35: Use the optimal hyperparameter combination and the corresponding loss function value to train the Transformer model to obtain the trained model.

[0015] As described above, in which, the expression of the loss function corresponding to the comprehensive optimization objective is as follows: ; Among them, is the loss function corresponding to the comprehensive optimization objective; is the weight corresponding to the th optimization objective; is the th optimization objective corresponding loss function, , is the total number of optimization objectives, is a natural number.

[0016] As described above, in which, the expression of the selection boundary value is: ; Among them, is the selection boundary value of the candidate point ; is the candidate point prediction mean of the candidate point ; is the candidate point standard deviation of the candidate point ; is a trade-off parameter.

[0017] The beneficial effects achieved by this application are as follows: (1)The automatic calibration method for the parameters of the water quality multivariate time series prediction AI model in this application can achieve the efficient and automatic calibration of the hyperparameters of the water quality multivariate time series prediction model based on the Transformer architecture, reducing manual intervention.

[0018] (2)The automatic calibration method for the parameters of the water quality multivariate time series prediction AI model in this application can simultaneously optimize multiple conflicting optimization objectives (such as prediction accuracy, model complexity, and computational efficiency), significantly improving the model performance and computational efficiency, and is applicable to complex time series prediction tasks.

[0019] (3)The automatic calibration method for the parameters of the water quality multivariate time series prediction AI model in this application can significantly reduce the computational cost and time consumption of hyperparameter search.

[0020] (4)The automatic calibration method for the parameters of the water quality multivariate time series prediction AI model in this application fully considers the mean square error between the test set and the validation set and the corresponding prediction results of the output of the trained model, can reduce the influence of outliers, fully considers the prediction stability of the trained model between different data sets, and the duration of the prediction process of the trained model between different data sets, improving the comprehensiveness and accuracy of the prediction performance analysis of the trained model. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings described below are only some embodiments recorded in this application. For those of ordinary skill in the art, other drawings can also be obtained based on these drawings.

[0022] Figure 1 FIG. is a flowchart of an embodiment of the automatic calibration method for the parameters of the water quality multivariate time series prediction AI model; Figure 2 FIG. is a schematic diagram of another embodiment of the automatic calibration method for the parameters of the water quality multivariate time series prediction AI model. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0023] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts fall within the protection scope of the present invention.

[0024] Such as Figure 1As shown in the figure, the present application provides an automatic calibration method for AI model parameters of water quality multivariate time series prediction, including the following steps: S1: Obtain water quality monitoring data, and use the water quality monitoring data to construct a multivariate time series dataset. Among them, the multivariate time series dataset includes: a training set, a validation set, and a test set.

[0025] Furthermore, the sub-steps of obtaining water quality monitoring data and using the water quality monitoring data to construct a multivariate time series dataset are as follows: S11: Determine the water quality indicators to be monitored. The water quality indicators to be monitored form the water quality monitoring data to be obtained. According to the water quality monitoring data to be obtained, obtain the water quality monitoring data from each monitoring site.

[0026] Specifically, according to the research purpose or actual needs, determine the water quality indicators to be monitored. The water quality indicators to be monitored form the water quality monitoring data to be obtained. According to the water quality monitoring data to be obtained, obtain the water quality monitoring data from each monitoring site.

[0027] The water quality monitoring data is the water quality data of each monitoring site collected at a predetermined time interval by using water quality monitoring equipment, sensors, or laboratory analysis means. Among them, the water quality monitoring data includes multiple water quality indicators, and the multiple water quality indicators include: pH value, dissolved oxygen, and turbidity, but are not limited to including: pH value, dissolved oxygen, and turbidity, and may also include other water quality indicators such as chemical oxygen demand (COD), ammonia nitrogen (NH 3 - N), heavy metal content, and total bacteria count.

[0028] S12: Preprocess the water quality monitoring data to obtain preprocessed data.

[0029] Furthermore, the sub-steps of preprocessing the water quality monitoring data to obtain preprocessed data are as follows: S121: Perform anomaly analysis on the water quality monitoring data. If there are missing values and / or outliers in the water quality monitoring data, then execute S122; if there are no missing values and outliers in the water quality monitoring data, then use the water quality monitoring data as the data to be processed and execute S123; Specifically, use a pre-trained artificial intelligence-based inspection model or existing methods to perform anomaly analysis on the water quality monitoring data.

[0030] S122: Process the missing values and / or outliers in the water quality monitoring data to obtain the data to be processed, and execute S123.

[0031] Furthermore, the sub-steps of processing the missing values and / or outliers in the water quality monitoring data to obtain the data to be processed are as follows: S1221: If there are outliers in the water quality monitoring data, then correct the outliers greater than the preset upper limit to the upper limit value, and correct the values less than the preset lower limit to the lower limit value. After the correction is completed, obtain the corrected data and execute S1222; if there are no outliers in the water quality monitoring data, then execute S1222.

[0032] Further, the expression for the upper limit value is: ; the expression for the lower limit value is: ; where is the mean value of the th water quality index in the water quality monitoring data; is the standard deviation of the th water quality index in the water quality monitoring data; is the correction threshold of the th water quality index in the water quality monitoring data.

[0033] Specifically, The specific value of

[0034] is determined according to experience or data characteristics. In this application, it is preferably 2 or 3.

[0035] Specifically, the interpolation method is used to interpolate the missing values in the water quality monitoring data, but it is not limited to the interpolation method, and other processing methods such as the deletion method and the model method can also be used.

[0036] S123: Perform standardization processing on the data to be processed, map the data to be processed to a preset interval or make the data to be processed have zero mean and unit variance, so as to obtain the preprocessed data.

[0037] Specifically, since the dimensions and value ranges of different water quality indicators are different, in order to avoid some water quality indicators having too much influence on the model, it is necessary to perform standardization processing on the data to be processed. For example, the Z-score standardization method or the Min-Max standardization method is used to map the data to be processed to a preset interval or make the data to be processed have zero mean and unit variance, but it is not limited to using the Z-score standardization method or the Min-Max standardization method.

[0038] S13: Extract features from the preprocessed data to obtain time features and derivative features.

[0039] Specifically, time-related features are extracted from the time series of the preprocessed data as time features, such as: hour, day, month, season, etc. There is an association between the time features and the water quality changes, which can help the model capture the time patterns.

[0040] According to various water quality indicators in the preprocessed data, new features are constructed as derivative features through mathematical operations or statistical methods. Among them, the derivative features include one or more of the difference of water quality indicators, the ratio of water quality indicators, the moving average of water quality indicators, and the standard deviation of water quality indicators, but are not limited to including one or more of the difference of water quality indicators, the ratio of water quality indicators, the moving average of water quality indicators, and the standard deviation of water quality indicators. The information content of the data is increased through the derivative features.

[0041] S14: Integrate the preprocessed data, time features, and derivative features into a dataset to form an initial multivariate time series dataset. Among them, each column in the initial multivariate time series dataset represents a variable, and each row in the initial multivariate time series dataset represents the observed values of multiple variables at a time point; among them, the variables include: various water quality indicators and derivative features; the variable observed values include: the monitored values of each water quality indicator and the calculated values of each derivative feature.

[0042] S15: Divide the initial multivariate time series dataset to obtain a training set, a validation set, and a test set, and use the training set, the validation set, and the test set as the multivariate time series dataset and store them.

[0043] Further, determine the time order of the initial multivariate time series dataset according to the time features of the initial multivariate time series dataset. The time order is from front to back. Sort the initial multivariate time series dataset according to the time order. After sorting, determine the division point according to the preset division ratio, and divide the initial multivariate time series dataset into a training set, a validation set, and a test set according to the division point. Use the training set, the validation set, and the test set as the multivariate time series dataset and store them.

[0044] Specifically, dividing the initial multivariate time series dataset into a training set, a validation set, and a test set according to the preset division ratio and time series can retain the order characteristics of the data itself, so that the model training and validation processes are more in line with the actual situation of data generation, which is beneficial to improving the prediction accuracy of the model.

[0045] Further, the specific value of the preset division ratio is set according to the actual situation. This application preferably uses a training set: validation set: test set = 8:1:1 or a training set: validation set: test set = 7:2:1.

[0046] Specifically, the training set, the validation set, and the test set are respectively used for the training, tuning, and evaluation of the model.

[0047] S2: Define the hyperparameter space of the Transformer model, set the value ranges of each hyperparameter to be optimized in the hyperparameter space, and obtain the initial hyperparameter combination and the initial loss function value.

[0048] Furthermore, the hyperparameters to be optimized at least include: the input time step, the output time step, the number of layers, the number of attention heads, the hidden layer dimension, the batch size, and the Dropout rate.

[0049] Specifically, the Dropout rate refers to the proportion of randomly setting neurons or connections to be invalid (i.e., "discarded") during the training process.

[0050] Furthermore, the initial hyperparameter combination has the expression: , where represents the th sub-hyperparameter combination; is the initial sample number of the sub-hyperparameter combination, and is a natural number.

[0051] As an example, the Sobol sequence or the Latin hypercube sampling method is used to generate the initial hyperparameter combination, but it is not limited to the Sobol sequence or the Latin hypercube sampling method.

[0052] Specifically, the Sobol sequence is a low-discrepancy sequence used to generate a uniformly distributed point set in a multi-dimensional space to improve the computational efficiency and accuracy.

[0053] Furthermore, the initial loss function value has the expression: ; where represents the sub-loss function value corresponding to the th sub-hyperparameter combination in the initial hyperparameter combination; is the initial sample number of the sub-hyperparameter combination, and is a natural number.

[0054] The expression of the sub-loss function value corresponding to the th sub-hyperparameter combination in the initial hyperparameter combination is: ; where is the sub-loss function value corresponding to the th sub-hyperparameter combination in the initial hyperparameter combination; is the multi-objective loss function; represents the Sub-hyperparameter combination; is the initial sample number of the sub-hyperparameter combination, which is a natural number.

[0055] Specifically, use the real multi-objective loss function to calculate the sub-loss function value corresponding to the th sub-hyperparameter combination in the initial hyperparameter combination.

[0056] S3: Optimize the initial hyperparameter combination to obtain the optimal hyperparameter combination, and use the optimal hyperparameter combination to train the Transformer model to obtain the trained model.

[0057] Among them, the optimal hyperparameter combination at least includes: the value of the input time step, the value of the output time step, the value of the number of layers, the value of the number of attention heads, the value of the hidden layer dimension, the value of the batch size, and the value of the Dropout rate.

[0058] Furthermore, as an embodiment, the sub-steps of optimizing the initial hyperparameter combination to obtain the optimal hyperparameter combination and using the optimal hyperparameter combination to train the Transformer model to obtain the trained model are as follows: S31: Define multiple optimization objectives, and obtain a comprehensive optimization objective according to the multiple optimization objectives. Among them, the multiple optimization objectives at least include: prediction accuracy, model complexity, and computational efficiency.

[0059] Specifically, prediction accuracy: It is measured by the mean squared error (MSE) or the mean absolute error (MAE), but is not limited to being measured by the mean squared error (MSE) or the mean absolute error (MAE).

[0060] Model complexity: It is measured by the number of model parameters or the amount of computation, but is not limited to being measured by the number of model parameters or the amount of computation.

[0061] Computational efficiency: It is measured by the training time or the inference time, but is not limited to being measured by the training time or the inference time.

[0062] As an embodiment, the multiple optimization objectives are integrated into a comprehensive optimization objective by the weighted summation or Pareto front optimization method, but are not limited to the weighted summation or Pareto front optimization method.

[0063] Furthermore, the expression of the loss function corresponding to the comprehensive optimization objective is as follows: ; Among them, is the loss function corresponding to the comprehensive optimization objective; is the weight corresponding to the th optimization objective; is the loss function corresponding to the th optimization objective, , is the total number of optimization objectives, being a natural number.

[0064] Specifically, the specific value of is set according to the relative importance of the optimization objective.

[0065] S32: Using the initial hyperparameter combination as the input and the initial loss function value as the output to form the fitting data; selecting a kernel function and fitting the simulator according to the kernel function and the fitting data to obtain the fitted simulator.

[0066] Specifically, the expression of the fitting data is: , where is the initial loss function value; is the initial hyperparameter combination. Select a specific kernel function according to the characteristics of the problem, such as: squared exponential kernel, Matern kernel, etc. The fitted simulator is used to model the relationship between the sub-hyperparameter combination and the sub-loss function value. Predict the sub-loss function value of each sub-hyperparameter combination in the initial hyperparameter combination through the fitted simulator, and calculate the uncertainty of the multi-objective loss function value.

[0067] S33: Using the fitted simulator to calculate the candidate point prediction mean and candidate point standard deviation of each candidate point in the hyperparameter space, and calculating the selection boundary value according to the candidate point prediction mean and candidate point standard deviation.

[0068] Furthermore, the expression of the selection boundary value is: ; where is the selection boundary value of candidate point ; is the candidate point prediction mean of candidate point ; is the candidate point standard deviation of candidate point ; is the trade-off parameter.

[0069] Specifically, the specific value of the trade-off parameter is set according to the number of iterations and is a parameter used to balance exploration and exploitation. The candidate point is a point selected from the value range of each hyperparameter to be optimized in the hyperparameter space as a potential value combination of the model hyperparameters.

[0070] S34: Obtain the iterative change value of the selection boundary value. If the iterative change value is less than the preset iterative threshold, the convergence condition is satisfied, stop the iteration, output the optimal hyperparameter combination and the corresponding loss function value, and execute S35; if the iterative change value is greater than or equal to the preset iterative threshold, the convergence condition is not satisfied, select the point with the largest selection boundary value as the new hyperparameter combination, and execute S33.

[0071] Further, the expression of the iterative change value is: ; where is the iterative change value of this round of iteration; is the selection boundary value obtained in the previous round of iteration; is the selection boundary value obtained in this round of iteration; is the iteration number corresponding to this round of iteration, is a natural number; is the iteration number corresponding to the previous round of iteration.

[0072] S35: Use the optimal hyperparameter combination and the corresponding loss function value to train the Transformer model to obtain the trained model.

[0073] Further, as Figure 2 shown, as another embodiment, by constructing a Gaussian process model, predict the multi-objective loss function values under different hyperparameter combinations, and calculate their uncertainties; use the acquisition function based on the upper confidence bound (UCB) to select a new hyperparameter combination (i.e., the most promising hyperparameter combination) for training and evaluation of the Transformer model to obtain the trained model. By combining multi-objective Bayesian optimization and the Gaussian process (GP) surrogate model simulator and the upper confidence bound acquisition function, it is possible to achieve efficient and automated calibration of the hyperparameters of the water quality multivariate time series prediction model based on the Transformer architecture, and it is possible to optimize multiple conflicting optimization objectives simultaneously, significantly improving the model performance and computational efficiency, and is applicable to complex time series prediction tasks.

[0074] S4: Use the test set and the validation set to analyze the prediction performance of the trained model to obtain the prediction performance value, and use the preset performance threshold to analyze the prediction performance value. If the prediction performance value is less than the preset performance threshold, use the trained model as the water quality multivariate time series prediction AI model; if the prediction performance value is greater than or equal to the preset performance threshold, re-execute S2.

[0075] Further, the expression of the prediction performance value is: ; where is the prediction performance value; For the confusion matrix in the test set At the row and the element in the column; For the confusion matrix in the validation set At the row and the element in the column; Is the stability weight; Is the mean squared logarithmic error obtained by calculating after inputting the test set into the trained model; Is the mean squared logarithmic error obtained by calculating after inputting the validation set into the trained model; Is the error weight; Is the duration between inputting the test set into the trained model and the trained model outputting the prediction result; Is the duration between inputting the validation set into the trained model and the trained model outputting the prediction result; Is the preset standard duration; Is the duration weight.

[0076] Specifically, And The specific values of both are set according to the actual situation. The smaller the prediction performance value, the better the prediction performance of the trained model. This application fully considers the mean squared error between the test set and the validation set and the corresponding prediction results output by the trained model, can reduce the influence of outliers, fully considers the prediction stability of the trained model between different data sets, and the duration of the trained model to complete the prediction process between different data sets, improving the comprehensiveness and accuracy of the prediction performance analysis of the trained model.

[0077] Specifically, after inputting water quality monitoring data (such as pH value, dissolved oxygen, turbidity, etc.) into the water quality multivariate time series prediction AI model, the water quality multivariate time series prediction AI model outputs the water quality prediction results for future time steps.

[0078] The method for automatic calibration of the parameters of the water quality multivariate time series prediction AI model in this application is not only applicable to water quality prediction tasks, but also applicable to other multivariate time series prediction scenarios.

[0079] The beneficial effects achieved by this application are as follows: (1) The method for automatic calibration of the parameters of the water quality multivariate time series prediction AI model in this application can achieve efficient and automatic calibration of the hyperparameters of the water quality multivariate time series prediction model based on the Transformer architecture, reducing manual intervention.

[0080] (2)The automatic calibration method for the parameters of the AI model for predicting water quality multivariate time series in this application can optimize multiple conflicting optimization objectives simultaneously (such as prediction accuracy, model complexity, and computational efficiency), significantly improving the model performance and computational efficiency, and is applicable to complex time series prediction tasks.

[0081] (3)The automatic calibration method for the parameters of the AI model for predicting water quality multivariate time series in this application can significantly reduce the computational cost and time consumption of hyperparameter search.

[0082] (4)The automatic calibration method for the parameters of the AI model for predicting water quality multivariate time series in this application fully considers the mean square error between the test set and the validation set and the corresponding prediction results of the output of the trained model, can reduce the influence of outliers, fully considers the prediction stability of the trained model between different data sets, and the duration of the prediction process of the trained model between different data sets, improving the comprehensiveness and accuracy of the prediction performance analysis of the trained model.

[0083] Although the preferred embodiments of this application have been described, those skilled in the art can make additional changes and modifications once they know the basic creative concept. Therefore, the protection scope of this application is intended to be interpreted as including the preferred embodiments and all changes and modifications falling within the scope of this application. Obviously, those skilled in the art can make various changes and variations to this application without departing from the spirit and scope of this application. Thus, if these modifications and variations of this application fall within the scope of the protection of this application and its equivalent technologies, this application also intends to include these modifications and variations.

Claims

1. A method for automatic calibration of AI model parameters for water quality multivariate time series prediction, characterized in that: The steps include: S1: Obtain water quality monitoring data, and use the water quality monitoring data to construct a multivariate time series data set, where the multivariate time series data set includes: a training set, a validation set, and a test set; S2: Define the hyperparameter space of the Transformer model, set the value range of each hyperparameter to be optimized in the hyperparameter space, and obtain the initial hyperparameter combination and initial loss function value; S3: Optimize the initial hyperparameter combination to obtain the optimal hyperparameter combination, use the optimal hyperparameter combination to train the Transformer model, and obtain the trained model; S4: Use the test set and validation set to perform predictive performance analysis on the trained model to obtain a predictive performance value, and use the preset performance threshold to analyze the predictive performance value. If the predictive performance value is less than the preset performance threshold, the trained model is used as a water quality multivariate time series prediction AI model; if the predictive performance value is greater than or equal to the preset performance threshold, re-execute S2.

2. The method for automatically calibrating parameters of the AI ​​model for water quality multivariate time series prediction according to claim 1 is characterized in that: The sub-steps of obtaining water quality monitoring data and using the water quality monitoring data to construct a multivariate time series dataset are as follows: S11: Determine the water quality indicators that need to be monitored, form the water quality monitoring data to be obtained based on the water quality indicators that need to be monitored, and obtain water quality monitoring data from each monitoring station based on the water quality monitoring data to be obtained; S12: preprocessing the water quality monitoring data to obtain preprocessed data; S13: extracting features from the preprocessed data to obtain time features and derivative features; S14: Integrate the preprocessed data, time features and derived features into one data set to form an initial multivariate time series data set, wherein each column in the initial multivariate time series data set represents a variable, and each row in the initial multivariate time series data set represents multiple variable observations at a time point; wherein the variables include: multiple water quality indicators and derived features; the variable observations include: the monitoring value of each water quality indicator and the calculated value of each derived feature; S15: Divide the initial multivariate time series data set to obtain a training set, a validation set, and a test set, and use the training set, the validation set, and the test set as the multivariate time series data set and store them.

3. The method for automatically calibrating parameters of the AI ​​model for water quality multivariate time series prediction according to claim 2 is characterized in that: The sub-steps of preprocessing water quality monitoring data to obtain preprocessed data are as follows: S121: Perform abnormal analysis on the water quality monitoring data. If there are missing values ​​and / or abnormal values ​​in the water quality monitoring data, execute S122; if there are no missing values ​​and abnormal values ​​in the water quality monitoring data, take the water quality monitoring data as data to be processed and execute S123; S122: Process missing values ​​and / or abnormal values ​​in the water quality monitoring data to obtain data to be processed, and execute S123; S123: Standardizing the data to be processed, mapping the data to be processed to a preset interval or making the data to be processed have a zero mean and a unit variance, thereby obtaining pre-processed data.

4. The method for automatically calibrating parameters of the AI ​​model for water quality multivariate time series prediction according to claim 3 is characterized in that: The sub-steps for processing missing values ​​and / or outliers in water quality monitoring data to obtain the data to be processed are as follows: S1221: If there are abnormal values ​​in the water quality monitoring data, the abnormal values ​​greater than the preset upper limit are corrected to the upper limit value, and the values ​​less than the preset lower limit are corrected to the lower limit value. After the correction is completed, the corrected data is obtained and S1222 is executed; if there are no abnormal values ​​in the water quality monitoring data, S1222 is executed; S1222: If there are missing values ​​in the water quality monitoring data or the corrected data, the missing values ​​in the water quality monitoring data are interpolated using the interpolation method. After the interpolation is completed, the data to be processed are obtained; if there are no missing values ​​in the corrected data, the corrected data are directly used as the data to be processed.

5. The method for automatic calibration of AI model parameters for water quality multivariate time series prediction according to claim 1 is characterized in that: The hyperparameters to be optimized include at least: input time step, output time step, number of layers, number of attention heads, hidden layer dimension, batch size and dropout rate.

6. The method for automatic calibration of AI model parameters for water quality multivariate time series prediction according to claim 1 is characterized in that: Generate initial hyperparameter combinations using Sobol sequence or Latin hypercube sampling methods.

7. The method for automatically calibrating parameters of the AI ​​model for water quality multivariate time series prediction according to claim 2 is characterized in that: Determine the time order of the initial multivariate time series data set according to the time characteristics of the initial multivariate time series data set, the time order is from front to back, sort the initial multivariate time series data set according to the time order, after the sorting is completed, determine the division point according to the preset division ratio, divide the initial multivariate time series data set into a training set, a validation set and a test set according to the division point, and use the training set, validation set and test set as the multivariate time series data set and store them.

8. The method for automatic calibration of AI model parameters for water quality multivariate time series prediction according to claim 1 is characterized in that: Optimize the initial hyperparameter combination to obtain the optimal hyperparameter combination, and use the optimal hyperparameter combination to train the Transformer model. The sub-steps for obtaining the trained model are as follows: S31: defining multiple optimization objectives, and obtaining a comprehensive optimization objective according to the multiple optimization objectives, wherein the multiple optimization objectives at least include: prediction accuracy, model complexity, and computational efficiency; S32: taking the initial hyperparameter combination as input and the initial loss function value as output to form fitting data; selecting a kernel function, fitting the simulator according to the kernel function and the fitting data, and obtaining a fitting simulator; S33: using the fitting simulator to calculate the candidate point prediction mean and the candidate point standard deviation of each candidate point in the hyperparameter space, and calculating the selection limit value according to the candidate point prediction mean and the candidate point standard deviation; S34: Obtain the iterative change value of the selected boundary value. If the iterative change value is less than the preset iteration threshold, the convergence condition is met, the iteration is stopped, the optimal hyperparameter combination and the corresponding loss function value are output, and S35 is executed; if the iterative change value is greater than or equal to the preset iteration threshold, the convergence condition is not met, the point with the largest boundary value is selected as the new hyperparameter combination, and S33 is executed; S35: Train the Transformer model using the optimal hyperparameter combination and the corresponding loss function value to obtain the trained model.

9. The method for automatic calibration of AI model parameters for water quality multivariate time series prediction according to claim 8 is characterized in that: The expression of the loss function corresponding to the comprehensive optimization objective is as follows: ； in, is the loss function corresponding to the comprehensive optimization objective; For the The weight corresponding to each optimization goal; For the The loss function corresponding to the optimization objective is , is the total number of optimization objectives, is a natural number.

10. The method for automatic calibration of AI model parameters for water quality multivariate time series prediction according to claim 8, characterized in that: The expression for selecting the limit value is: ； in, Candidate point The selection threshold value of Candidate point The predicted mean of candidate points; Candidate point The standard deviation of candidate points; is a trade-off parameter.

Images