Control parameter setting method based on proxy model

Abstract

The invention relates to a control parameter setting method based on an agent model. The method comprises the following steps: constructing a correction model; simulating the controlled object by applying the initial control parameters in different groups of variable working condition processes to obtain a total utility value of each group of scheme; constructing a data set based on the error between the controlled variables in each group of variable working condition process, the corresponding initial control parameter and the total utility value of each group of scheme; using the data set to train a machine learning agent model; performing data interaction on the trained machine learning agent model by using an optimization algorithm to obtain an optimal initial control parameter combination with maximized utility; and performing nuclear reactor water supply control based on the optimal initial control parameter combination and the correction model. By fusing simulation data, a machine learning agent model and an optimization algorithm, an efficient and intelligent solution is provided for control parameter setting in different variable working condition control processes of the nuclear reactor water supply system, and the control efficiency and accuracy are effectively improved.

Description

Technical Field

[0001] The invention relates to the technical field of flow rate and pressure control method optimization of a nuclear reactor water supply system, and in particular to a control parameter setting method based on an agent model. Background Art

[0002] In the actual control process of the nuclear reactor water supply system, whether the flow and pressure are controlled timely and accurately is an important condition to ensure the safe and normal operation of the system and good dynamic performance. The automatic control process of the nuclear reactor water supply system is usually completed by the PID control algorithm. However, the main reasons for the poor automatic control effect of the nuclear reactor water supply system under various variable conditions are that the initial parameters of the control algorithm are difficult to adjust and select under different variable conditions, and the algorithm parameters cannot be adaptively changed during the control process.

[0003] Since the nuclear reactor water supply system has strong coupling characteristics and complex nonlinearity, the initial parameters of the PID control algorithm under different variable operating conditions need to be repeatedly debugged and adjusted. If the experimental method is used to debug and adjust the control algorithm parameters under different variable operating conditions, there are many shortcomings such as strict experimental conditions, high cost, long cycle, and limited scope of application, which makes it difficult to directly apply it to engineering practice.

[0004] Different variable operating conditions also have different requirements for control algorithm parameters. For conditions where the control target changes slightly and needs to be adjusted quickly, larger initial control parameters need to be applied to speed up the control speed and improve the responsiveness of the system. For conditions where the control target changes significantly, smaller initial control parameters need to be applied to keep the control process smooth to avoid serious overshoot and improve the stability of the system.

[0005] With the development of the control process, the error term input of the PID control algorithm continues to decrease, and the integral term input continues to increase. If fixed control algorithm parameters are applied, the overshoot phenomenon of the integral term being too large and the error term being too small is likely to occur in the later stage of the control process, which is difficult to meet the control requirements. Therefore, it is necessary to study the control parameter setting technology of the control algorithm under different variable conditions. By continuously and reasonably changing the control parameters during the control process, the nuclear reactor water supply system can be quickly, safely and smoothly transitioned from the current operating state to the target operating state. Summary of the invention

[0006] The purpose of the present invention is to solve the problems that the initial parameters of the control algorithm of the nuclear reactor water supply system under different operating conditions are difficult to adjust and the algorithm parameters cannot be changed with the development of the control process. The present invention provides a control algorithm parameter adaptive adjustment method based on an agent model and its application. By combining a hybrid method of simulation, fuzzy control method, neural network agent model, and optimization algorithm, real-time and rapid adaptive adjustment of control parameters can be achieved, thereby improving control efficiency and accuracy.

[0007] To achieve the above object, the present invention provides the following solutions:

[0008] A control parameter tuning method based on an agent model, comprising:

[0009] Constructing a correction model; wherein the correction model is used to optimize the control algorithm parameters during the control process according to the control error and the error change rate, so as to reduce the control overshoot and the adjustment time;

[0010] Apply the initial control parameters to simulate the controlled object under different groups of variable working conditions to obtain the total utility value of each group of schemes;

[0011] Based on the errors between the initial flow rate and initial pressure of the controlled pipeline and the target flow rate and target pressure during each set of variable operating conditions and the corresponding initial control parameters, as well as the total utility value of each set of solutions, a data set is constructed;

[0012] Using the data set, training a machine learning agent model; wherein the machine learning agent model is constructed based on a convolutional neural network;

[0013] Using the optimization algorithm, the trained machine learning agent model is interacted with data to obtain the best initial control parameter combination that maximizes utility.

[0014] Based on the optimal initial control parameter combination and the correction model, nuclear reactor water supply control is performed.

[0015] Optionally, taking the total utility value of each group of solutions includes:

[0016] Apply initial control parameters to simulate the controlled object under different group-changing working conditions to obtain the adjustment time and overshoot;

[0017] According to the adjustment time and overshoot, an evaluation system of the control scheme is constructed, and the total utility value of each group of schemes is calculated.

[0018] Optionally, the total utility value is:

[0019]

[0020] u i (r ij ) = arij +b

[0021] Among them, r ij is the i-th performance index value under the j-th scheme, a and b are coefficients, and w i is the weight value of the i-th performance indicator, u i is the utility value of each performance indicator of the control scheme, n is the number of performance indicators, U j is the total utility value of all performance indicators.

[0022] Optionally, the initial control error is:

[0023] e0=yset-y0

[0024] where y set is the target value of the controlled variable, y 0 is the initial value of the controlled quantity, e 0 is the initial control error.

[0025] Optionally, using the data set to train the machine learning agent model includes:

[0026] The initial error between the controlled variable and the target value in each set of variable operating conditions and the corresponding initial control parameters are used as model input, and the total utility value is used as model output. The machine learning agent model is trained and iterated repeatedly until the prediction accuracy meets the preset requirements.

[0027] Optionally, repeatedly iterating until the prediction accuracy meets the preset requirement includes:

[0028] The number of training iterations is preset, and the prediction accuracy of the control scheme utility value prediction model is evaluated using evaluation indicators until the prediction accuracy reaches the set range, and the machine learning model is judged to have converged.

[0029] Optionally, using an optimization algorithm, data interaction with the trained machine learning agent model includes:

[0030] Using the optimization algorithm, the initial error and initial control parameters are input into the trained machine learning agent model, and the trained machine learning agent model is fed back with the predicted utility value. The optimization algorithm is iteratively updated according to the feedback predicted utility value until the best initial control parameter combination that maximizes the utility is found.

[0031] Optionally, the criterion for determining whether the optimization algorithm finds the maximum utility value is the utility value obtained in the n+1th iteration. and the utility value of the nth iteration The error between them is less than the preset threshold.

[0032] Optionally, based on the optimal initial control parameter combination and the correction model, performing water supply control for the nuclear reactor includes:

[0033] Based on the optimal initial control parameter combination and the correction model, the initial control parameter selection and correction of the PID control algorithm of the pipeline regulating valve of the nuclear reactor water supply system are performed;

[0034] The opening of the regulating valve in the water supply pipeline is controlled by the PID control algorithm to control the water supply to the nuclear reactor system.

[0035] The beneficial effects of the present invention are:

[0036] The present invention adopts a simulation method to apply different control algorithm parameters to different variable operating conditions of the nuclear reactor water supply system and record overshoot and adjustment time data, which can effectively avoid the safety hazards and equipment losses caused by conducting experiments on the physical system and effectively shorten the experimental cycle;

[0037] The correction model that updates the control algorithm parameters based on the control error and the error change rate can realize the update of control parameters to better match the real-time control requirements and improve the control efficiency and accuracy;

[0038] The control scheme evaluation system is introduced to simply and efficiently evaluate the performance of each group of control algorithm parameters under different variable working conditions;

[0039] The control effect proxy prediction model based on the machine learning model can quickly obtain the performance of the control algorithm when applying specific initial control parameters under different degrees of change of the control target; on this basis, combined with the optimization algorithm, the optimal initial parameters of the control algorithm under different variable operating conditions can be quickly obtained, further improving the performance and efficiency of the control algorithm. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0041] Figure 1 It is a flow chart of a control parameter setting method based on an agent model according to an embodiment of the present invention;

[0042] Figure 2 It is a schematic diagram of control test results of the control parameter setting method under 60% working condition according to an embodiment of the present invention. DETAILED DESCRIPTION

[0043] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0044] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0045] like Figure 1 As shown, this embodiment proposes a control parameter setting method based on an agent model, which is characterized by comprising:

[0046] Constructing a correction model; wherein the correction model is used to optimize the control algorithm parameters during the control process according to the control error and the error change rate, so as to reduce the control overshoot and the adjustment time;

[0047] Apply the initial control parameters to simulate the controlled object under different groups of variable working conditions to obtain the total utility value of each group of schemes;

[0048] Based on the errors between the initial flow rate and initial pressure of the controlled pipeline and the target flow rate and target pressure during each set of variable operating conditions and the corresponding initial control parameters, as well as the total utility value of each set of solutions, a data set is constructed;

[0049] Using the data set, training a machine learning agent model; wherein the machine learning agent model is constructed based on a convolutional neural network;

[0050] Using the optimization algorithm, the trained machine learning agent model is interacted with data to obtain the best initial control parameter combination that maximizes utility.

[0051] Based on the optimal initial control parameter combination and the correction model, nuclear reactor water supply control is performed.

[0052] Furthermore, the total utility value of each group of solutions includes:

[0053] Apply initial control parameters to simulate the controlled object under different group-changing working conditions to obtain the adjustment time and overshoot;

[0054] According to the adjustment time and overshoot, an evaluation system of the control scheme is constructed, and the total utility value of each group of schemes is calculated.

[0055] Furthermore, using the data set, training the machine learning agent model includes:

[0056] The initial error between the controlled variable and the target value in each set of variable operating conditions and the corresponding initial control parameters are used as model input, and the total utility value is used as model output. The machine learning agent model is trained and iterated repeatedly until the prediction accuracy meets the preset requirements.

[0057] Further, repeatedly iterating until the prediction accuracy meets the preset requirements includes:

[0058] The number of training iterations is preset, and the prediction accuracy of the control scheme utility value prediction model is evaluated using evaluation indicators until the prediction accuracy reaches the set range, and the machine learning model is judged to have converged.

[0059] Furthermore, using the optimization algorithm, data interaction is performed on the trained machine learning agent model, including:

[0060] Using the optimization algorithm, the initial error and initial control parameters are input into the trained machine learning agent model, and the trained machine learning agent model is fed back with the predicted utility value. The optimization algorithm is iteratively updated according to the feedback predicted utility value until the best initial control parameter combination that maximizes the utility is found.

[0061] Furthermore, the criterion for determining whether the optimization algorithm has found the maximum utility value is the utility value obtained at the n+1th iteration. and the utility value of the nth iteration The error between them is less than the preset threshold.

[0062] Furthermore, based on the optimal initial control parameter combination and the correction model, the water supply control of the nuclear reactor includes:

[0063] Based on the optimal initial control parameter combination and correction model, the initial control parameter selection and correction of the PID control algorithm of the pipeline regulating valve of the nuclear reactor water supply system are performed;

[0064] The opening of the regulating valve in the water supply pipeline is controlled by the PID control algorithm to control the water supply to the nuclear reactor system.

[0065] The development of a control parameter setting method based on an agent model can realize the selection of optimal initial parameters of the control algorithm under different variable operating conditions and the real-time update of control parameters, and can fully combine the advantages of various methods to improve the efficiency and accuracy of the control algorithm. The present invention provides an efficient and intelligent solution for the control parameter setting in the process of different variable operating conditions of the nuclear reactor water supply system by integrating simulation data, machine learning agent models and optimization algorithms, effectively improving the control efficiency and accuracy. The specific implementation steps of this embodiment are as follows:

[0066] S1, establishing a correction model that can update the control algorithm parameters according to the control error and the error change rate; wherein the correction model is used to optimize the initial control algorithm parameters in the control process according to the control error and the error change rate, and is finally used in S7 to reduce the control overshoot and adjustment time;

[0067] The correction method of the control algorithm parameters in step S1 is a fuzzy control method. The adjustment range of the fuzzy control for the control parameters is 20% to 100%, and the updated control algorithm parameters are the proportional term coefficient K in the PID control algorithm. p And the integral coefficient K i The control error is the error between the target flow value, pressure value and the real-time flow value, pressure value. The control error e and error change rate e c The calculation formula is as follows

[0068] e=y set -y t

[0069]

[0070] In the formula, y set is the target value of the controlled variable, y t It is the real-time value of the controlled quantity.

[0071] S2. Using the water supply system simulation model, various initial control parameters are applied to simulate the controlled object under different variable working conditions and the adjustment time and overshoot are recorded to form a database;

[0072] In step S2, the different variable operating conditions of the nuclear reactor water supply system in the simulation experiment include five types of load variations: 25%, 50%, 75%, and 100%. The initial parameter K of the control algorithm is p , K i Taking their respective maximum values ​​as the benchmark, they are divided into 20%, 40%, 60%, 80%, and 100% based on experience, and a total of 100 groups of simulation results recording flow and pressure regulation time and overshoot are arranged and combined.

[0073] S3. Design an evaluation system for the control scheme based on key performance indicators such as adjustment time and overshoot, and calculate the total utility value for each group of schemes to quantify the evaluation effect;

[0074] The utility value u of each performance indicator of the control scheme in step S3 i And the total utility value U of all performance indicators j The calculation formula is as follows:

[0075] u i (r ij ) = ar ij +b

[0076]

[0077] In the formula, r ij is the i-th performance indicator value under the j-th scheme; a, b are coefficients; w i is the weight value of the i-th performance indicator.

[0078] For different working conditions and different properties, the final selection of each coefficient in the utility value calculation formula is shown in Tables 1 and 2:

[0079] Table 1 Utility value calculation coefficient selection (overshoot)

[0080]

[0081]

[0082] Table 2 Utility value calculation coefficient selection (adjustment time)

[0083]

[0084] S4, establishing a utility value prediction model for the control scheme of the nuclear reactor water supply system based on the machine learning model, taking the initial error between the controlled quantity and the target value in each group of variable operating conditions and the corresponding initial control parameters as input, and the total utility value calculated in step S3 as output;

[0085] The machine learning prediction agent model in step S4 is selected as a convolutional neural network model. The number of output channels of the convolutional layer of the neural network is set to 57, the number of units of the fully connected layer is set to 60, and the learning rate is set to 1.06×10-4.

[0086] S5. Train the machine learning agent model, iterate repeatedly until its prediction accuracy meets the requirements, and finally output the trained machine learning agent model;

[0087] In step S5, the preprocessed database is divided into a training set and a test set by random partitioning at a ratio of 4:1, and normalization preprocessing is performed based on the maximum value; the prediction accuracy of the control scheme utility value prediction model is evaluated by the evaluation index until the prediction accuracy reaches the set range, and the machine learning model is determined to converge, and the training iteration process is set to 9500 times;

[0088] The evaluation index is the mean square error RMSE, and the Euclidean Loss function is used as the calculation function:

[0089]

[0090] Where n is the number of test points; Y iis the true value of the i-th sample; is the predicted value of the i-th sample.

[0091] S6. Establish an optimization algorithm, and interact with the data of the agent model through the algorithm, that is, the algorithm inputs the initial error and initial control parameters to the model, and the model feeds back the predicted utility value, and the algorithm iteratively updates until the best initial control parameter combination that maximizes the utility is found;

[0092] The optimization algorithm in step S6 is selected as the Bayesian optimization algorithm, the number of iterations is selected as 40, and the criterion for judging whether the optimization algorithm finds the maximum utility value is the utility value obtained in the n+1th iteration. and the utility value of the nth iteration The error between them is less than 1%, that is:

[0093]

[0094] S7. Integrate the optimization algorithm with the prediction model and the correction model into an adaptive tuning model, which can exchange data with the control algorithm and dynamically optimize the control parameters in the control process of different variable operating conditions.

[0095] Application of control parameter tuning method based on surrogate model in nuclear reactor water supply control process.

[0096] like Figure 2 As shown, it is a schematic diagram of the control test results under 60% working conditions using the control parameter tuning method.

[0097] This embodiment adopts a simulation method to apply different control algorithm parameters to different variable operating conditions of the nuclear reactor water supply system and record overshoot and adjustment time data, which can effectively avoid the safety hazards and equipment losses caused by conducting experiments on the physical system and effectively shorten the experimental cycle;

[0098] The correction model that updates the control algorithm parameters based on the control error and the error change rate can realize the update of control parameters to better match the real-time control requirements and improve the control efficiency and accuracy;

[0099] The control scheme evaluation system is introduced to simply and efficiently evaluate the performance of each group of control algorithm parameters under different variable working conditions;

[0100] The control effect proxy prediction model based on the machine learning model can quickly obtain the performance of the control algorithm when applying specific initial control parameters under different degrees of change of the control target; on this basis, combined with the optimization algorithm, the optimal initial parameters of the control algorithm under different variable operating conditions can be quickly obtained, further improving the performance and efficiency of the control algorithm.

[0101] Therefore, this embodiment adopts the above-mentioned control parameter tuning method based on the agent model and its application, and through the hybrid method combining fuzzy control, machine learning and optimization algorithm, it can make up for the limitations of a single method to a certain extent and improve the efficiency of the control algorithm.

[0102] The embodiments described above are only descriptions of the preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Without departing from the design spirit of the present invention, various modifications and improvements made to the technical solutions of the present invention by ordinary technicians in this field should all fall within the protection scope determined by the claims of the present invention.

Claims

1. A control parameter setting method based on an agent model, characterized in that: include: Constructing a correction model; wherein the correction model is used to optimize the initial control algorithm parameters during the control process according to the control error and the error change rate, and reduce the control overshoot and adjustment time; Apply the initial control parameters to simulate the controlled object under different groups of variable working conditions to obtain the total utility value of each group of schemes; Based on the initial control errors between the initial flow rate and initial pressure of the controlled pipeline and the target flow rate and target pressure during each set of variable operating conditions and the corresponding initial control parameters, as well as the total utility value of each set of solutions, a data set is constructed; Using the data set, training a machine learning agent model; wherein the machine learning agent model is constructed based on a convolutional neural network; Using the optimization algorithm, the trained machine learning agent model is interacted with data to obtain the best initial control parameter combination that maximizes utility. Based on the optimal initial control parameter combination and the correction model, nuclear reactor water supply control is performed.

2. The control parameter setting method based on the agent model according to claim 1 is characterized in that: The total utility value of each group of solutions includes: Apply initial control parameters to simulate the controlled object under different group-changing working conditions to obtain the adjustment time and overshoot; According to the adjustment time and overshoot, an evaluation system of the control scheme is constructed, and the total utility value of each group of schemes is calculated.

3. The control parameter setting method based on the agent model according to claim 1 is characterized in that: The total utility value is: u i (r ij )=ar ij +b Among them, r ij is the i-th performance index value under the j-th scheme, a and b are coefficients, and w i is the weight value of the i-th performance indicator, u i is the utility value of each performance indicator of the control scheme, n is the number of performance indicators, U j is the total utility value of all performance indicators; The initial control error is: e0=yset-y0 Among them, y set is the target value of the controlled variable, y0 is the initial value of the controlled variable, and e0 is the initial control error.

4. The control parameter setting method based on the agent model according to claim 1 is characterized in that: Using the data set, training the machine learning agent model includes: The initial control error between the controlled quantity and the target value in each set of variable operating conditions, as well as the corresponding initial control parameters are used as model input, and the total utility value is used as model output. The machine learning agent model is trained and iterated repeatedly until the prediction accuracy meets the preset requirements.

5. The control parameter setting method based on the agent model according to claim 4 is characterized in that: Repeated iterations until the prediction accuracy meets the preset requirements include: The number of training iterations is preset, and the prediction accuracy of the control scheme utility value prediction model is evaluated using evaluation indicators until the prediction accuracy reaches the set range, and the machine learning model is judged to have converged.

6. The control parameter setting method based on the agent model according to claim 1 is characterized in that: Using the optimization algorithm, data interaction with the trained machine learning agent model includes: Using the optimization algorithm, the initial error and initial control parameters are input into the trained machine learning agent model, and the trained machine learning agent model is fed back with the predicted utility value. The optimization algorithm is iteratively updated according to the feedback predicted utility value until the best initial control parameter combination that maximizes the utility is found.

7. The control parameter setting method based on the agent model according to claim 6 is characterized in that: The criterion for judging whether the optimization algorithm has found the maximum utility value is the utility value U obtained in the n+1th iteration. j n+1 and the utility value U of the nth iteration j n The error between them is less than the preset threshold.

8. The control parameter setting method based on the agent model according to claim 1 is characterized in that: Based on the optimal initial control parameter combination and the correction model, the water supply control of the nuclear reactor includes: Based on the optimal initial control parameter combination and correction model, the initial control parameter selection and correction of the PID control algorithm of the pipeline regulating valve of the nuclear reactor water supply system are performed; The opening of the regulating valve in the water supply pipeline is controlled by the PID control algorithm to control the water supply to the nuclear reactor system.

Images