PID control loop tuning method and device

Abstract

This application discloses a method and apparatus for tuning a PID control loop. The method includes: obtaining multiple sets of control deviation values ​​for the PID control loop; sequentially training an initial model in multiple stages using the multiple sets of control deviation values ​​and a pre-determined tuning strategy for the PID control loop to obtain a target model, wherein the initial model trained in the current stage is the same as the initial model trained in the previous stage; inputting the multiple sets of control deviation values ​​for the PID control loop within a target time period into the target model to obtain a target tuning strategy for the PID control loop within the target time period; and tuning the PID control loop according to the target tuning strategy. This method at least solves the technical problem of low training efficiency of the tuning model due to the complexity of the control loop tuning task.

Description

Technical Field

[0001] This application relates to the field of automation control, and more specifically, to a method and apparatus for tuning a PID control loop. Background Technology

[0002] Reinforcement learning requires building an agent that interacts with the external environment to generate failure / success case data. This data is then used for training to learn strategies for solving tasks. In the initial stage of interaction, the agent explores the environment based on its own randomness, accumulating mostly failure case data. Therefore, in the initial stage, the agent's tendency to take the correct action based on the current state is nonexistent. It requires a long period of exploration and accumulation of a sufficient number of success case data. In increasingly difficult and complex environments, the time required for the agent to accumulate success experience through its own exploration ability is also longer. In some cases, only sporadic success case data can be accumulated over a long period of time. This leads to the problems of high training time cost and poor training effect for reinforcement learning agents.

[0003] There is currently no effective solution to the above problems. Summary of the Invention

[0004] This application provides a method and apparatus for tuning a PID control loop, which at least solves the technical problem of low training efficiency of the tuning model due to the complexity of the control loop tuning task.

[0005] According to one aspect of the embodiments of this application, a method for tuning a PID control loop is provided, comprising: obtaining a plurality of control deviation value sets of the PID control loop; sequentially training an initial model in multiple stages using the plurality of control deviation value sets of the PID control loop and a predetermined tuning strategy of the PID control loop to obtain a target model, wherein the initial model trained in the current stage is the initial model trained in the previous stage; inputting the plurality of control deviation value sets of the PID control loop in a target time period into the target model to obtain a target tuning strategy of the PID control loop in the target time period; and tuning the PID control loop according to the target tuning strategy.

[0006] Optionally, the predetermined tuning strategy for the PID control loop includes: determining the state of the PID control loop based on multiple sets of control deviation values; when the state of the PID control loop is in a state of oscillating decay, determining the control parameter with the largest cumulative value within a set time period among the control parameters of the PID control loop, and reducing the control parameter with the largest cumulative value according to a first preset ratio; when the state of the PID control loop is in a state of constant amplitude oscillation, reducing the control parameter with the largest cumulative value according to a second preset ratio; when the state of the PID control loop is in a state of divergence, reducing the control parameter with the largest cumulative value according to a third preset ratio; and when the state of the PID control loop is in a state of slow adjustment, increasing the control parameter with the smallest cumulative value according to a fourth preset ratio, wherein the first preset ratio is greater than the second preset ratio and the second preset ratio is greater than the third preset ratio.

[0007] Optionally, the state of the PID control loop is determined by the following methods: determining multiple maximum control deviation values ​​of the PID control loop at multiple sampling points within a preset time period from the multiple sets of control deviation values; determining that the state of the PID control loop is oscillating attenuation when the response curve of the PID control loop is in an oscillating state and the multiple maximum control deviation values ​​gradually decrease in chronological order; determining that the state of the PID control loop is constant amplitude oscillation when the response curve of the PID control loop is in an oscillating state and the multiple maximum control deviation values ​​are equal; determining that the state of the PID control loop is diverging when the multiple maximum control deviation values ​​gradually increase in chronological order; and determining that the state of the PID control loop is slow adjustment when the multiple maximum control deviation values ​​gradually decrease in chronological order and the minimum value of the multiple maximum control deviation values ​​is greater than a set threshold.

[0008] Optionally, the initial model is trained in multiple stages using multiple sets of control deviation values ​​from the PID control loop and a pre-determined tuning strategy for the PID control loop to obtain the target model. This includes: determining a target set from the multiple sets of control deviation values ​​of the PID control loop, wherein the target set includes at least: a first set obtained by inputting the control parameters of the PID control loop within a first interval into the preset model; a second set obtained by inputting the control parameters of the PID control loop within a second interval into the preset model; a third set obtained by adding noise to the second set; and a third set obtained by inputting the control parameters of the PID control loop within the first interval into the target object. The fourth set of models includes: a target object identification model, where the target object is controlled by a PID control loop; the model trained in the target stage is trained according to a predetermined training order using the target set and a predetermined tuning strategy of the PID control loop to obtain the target model. The predetermined training order is as follows: first stage, second stage, third stage, and fourth stage. The first stage trains the first set, the second stage trains using the second set, the third stage trains using the third set, and the fourth stage trains using the fourth set. The training task difficulty corresponding to the first set, second set, third set, and fourth set increases sequentially.

[0009] Optionally, the model trained in the target stage is trained using a target set and a pre-determined tuning strategy for the PID control loop, including: inputting the target set into the model trained in the target stage to obtain a first tuning strategy; determining a second tuning strategy corresponding to the target set using the pre-determined tuning strategy for the PID control loop; selecting one tuning strategy from the first and second tuning strategies based on a pre-determined selection probability function and determining it as the output result of the model trained in the target stage, until the probability of the first tuning strategy passing verification is greater than a set threshold, thus completing the training of the model trained in the target stage.

[0010] Optionally, after determining the output result, the method further includes: tuning the PID control loop in the target sample set according to the tuning strategy corresponding to the output result; if the tuning result does not reach the preset tuning index, determining the tuning index weight corresponding to the next tuning operation based on the tuning result; the tuning result is used to characterize whether the tuning index corresponding to the current tuning operation has been reached.

[0011] Optionally, reducing the control parameter with the largest cumulative value according to a preset first preset ratio includes: determining the product of the minimum value of the control parameter adjustment and the first preset ratio as the target parameter; and reducing the control parameter with the largest cumulative value to the target parameter.

[0012] According to another aspect of the embodiments of this application, a PID control loop tuning device is provided, comprising: acquiring a plurality of control deviation value sets of the PID control loop, wherein the control deviation values ​​are used to determine the state of the PID control loop; sequentially training an initial model in multiple stages using the plurality of control deviation value sets of the PID control loop and a predetermined tuning strategy of the PID control loop to obtain a target model, wherein the initial model trained in the current stage is the initial model trained in the previous stage; inputting the state of the PID control loop in a target time period into the target model to obtain a target tuning strategy of the PID control loop in the target time period, and tuning the PID control loop according to the target tuning strategy.

[0013] According to another aspect of the embodiments of this application, an electronic device is provided, including a memory and a processor, the processor being used to run a program, wherein the program executes the tuning method of the PID control loop described above when it runs.

[0014] According to another aspect of the embodiments of this application, a non-volatile storage medium is provided, the non-volatile storage medium including a stored program, wherein, when the program is running, the device where the non-volatile storage medium is located executes the tuning method of the above-mentioned PID control loop.

[0015] In this embodiment, multiple training sample sets of the PID control loop are obtained; multiple control deviation value sets of the PID control loop are obtained, and the initial model is trained in multiple stages using the multiple control deviation value sets of the PID control loop and a predetermined tuning strategy of the PID control loop to obtain the target model. The initial model trained in the current stage is the initial model trained in the previous stage. Multiple control deviation value sets of the PID control loop within the target time period are input into the target model to obtain the target tuning strategy of the PID control loop within the target time period. The PID control loop is tuned according to the target tuning strategy. By using different training sample sets to train the initial model, the task complexity is gradually increased, achieving the effect of hierarchical training of complex tasks. This at least solves the technical problem of low training efficiency of the tuning model caused by the complexity of the control loop tuning task. Attached Figure Description

[0016] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0017] Figure 1 This is a hardware structure block diagram of a computer terminal (or mobile device) for a PID control loop tuning method according to an embodiment of this application.

[0018] Figure 2 This is a schematic diagram of a PID control loop tuning method according to this application;

[0019] Figure 3 A schematic diagram of the response curve of an optional PID control loop according to an embodiment of this application;

[0020] Figure 4 An optional initial model training flowchart according to an embodiment of this application;

[0021] Figure 5 This is a schematic diagram of an optional PID control loop tuning device according to an embodiment of this application. Detailed Implementation

[0022] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present application.

[0023] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0024] According to an embodiment of this application, an embodiment of a PID control loop tuning method is also provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Also, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0025] The methods and embodiments provided in this application can be executed on mobile terminals, computer terminals, cloud servers, or similar computing devices. Figure 1A hardware block diagram of a computer terminal (or mobile device) for implementing a PID control loop tuning method is shown. Figure 1 As shown, the computer terminal 10 (or mobile device 10) may include one or more processors 102 (shown as 102a, 102b, ..., 102n in the figure) 102 (processor 102 may include, but is not limited to, a microprocessor MCU or a programmable logic device FPGA, etc.), a memory 104 for storing data, and a transmission module 106 for communication functions. In addition, it may also include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of the I / O interface), a network interface, a power supply, and / or a camera. Those skilled in the art will understand that... Figure 1 The structure shown is for illustrative purposes only and does not limit the structure of the aforementioned electronic device. For example, computer terminal 10 may also include... Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown.

[0026] It should be noted that the aforementioned one or more processors 102 and / or other data processing circuits are generally referred to herein as "data processing circuits". These data processing circuits may be embodied, in whole or in part, in software, hardware, firmware, or any other combination thereof. Furthermore, the data processing circuits may be a single, independent processing module, or may be integrated, in whole or in part, into any other element within the computer terminal 10 (or mobile device). As involved in the embodiments of this application, the data processing circuits serve as a processor control mechanism (e.g., selection of a variable resistor termination path connected to an interface).

[0027] The memory 104 can be used to store software programs and modules of application software, such as the program instructions / data storage device corresponding to the PID control loop tuning method in the embodiments of this application. The processor 102 executes various functional applications and data processing by running the software programs and modules stored in the memory 104, thereby implementing the PID control loop tuning method of the above-mentioned application. The memory 104 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory remotely located relative to the processor 102, and these remote memories can be connected to the computer terminal 10 via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0028] The transmission module 106 is used to receive or send data via a network. Specific examples of the network described above may include a wireless network provided by the communication provider of the computer terminal 10. In one example, the transmission module 106 includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission module 106 may be a Radio Frequency (RF) module, used for wireless communication with the Internet.

[0029] The display can be, for example, a touchscreen liquid crystal display (LCD) that allows the user to interact with the user interface of the computer terminal 10 (or mobile device).

[0030] According to an embodiment of this application, an embodiment of a PID control loop tuning method is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Also, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0031] Figure 2 This is a flowchart of a PID control loop tuning method according to an embodiment of this application, as shown below. Figure 2 As shown, the method includes the following steps:

[0032] Step S202: Obtain a set of multiple control deviation values ​​for the PID control loop;

[0033] Step S204: The initial model is trained in multiple stages using multiple sets of control deviation values ​​of the PID control loop and a predetermined tuning strategy of the PID control loop to obtain the target model. The initial model being trained in the current stage is the initial model that was trained in the previous stage.

[0034] Step S206: Input the set of multiple control deviation values ​​of the PID control loop within the target time period into the target model to obtain the target tuning strategy of the PID control loop within the target time period, and tune the PID control loop according to the target tuning strategy.

[0035] Through the above steps, by using different sets of control deviation values ​​to train the initial model and gradually increasing the task complexity, the effect of hierarchical training of complex tasks is achieved, thereby at least solving the technical problem of low training efficiency of the tuning model caused by the complexity of the control loop tuning task.

[0036] It should be noted that the above method also improves the training speed and further improves the training efficiency by adding a pre-determined tuning strategy (expert experience) to the initial model's recognition results during the training process. This is achieved through the PID (Proportion Integral Differential) control algorithm.

[0037] It should be further explained that in the related technologies, the multi-agent deep deterministic policy gradient method is used to construct a sample complexity standard using information in the experience replay pool. The agent samples the corresponding dataset according to the complexity and trains to learn, thereby progressively learning complex knowledge. However, this artificially defined standard sample complexity may not be suitable for the agent's learning, and it lacks expert experience, resulting in high training time costs.

[0038] In step S202, multiple sets of control deviation values ​​include multiple sets of control deviation values ​​with increasing task difficulty, such as: control deviation values ​​output from the identification model by parameter samples with small PID parameter changes, control deviation values ​​output from the identification model by parameter samples with large PID parameter changes, and other training samples with added noise, etc., which increase the task difficulty of the control deviation values ​​participating in the training in an orderly manner. The control deviation values ​​reflect the environmental state of the PID control loop and utilize different environments.

[0039] In step S204, after each training stage is completed, the model parameters obtained from the current stage are fixed. Then, training samples with higher task difficulty are used to train the model in the next stage, gradually increasing the task difficulty until the model can complete the final task. This adopts a tiered training mode, decomposing the actual complex environment (the state of the PID control loop) into environments of different difficulty levels. The model starts training in the simplest environment and eventually enables it to complete the complex task, providing the direction for adjusting the PID parameters. Simultaneously, each set of control deviation values ​​contains a pre-determined PID control loop tuning strategy, which is the tuning experience (expert experience) accumulated during the PID parameter tuning process of the industrial object being studied. The pre-determined tuning strategy classifies the loop state by the state of the PID control loop and, in conjunction with the PID controller, outputs guiding actions to teach the model, improving training efficiency.

[0040] In step S206, after the initial model completes multiple stages of training, the target model is obtained. At this time, the accuracy of the target model in completing the task reaches the preset standard. The target model can be used to process independently to obtain the target tuning strategy, and the PID control loop is tuned according to the target tuning strategy.

[0041] The method provided in this application applies expert experience in loop tuning throughout the entire model training process. Expert experience is used based on the initial model's learning performance. Simultaneously, complex environmental tasks are tiered, gradually increasing in complexity, enabling the model to progressively learn strategies for solving complex environmental tasks. A learning framework combining tiered training and prior knowledge is used to complete the reinforcement learning training process for PID control parameter tuning of process industrial control objects, primarily addressing the problem of low efficiency in directly learning from complex and challenging environmental tasks.

[0042] The following explains steps S202 to S206 in detail.

[0043] In step S204, the state of the PID control loop is determined based on a set of multiple control deviation values ​​of the PID control loop; when the state of the PID control loop is in the state of oscillation decay, the control parameter with the largest cumulative value within a set time period is determined, and the control parameter with the largest cumulative value is reduced according to a first preset ratio; when the state of the PID control loop is in the state of constant amplitude oscillation, the control parameter with the largest cumulative value is reduced according to a second preset ratio; when the state of the PID control loop is in the state of divergence, the control parameter with the largest cumulative value is reduced according to a third preset ratio; when the state of the PID control loop is in the state of slow adjustment, the control parameter with the largest cumulative value is increased according to a fourth preset ratio, wherein the first preset ratio is greater than the second preset ratio and the second preset ratio is greater than the third preset ratio.

[0044] It should be noted that the states of the PID control loop include: oscillation decay, constant amplitude oscillation, divergence, and slow regulation. The pre-determined tuning strategy for the PID control loop can be determined based on the various states of the PID control loop. The method provided in this application does not directly output the tuning parameters, but rather tunes the PID control loop by increasing or decreasing the initial parameters of the PID control loop.

[0045] It should be noted that the control deviation value is determined by the difference between the PID controller's setpoint and the feedback value, as shown in the following formula:

[0046] e(t) = sv(t) - y(t)

[0047] In the formula, t represents the sampling time, e(t) represents the control deviation value at time t, sv(t) represents the set value at time t, and y(t) represents the feedback value at time t.

[0048] In some embodiments of this application, the state of the PID control loop can be reflected by the control deviation values ​​of multiple sampling points within a preset time period.

[0049] It should be further explained that the output value of a PID control loop includes three parts: proportional, integral, and derivative, as shown in the following formula:

[0050]

[0051] Ratio u p (t), integral u i (t) and differential u d The function of the three parts (t) can be expressed as:

[0052] u p (t)=K P e(t);

[0053]

[0054]

[0055] In the formula, K P T represents the proportional adjustment coefficient, and T represents the sampling period. i T represents the integration time. d This represents the differential time.

[0056] As can be seen from the above formula, based on the measured output value u(t) of the PID controller, the control deviation value e(t) at time t, the control deviation value e(t-1) at time t-1, and the proportional adjustment coefficient, sampling period, integral time, and derivative time at the current time, the proportional action, integral action, and derivative action can be obtained. By adjusting these three parameters, the output value of the controller can be obtained.

[0057] Figure 3 The diagram shows the response curve of a PID control loop. The distance between the curve and the setpoint is the control deviation value e(t). It can be seen that the state of the control loop is reflected by the control deviation value.

[0058] In one alternative approach, the state of the PID control loop can be determined as follows: From the set of multiple control deviation values, determine the maximum control deviation values ​​of the PID control loop at multiple sampling points within a preset time period; if the response curve of the PID control loop is in an oscillating state and the multiple maximum control deviation values ​​gradually decrease in chronological order, determine that the state of the PID control loop is oscillating attenuation; if the response curve of the PID control loop is in an oscillating state and the multiple maximum control deviation values ​​are equal, determine that the state of the PID control loop is constant-amplitude oscillation; if the multiple maximum control deviation values ​​gradually increase in chronological order, determine that the state of the PID control loop is diverging; if the multiple maximum control deviation values ​​gradually decrease in chronological order and the minimum value of the multiple maximum control deviation values ​​is greater than a set threshold, determine that the state of the PID control loop is slow adjustment.

[0059] Specifically, the preset time period is [t] i-1 ,t i For example, there are multiple sampling points within a preset time period, such as 10 sampling points, and 10 maximum control deviation values ​​for these 10 sampling points. Assuming the values ​​decrease sequentially over time, the control loop state within the preset time period is determined to be oscillation decay. Therefore, it can be known that... Figure 3 As can be seen, the response curve exhibits an oscillating decay state.

[0060] At 10 maximum control deviation values Under the condition that they are equal, the control loop state within the preset time period is determined to be constant amplitude oscillation. It should be noted that the above equality can be approximate equality. One optional method is to use 10 maximum control deviation values. If the difference is less than the preset difference, then 10 maximum control deviation values ​​are determined. equal.

[0061] At 10 maximum control deviation values When the values ​​increase sequentially over time, the state of the control loop is determined to be divergent.

[0062] At 10 maximum control deviation values The 10 maximum control deviation values ​​that decrease sequentially over time and within a preset time period. If the minimum value is greater than the preset threshold, it is determined to be slow adjustment. In actual application scenarios, since the maximum control deviation value decreases sequentially in time, the maximum control deviation value is the maximum control deviation value of the last sampling point within the preset time period. In some embodiments of this application, the state of the control loop can also be determined to be slow adjustment if the maximum control deviation value is still greater than the preset threshold after adjustment within the preset time period.

[0063] After determining the state of the PID control loop, the control loop is tuned based on the state of the control loop, and the initial parameters of the control loop are adjusted. Specifically, the minimum value of the control parameter adjustment is determined by multiplying it by the first preset ratio; the difference between the value of the control parameter with the largest cumulative value and the product is determined as the target parameter; and the control parameter with the largest cumulative value is adjusted as the target parameter.

[0064] The minimum value of the control parameter adjustment is a. min The maximum value of the control parameter adjustment is a. max Taking a preset ratio of K1, K2, K3, and K4 as an example, in practical application scenarios, a can be... min and a max The value range is set between (0, 2), and the value range can also be adjusted according to actual needs. K1, K2, K3, and K4 are all constants and can be adjusted according to actual needs, where K1 > K2 > K3.

[0065] When the control loop is in a state of oscillation decay, determine the respective weights of the proportional cumulative value, integral cumulative value, and derivative cumulative value in the output value, and select the active term with the largest weight, decreasing its corresponding parameter. Taking the proportional cumulative value as having the largest weight in the output as an example, only the proportional parameter can be adjusted. The adjustment value can be expressed as K1a. min Among them, K1a min <1;

[0066] When the control loop is in a constant-amplitude oscillation, select the active term with the largest proportion and decrease its corresponding parameter. The magnitude of the decrease is different from the magnitude of the oscillation decay. Taking the integral cumulative value as the largest proportion of the output as an example, only the integral parameter can be adjusted. The adjustment value can be expressed as: K2a min Among them, K2a min <1;

[0067] When the control loop is divergent, select the active term with the largest weight and decrease its corresponding parameter. The reduction magnitude differs from the two states mentioned above. Taking the derivative cumulative value having the largest weight in the output as an example, only the derivative parameter can be adjusted. The adjustment value can be expressed as: K3a min Among them, K3a min <1;

[0068] When the control loop is in a slow adjustment state, select the action with the smallest proportion and increase its corresponding parameter. Taking the proportional cumulative value as the smallest proportion of the output as an example, only the proportional parameter can be adjusted. The adjustment value can be expressed as: K4a max Among them, K4a max >1.

[0069] In some embodiments of this application, when the control loop is in the slow adjustment state, the parameter of the action item with the specific gravity in the middle value can also be increased. The increase method is similar to the above method and will not be described again here.

[0070] In the process of training the initial model in multiple stages using multiple sets of control deviation values ​​of the PID control loop to obtain the target model, a target set is determined among the multiple sets of control deviation values ​​of the PID control loop. The target set includes at least: a first set obtained by inputting the control parameters of the PID control loop in the first interval into the preset model; a second set obtained by inputting the control parameters of the PID control loop in the second interval into the preset model; a third set obtained by adding noise to the second set; and a fourth set obtained by inputting the control parameters of the PID control loop in the first interval into the target object. The preset model includes: the identification model of the target object, the target object being controlled by the PID control loop, and the training task difficulty corresponding to the first set, the second set, the third set, and the fourth set increases sequentially.

[0071] It should be noted that the preset model can be an input-output model, such as a transfer function model, used to identify the PID loop state obtained by inputting PID control parameters. In one optional approach, it can be trained using historical data of the controlled object.

[0072] Specifically, the tiered training model uses a teacher-like approach, breaking down a complex task environment. The first, more basic task environment is used to teach the model. Once the agent reaches a certain level of proficiency in the simple environment, its acquired experience and skills are retained. Complexity is then increased based on the previous task environment, gradually accumulating complexity until the model can complete the final task. For PID tuning of industrial process control objects, it is necessary to overcome the complex environment of large PID parameter adjustment space, noisy output, and interference from the input. In specific application environments, the training process can be divided into multiple stages, such as three or five stages, depending on the difficulty of the training task and the training effect. This application proposes a staged reinforcement learning method, which includes:

[0073] Step S402: Based on the historical data or experimental data of the controlled object, establish an identification model G for the controlled object;

[0074] The model can be established using classic system identification methods. The identification model is generally an input-output model, such as a transfer function model.

[0075] Step S404: Perform a step response test on the identification model G of the controlled object, set the initial PID control parameters that vary in the interval [a1, b1], input the output value of the identification model G into the initial model, train for multiple rounds, and retain the parameters of the model, including the weight ratio W1 and the skew amount B1.

[0076] Step S406: Load the first-stage model parameters onto the initial model, perform a step response test on the identification model G of the controlled object, set the initial PID control parameters that vary in the interval [a2, b2], input the output value of the identification model G into the initial model of the current stage, train for multiple rounds, and retain the parameters of the model, including the weight ratio W2 and the skew amount B2.

[0077] Step S408: On the model that has completed the first stage of training, load the second stage model parameters, perform a step response on the identification model G of the controlled object, set the initial PID control parameters that vary in the interval [a2, b2], add random noise to the output value, input the output value of the identification model G into the initial model of the current stage, train for multiple rounds, and retain the parameters of the model, including the weight ratio W3 and the skew amount B3.

[0078] Step S410: On the model that has completed the second stage training, load the third stage model parameters, perform a step response on the identification model G of the controlled object, set the initial PID control parameters that vary in the interval [a2, b2], add random noise to the output value of the identification model G, add random noise to the input value of the identification model G, input the actual output value of the identification model G into the initial model of the current stage, train for multiple rounds, and retain the parameters of the model, including the weight ratio W4 and the skew amount B4;

[0079] Step S412: On the model that has completed the third stage training, load the fourth stage model parameters, conduct a step experiment on the real controlled object, set the initial PID control parameters that vary in the interval [a1, b1], input the actual output value of the real controlled object into the initial model of the current stage, train for multiple rounds, and retain the parameters of the model, including the weight ratio W5 and the skew amount B5.

[0080] Step S414: On the model that has completed the fourth stage training, load the fifth stage model parameters, conduct a step experiment on the real controlled object, set the initial PID control parameters that vary in the interval [a2, b2], input the actual output value of the real controlled object into the initial model of the current stage, train for multiple rounds, and retain the parameters of the model, including the weight ratio W6 and the skew amount B6.

[0081] It should be noted that during the model training process in each of the above stages, a preset success rate, such as 98% or 100%, must be achieved before the model parameters can be saved and the next stage of training can begin. The interval [a1, b1] must be smaller than the interval [a2, b2].

[0082] The initial model trained in the target stage is trained using the target sample set, including: inputting the target state of the PID control loop in the target sample set into the initial model trained in the target stage to obtain the first tuning strategy; determining the second tuning strategy corresponding to the target state using the pre-determined tuning strategy model of the PID control loop; and selecting one tuning strategy from the first and second tuning strategies based on a pre-determined selection probability function to determine the output result of the model trained in the target stage, thereby completing the training of the model trained in the target stage.

[0083] It should be noted that the target stage can be any stage in the initial model's hierarchical training. One optional approach is to select a tuning strategy from the first and second tuning strategies as the target tuning strategy based on a pre-determined selection probability function. In the early stages of training, the pre-determined PID control loop tuning strategy model can serve as the dominant decision-making strategy. For example, in the early stages of training, the probability of selecting the second tuning strategy as the target strategy can be set to 100%. As the success rate of the first tuning strategy increases, the probability of the second tuning strategy as the target strategy is gradually decreased, while the probability of the first tuning strategy as the target strategy is increased. For example, after n... test If the parameter tuning success rate is above p after a number of rounds of testing, the probability of using the second tuning strategy as the target strategy is reduced to N times the original.

[0084] After determining the output result, the method further includes: tuning the PID control loop in the target sample set according to the tuning strategy corresponding to the output result; if the tuning result does not reach the preset tuning index, determining the tuning index weight corresponding to the next tuning operation based on the tuning result; the tuning result is used to characterize whether the tuning index corresponding to the current tuning operation has been reached.

[0085] Specifically, this application also accelerates the parameter tuning speed of the initial model by setting a reward mechanism. In one optional approach, the reward mechanism can be set up with two parts: a positive reward for successful tuning and a negative penalty for unsuccessful tuning. The tuning index weights include: positive reward value and negative penalty value.

[0086] Negative penalty rewards are mainly defined based on the characteristics of the response curve and consist of general control performance indicators (tuning indicators): overshoot, rise time, number of peaks and troughs of the curve, residual error, and a penalty coefficient X when the model adjustment parameters exceed the safety threshold.

[0087] Wherein, overshoot represents the percentage by which the maximum value of the system's step response exceeds the steady-state value, denoted as δ; rise time represents the time it takes for the system's step response to first reach the steady-state value, denoted as t. r The number of peaks and troughs in the curve represents the oscillation frequency of the curve, denoted as k; the residual error represents the error between the system output and the steady-state value when the reaction time reaches the maximum specified value, denoted as e. t The different indicators have different weights for penalties and rewards, and these weights are adjusted according to the aspects that the system needs to focus on controlling.

[0088] R neg (t)=μ1r1(t)+μ2r2(t)+μ3r3(t)+μ4r4(t)+X

[0089]

[0090]

[0091]

[0092]

[0093] Where p1, p2, p3, and p4 represent constant terms, μ1, μ2, μ3, and μ4 represent weight terms, and R neg (t) represents the negative penalty value.

[0094] It should be noted that the positive reward value is only related to the number of steps taken after successful parameter tuning. This setting is intended to encourage the agent to complete the tuning as quickly as possible.

[0095]

[0096] Where, n step n represents the number of adjustment steps required for successful tuning. max R represents the maximum number of adjustment steps in a round. pos (t) represents the positive reward value.

[0097] It should also be noted that, for the initial model training, after obtaining the output results, the obtained tuning index weights are added, along with the set of control deviation values ​​corresponding to the PID control loop in the next time period as the next training sample. Each step uses the set of control deviation values ​​corresponding to the PID control loop as the training sample instead of directly using the control parameters of the PID control loop. This fully considers the actual tuning scenario and makes the training results more accurate.

[0098] In practical application scenarios, Figure 4 An optional initial model training flowchart is shown, such as Figure 4As shown, the set of control deviation values ​​for a preset time period is input into the initial network and the expert experience base (predetermined tuning strategy), generating a first tuning strategy and a second tuning strategy respectively. A target tuning strategy is selected from these strategies, and after multiple training stages, the tuning index weights are obtained. These weights are then transferred to the control deviation value set for the next time period, resulting in tuple data containing the control deviation value set for the preset time period, the target tuning strategy, the tuning index weights, and the control deviation value set for the next time period. This tuple data is stored in the experience buffer. Each time the model interacts with the control deviation value set, it samples a batch of tuple data from the experience buffer for training.

[0099] The control deviation value set for the next time period is input into the policy network of the target network to obtain the third tuning policy. Then, the third tuning policy and the control deviation value set for the next time period are input into the evaluation network 1 and evaluation network 2 of the target network to obtain two tuning index weights. The weight with the smaller weight value is selected to calculate the feedback value. The feedback value is then used to update the two evaluation networks in the initial network. After a delay of K steps, the control deviation value set for the preset time period is input into the policy network of the initial network to obtain the current optimal policy. The current optimal policy is input into the two evaluation networks of the initial network to calculate the gradient of the relative optimal action. The gradient of the relative optimal action is used to update the policy network in the initial network. The update method for the policy network and evaluation network in the target network adopts the shadow averaging method.

[0100] According to another aspect of the embodiments of this application, a non-volatile storage medium is also provided, including a stored program, wherein, when the program is running, it controls the device where the non-volatile storage medium is located to execute the above-mentioned ID control loop tuning method.

[0101] According to another aspect of the embodiments of this application, an electronic device is also provided, including: a memory and a processor, wherein the memory is used to store program instructions; and the processor is connected to the memory and used to run the program, wherein the program executes the above-described ID control loop tuning method during runtime.

[0102] This application also provides a tuning device for a PID control loop, such as... Figure 5As shown, it includes: an acquisition module 50, used to acquire a set of multiple control deviation values ​​of the PID control loop, the control deviation values ​​being used to determine the state of the PID control loop; a training module 52, used to sequentially train the initial model in multiple stages using the set of multiple control deviation values ​​of the PID control loop and a pre-determined tuning strategy of the PID control loop to obtain a target model, wherein the initial model trained in the current stage is the initial model trained in the previous stage; and a tuning module 54, used to input the state of the PID control loop in the target time period into the target model to obtain the target tuning strategy of the PID control loop in the target time period, and tune the PID control loop according to the target tuning strategy.

[0103] The aforementioned tuning device is applied to the tuning method of the PID control loop proposed in this application. The method and device proposed in this application correspond one-to-one, and will not be described in detail here.

[0104] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0105] In the above embodiments of this application, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0106] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units can be a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling, direct coupling, or communication connection may be through some interfaces; the indirect coupling or communication connection of units or modules may be electrical or other forms.

[0107] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0108] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0109] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard drive, magnetic disk, or optical disk.

[0110] The above are merely preferred embodiments of this application. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A method for tuning a PID control loop, characterized in that, include: Obtain a set of multiple control deviation values ​​for the PID control loop; The target model is obtained by sequentially training the initial model in multiple stages using multiple sets of control deviation values ​​of the PID control loop and a predetermined tuning strategy of the PID control loop. The initial model being trained in the current stage is the initial model that was trained in the previous stage. Multiple control deviation value sets of the PID control loop within the target time period are input into the target model to obtain the target tuning strategy of the PID control loop within the target time period. The PID control loop is tuned according to the target tuning strategy. The multiple control deviation value sets include multiple control deviation value sets with increasing task difficulty. After each stage of training is completed, the model parameters obtained from the current stage are fixed, and the model trained in the current stage is trained in the next stage using training samples with higher task difficulty. The model trained in the target stage is trained using the target set and the pre-determined tuning strategy of the PID control loop. This includes: inputting the target set into the model trained in the target stage to obtain a first tuning strategy; determining a second tuning strategy corresponding to the target set using the pre-determined tuning strategy of the PID control loop; selecting one tuning strategy from the first tuning strategy and the second tuning strategy based on a pre-determined selection probability function to determine the output result of the model trained in the target stage, until the probability of the first tuning strategy passing the verification is greater than a set threshold, thus completing the training of the model trained in the target stage. The initial model is trained in multiple stages using multiple sets of control deviation values ​​from the PID control loop and a pre-determined tuning strategy for the PID control loop to obtain a target model. This includes: determining a target set from the multiple sets of control deviation values ​​of the PID control loop, wherein the target set includes at least: a first set obtained by inputting control parameters of the PID control loop within a first interval into the preset model; a second set obtained by inputting control parameters of the PID control loop within a second interval into the preset model; a third set obtained by adding noise to the second set; and a fourth set obtained by inputting control parameters of the PID control loop within the first interval into a target object. The preset model includes: the target... The target object identification model is controlled by the PID control loop. The training task difficulty corresponding to the first set, the second set, the third set, and the fourth set increases sequentially. The model trained in the target stage is trained according to a predetermined training order using the target set and the predetermined tuning strategy of the PID control loop to obtain the target model. The predetermined training order is as follows: first stage, second stage, third stage, and fourth stage. In the first stage, the first set is trained; in the second stage, the second set is trained; in the third stage, the third set is trained; and in the fourth stage, the fourth set is trained.

2. The method according to claim 1, characterized in that, The predetermined tuning strategy for the PID control loop includes: The state of the PID control loop is determined based on a set of multiple control deviation values ​​of the PID control loop. When the state of the PID control loop is in an oscillating decay state, the control parameter with the largest cumulative value in the set time period is determined among the control parameters of the PID control loop, and the control parameter with the largest cumulative value is reduced according to a first preset ratio. When the state of the PID control loop is in constant amplitude oscillation, the control parameter with the largest cumulative value is reduced according to the second preset ratio. When the state of the PID control loop is divergent, the control parameter with the largest cumulative value is reduced according to a third preset ratio. When the PID control loop is in a slow adjustment state, the control parameter with the smallest cumulative value is increased by a fourth preset ratio, wherein the first preset ratio is greater than the second preset ratio and the second preset ratio is greater than the third preset ratio.

3. The method according to claim 2, characterized in that, Determining the state of the PID control loop based on a set of multiple control deviation values ​​includes: Determine multiple maximum control deviation values ​​of the PID control loop at multiple sampling points within a preset time period from the multiple sets of control deviation values; If the response curve of the PID control loop is in an oscillating state and the multiple maximum control deviation values ​​gradually decrease in time sequence, it is determined that the state of the PID control loop is in an oscillating decay state. If the response curve of the PID control loop is in an oscillating state and the multiple maximum control deviation values ​​are equal, then the state of the PID control loop is determined to be in a constant amplitude oscillation. When the multiple maximum control deviation values ​​gradually increase in chronological order over time, it is determined that the state of the PID control loop is diverging. If the multiple maximum control deviation values ​​decrease gradually in chronological order and the minimum value of the multiple maximum control deviation values ​​is greater than a set threshold, the state of the PID control loop is determined to be in slow regulation.

4. The method according to claim 1, characterized in that, After determining the output result, the method further includes: The PID control loop corresponding to the target set is tuned according to the tuning strategy corresponding to the output result. If the tuning result does not reach the preset tuning index, the tuning index weight corresponding to the next tuning operation is determined based on the tuning result. The tuning result is used to characterize whether the tuning index corresponding to the current tuning operation is reached.

5. The method according to claim 2, characterized in that, The control parameter that reduces the cumulative value by the first preset ratio includes: The product of the minimum value of the control parameter adjustment and the first preset ratio is determined as the target parameter; The control parameter with the largest cumulative value is reduced to the target parameter.

6. A tuning device for a PID control loop, characterized in that, include: The acquisition module is used to acquire a set of multiple control deviation values ​​for the PID control loop; The training module is used to sequentially train the initial model in multiple stages using multiple sets of control deviation values ​​of the PID control loop and a pre-determined tuning strategy of the PID control loop to obtain the target model. The initial model being trained in the current stage is the initial model that was trained in the previous stage. The tuning module is used to input multiple sets of control deviation values ​​of the PID control loop within the target time period into the target model to obtain the target tuning strategy of the PID control loop within the target time period, and to tune the PID control loop according to the target tuning strategy. The multiple sets of control deviation values ​​include multiple sets of control deviation values ​​with progressively increasing task difficulty. After each training stage is completed, the model parameters obtained in the current stage are fixed, and then training samples with higher task difficulty are used to train the model trained in the current stage for the next stage. The model trained in the target stage is trained using the target set and the pre-determined tuning strategy of the PID control loop. This includes: inputting the target set into the model trained in the target stage to obtain a first tuning strategy; determining a second tuning strategy corresponding to the target set using the pre-determined tuning strategy of the PID control loop; selecting one tuning strategy from the first and second tuning strategies based on a pre-determined selection probability function as the output result of the model trained in the target stage, until the probability of the first tuning strategy passing verification is greater than a set threshold, thus completing the training of the model trained in the target stage. The initial model is trained in multiple stages using multiple sets of control deviation values ​​from the PID control loop and a pre-determined tuning strategy for the PID control loop to obtain a target model. This includes: determining a target set from the multiple sets of control deviation values ​​of the PID control loop, wherein the target set includes at least: a first set obtained by inputting control parameters of the PID control loop within a first interval into the preset model; a second set obtained by inputting control parameters of the PID control loop within a second interval into the preset model; a third set obtained by adding noise to the second set; and a fourth set obtained by inputting control parameters of the PID control loop within the first interval into a target object. The preset model includes: the target... The target object identification model is controlled by the PID control loop. The training task difficulty corresponding to the first set, the second set, the third set, and the fourth set increases sequentially. The model trained in the target stage is trained according to a predetermined training order using the target set and the predetermined tuning strategy of the PID control loop to obtain the target model. The predetermined training order is as follows: first stage, second stage, third stage, and fourth stage. In the first stage, the first set is trained; in the second stage, the second set is trained; in the third stage, the third set is trained; and in the fourth stage, the fourth set is trained.

7. An electronic device, characterized in that, It includes a memory and a processor, the processor being used to run a program, wherein the program, when running, executes the tuning method of the PID control loop according to any one of claims 1 to 5.

8. A non-volatile storage medium, characterized in that, The non-volatile storage medium includes a stored program, wherein, when the program is executed, it controls the device containing the non-volatile storage medium to perform the tuning method of the PID control loop according to any one of claims 1 to 5.

Images