An automatic control method for flotation process based on intelligent optimization reinforcement learning algorithm

Abstract

The present invention relates to an automatic control method for a flotation process based on an intelligent optimization reinforcement learning algorithm. The present invention aims at the need to precisely control the flotation reagent system during the mineral flotation operation, such as adjusting the dosage of reagents such as collectors, inhibitors and frothers, or process reaction parameters such as pulp pH, flotation temperature, and reagent reaction time, so as to optimize the flotation index and maximize the utilization of mineral resources. The main equipment for adding flotation reagents is a variable frequency pump, and the dosage is adjusted by PID. At the same time, the process index data of the flotation tank is collected by sensors. The improved FPA reinforcement learning is used to optimize the entire control algorithm, and the flotation process index data parameters are used as the environmental state observation values ​​of the reinforcement learning agent. At the same time, the environment reward is designed according to the conditions that meet the flotation requirements, and the agent is trained through repeated environmental interactions. Finally, during operation, the agent continuously gives the optimal action value parameters of the flotation tank actuator in real time, realizing unmanned reinforcement learning optimization control of mineral flotation.

Description

Technical Field

[0001] The present invention relates to the technical field of mineral flotation operation algorithm learning, and in particular to an automatic control method for a flotation process using an intelligently optimized reinforcement learning algorithm. Background Art

[0002] During the mineral flotation operation, the flotation reagent system needs to be precisely controlled, such as adjusting the dosage of reagents such as collectors, depressants and frothers, the pH value of the ore pulp, the flotation temperature, the reagent reaction time, etc., so as to achieve the best flotation indicators and maximize the utilization of mineral resources.

[0003] This invention is the application of intelligent optimization deep reinforcement learning in flotation process control. Combining the characteristics of the flotation process, an efficient and stable deep reinforcement learning control algorithm is designed. With the rapid development of artificial intelligence technology, deep reinforcement learning (DRL) provides a new approach to solving complex control problems in such flotation processes. The main equipment for flotation reagent addition is a variable frequency pump. The reagent dosage is adjusted by PID. The improved FPA algorithm is used to optimize the PID control algorithm of deep reinforcement learning PPO, achieving significant results in improving flotation efficiency, optimizing resource utilization, and enhancing system stability. The reagent addition control of the flotation process uses sensors to collect process index parameters of the flotation cell. Then, the IFPA-PPO-PID control algorithm is used for automatic adjustment. The flotation process index data parameters are used as environmental state observation values ​​of the reinforcement learning agent. At the same time, the environment is rewarded according to the conditions that meet the flotation requirements. After repeated environmental interaction training, the agent is finally run. Finally, the agent continuously gives the optimal action value parameters of the flotation cell control actuator in real time. The reinforcement learning optimization control of metal mineral flotation achieves refined automatic control and realizes unmanned reinforcement learning optimization control of mineral flotation. Summary of the Invention

[0004] The technical problem to be solved by the present invention is to provide an automatic control method for a flotation process for precisely controlling the flotation reagent system, thereby achieving the best flotation index, in view of the deficiencies of the existing technology.

[0005] To solve the above technical problems, the present invention provides a technical solution: a flotation process automatic control method based on an intelligent optimization reinforcement learning algorithm. During the mineral flotation operation in the field of lead and zinc metal mineral resource processing, the method achieves optimal flotation indicators and maximizes the utilization of mineral resources by precisely controlling the dosage of flotation reagents (such as collectors, depressants, frothers, etc.) and process parameters such as pulp pH, flotation temperature, and reagent reaction time. The specific implementation content is as follows:

[0006] A. Preliminary Analysis and Technology Introduction: First, we conduct an in-depth analysis of the characteristics and control difficulties of the flotation process, and then apply deep reinforcement learning technology to flotation process control;

[0007] B. Algorithm Selection and Architecture Determination: A PID control strategy is adopted for flotation system process control, and a hybrid reinforcement learning PPO method combining policy rules and execution evaluation is selected. Given the strong coupling, complex nonlinearity, and time-varying parameter characteristics of flotation process control systems, an Actor-Critic (AC) network architecture suitable for outputting a continuous action space is selected.

[0008] C. Algorithm Improvement and Parameter Optimization: We use a variety of intelligent optimization algorithms to improve the deep reinforcement learning PPO algorithm. This optimized algorithm is used to automatically optimize the PID control parameters, thereby improving the control effect of the flotation process and enhancing the automation level and control accuracy.

[0009] Furthermore, the algorithm selection is specifically as follows: in the corresponding flotation system process control, a PID control strategy is adopted, an evaluation strategy network that gives rewards to the fitting environment is continuously trained and iteratively optimized with the agent action strategy rule network, and a hybrid reinforcement learning PPO method of strategy rules and execution evaluation is selected.

[0010] Furthermore, the algorithm selection is specifically for the flotation process control system, whose characteristics such as strong coupling, complex nonlinearity and time-varying parameters constitute significant operational control difficulties. A reinforcement learning strategy suitable for outputting a continuous action space, namely the Actor-Critic (AC) network architecture, is selected.

[0011] Furthermore, in the reinforcement learning agent of the AC network, the training iterative update gradient strategy of the action policy function and the state value function mainly includes three methods:

[0012] The first is the update of the policy network (Actor): through the policy gradient algorithm, such as the REINFORCE algorithm or its improved version, the parameters of the policy network are updated according to the evaluation signal provided by the value function network (Critic). The policy gradient algorithm aims to maximize the expectation of the cumulative reward and adjusts the policy parameters through the gradient ascent method;

[0013] The second is updating the value function network (Critic): using temporal difference learning or Monte Carlo methods, the parameters of the value function deep learning network are updated based on the data generated by the interaction between the agent and the environment. The goal of the Critic network is to accurately estimate the value of a given state or state-action pair to guide the update of the Actor network;

[0014] The third is joint update: In the AC framework, the Actor and Critic networks are usually updated alternately and simultaneously. That is, the Actor network adjusts its strategy according to the evaluation results of the Critic network, and the Critic network updates its value function estimate based on the interaction data under the new strategy generated by the Actor network. This joint update mechanism helps the intelligent agent find a balance between exploration and exploitation, thereby learning the optimal strategy more effectively.

[0015] Furthermore, in order to obtain the optimal strategy to maximize the reward, according to the training of the action policy network in the previous section, a gradient algorithm is required to update the policy parameters by optimizing the performance index gradient to maximize the reward index obtained by the agent;

[0016] The PPO algorithm combines the advantages of the policy gradient method and importance sampling. It ensures the stability of policy updates by limiting the difference between the new and old policies. The PPO algorithm includes an Actor network and a Critic network, which are used to generate actions and evaluate the value of actions respectively.

[0017] During the training process, PPO updates the policy parameters by optimizing the alternative objective function, and uses the Clip truncation mechanism to control the amplitude of the policy update;

[0018] The PPO algorithm allows the same data to be used multiple times for strategy updates in each iteration, which improves data utilization efficiency and makes the PID parameter optimization process more efficient;

[0019] At the same time, the probability density function of Gaussian distribution is used as the Actor action strategy function, so that the control system can better adapt to complex and changing environments and working conditions.

[0020] Furthermore, the PPO algorithm is applied to the flotation process control system framework for adaptively adjusting PID parameters. The PPO algorithm, as a parameter adjustment mechanism for PID control, obtains environmental state information of the flotation control process. The action information is the output of the action strategy a, which is a three-dimensional feature vector array, including kp, ki, and kd composed of the three PID parameters. The action space is:

[0021] A={k p ,k i ,k d}

[0022] Then, it continuously interacts with the environment of the flotation control process, collects data and updates network parameters to achieve learning and training of the PPO agent, and continuously optimizes the PPO strategy network and evaluation network until the preset number of training rounds or reward threshold is reached. After training, the PPO-PID algorithm model will provide the best adjustment parameters for flotation optimization control.

[0023] The entire algorithm model framework of the autonomous unmanned system with optimized control established by reinforcement learning uses various sensors to collect perception data of the environment, inputs it into the constructed reinforcement learning agent, makes decisions and controls through continuous action strategies, and calculates rewards with the value function network, thereby iteratively updating the parameters of the reinforcement learning training to achieve the highest reward of the learning control strategy. Finally, the trained agent is used to achieve intelligent real-time control in interaction with the environment.

[0024] Furthermore, the algorithm improvement and parameter optimization are specifically to improve the chaotic optimization FPA algorithm, use piecewise chaotic mapping to improve the FPA algorithm (IPFA) initialization, and improve the parameter optimization effect and fitness curve convergence speed;

[0025] The IFPA algorithm is introduced to optimize reinforcement learning PPO (IFPA-PPO-PID). By utilizing the excellent parameter iterative optimization characteristics of IFPA, the global optimal parameters are selected in the PPO hyperparameter adjustment link, which improves the training reward value, accelerates the convergence speed, and reduces the training time consumption.

[0026] Furthermore, the flotation process indicator data parameters are used as the environmental state observation values ​​of the reinforcement learning agent. At the same time, the environmental rewards are designed according to the conditions that meet the flotation requirements. After repeated environmental interaction training, the agent is trained and the improved IFPA algorithm is used to continuously improve and adjust the hyperparameters of the PPO reinforcement learning. Finally, during operation, the agent continuously gives the optimal action value parameters of the flotation tank actuator in real time, realizing unmanned reinforcement learning optimization control of mineral flotation and solving the problem of poor control effect under nonlinear time-varying mineral flotation process.

[0027] After adopting the above algorithm, the present invention has the following advantages: In the field of lead and zinc metal mineral resource processing, the flotation process is a key link in extracting valuable minerals, and its control effect directly affects the quality and recovery rate of metal resource products. The method proposed in the present invention targets the mineral flotation operation process and uses precise algorithm control of the flotation reagent system, such as adjusting the dosage of reagents such as collectors, depressants, and frothers, the pH value of the ore pulp, the flotation temperature, and the reagent reaction time, so as to achieve the best flotation indicators and maximize the utilization of mineral resources. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 Schematic diagram of the model for the interaction between the intelligent agent and the environment;

[0029] Figure 2 This is a schematic diagram of the critic evaluation algorithm network structure;

[0030] Figure 3 This is a diagram of the network structure of the actor action strategy algorithm;

[0031] Figure 4 This is a logical diagram of the deep reinforcement learning PPO strategy algorithm;

[0032] Figure 5 Schematic diagram of the algorithm model framework for deep reinforcement learning optimization;

[0033] Figure 6 Schematic diagram of the control algorithm framework for deep reinforcement learning optimization;

[0034] Figure 7 Schematic diagram of the deep reinforcement learning PPO adaptive optimization PID control system framework;

[0035] Figure 8 Schematic diagram of piecewise chaotic mapping distribution;

[0036] Figure 9 This is a diagram of the frequency distribution of piecewise chaotic mapping;

[0037] Figure 10 Schematic diagram of the F1(x) function and fitness value iteration curve.

[0038] Figure 11 Schematic diagram of the F2(x) function and fitness value iteration curve;

[0039] Figure 12 Schematic diagram of the results of the F3(x) function and the fitness value iteration curve;

[0040] Figure 13 Schematic diagram of the F4(x) function and fitness value iteration curve;

[0041] Figure 14 Schematic diagram of the deep reinforcement learning PPO adaptive optimization PID control system framework;

[0042] Figure 15 Schematic diagram of the fitness curve for improving the deep reinforcement learning IFPA-PPO-PID control algorithm;

[0043] Figure 16 Schematic diagram of the fitness curve for improving the deep reinforcement learning IFPA-PPO-PID control algorithm;

[0044] Figure 17 Schematic diagram of the fitness curve for improving the deep reinforcement learning IFPA-PPO-PID control algorithm;

[0045] Figure 18 Schematic diagram of IFPA-PPO-PID control parameter output changes. DETAILED DESCRIPTION

[0046] The present invention will be described in further detail below with reference to the accompanying drawings.

[0047] Combined with attachment Figure 1-18 The object of the present invention is achieved by the following manner: regulating the flotation process of metal minerals, especially the flotation of lead and zinc ores, and adjusting the flotation reaction reagents such as collectors, depressants and frothers as well as the flotation reaction conditions such as pH and temperature during the separation and purification of the flotation tank to achieve the best flotation effect, that is, to improve the indicators such as the lead and zinc recovery rate.

[0048] The deep reinforcement learning in this invention is based on the Markov decision process for algorithm architecture design. Reinforcement learning allows the agent to learn by trial and error in the environment and continuously optimize its behavior strategy to maximize the cumulative reward. The agent and the environment continuously interact through state and action. The state of the environment is random, and the action strategy of the agent is also random. The state and action time t, the state of the environment s t , the action output of the agent actuator of reinforcement learning at, the action set A made by the agent, the reward value rt given by the environment to reinforcement learning after the agent performs the action at time t, the reward evaluation function R, E is the target expected value of the reinforcement learning training process, the value function Vy of the state at this moment t, the action strategy y decided by the agent state actuator, the network parameter of the strategy function y is θ, the value function qy of the permanent reward after the state at time t performs the action, p is the possibility of state change after executing the action, the best strategy y*, the best value function q*, and the evaluation attenuation factor γ of the execution effect.

[0049] According to the probability density function distribution of the environmental state is discrete or continuous, the formula for obtaining the environmental state is expressed as follows

[0050] p(s t |s t-1 ,a t-1 )→s t (0-1)

[0051] The random sampling of the actuator action output and the actuator action strategy probability function of reinforcement learning is obtained

[0052] y(a t |s t )→a t (0-2)

[0053] The rewards set by the reinforcement learning action are obtained by random sampling of the state sampling and execution dynamics of the environment.

[0054] R(rt|st,at)→rt (0-3)

[0055] The state value function at time t is the expectation of the actuator reward and the subsequent continuous action to obtain the decaying prospective reward q, which is obtained using the Bellman expectation equation.

[0056]

[0057] The action value function after the state executes the action at time t is obtained by the expectation of the actuator reward and the decayed prospective return Ut

[0058]

[0059] The best actuator action strategy is selected in any state s obtained through the final reinforcement learning training.

[0060]

[0061] At this time, the corresponding optimal value function will also be obtained by any state and any action of the intelligent agent.

[0062]

[0063] Reinforcement learning is generally carried out around the iteration of action value function and state value function. As the evaluation body of reinforcement learning referee, critic uses value function network to evaluate state and action at the same time. This is evaluated by deep learning network to obtain q y (s t ,a)≈q(s t ,a;w), w is the weight parameter of the deep learning network, such as Figure 2 The evaluation value function of reinforcement learning is shown in Figure 2.

[0064] The reinforcement learning algorithm for training action strategy improvement continuously improves the actor strategy, that is, through complete output actions, in a complete interaction with the environment, the cumulative reward is obtained to achieve the optimal value. Then the goal of training reinforcement learning is the optimal action strategy function distribution, which can be obtained

[0065]

[0066] Therefore, the best action of reinforcement learning is obtained under the optimal action strategy function.

[0067]

[0068] From the above, we can see that the value learning of the evaluation function network requires iterative calculation of the gradient training of the value function, where the error is also the loss function That is, the estimated value given by the deep learning value network, so the gradient of the iteratively updated parameters is

[0069] Then in order to reduce the error of the evaluation function, according to the principle of gradient descent, the learning rate, that is, the step size, is α, so the iterative update of the evaluation function network parameter w is

[0070] Typical actor networks for reinforcement learning are as follows: Figure 3 shown.

[0071] By utilizing the powerful fitting calculation capability of the deep learning network and performing approximate calculation of the action strategy based on the policy network parameters, we can obtain

[0072] y(a|s t )≈y(a|s t ;θ)(0-10)

[0073] And the probability density of the action policy network

[0074]

[0075] Then, the state value function is obtained from formula (0-4) as follows

[0076]

[0077] However, the strategy gradient is obtained randomly, as shown below:

[0078]

[0079] According to the results of the policy gradient, we need to maximize the reward after the action is executed, so the update can be obtained as follows: The above constitutes the most basic AC algorithm framework and training method for deep reinforcement learning.

[0080] For the flotation process control system, it is necessary to select a reinforcement learning strategy suitable for outputting a continuous action space, namely the Actor-Critic (AC) network architecture. The PPO algorithm includes an Actor network and a Critic network, which are used to generate actions and evaluate the value of actions, respectively.

[0081] PPO algorithm optimization process architecture Figure 4 As shown in Figure 1, it is divided into two parts: one is the interaction between the agent and the environment, and the other is the iterative improvement of the training network parameters. When training reinforcement learning, the state value function V estimated by the Critic evaluation network is discounted and added with the reward rt to obtain the reward function R. The advantage function is estimated by calculating the time difference error between the reward function R and the state value function V, and then performing the generalized advantage estimation gae calculation. for:

[0082]

[0083] Then back propagate the gradient descent to update the Critic evaluation network parameter w, and the loss function of the updated Critic evaluation network parameter w is:

[0084]

[0085] At the same time, the distribution probability of new and old actions is sampled by the importance weight δ, and then the first-order optimization constraint and truncation limit function operation coefficient ε are used. ε is generally taken as 0.2, and the loss function is obtained as follows:

[0086]

[0087] The loss function is used as the objective function to perform gradient updates, and the truncation operation better limits the update of the new strategy. Is positive, the reward estimate generated by the current action is greater than the return estimate of the old action strategy, the new action strategy needs to be updated, and the probability of the action should be larger in distribution, but it needs to be restricted and truncated within the range of (1-ε,1+ε) multiples of the old strategy. Similarly, when If ∈R is positive, the action probability is smaller in distribution and the update is truncated within the range of (1-ε,1+ε) multiples of the old policy.

[0088] The design uses the deep reinforcement learning PPO algorithm to adaptively adjust the PID control model (PPO-PID). Taking the flotation dosing pH control system as an example, the control goal is to achieve the control process index evaluation of the desired reaction parameters of the flotation system based on multiple requirements. The environmental reward function of reinforcement learning is very important. For the pH adjustment automatic process control of the flotation system, the expected pH value is TpHq and the actual pH is TpH. Then the discretization is

[0089] e(n)=[T pHq (n)-T ph (n)] (0-17)

[0090] Then the state observation space S is calculated according to the error e between the actual pH value TpHq and the expected pH value of the flotation system and the change rate To design:

[0091]

[0092] In addition, the termination signal of the reinforcement learning agent is designed to stop in advance, that is, to stop the sign according to the control boundary conditions to prevent exceeding the safety range. At the same time, it prevents errors and poor training data from affecting the learning update of the agent. According to the process conditions of flotation dosing to control pH, the calculation of the stop sign is:

[0093]

[0094] The safety margin reward according to the stop sign is rd:

[0095] r d =-20I done (0-20)

[0096] The control error reward re is:

[0097]

[0098] The training ends at time tq, and the response reward rt at time t when the control response reaches the target value within 50% of the error is:

[0099]

[0100] The reward rq at the end of training is:

[0101]

[0102] The faster the response speed, the better, and the smaller the steady-state fluctuation and overshoot, the better. Then the reward function of the entire control process is

[0103] r=r d +r e +r t +r q (0-24)

[0104] The flotation control system uses PID controller for basic control operations. The PID control formula has three parameters: kp, ki, and kd. After discretization, it is

[0105]

[0106] Then the formula for the pH PID controller is

[0107]

[0108] The flotation process control system framework using the PPO algorithm to adaptively adjust PID parameters is as follows: Figure 7 As shown in the figure, the PPO algorithm is used as a parameter adjustment mechanism for PID control to obtain the environmental state information of the flotation control process. The action information is the output of the action strategy. a is a three-dimensional feature vector array, which contains kp, ki, and kd composed of the three PID parameters. The action space is:

[0109] A={k p ,k i ,k d}(0-27)

[0110] Then, it continuously interacts with the environment of the flotation control process, collects data and updates network parameters to realize the learning and training of the PPO agent, and continuously optimizes the PPO strategy network and evaluation network until the preset number of training rounds or reward threshold is reached. After training, the PPO-PID algorithm model will provide the best adjustment parameters for flotation optimization control.

[0111] In the PPO algorithm, the PID parameters are adaptively adjusted. Its parameter space is continuous and a continuous Gaussian probability density function is used for sampling. Therefore, the function distribution of the action strategy y is

[0112]

[0113] Where a is the action output. In policy-based reinforcement learning algorithms, policies are often expressed as distributions. By combining distribution transformation with reversible compression functions, the PPO algorithm is applied to the flotation process control. The detailed process steps of the algorithm pseudo code are shown in Algorithm Table (1). This achieves adaptive adjustment of PID parameters, improves control accuracy, and maintains the continuity of the distribution.

[0114] Table 1: Detailed process steps of reinforcement learning flotation control algorithm pseudo code

[0115]

[0116] The Flower Pollenation Algorithm (FPA) introduces a parameter p, initially set to 0.5, to slightly bias the switching probability towards local pollination. This parameter adjusts the frequency of switching between local and global pollination, improving optimization efficiency. The global and flower-specific nature of biological pollination can be mathematically described, and the update formula is expressed as:

[0117]

[0118] Here, Represents the solution vector The update during the t iterations, x g is the optimal solution for the current fitness function evaluation, and λ is the proportional coefficient for adjusting the step size. Function T() represents the step size during the pollination process, and the random step size sampled based on Levy flight reflects the intensity of pollination activity.

[0119] The formula for the Levy flight distribution is as follows:

[0120] [T~Levy(ξ),(ξ>0)] (0-30)

[0121] Levy flight is calculated as follows:

[0122]

[0123] The distribution is valid for ξ>0, where Γ() represents the standard Gamma function.

[0124] Another equation is the local self-pollination process, using the switching probability parameter p, and its self-pollination process formula is:

[0125]

[0126] in, and represents two solutions drawn randomly, Follow the principle of randomly sampling values ​​uniformly distributed between [0,1]. The parameter p is introduced, and its initial value is set to 0.5. FPA is described by four limiting conditions based on natural laws, which are expressed as the four equations above. In addition, the initialization position distribution of the general pollen population will not be uniform if the coefficients randomly generated from [0,1] are extracted within the spatial range. Therefore, piecewise chaotic mapping is used to improve the initialization of the FPA algorithm (Improved Flowerpollenation algorithm, IPFA), enabling the IPFA algorithm to improve parameter optimization and accelerate the convergence of the fitness curve. The derivation formula of piecewise chaotic mapping is as follows:

[0127]

[0128] Where n is the number of pollen populations, and 1≤i<n, then the initial distribution of the population is

[0129]

[0130] According to the above formula, the code is written and run. The piecewise chaotic map obtains the [0,1] population distribution according to the population size of 30 as follows Figure 8 and Figure 9 , in the initial stage of the algorithm operation, the best parameter solution space segment can be found as quickly as possible, increasing the probability of solving the global optimal parameters.

[0131] To verify the performance of the improved IFPA algorithm, the experiment selected four representative standard performance test functions: Generalized Rastrigin (F1(x)), Ackley (F2(x)), Step (F3(x)), and Griewank (F4(x)). F1(x) and F2(x) are unimodal functions that can test the optimization algorithm's ability to find the best solution in the local solution space. F3(x) and F4(x) are multimodal functions that can evaluate the optimization algorithm's ability to escape local extreme points and obtain the global optimal parameters. IFPA's ability to find the optimal parameters was also tested, and then compared with the other three algorithms on standard test functions. The experimental parameter settings are shown in Table 2.

[0132] Table 2 Algorithm test comparison experiment settings

[0133]

[0134] According to the unified experimental parameter settings, the algorithm experimental function and running results are as follows Figures 10 to 13 As shown in the running curve results, it can be seen that the improved IFPA algorithm can iteratively solve the optimal solution of the function. Compared with the unimproved FPA algorithm, the optimization ability is significantly improved, and it can quickly iterate and converge to the optimal solution, achieving better algorithm optimization performance for solving the optimal parameters.

[0135] In the process of training the IFPA-PPO-PID algorithm, the learning rate of PPO-PID is first selected using IFPA, and the population number of IFPA is set to n, the search range is [LB, UB], the maximum number of iterations IterMax, the transition probability p and other parameters are set. Then, the piecewise chaotic mapping is used to initialize the distribution of the population, and the initialized fitness function value is calculated. Here, the reward of reinforcement learning is used as the fitness function, and PPO-PID is called to initialize the fitness calculation to obtain the fitness value of each particle in the population and obtain the global optimal solution. After that, FPA is used to update the particle distribution position, and the fitness function is calculated again to iteratively update the particle distribution position. The particle distribution position and fitness function value are continuously iteratively updated until the maximum number of iterations IterMax is reached and the update iteration is stopped. At this time, the optimal PPO-PID learning rate parameter is output, such as Figure 14 .

[0136] To verify the effectiveness of the IFPA-PPO-PID algorithm, a commonly used first-order time-delay model was selected as the controlled object. First, the parameters of the PPO-PID with and without IFPA were set as shown in Table 3. The flotation process control experiment was conducted under the same initial conditions and experimental parameters. The learning rate range was [0.05, 0.0001]. Without IFPA, the learning rate was generally set to 0.01.

[0137] Table 3 Reinforcement learning algorithm experimental settings

[0138]

[0139]

[0140] The algorithm fitness function curve using IFPA is as follows: Figure 15 As shown in the figure, the algorithm ran 20 iterations, achieving a rapid decrease in fitness at the beginning of the iteration. After finding the local optimal parameters, the algorithm continued searching until it converged after obtaining the global optimal parameters, achieving a good balance between local and global search. The PPO-PID reward values ​​with and without IFPA were -134.5 and -130.7, respectively, a 2.9% performance improvement.

[0141] After further training the IFPA-PPO-PID algorithm 30,000 times, Figure 16 The reward value has converged after 12,000 times, indicating that the improved deep reinforcement learning control algorithm can achieve good control effects in flotation process control. After training the saved IFPA-PPO-PID algorithm model, the flotation process control simulation is carried out and compared with the PID control after manual tuning. The results are as follows Figure 17 As shown, the control effect is greatly improved.

[0142] In the flotation process control simulation of the IFPA-PPO-PID algorithm model, the effect of its action space, i.e., the output of PID parameters, is as follows: Figure 18 As shown in the figure, compared with the PID control algorithm, the parameters are continuously and adaptively adjusted in real time, achieving better results than manual adjustment.

[0143] The IFPA-PPO-PID improved reinforcement learning optimization control algorithm for mineral flotation proposed in the present invention uses the flotation process index data parameters as the environmental state observation values ​​of the reinforcement learning agent, and at the same time designs the environmental rewards according to the conditions that meet the flotation requirements. After repeated environmental interaction training, the agent is trained, and the improved IFPA algorithm is used to continuously improve and adjust the hyperparameters of the PPO reinforcement learning. Finally, during operation, the agent continuously gives the optimal action value parameters of the flotation tank actuator in real time, realizing unmanned reinforcement learning optimization control of mineral flotation and solving the problem of poor control effect under nonlinear time-varying mineral flotation process. The reagent addition control of the flotation process collects the process index parameters of the flotation tank through sensors, and then uses the IFPA-PPO-PID control algorithm for automatic adjustment. The reinforcement learning optimization control of metal mineral flotation realizes refined automatic control.

[0144] The above description of the present invention and its embodiments is non-limiting, and the actual structure is not limited thereto. In short, if a person skilled in the art is inspired by the above, and does not deviate from the purpose of the invention, without creatively designing a structure and embodiment similar to the technical solution, they shall fall within the scope of protection of the present invention.

Claims

1. A flotation process automatic control method based on intelligent optimization reinforcement learning algorithm, characterized by: In the mineral flotation process of lead and zinc metal mineral resource processing, by precisely controlling the flotation reagent dosage and the process parameters of the pulp pH, flotation temperature, and reagent reaction time, the flotation indicators are optimized to maximize the utilization of mineral resources. The specific implementation contents are as follows: A. Preliminary Analysis and Technology Introduction: First, we conduct an in-depth analysis of the characteristics and control difficulties of the flotation process, and then apply deep reinforcement learning technology to flotation process control; B. Algorithm Selection and Architecture Determination: A PID control strategy is adopted for flotation system process control, and a hybrid reinforcement learning PPO method combining policy rules and execution evaluation is selected. Given the strong coupling, complex nonlinearity, and time-varying parameter characteristics of flotation process control systems, an Actor-Critic (AC) network architecture suitable for outputting a continuous action space is selected. C. Algorithm Improvement and Parameter Optimization: We use a variety of intelligent optimization algorithms to improve the deep reinforcement learning PPO algorithm. This optimized algorithm is used to automatically optimize the PID control parameters, thereby improving the control effect of the flotation process, enhancing the automation level and control accuracy. Specifically, the algorithm selection adopts a PID control strategy in the corresponding flotation system process control, and continuously trains and iterates the optimization of the evaluation strategy network that rewards the fitting environment and the agent action strategy rule network, and selects a hybrid reinforcement learning PPO method that combines strategy rules and execution evaluation. Specifically, the algorithm selection is based on the flotation process control system, whose strong coupling, complex nonlinearity, and time-varying parameter characteristics constitute significant operational control difficulties. A reinforcement learning strategy suitable for outputting a continuous action space, namely the Actor-Critic (AC) network architecture, is selected. The PPO algorithm is applied to the flotation process control system framework for adaptively adjusting PID parameters. The PPO algorithm is used as a parameter adjustment mechanism for PID control to obtain environmental state information of the flotation control process. The action information is the output of the action strategy. a is a three-dimensional feature vector array, which is composed of the three PID parameters k p 、k i 、k d , the action space is: A={k p ，k i ，k d } ； Then, it continuously interacts with the flotation control process environment, collects data and updates network parameters to implement PPO agent learning and training, continuously optimizing the PPO policy network and evaluation network until the preset number of training rounds or reward threshold is reached. After training, the PPO-PID algorithm model will provide the best adjustment parameters for flotation optimization control. The entire algorithm model framework of the autonomous unmanned system with optimized control established by reinforcement learning uses various sensors to collect perception data of the environment, inputs it into the constructed reinforcement learning agent, makes decisions and controls through continuous action strategies, and calculates rewards with the value function network, thereby iteratively updating the parameters of the reinforcement learning training to achieve the highest reward of the learning control strategy. Finally, the trained agent is used to achieve intelligent real-time control in interaction with the environment.

2. The flotation process automatic control method based on an intelligent optimization reinforcement learning algorithm according to claim 1 is characterized in that: In the reinforcement learning agent of the AC network, the training iterative update gradient strategy of the action policy function and the state value function includes three methods: The first is the update of the policy network (Actor): the policy network parameters are updated based on the evaluation signal provided by the value function network (Critic) through the policy gradient algorithm. The policy gradient algorithm aims to maximize the expected cumulative reward and adjusts the policy parameters through the gradient ascent method. The second is the update of the value function network (Critic): using temporal difference learning or Monte Carlo methods, the parameters of the value function deep learning network are updated according to the data generated by the interaction between the agent and the environment. The goal of the Critic network is to accurately estimate the value of a given state or state-action pair to guide the update of the Actor network; The third is joint update: In the AC framework, the Actor and Critic networks are usually updated alternately and simultaneously. That is, the Actor network adjusts its strategy according to the evaluation results of the Critic network, and the Critic network updates its value function estimation based on the interaction data under the new strategy generated by the Actor network. This joint update mechanism helps the intelligent agent find a balance between exploration and exploitation, thereby learning the optimal strategy more effectively.

3. The flotation process automatic control method based on an intelligent optimization reinforcement learning algorithm according to claim 2 is characterized in that: In order to obtain the optimal strategy to maximize the reward, according to the training of the action-policy network in the previous section, a gradient algorithm is required. By optimizing the performance index gradient, the policy parameters are updated to maximize the reward index obtained by the agent. The PPO algorithm combines the advantages of the policy gradient method and importance sampling. It ensures the stability of policy updates by limiting the difference between the new and old policies. The PPO algorithm includes an Actor network and a Critic network, which are used to generate actions and evaluate the value of actions respectively. During the training process, PPO updates the policy parameters by optimizing the alternative objective function, and uses the Clip truncation mechanism to control the amplitude of the policy update; The PPO algorithm allows the same data to be used multiple times for strategy updates in each iteration, which improves data utilization efficiency and makes the PID parameter optimization process more efficient; At the same time, the probability density function of Gaussian distribution is used as the Actor action strategy function, so that the control system can better adapt to complex and changing environments and working conditions.

4. The flotation process automatic control method using an intelligent optimization reinforcement learning algorithm according to claim 1 is characterized in that: The algorithm improvement and parameter optimization are specifically to improve the chaotic optimization FPA algorithm, use piecewise chaotic mapping to improve the FPA algorithm (IPFA) initialization, and improve the parameter optimization effect and fitness curve convergence speed; The IFPA algorithm is introduced to optimize the reinforcement learning PPO (IFPA - PPO - PID). By utilizing the excellent parameter iterative optimization characteristics of IFPA, the global optimal parameters are selected in the PPO hyperparameter adjustment phase, which improves the training reward value, accelerates the convergence speed, and reduces the training time consumption.

5. The automatic control method for a flotation process using an intelligent optimization reinforcement learning algorithm according to any one of claims 1 to 4, characterized in that: The flotation process indicator data parameters are used as the environmental state observation values ​​of the reinforcement learning agent. At the same time, the environmental rewards are designed according to the conditions that meet the flotation requirements. The agent is trained through repeated environmental interactions, and the improved IFPA algorithm is used to continuously improve and adjust the hyperparameters of the PPO reinforcement learning. Finally, during runtime, the agent continuously gives the optimal action value parameters of the flotation tank actuator in real time, realizing unmanned reinforcement learning optimization control of mineral flotation and solving the problem of poor control effect under nonlinear time-varying mineral flotation process.

Images