Ore grinding control parameter calculation method and system based on deep reinforcement learning

Abstract

The invention provides an ore grinding control parameter calculation method and system based on deep reinforcement learning, and the method comprises the steps: 1, collecting the point data of a ball mill, a pump pool and a cyclone in a grinding and selecting stage, and carrying out the data cleaning and data preprocessing; 2, modeling control problems of four control quantities including the ore feeding quantity of the ball mill, the water feeding quantity of the ball mill, the water supplementing and adding quantity of a pump pool and the pressure of a hydrocyclone by adopting deep reinforcement learning; and 3, the deep reinforcement learning model is trained offline based on the collected data, the trained deep reinforcement learning model outputs the ore feeding amount of the ball mill, the water feeding amount of the ball mill, pump pool water supplementing and adding and the pressure of a cyclone under the current working condition, and ore grinding control is achieved. The dynamic time-varying control problem of multi-parameter coupling in the grinding and separation stage is solved by adopting deep reinforcement learning modeling multi-parameter cooperative control (the ore feeding amount of the ball mill, the water feeding amount, the pump pool water supplementing and adding amount and the pressure of the hydrocyclone).

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of deep learning and data processing, in particular, to a grinding control parameter calculation method and system based on deep reinforcement learning. BACKGROUND

[0002] Ore needs to go through multiple stages of grinding process before becoming high-grade concentrate pellets, including grinding and separation, magnetic separation, flotation, etc. The grinding and separation stage is the most critical, because it is at the forefront of the grinding process, and the accuracy of the control of this stage is crucial to the entire subsequent grinding process. The grinding and separation process is a highly closed-loop process. Ore first enters the ball mill. In order to ensure a certain grinding concentration, water is added to the ball mill while grinding. After grinding, the ore flows into a buffer pump pool waiting for classification by the cyclone. The cyclone uses centrifugal force to classify the pump pool slurry. The qualified slurry will flow into the next stage of the magnetic separation process, and the unqualified slurry will flow back to the ball mill for further grinding.

[0003] The important control quantities of the grinding and separation process include the ball mill feed quantity, the ball mill water quantity, the pump pool water supplement, and the cyclone pressure. A larger feed quantity will cause the ball mill to swell, but a smaller feed quantity will result in low grinding efficiency. A larger ball mill water quantity will result in a lower grinding concentration in the ball mill, resulting in low grinding efficiency, while a smaller ball mill water quantity will result in a higher grinding concentration in the ball mill, causing dry grinding. The pump pool water supplement and the cyclone pressure will both affect the slurry concentration in the pump pool, which will affect the return sand and overflow quantity obtained by the cyclone classification, and the return sand quantity will further affect the ball mill feed quantity and water quantity. In production, the four control quantities are coupled with each other, and the response to changes in conditions such as incoming material is overall lagging, and the lag length is uncertain. In order to improve grinding efficiency, the four parameters must be optimized in real time.

[0004] Patent document (application number 201610445622.4) discloses an energy-saving optimization method for grinding and classification system of a concentrator based on decision rule extraction. A decision rule set is established using a data-oriented decision contingency table for production process record data. The rules that can be listed by this decision rule set are very limited, and the three parameters controlled: feed quantity, return sand water, and discharge water are not optimized in a coordinated manner, but are optimized separately, so the optimal coordinated control cannot be achieved.

[0005] The patent document (application number 202111048650.X) discloses a matching method, device and electronic equipment for grinding control parameters, determines the working condition category in the grinding classification process based on historical data, and matches the grinding control parameters and the corresponding parameter values corresponding to the working condition category for the current data. The mode recognition of the working condition category in this method mainly relies on the realization of the clustering method, and a set of setting values belonging to this category are obtained after the new working condition is compared with the clustering center. The disadvantage of this method is that the number of working condition modes is difficult to determine, and the comparison with the clustering center is only relied on to determine the setting value, which is relatively rough and cannot realize fine control.

[0006] The patent application document CN108469797A discloses a grinding process modeling method based on neural network and evolutionary calculation, relating to the technical field of iron ore grinding. The method first establishes a case library, and retrieves a reasonable ball mill feed quantity from the case library by using a case retrieval method; then, a mathematical model of the ball mill grinding process is established by using a neural network method, and the relationship between the ball mill feed quantity, water quantity and grinding effect is established; taking the maximum ball mill specific productivity and the optimal ore particle size distribution as optimization objectives, the constraint conditions are determined in combination with the actual working conditions, a set of non-inferior solution sets are obtained by using a non-inferior sorting genetic algorithm with an elite strategy, and the optimal solution is determined by using a TOPSIS algorithm. However, this patent cannot completely solve the existing technical problems and cannot meet the needs of the present application. SUMMARY

[0007] In view of the defects in the prior art, the purpose of the present application is to provide a grinding control parameter calculation method and system based on deep reinforcement learning.

[0008] The grinding control parameter calculation method based on deep reinforcement learning provided by the present application comprises: Step 1: collecting point data of ball mills, pump pools and cyclones in the grinding and separation stage, and performing data cleaning and data preprocessing; Step 2: modeling the control problem of the four control quantities of ball mill feed quantity, ball mill water quantity, pump pool water supplement and cyclone pressure by using deep reinforcement learning; Step 3: training the deep reinforcement learning model based on the collected data offline, and the trained deep reinforcement learning model outputs the ball mill feed quantity, ball mill water quantity, pump pool water supplement and cyclone pressure of the current working condition, thereby realizing the control of the grinding.

[0009] Preferably, the data cleaning and data preprocessing comprise: using a forward filling method to fill in the null values, screening the running state data according to the ball mill running signal flag, performing additive merging on the point data, and removing the transitional operation of the four control quantity setting values; The additive merging is for one standby one use device, and the point data of the working device at the same time is merged. The transitional operation of removing the four control quantity set values includes: cutting each control quantity set value in sequence with a short duration one by one.

[0010] Preferably, the step 2 includes: Step 2.1: defining the state of reinforcement learning The point positions associated with the intensity Top10 of the ore feeding quantity set value, the mill water feeding quantity set value, the pump pool makeup water set value and the cyclone pressure set value are collected; The action is defined as four continuous variables: the ore feeding quantity set value, the mill water feeding quantity set value, the pump pool makeup water set value and the cyclone pressure set value; The reward function is defined as , wherein is the ore grinding throughput, is the average liquid level in the mill, The return function is defined as , wherein is the action sequence, is the discount factor; is the number of operations; represents the time of the current action sampling; Step 2.2: taking the action change point as the segmentation point, on the historical sequence data: from the previous period , N equally spaced points are sampled to form the sequence ; from the next period , N points are sampled to form the sequence ; wherein is the average duration of the action, and the corresponding actions are all ; Step 2.3: design a policy network, the input includes the working condition parameter sequence and the current action value, the network structure includes an LSTM layer for processing time sequence input and a fully connected layer FC for processing structured action input.

[0011] Preferably, the ore grinding throughput is expressed as:

[0012] , wherein, is the duration of the action , is the weight of the ore meeting the particle size requirement overflowed from the cyclone in the time, is the weight of the ore fed in the time; The estimation method is: ​​

[0013] wherein, is the cyclone flow rate, is the overflow concentration; the average liquid level in the pump tank The expression is:

[0014] wherein, represents the pump tank liquid level at time t.

[0015] Preferably, the step 3 comprises: Step 3.1: training the policy network based on the Actor-Critic framework; wherein, the Actor network objective function is:

[0016] wherein, is the number of samples, is the index of the sample, is the policy network, is the learnable parameter of the Actor network, is the state of reinforcement learning at time t, represents the state value function; wherein, the Critic network is trained by minimizing the TD error function, the expression is:

[0017] wherein, represents the index at a certain time in the future, is the reward function at a certain time in the future, is the trainable parameter of the Critic network; Step 3.2: inputting the sequence data , and the corresponding action into the policy network to obtain the ball mill feed rate, the ball mill water feed rate, the pump tank water supplement, and the cyclone pressure of the current working condition.

[0018] The grinding control parameter calculation system based on deep reinforcement learning provided by the present application comprises: Module M1: collecting point data of ball mill, pump tank, and cyclone in the grinding and beneficiation stage, and performing data cleaning and data preprocessing; Module M2: modeling the control problem of the four control quantities of ball mill feed rate, ball mill water feed rate, pump tank water supplement, and cyclone pressure by using deep reinforcement learning; Module M3: training a deep reinforcement learning model offline based on the collected data, the trained deep reinforcement learning model outputting the ball mill feed rate, the ball mill water feed rate, the pump pool water supplement and the cyclone pressure in the current working condition, so as to realize control of the ore grinding.

[0019] Preferably, the data cleaning and data preprocessing include: filling in the null values using the forward filling method, screening the running state data according to the ball mill running signal flag, additive merging the point data, and removing the transitional operation of the four control quantity set values; The additive merging is for a one-backup-one-use device, and the point data of the working device at the same time is merged. The removing of the transitional operation of the four control quantity set values includes: trimming each control quantity set value sequence with short duration one by one.

[0020] Preferably, the module M2 includes: Module M2.1: defining the state of reinforcement learning The point set associated with the intensity Top10 of the feed rate set value, the mill water feed rate set value, the pump pool water supplement set value and the cyclone pressure set value; Defining the action as four continuous variables: the feed rate set value, the mill water feed rate set value, the pump pool water supplement set value and the cyclone pressure set value; Defining the reward function , wherein is the ore grinding throughput, is the average liquid level in the pump pool, Defining the return function as , wherein is the action sequence, is the discount factor; is the number of operations; represents the time when the current action is sampled; Module M2.2: taking the action change point as the segmentation point, on the historical sequence data: from the previous period , sampling N points at equal intervals to form a sequence ; from the next period , sampling N points to form a sequence ; wherein is the average duration of the action, and the corresponding action is ; Module M2.3: designing a policy network, the input including the working condition parameter sequence and the current action value, the network structure including an LSTM layer for processing time sequence input and a fully connected layer FC for processing structured action input.

[0021] Preferably, the ore grinding throughput​ The expression is:

[0022] wherein, is the action the duration of time, is the weight of the ore meeting the particle size requirement overflowed from the cyclone within the time, is the weight of the ore given within the time; The estimation method is:

[0023] wherein, is the cyclone flow rate, is the overflow concentration; the average liquid level in the time The expression is:

[0024] wherein, represents the pump pool liquid level at time t.

[0025] Preferably, the module M3 comprises: Module M3.1: training the policy network based on the Actor-Critic framework; wherein, the Actor network objective function is:

[0026] wherein, is the number of samples, is the index of the sample, is the policy network, is the learnable parameter of the Actor network, is the state of reinforcement learning at time t, represents the state value function; wherein, the Critic network is trained by minimizing the TD error function, the expression is:

[0027] wherein, represents the index at a certain time in the future, is the reward function at a certain time in the future, is the trainable parameter of the Critic network; Module M3.2: sequence data , and the corresponding action An input strategy network is obtained to obtain the ball mill feed quantity, ball mill water quantity, pump pool water supplement and cyclone pressure of the current working condition.

[0028] Compared with the prior art, the present application has the following beneficial effects: (1) The present application solves the dynamic time-varying control problem of multi-parameter coupling in the grinding and separation stage by using deep reinforcement learning to model multi-parameter collaborative control (ball mill feed quantity, water quantity, pump pool water supplement and cyclone pressure). (2) The present application solves the problem of device redundancy and operation jitter interference in historical data by using a data preprocessing method of redundant device data merging and transition set value clipping, and realizes the extraction of high-quality training data. (3) The present application solves the reward feedback time lag problem of the grinding and separation process by using a reward function defined based on the grinding throughput and pump pool liquid level, and realizes the quantitative evaluation of the grinding efficiency; solves the time sequence correlation problem of the reinforcement learning training sample by using a sequence sampling method with the action change point as the segmentation point, and realizes the effective representation of the working condition before and after the action; solves the collaborative processing problem of time sequence working condition parameters and structured action input by combining the strategy network architecture of LSTM and full connection layer, and realizes the fusion analysis of multi-source heterogeneous data. BRIEF DESCRIPTION OF DRAWINGS

[0029] Other features, objects and advantages of the present application will become more apparent from the following detailed description of non-limiting embodiments, made with reference to the accompanying drawings: Figure 1 An example graph of the correlation strength analysis result between the control quantity and the working condition parameter is shown. Figure 2 A schematic diagram of the strategy network architecture involved in the present application is shown. DETAILED DESCRIPTION

[0030] The present application will be described in detail below with reference to specific embodiments. The following embodiments will help those skilled in the art to further understand the present application, but do not limit the present application in any form. It should be noted that those skilled in the art can make several changes and improvements without departing from the concept of the present application. These all belong to the protection scope of the present application.

[0031] Example 1 The present application provides a grinding control parameter calculation method based on deep reinforcement learning, comprising: Step 1: Collect key point data of ball mill, pump pool and cyclone in the grinding and separation stage, and perform data cleaning and data preprocessing; Step 2: Use deep reinforcement learning to model the ball mill feed quantity, ball mill water quantity, pump pool water supplement and cyclone pressure control problem; Step 3: offline training of a deep reinforcement learning model based on the collected data, the model outputting a ball mill feed amount, a ball mill water feed amount, a pump pool water supplement amount and a cyclone pressure for the current working condition, and the trained model being deployed to the field to realize control of the grinding process.

[0032] The step 1 comprises the following steps: Step 1.1: Collecting historical working condition parameters and corresponding set values of four control parameters by an operator. A total of 99 working condition parameters are collected, mainly including four powder ore bin levels, a cumulative value of a belt feed amount, a ball mill sound, a ball mill feed water flow, a pump pool liquid level, a cyclone feed flow, a slurry pump frequency and a slurry pump current. Corresponding control amounts are a feed amount set value, a mill water feed set value, a pump pool water supplement set value and a cyclone pressure set value.

[0033] Step 1.2: cleaning and preprocessing the historical data collected in the above step. First, the forward filling method is used to fill in the null values in the data; second, the data in the running state is selected according to the ball mill running signal flag, and the shutdown state data is filtered out; then, part of the point data is added and combined, which is mainly because the devices in the grinding and separation stage may be one standby and one in use, for example, there are multiple ball mills and slurry pumps, and only one device is working at the same time, and the point data of other devices is zero, so it needs to be combined; finally, the set values of the four control amounts are identified and the transitional operation is removed, which is mainly because the operator has a lot of trial behaviors in the operation process, resulting in a lot of transitional operation / shaking. Based on the feature that the transitional operation generally lasts for a short time, the sequences with a short duration of each control amount set value are cut off one by one to make the operation one step to the destination.

[0034] The step 2 comprises the following steps: Step 2.1: defining key concepts of reinforcement learning. The key concepts of reinforcement learning include state, action, reward and return, which need to be defined in combination with the grinding and separation process.

[0035] State : The state is a description of the working condition, which is a key observation basis for the operator to control. In order to analyze the dependence between the control amount and the working condition parameter, the decision tree analysis method is used to analyze the correlation strength between the single control amount and the working condition parameter, and the analysis result is shown in Figure 1 From Figure 1 it can be seen that the working condition parameters dependent on each control amount are repeated, and we finally set the union of the Top10 point positions associated with each control amount as the state vector.

[0036] Action The action is four continuous variables, namely the setting value of the feed ore, the setting value of the mill water, the setting value of the pump pool water, and the setting value of the cyclone pressure. As long as one or more of the above four setting values changes, the action changes.

[0037] Reward: The reward is mainly given to the action with higher grinding efficiency, and vice versa. Combined with the grinding and separation process, the present application proposes the concept of grinding throughput, which is used as the reward function. The grinding throughput is defined as follows:

[0038] wherein, is the duration of the action , is the weight of the ore overflowing from the cyclone within the time , is the weight of the ore given within the time . The data can be directly measured by a belt scale, and the estimation method is as follows:

[0039] wherein, is the cyclone flow, and the overflow concentration. In addition, the pump pool liquid level is another key variable that the operator pays attention to. Generally, a higher liquid level height means high load and saturated production, and vice versa. Because the grinding throughput is corrected by the liquid level, the average liquid level in the time is represented as follows:

[0040] wherein, represents the pump pool liquid level at time t; The corrected reward function is as follows:

[0041] Return: The return of reinforcement learning corresponds to the cumulative discounted reward, which is also the objective function to be optimized by reinforcement learning. The purpose is that the current action will make the maximum cumulative discounted reward from the current time to the end. The corresponding return function in this paper is as follows:

[0042] wherein, is the action sequence, is the discount factor; is the number of operations; represents the time when the current action is sampled.

[0043] Step 2.2: Organize the reinforcement learning sequence data. First, calculate the average duration of action changes on the historical operation sequence data, using... This represents the average duration of an action. It uses the point of action change as a dividing point, starting from before each action change... Equal interval sampling within a time period Points ,like Then fill in the blanks forward. There are 1 point, and each point uses its vector. describe, Let such a sequence be denoted as Similarly, after each change in action... It is also possible to sample a content containing A sequence of points . and The corresponding actions are all .

[0044] Step 2.3: Design the reinforcement learning network architecture. When adjusting parameters, operators not only focus on the operating conditions but also on the current control variable's value, primarily adjusting parameters based on these two pieces of information. Therefore, our designed policy network also uses these two pieces of information as input. The designed policy network architecture is as follows: Figure 2 As shown, LSTM is used to process time series inputs, and fully connected layers (FC) are used to process structured action inputs.

[0045] Step 3 includes the following steps: Step 3.1: Train the policy network proposed in Step 2 based on Actor-Critic. The objective of the Actor network is to maximize the following objective function:

[0046] in, For the sample size, For policy networks, These are the learnable parameters of the network. This is a state-value network, which serves as the Critic network in this invention. It is implemented using a three-layer fully connected architecture. The network achieves this by minimizing the following TD error function:

[0047] in, Indicates the first Index of each sample, Represents the state value function. An index representing a future point in time; Step 3.2: input the data processed in step 2.3 into the reinforcement learning network defined in step 3.1 for model training.

[0048] Embodiment 2 The application also provides a deep reinforcement learning-based grinding control parameter calculation system, which can be realized by performing the process steps of the deep reinforcement learning-based grinding control parameter calculation method, i.e., the deep reinforcement learning-based grinding control parameter calculation method can be understood by those skilled in the art as a preferred embodiment of the deep reinforcement learning-based grinding control parameter calculation system.

[0049] The system comprises: module M1: collecting point data of three devices of a ball mill, a pump pool and a cyclone in a grinding and separation stage, and performing data cleaning and data preprocessing; module M2: using deep reinforcement learning to model the control problem of four control quantities of the ball mill feed rate, the ball mill water feed rate, the pump pool water supplement and the cyclone pressure; module M3: training a deep reinforcement learning model based on the collected data offline, and outputting the ball mill feed rate, the ball mill water feed rate, the pump pool water supplement and the cyclone pressure of the current working condition of the trained deep reinforcement learning model to realize control of the grinding.

[0050] The data cleaning and data preprocessing comprise: using a forward filling method to fill in the null values, screening the running state data according to the ball mill running signal flag, performing additive merging on the point data, and removing the transitional operation of the four control quantity set values; The additive merging is for one standby one use device, and the point data of the working device at the same time is merged; The removing of the transitional operation of the four control quantity set values comprises: trimming each control quantity set value sequence with a short duration one by one.

[0051] The module M2 comprises: Module M2.1: defining the state of reinforcement learning The point set associated with the Top10 intensity of the ball mill feed rate set value, the ball mill water feed rate set value, the pump pool water supplement set value and the cyclone pressure set value; Defining the action as four continuous variables: the ball mill feed rate set value, the ball mill water feed rate set value, the pump pool water supplement set value and the cyclone pressure set value; Defining the reward function , wherein is the grinding throughput, is the average liquid level in the pump pool; Defining the return function as , wherein is the action sequence, is the return function of the action sequence, is the discount factor; is the number of operations; denotes the time of the current action sample; Module M2.2: Taking the action change point as the segmentation point, on the historical sequence data: from the previous period sampling N points at equal intervals to form a sequence ; from the next period sampling N points to form a sequence ; wherein is the average duration of the action, and the corresponding action is ; Module M2.3: Design a policy network, input including working condition parameter sequence and current action value, network structure contains LSTM layer processing time sequence input and fully connected layer FC processing structured action input.

[0052] Grinding throughput The expression is:

[0053] wherein, is the duration of the action , is the weight of the ore meeting the particle size requirement overflowing from the cyclone in , is the weight of the ore given in ; and The estimation method is:

[0054] wherein, is the cyclone flow rate, is the overflow concentration; the average liquid level in The expression is:

[0055] wherein, denotes the pump pool liquid level at t time.

[0056] The module M3 includes: Module M3.1: Train the policy network based on the Actor-Critic framework; wherein, the Actor network objective function is:

[0057] wherein, is the number of samples, ​​is an index of the sample, is a policy network, is a learnable parameter of the Actor network, is a is a state of the reinforcement learning at the moment, represents a state value function; The Critic network is trained by minimizing the TD error function, expressed as:

[0058] wherein, is an index of the future moment, is a reward function of the future moment, is a trainable parameter of the Critic network; Module M3.2: input the sequence data , and the corresponding action into the policy network to obtain the ball mill feed, the ball mill water, the pump pool water and the cyclone pressure of the current working condition.

[0059] Those skilled in the art know that, in addition to implementing the system, device and each module thereof provided by the present application in the form of pure computer readable program code, the same program can also be realized by logically programming the method steps in the form of logic gates, switches, application specific integrated circuits, programmable logic controllers and embedded microcontrollers. Therefore, the system, device and each module thereof provided by the present application can be considered as a hardware component, and the modules included therein for realizing various programs can also be considered as structures within the hardware component; the modules for realizing various functions can also be considered as both software programs for realizing methods and structures within the hardware component.

[0060] The specific embodiments of the present application are described above. It should be understood that the present application is not limited to the specific embodiments described above, and various changes or modifications can be made by those skilled in the art within the scope of the claims, which do not affect the essential content of the present application. In the case of no conflict, the embodiments of the present application and the features in the embodiments can be combined with each other arbitrarily.

Claims

1. A method for calculating grinding control parameters based on deep reinforcement learning, characterized in that, include: Step 1: Collect location data for the ball mill, pump tank, and hydrocyclone during the grinding and separation stage, and perform data cleaning and preprocessing. Step 2: Use deep reinforcement learning to model the control problem of four control variables: ball mill feed rate, ball mill water feed rate, pump pool makeup water, and hydrocyclone pressure; Step 3: Train a deep reinforcement learning model offline based on the collected data. The trained deep reinforcement learning model outputs the ball mill feed rate, ball mill water feed rate, pump pool water replenishment, and hydrocyclone pressure under the current operating conditions, thereby controlling the grinding process.

2. The grinding control parameter calculation method based on deep reinforcement learning according to claim 1, characterized in that, Data cleaning and preprocessing include: filling in missing values ​​using the forward filling method, filtering operating status data based on the ball mill operating signal flags, additively merging point data, and removing transient operations of the four control setpoints; The additive merging is applied to one standby device and one active device, merging the location data of devices working at the same time. The transitional operation of removing the four control value settings includes: trimming the sequence of control value settings with shorter durations one by one.

3. The grinding control parameter calculation method based on deep reinforcement learning according to claim 1, characterized in that, Step 2 includes: Step 2.1: Define the state for reinforcement learning The union of the top 10 points with the strongest correlation strength with the feed rate setting, mill water supply setting, pump pool makeup water setting, and hydrocyclone pressure setting; The action is defined as four continuous variables: feed rate setpoint, mill water feed rate setpoint, pump pool makeup water setpoint, and hydrocyclone pressure setpoint; Define reward function ,in For grinding throughput, for Average liquid level inside; Define the reward function as ,in For action sequences, Discount factor; Number of operations; Indicates the time of the current action sampling; Step 2.2: Using the point of action change as the dividing point, in the historical sequence data: from the previous time period A sequence is formed by sampling N points at equal intervals within the inner region. From the later period The sequence is formed by sampling N points within the inner sample. ;in The average duration of the action. and The corresponding actions are all ; Step 2.3: Design the strategy network. The input includes the sequence of operating parameters and the current action value. The network structure includes an LSTM layer for processing the time sequence input and a fully connected FC layer for processing the structured action input.

4. The grinding control parameter calculation method based on deep reinforcement learning according to claim 3, characterized in that, Grinding throughput The expression is: in, For action Duration for The weight of ore meeting the particle size requirements that overflows from the hydrocyclone within a given time period. for The weight of ore given within a given time period; The estimation method is as follows: in, For hydrocyclone flow rate, Overflow concentration; Average liquid level inside The expression is: in, This represents the liquid level in the pump tank at time t.

5. The grinding control parameter calculation method based on deep reinforcement learning according to claim 4, characterized in that, Step 3 includes: Step 3.1: Train the policy network based on the Actor-Critic framework; The objective function of the Actor network is: in, For the sample size, For the index of the sample, For policy networks, For the learnable parameters of the Actor network, for Constantly reinforce the state of learning. Represents the state value function; The Critic network is trained by minimizing the TD error function, expressed as: in, Indicates an index at a future point in time. Let the reward function be at some future moment. These are the trainable parameters of the Critic network; Step 3.2: Transfer the sequence data , and corresponding actions Input the strategy network to obtain the ball mill feed rate, ball mill water feed rate, pump pool makeup water, and hydrocyclone pressure under the current operating conditions.

6. A grinding control parameter calculation system based on deep reinforcement learning, characterized in that, include: Module M1: Collects location data for three types of equipment during the grinding and selection stage: ball mill, pump tank, and hydrocyclone, and performs data cleaning and preprocessing. Module M2: Deep reinforcement learning is used to model the control problem of four control variables: ball mill feed rate, ball mill water feed rate, pump pool makeup water, and hydrocyclone pressure. Module M3: Based on the collected data, an offline deep reinforcement learning model is trained. The trained deep reinforcement learning model outputs the ball mill feed rate, ball mill water feed rate, pump pool makeup water, and hydrocyclone pressure under the current operating conditions, thereby realizing the control of grinding.

7. The grinding control parameter calculation system based on deep reinforcement learning according to claim 6, characterized in that, Data cleaning and preprocessing include: filling in missing values ​​using the forward filling method, filtering operating status data based on the ball mill operating signal flags, additively merging point data, and removing transient operations of the four control setpoints; The additive merging is applied to one standby device and one active device, merging the location data of devices working at the same time. The transitional operation of removing the four control value settings includes: trimming the sequence of control value settings with shorter durations one by one.

8. The grinding control parameter calculation system based on deep reinforcement learning according to claim 6, characterized in that, The module M2 includes: Module M2.1: Defines the state for reinforcement learning The union of the top 10 points with the strongest correlation strength with the feed rate setting, mill water supply setting, pump pool makeup water setting, and hydrocyclone pressure setting; The action is defined as four continuous variables: feed rate setpoint, mill water feed rate setpoint, pump pool makeup water setpoint, and hydrocyclone pressure setpoint; Define reward function ,in For grinding throughput, for Average liquid level inside; Define the reward function as ,in For action sequences, Discount factor; Number of operations; Indicates the time of the current action sampling; Module M2.2: Using the point of action change as the dividing point, in historical sequence data: from the previous time period A sequence is formed by sampling N points at equal intervals within the inner region. From the later period The sequence is formed by sampling N points within the inner sample. ;in The average duration of the action. and The corresponding actions are all ; Module M2.3: Design strategy network. The input includes the sequence of operating parameters and the current action value. The network structure includes an LSTM layer for processing the time sequence input and a fully connected FC layer for processing the structured action input.

9. The grinding control parameter calculation system based on deep reinforcement learning according to claim 8, characterized in that, Grinding throughput The expression is: in, For action Duration for The weight of ore meeting the particle size requirements that overflows from the hydrocyclone within a given time period. for The weight of ore given within a given time period; The estimation method is as follows: in, For hydrocyclone flow rate, Overflow concentration; Average liquid level inside The expression is: in, This represents the liquid level in the pump tank at time t.

10. The grinding control parameter calculation system based on deep reinforcement learning according to claim 9, characterized in that, The module M3 includes: Module M3.1: Training the policy network based on the Actor-Critic framework; The objective function of the Actor network is: in, For the sample size, For the index of the sample, For policy networks, For the learnable parameters of the Actor network, for Constantly reinforce the state of learning. Represents the state value function; The Critic network is trained by minimizing the TD error function, expressed as: in, Indicates an index at a future point in time. Let the reward function be at some future moment. These are the trainable parameters of the Critic network; Module M3.2: Transfers sequence data , and corresponding actions Input the strategy network to obtain the ball mill feed rate, ball mill water feed rate, pump pool makeup water, and hydrocyclone pressure under the current operating conditions.

Images