Intelligent control method and system for fly ash activation treatment based on security constraint reinforcement learning

Abstract

This invention proposes an intelligent control method and system for fly ash activation treatment based on safety-constrained reinforcement learning. The method includes: acquiring the current activation process state within the current control cycle and the previous activation process state within the previous control cycle; calculating the difference between the current and previous activation process states to obtain a temperature increment index; calculating the current safe operating boundary based on a preset material safety temperature, the current activation process state, the temperature increment index, a preset state contraction coefficient, and a preset trend suppression coefficient; inputting the current activation process state and the current safe operating boundary into a preset reinforcement learning strategy model to obtain the current control strategy; and controlling the heating power of the activation reaction zone according to the current control strategy. This improves the activation efficiency of fly ash activation treatment and reduces unit energy consumption.

Description

Technical Field

[0001] This invention belongs to the field of new environmentally friendly building materials, and in particular relates to an intelligent control method and system for the activation treatment of fly ash based on safety constraint reinforcement learning. Background Technology

[0002] Fly ash is a significant solid waste generated in large quantities during industrial processes such as coal-fired power plants. Its main components include a certain proportion of glassy phase and silica-alumina active components. After appropriate activation treatment, it can be utilized in cement admixtures, road materials, and solid waste resource utilization. Therefore, the treatment process for enhancing fly ash activation has always been a key focus of industry. The fly ash activation process is essentially a continuous process with significant time-varying characteristics and thermal inertia. The temperature and its changing trend in the reaction zone directly affect the activation effect, energy consumption level, and operational safety. In related technologies, temperature curves are set based on experience, or alarms and limit controls are implemented around a fixed upper temperature limit to achieve fly ash activation. While these methods can maintain basic operation, they often struggle to simultaneously achieve activation efficiency, unit energy consumption, and equipment safety when fly ash sources fluctuate, equipment thermal inertia is significant, and the activation process gradually approaches the high-temperature range. Therefore, how to improve the activation efficiency and reduce unit energy consumption during fly ash activation treatment has become an urgent technical problem to be solved. Summary of the Invention

[0003] The purpose of this invention is to design an intelligent control method and system for fly ash activation treatment based on safety constraint reinforcement learning, which can improve the activation efficiency and reduce the unit energy consumption during fly ash activation treatment.

[0004] To achieve the above objectives, a first aspect of the present invention provides an intelligent control method for fly ash activation treatment based on security constraint reinforcement learning, the method comprising:

[0005] Obtain the current activation process status within the current control cycle and the previous activation process status within the previous control cycle;

[0006] The temperature increment index is obtained by calculating the difference between the current activation process state and the previous activation process state.

[0007] The current safe operating boundary is calculated based on the preset material safety temperature, the current activation process state, the temperature increment index, the preset state shrinkage coefficient, and the preset trend suppression coefficient.

[0008] The current activation process state and the current safe operation boundary are input into a preset reinforcement learning strategy model to obtain the current control strategy;

[0009] The heating power of the activation reaction zone is controlled according to the current control strategy.

[0010] Further, controlling the heating power of the activation reaction region according to the current control strategy includes:

[0011] If the current control strategy is greater than or equal to the first preset value, then the heating power of the activation reaction region is increased;

[0012] If the current control strategy is less than or equal to the second preset value, then the heating power of the activation reaction zone is reduced; wherein the second preset value is less than the first preset value;

[0013] If the current control strategy is less than the first preset value and greater than the second preset value, then the current heating power of the activation reaction region is maintained.

[0014] Further, controlling the heating power of the activation reaction region according to the current control strategy includes:

[0015] Obtain the heating power from the previous control cycle;

[0016] The target heating power is obtained by multiplying the preset power mapping coefficient by the current control strategy and adding the previous heating power.

[0017] The heating power of the activation reaction region is controlled according to the target heating power.

[0018] Furthermore, after multiplying the preset power mapping coefficient by the current control strategy and adding the previous heating power to obtain the target heating power, the method further includes:

[0019] Obtain the rated maximum heating power and the rated minimum heating power;

[0020] The first heating power is obtained by selecting the maximum heating power from the rated minimum heating power and the target heating power;

[0021] The minimum heating power is selected from the first heating power and the rated maximum heating power to obtain the updated target heating power.

[0022] Further, the calculation of the current safe operating boundary based on the preset material safety temperature, the current activation process state, the temperature increment index, the preset state shrinkage coefficient, and the preset trend suppression coefficient includes:

[0023] The difference between the material's safe temperature and the current activation process state is calculated, and then multiplied by the state shrinkage coefficient to obtain the first data;

[0024] A one-sided trend term is constructed based on the temperature increment index, and then multiplied by the trend suppression coefficient to obtain the second data;

[0025] The current safe operating boundary is obtained by subtracting the first data and the second data from the material's safe temperature.

[0026] Furthermore, the reinforcement learning policy model includes a normalization module and a policy output module. The step of inputting the current activation process state and the current safe operating boundary into the preset reinforcement learning policy model to obtain the current control policy includes:

[0027] The current activation process status and the current safe operating boundary are input into the normalization module to obtain the normalized status index;

[0028] The normalized state index is input to the strategy output module to obtain the current control strategy.

[0029] Further, obtaining the current activation process state within the current control cycle includes:

[0030] Acquire multiple temperature sample values ​​within the current control cycle;

[0031] Anomaly removal is performed on the multiple temperature sample values ​​to obtain updated multiple temperature sample values;

[0032] The current activation process state is obtained by averaging multiple temperature sampling values.

[0033] In a second aspect, the present invention provides an intelligent control system for fly ash activation treatment based on security constraint reinforcement learning, the system comprising:

[0034] The acquisition unit is used to acquire the current activation process status within the current control cycle and the previous activation process status within the previous control cycle.

[0035] The first calculation unit is used to calculate the difference between the current activation process state and the previous activation process state to obtain the temperature increment index.

[0036] The second calculation unit is used to calculate the current safe operating boundary based on the preset material safety temperature, the current activation process state, the temperature increment index, the preset state shrinkage coefficient, and the preset trend suppression coefficient.

[0037] The input unit is used to input the current activation process state and the current safe operation boundary into a preset reinforcement learning strategy model to obtain the current control strategy;

[0038] The control unit is used to control the heating power of the activation reaction zone according to the current control strategy.

[0039] In a third aspect of the invention, an electronic device is provided, the electronic device including a memory and a processor, the memory storing a computer program, the processor executing the computer program to implement the method described in the first aspect above.

[0040] In a fourth aspect of the invention, a computer-readable storage medium is provided, the computer-readable storage medium storing a computer program that, when executed by a processor, implements the method described in the first aspect.

[0041] The beneficial technical effects of the present invention are at least as follows:

[0042] To address the aforementioned issues, this invention provides an intelligent control method and system for fly ash activation treatment based on safety constraint reinforcement learning. Its core lies in forming the current activation process state, which characterizes the overall heating level of the current control cycle, based on continuous temperature acquisition results of the reaction zone. This avoids misjudgments caused by directly driving control with single instantaneous measurements. Furthermore, by combining the material safety temperature determined during equipment commissioning and the state change trend between adjacent control cycles, the current safe operating boundary is dynamically determined. This ensures that the safety constraint is no longer a fixed upper limit, but an operating boundary that can contract synchronously with the current reaction position and heating trend. Subsequently, the current process state and the current safe operating boundary are input into the safety constraint reinforcement learning strategy generation process, enabling the control strategy to proactively suppress the risk of high-temperature approach from the outset, rather than simply adding a limit after the strategy output. Finally, the generated control strategy is converted into an incremental adjustment command for the target heating power, and execution constraints are applied in conjunction with the upper and lower limits of the equipment power, ensuring that the strategy results can smoothly and continuously act on the activation treatment equipment. Through the above methods, the present invention organically connects process state construction, safety boundary generation, reinforcement learning decision-making, and power execution control, so that safety constraints, optimization decisions, and equipment execution in the fly ash activation process form a unified link, thereby achieving coordinated control of activation effect, energy consumption level, and operational stability, which is more in line with the actual working conditions of continuous operation, significant thermal inertia, and sensitivity to high temperature zone in fly ash activation treatment. Attached Figure Description

[0043] The present invention will be further described with reference to the accompanying drawings, but the embodiments in the drawings do not constitute any limitation on the present invention. For those skilled in the art, other drawings can be obtained based on the following drawings without creative effort.

[0044] Figure 1This is a flowchart of an intelligent control method for fly ash activation treatment based on security constraint reinforcement learning, provided in an embodiment of this application.

[0045] Figure 2 This is a schematic diagram of the structure of an intelligent control system for fly ash activation treatment based on security constraint reinforcement learning, provided in an embodiment of this application. Detailed Implementation

[0046] Embodiments of the present invention are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and should not be construed as limiting the present invention.

[0047] Please refer to Figure 1 , Figure 1 This is a flowchart illustrating an intelligent control method for fly ash activation treatment based on security constraint reinforcement learning, as provided in an embodiment of this application. Figure 1 The method may include, but is not limited to, steps S101 to S105.

[0048] Step S101: Obtain the current activation process status within the current control cycle and the previous activation process status within the previous control cycle;

[0049] Step S102: Calculate the difference between the current activation process state and the previous activation process state to obtain the temperature increment index;

[0050] Step S103: Calculate the current safe operating boundary based on the preset material safety temperature, current activation process state, temperature increment index, preset state shrinkage coefficient, and preset trend inhibition coefficient.

[0051] Step S104: Input the current activation process state and the current safe operation boundary into the preset reinforcement learning strategy model to obtain the current control strategy;

[0052] Step S105: Control the heating power of the activation reaction zone according to the current control strategy.

[0053] In step S101 of some embodiments, the objective is to establish an activation process state that accurately represents the current activation level of fly ash. In fly ash activation treatment, temperature is the most direct, stable, and easily obtainable process quantity online. Regardless of whether electric heating, microwave heating, or other thermal activation methods are used, the first manifestation of fly ash transitioning from a low-activity state to a high-activity state is the continuous change in the temperature of the reaction zone. Therefore, focusing on the core information of the activation zone temperature, a complete set of operations is completed, including "sensor placement—continuous acquisition—anomaly rejection—state formation," ultimately outputting the current activation process state. .

[0054] Specifically, the temperature sampling values ​​of the reaction zone in the fly ash activation treatment equipment are directly read. Fly ash activation treatment equipment typically includes a reaction chamber, heating unit, temperature acquisition unit, and control unit. The temperature data in this step is not manually input, but is acquired in real time by temperature sensors installed within the activation reaction zone. Sensors are preferentially placed in the area where the material is most concentratedly heated, rather than simply on the furnace wall or equipment casing. This is because furnace wall temperature more easily reflects the equipment's temperature rise but cannot accurately reflect the activation state of the fly ash material itself. For fixed-bed equipment, the temperature sensor is inserted into the center of the material layer; for continuous conveying equipment, the temperature sensor is placed in the stable heating zone along the main material path; for microwave activation equipment, the temperature sensor is placed at the material's residence position corresponding to the microwave action zone, and a heat-insulating protection structure is used to prevent metal exposure that could cause measurement distortion.

[0055] The data collected by the temperature sensor first enters the equipment's data acquisition module, and then is read by the control unit at fixed time intervals. Within a control cycle, instead of directly using a single instantaneous temperature sample value, several temperature sample values ​​are read continuously. This is because during the fly ash activation process, local particle agitation, short-term heat exchange, or surface disturbances often occur, causing a single sample value to suddenly be too high or too low. If a single-point temperature sample value is directly used as the current activation process state, subsequent calculations of safe operating boundaries and control strategies will be skewed by short-term fluctuations, leading to situations where power reduction is not needed but overheating is mistakenly identified, or where continued heating should be considered but the temperature is mistakenly considered sufficient. Therefore, within a control cycle, the control unit continuously receives... Each temperature sample value is analyzed, and a simple anomaly removal process is performed first. The anomaly removal adopts rules that are easy to implement on-site. For example, when a temperature sample value shows a significant isolated jump compared to its adjacent temperature sample values, while the overall adjacent temperature sample values ​​remain stable, this point is regarded as an instantaneous disturbance and is not included in the state calculation for this cycle.

[0056] For example, if the temperature samples collected sequentially within a certain control cycle are 812, 813, 846, 814, and 813, where 846 is significantly higher than the preceding and following values ​​and does not constitute a continuous temperature increase trend, then this value will not be used as a representative value for the current state in the calculation. After anomaly removal processing, the average value of the remaining temperature samples is calculated to obtain the current activation process state. As shown in the following formula:

[0057] ;

[0058] in, This indicates the current activation process status, used to represent the overall heating status of fly ash within the current control cycle; Indicates the number of times within the current control cycle. The temperature sampling value obtained in this acquisition is collected by a temperature sensor arranged in the fly ash activation reaction zone and sent to the control unit via the data acquisition module. This indicates the number of valid temperature sample values ​​used in the calculation within the current control cycle. This number is determined by the control unit according to a preset sampling rhythm. This represents the count sequence number of the effective temperature sample value within the current control cycle, and its value ranges from 1 to... .

[0059] It should be noted that the above formula uses this calculation method not for the purpose of making a conventional average, but to reflect the current activation process state. This more closely approximates the actual thermal reaction level of current fly ash materials. For example, if five consecutive effective temperature sampling values ​​of 818, 819, 820, 820, and 821 are obtained within a control cycle, then the current cycle... These five values ​​collectively determine the state; compared to taking only the last sample, this method yields a more stable state and better reflects the actual characteristic of "continuous rather than instantaneous temperature changes" in fly ash activation treatment. For example, during microwave activation, local fly ash particles may sometimes experience short-term temperature spikes due to instantaneous energy concentration. Without this step, the control system might mistakenly believe the entire material has entered a high-temperature risk zone. However, through continuous sampling and anomaly removal, It can better reflect the overall activation temperature level of the material.

[0060] This step does not simply involve averaging, but rather, considering the characteristics of fly ash activation treatment—namely, significant local thermal disturbances, easily distorted instantaneous temperatures, and the necessity for control to rely on steady-state quantities—it involves a unified design of sensor placement, sampling cycle organization, and state formation rules. This approach yields… This approach avoids introducing additional data types and directly driving subsequent control with single temperature samples, thus providing a stable and reliable state input for determining the current safe operating boundary in the next step. Current activation process status. This is not a single instantaneous measurement, but a state result that represents the overall heating level of fly ash within this control cycle. The next step will be directly based on the current activation process state. Determine the current safe operating boundaries.

[0061] In some embodiments, steps S102 to S103, the current activation process state output in the previous step The overall heating level of fly ash within this control cycle has already been given. This step further determines the upper limit of the safe temperature that can be reached in the current control cycle, i.e., the current safe operating boundary. The object of treatment here is not a fixed upper limit, but a safe operating boundary that dynamically contracts with the current reaction state. The reason for this design is that what truly needs to be avoided in fly ash activation treatment is not simply "reaching a certain high temperature," but rather "the control system continuing to rapidly increase the temperature when it is already close to the material's structural risk zone." Therefore, this step considers the current activation process state... Moving forward, this translates into the current safe operating boundary that can be directly used by subsequent control strategies. This allows the next step of generating a reinforcement learning control policy to no longer face an open temperature space, but to make decisions within a safe range consistent with the current reaction state.

[0062] Specifically, during the installation and commissioning phase, the equipment will undergo a standard activation test to pre-determine a safe material temperature. This temperature is not given verbally based on experience, but is a parameter value determined by combining changes in activity and particle structure after activating multiple batches of fly ash samples under progressively increasing temperatures. In practice, fly ash samples from the same source are treated multiple times according to a fixed heating program. At each treatment temperature, it is observed whether the particles show obvious sintering, agglomeration, or an inflection point of decreased activity. When the sample begins to show stable signs of structural densification, the corresponding temperature is recorded as the safe temperature for this equipment and this type of fly ash combination. This parameter is written into the control system parameter table as a fixed input value during operation. Current activation process status. The data is derived from the continuous sampling and calculation results of the temperature sensor in the reaction zone during the current control cycle. The upper limit of material safety, which is jointly determined by equipment and materials. Reflecting the actual reaction state at the current moment, the combination of the two can form a boundary calculation basis that is closer to the actual process site.

[0063] In the activation process, the risk of temperature fluctuations depends not only on the current temperature itself, but also on whether the temperature approaches the high-temperature zone gradually or rapidly. Therefore, this step further incorporates the activation process state from the previous control cycle. This value is saved by the control system at the end of the previous control cycle. The temperature increment index within the current control cycle is obtained using the difference between the current activation process state and the previous activation process state. This formula originates from the first-order backward difference concept in numerical analysis. Under the condition of a fixed control period, it uses the state difference between two adjacent periods to represent the current temperature rise trend. Its expression is:

[0064] ;

[0065] in, This indicator represents the temperature increment within the current control cycle and is used to characterize the direction and magnitude of change in the current activation process state relative to the previous cycle. Indicates the current activation process status; This indicates the activation process state saved from the previous control cycle; this value is read from the control system's history buffer. Because... and They represent the same type of state variable, therefore the difference between the two is... Dimensions and Maintaining consistency allows for direct participation in subsequent boundary formula calculations. The direct reason for adopting this form is that the control period is fixed in the system, and the boundary calculation is concerned with whether the temperature continues to rise within the current period and by how much, rather than introducing additional time variables to increase control complexity.

[0066] Furthermore, to obtain Then, the current safe operating boundary The safety margin is determined by the material's safe operating temperature, the current activation process state, and the temperature increment index. This formula is based on the concept of "safety margin" in thermal control, meaning the upper safety limit should gradually shrink as the system approaches the danger zone. On this basis, a unilateral suppression term that only affects the temperature rise trend is superimposed, making the boundary more sensitive to rapid temperature increases. Specifically, the difference between the material's safe operating temperature and the current activation process state is calculated and multiplied by a state shrinkage coefficient to obtain the first data; a unilateral trend term is constructed based on the temperature increment index and multiplied by a trend suppression coefficient to obtain the second data; the current safe operating boundary is obtained by subtracting the first and second data from the material's safe operating temperature. Its expression is:

[0067] ;

[0068] in, This represents the current safe operating boundary, which is the highest safe temperature that can be referenced when generating the reinforcement learning control strategy in the next step. This indicates the material safety temperature determined and stored in the parameter table during the equipment commissioning phase; Indicates the current activation process status; This represents the shrinkage coefficient, used to control the magnitude of boundary shrinkage when the current activation process state approaches the material's safe temperature; This represents the trend suppression coefficient, used to control the additional impact of warming trends on boundary contraction; This represents a unilateral trend term, which is only included in the calculation when the temperature increment is positive; when the temperature remains constant or decreases, this term is zero. This formula can be understood in two layers. The first layer is... It stems from a linear compression of the safety margin: the current activation process state. At lower levels, The current safety operating boundary remains high; when near At that time, the current safe operating boundary automatically shrinks downward, thus making the control system more cautious in the high-temperature range. The second layer is... It is equivalent to adding a one-sided regularization term for the heating trend in the boundary calculation: additional contraction is only triggered when the system continues to heat up, thus prioritizing the identification of the state of "approaching the high temperature zone and still heating up" as a high-risk state.

[0069] Taking a set of implementation calculations as an example, suppose a certain activation device determines the material safety temperature for the target fly ash. The previous activation process state saved in the previous control cycle Current activation process status The control system's preset state contraction coefficient Trend suppression coefficient First, substituting into the temperature increment formula, we get... Then, substituting it into the boundary formula, we get:

[0070] ;

[0071] When continuing the calculation, , , ,therefore This result indicates that, although the theoretical safe material temperature for the system is 900°C in the current control cycle, the current activation process has already reached 854°C and is still showing a significant upward trend. Therefore, the current safe operating boundary has been dynamically narrowed to 875.6°C. In the next step, when generating the reinforcement learning control strategy, this boundary will be used as the highest acceptable temperature reference, and the control system will prioritize power regulation schemes that keep the temperature below this boundary. To provide another comparative example, if the current activation process state is still... However, the previous activation process state ,but At this point, the unilateral trend term is set to zero, and the boundary becomes... This indicates that when the system has entered a cooling or stabilizing phase, the boundary will not shrink further due to the trend term, thus avoiding excessive conservatism and preserving the necessary space for activation.

[0072] Through the above processing, this step changes the current activation process state. Further transformed into an executable current security operation boundary .in, The introduction of this feature ensures that the boundary changes with the current degree of reaction. The introduction of this ensures that the boundary is forward-looking regarding warming trends. This solidifies the test results of materials and equipment into the basis of operational constraints. This is how... Instead of isolated, fixed limits, these are dynamic safety boundaries that reflect both the "current temperature position" and the "current temperature trend." The next step, when calling the reinforcement learning policy model, will directly use the values ​​obtained in this step. As a safety constraint input, the control decisions are made both around the activation objective and always within the temperature range acceptable to the fly ash material structure.

[0073] In step S104 of some embodiments, the current activation process state is obtained. and current safe operating boundaries Next, this step generates a control strategy based on these two quantities. .in Reflects the current temperature level of the reaction zone. This indicates the highest safe temperature the system can achieve under the current activation process conditions, therefore and The relationship between these two inputs directly determines whether the current control cycle allows for continued temperature increase and the magnitude of that increase. This step utilizes a reinforcement learning policy model to generate a control policy based on these two inputs. However, the core expression of the policy is not a simple empirical rule, but rather derived from the normalized state representation and constraint control concepts in control theory. This allows the policy to reflect the distance between the current temperature and the safety boundary, and to automatically enhance its suppression capability when the temperature approaches the safety boundary.

[0074] The reinforcement learning strategy model includes a normalization module and a strategy output module. The current activation process state and the current safe operating boundary are input into the normalization module to obtain a normalized state index; the normalized state index is then input into the strategy output module to obtain the current control strategy. Specifically, the normalization module... and A dimensionless state index is constructed. This index is derived from the relative error normalization method commonly used in control theory, which describes the system position using the proportional relationship between the current activation process state and the current safe operating boundary. The purpose of normalization is to make the strategy model independent of absolute temperature values, focusing only on "how much space is left between the current activation process state and the current safe operating boundary". This normalized state index is calculated using the following formula:

[0075] ;

[0076] in, This represents a normalized state index used to describe the proportion of the current temperature within the safety boundary; Indicates the current activation process status; This indicates the current safe operating boundary. Because... and When physical quantities of the same type have the same dimensions, the ratio is... It is a dimensionless quantity. When A temperature close to 0 indicates that the current temperature is far below the safety boundary, and the system still has considerable room for temperature increase; when When the temperature approaches 1, it indicates that the temperature is nearing the safe boundary and the rising trend needs to be gradually suppressed.

[0077] After obtaining the normalized state index Next, the current control strategy The generation is based on the policy output module in reinforcement learning. Classical reinforcement learning policies typically output action values ​​through a policy network. This step, building upon this, incorporates the barrier function concept from constraint control to structurally modify the policy output form, enabling the policy to automatically enhance its suppression effect when the current activation state approaches the current safe operating boundary. The specific expression is as follows:

[0078] ;

[0079] in, This indicates the current control strategy, used to determine the direction of heating power adjustment in the next control cycle; Indicates a normalized state index; This represents the temperature rise driving coefficient, which is obtained from the reinforcement learning training process and is used to control the temperature rise tendency when the temperature is far from the boundary. This represents the linear suppression coefficient, used to control the power reduction intensity after the temperature enters the warning range; This represents the boundary approach suppression coefficient, used to enhance the suppression effect when the temperature (current activation process state) approaches the safety boundary (current safe operating boundary); This indicates the safety warning ratio, used to determine the point at which the temperature begins to enter the safety warning range; This represents the stability coefficient, used to avoid excessively large calculation results when the denominator is close to zero.

[0080] The strategy expression consists of three parts. The first part... Derived from the linear action output structure commonly found in reinforcement learning policy gradients, its function is to drive the system to continue heating up when the temperature is far below the safety boundary. When smaller, The larger the value, the more the strategy output tends to increase power. Part Two Derived from the unilateral penalty function in constraint control, its function is to apply a penalty when the temperature exceeds the safety warning ratio. Then, the temperature rise was gradually suppressed. Part Three Derived from the idea of ​​barrier functions, this term is... The value increases rapidly when it approaches 1, thus causing the strategy to exhibit a significant inhibitory effect when approaching the safety boundary. This nonlinear term makes the strategy more sensitive to high-temperature approach states, thus better meeting the actual need to rapidly reduce power when the temperature approaches the sintering range during the fly ash activation process.

[0081] To illustrate the strategy calculation process, a specific calculation example is given. Assume that within a certain control cycle, the current activation process state... Current safe operating boundary Then the normalized state index is:

[0082] ;

[0083] Let the safety early warning ratio in the control system be . The parameters obtained from reinforcement learning training are , , Stability coefficient is taken as First calculate ,get . Recalculate ,get Since the value is positive, the suppression term takes effect. The contribution of each part is then calculated separately: Part 1 Part Two The denominator of the third part is Therefore, the third part is Substituting the three parts into the policy expression yields the current control policy. A negative strategy value indicates that the system should reduce heating power to prevent the temperature from approaching the safety boundary. Conversely, if the current temperature is lower in another control cycle, for example... The current safe operating boundary is still ,but Because this value is lower than the safety warning ratio. The suppression term is not included in the calculation. In this case, the strategy is mainly determined by the first part, resulting in... A positive value indicates that the system allows for a moderate increase in heating power to continue driving the activation reaction.

[0084] As can be seen from the above calculation process, this step will determine the current activation process state. With respect to the current safe operating boundary By embedding these elements into the reinforcement learning policy expression, the policy can not only adjust according to the current temperature but also automatically change its control behavior based on how close the temperature is to the safety boundary. This results in a control policy... In the next step, it is converted into a specific heating power adjustment command, thereby realizing safe and intelligent control of the fly ash activation process.

[0085] In step S105 of some embodiments, the current control strategy output by the above steps The current activation process status has been updated. With respect to the current safe operating boundary The relationship between them is compressed into an executable adjustment direction, so this step no longer re-evaluates whether the temperature is safe, but instead... This is directly converted into a heating power command that the device can execute. Specifically, if the current control strategy is greater than or equal to a first preset value, the heating power of the activation reaction zone is increased; if the current control strategy is less than or equal to a second preset value, the heating power of the activation reaction zone is decreased; wherein, the second preset value is less than the first preset value; if the current control strategy is less than the first preset value but greater than the second preset value, the current heating power of the activation reaction zone is maintained. In one example, the first preset value can be set to 0.1, and the second preset value can be set to -0.1. When, it indicates that power increase is permitted. When this happens, it indicates that power needs to be reduced. When the power level is low, it indicates that the system will maintain operation near the current power level.

[0086] In some embodiments, an "incremental adjustment-boundary projection" execution method is adopted: first, the power change of the current control cycle relative to the previous control cycle is calculated based on the strategy value, and then the change is applied to the actual heating unit according to the rated power range of the equipment. The reason for this approach is that the fly ash activation process has significant thermal inertia. If the power is directly switched to a completely new absolute value in each control cycle, it is easy to cause temperature fluctuations and local thermal shocks. Incremental execution is more in line with the "continuous, small-step, and reversible" control law of industrial thermal processes, and it is also easier to connect with reinforcement learning strategies.

[0087] First, read the heating power from the previous control cycle that was executed in the control system's buffer. At the same time, read the current control policy. .in, The device controller automatically stores the data in the register area at the end of the previous control cycle. The regulation strength and direction are obtained through online inference from the reinforcement learning policy model. To convert the dimensionless policy output into an actual power change, this step introduces a power mapping coefficient. This coefficient is determined during the equipment commissioning phase through no-load tests and material-loaded heating tests. Its function is to map a "one-step change in strategy" to the actual adjustment range of the equipment power that should be adjusted. In this way, a stable numerical interface is established between the reinforcement learning strategy layer and the equipment execution layer.

[0088] The target heating power for the current control cycle is derived from the classic incremental control formula, which states that "the new control quantity equals the previous control quantity plus the current incremental term." In this step, the incremental term is not given by human experience, but rather by the current control strategy. Power mapping coefficient Multiplying these together, the target heating power can be calculated using the following formula:

[0089] ;

[0090] in, This represents the target heating power calculated in the current control cycle; This indicates the heating power that was executed in the previous control cycle, which is read from the historical register of the power control module; This represents the power mapping coefficient, which is determined during the equipment commissioning phase and written into the control parameter table. This represents the current control strategy. The formula originates from the discrete incremental control concept in automatic control. The modification in this step is to replace the fixed increment in the traditional controller with the output of a safety-constrained reinforcement learning strategy. This ensures that the power increment is no longer determined by manual rules, but rather by the combined effects of the "current state—safety boundary—strategy model." Because... It is a dimensionless quantity. This represents the power mapping scale, therefore and Since they have the same dimensions, the dimensions of both sides of the above equation are consistent. After calculating the target heating power, the heating power of the activation reaction region is controlled according to the target heating power.

[0091] In one example, since all actual heating units have a permissible operating range, for electric heating equipment, this range is determined by the rated output capacity of the thyristor power control module or frequency converter; for microwave activation equipment, this range is determined by the rated power and cooling capacity of the microwave source. Therefore, further adjustments to the target heating power are necessary. Perform interval projection processing to constrain the power to the upper and lower limits allowed by the equipment. This processing originates from the interval projection method in mathematics and can also be understood as a saturation element in engineering control. Specifically, obtain the actual rated maximum and minimum heating power of the heating unit; select the maximum heating power from the rated minimum and target heating power to obtain the first heating power; and select the minimum heating power from the first heating power and the rated maximum to obtain the updated target heating power. The expression is as follows:

[0092] ;

[0093] in, This indicates the updated target heating power; Rated maximum heating power indicates the maximum allowable heating power of the equipment, which is given in the equipment rating parameter table; Rated minimum heating power, representing the minimum stable heating power allowed by the equipment, is determined by low-power operation tests of the equipment; The target heating power before the update. The logic of this formula is: first use... Ensure that the target heating power before the update is not lower than the lower limit of stable equipment operation, and then use... Ensure it does not exceed the device's allowed limit. Because... 、; and Since they are all of the same type of power, the dimensions on both sides of the equation are consistent. This step can strictly limit the output of the reinforcement learning policy to the physical capabilities of the equipment, so that the policy execution retains the characteristics of intelligent decision-making while conforming to the actual constraints of industrial equipment.

[0094] To illustrate the specific calculation process, a set of examples are given. Let the heating power executed in the previous control cycle be... Current control strategy The power mapping coefficient determined during equipment commissioning The target heating power is calculated as follows:

[0095] ;

[0096] Rated minimum heating power Rated maximum heating power ,but , Therefore, the updated target heating power After the controller writes this value into the power regulation module, the electric heating device will reduce its output power to the corresponding level through voltage or current regulation, while the microwave device will reduce its power to that value by adjusting the microwave source duty cycle or the output stage drive level. If the value calculated in another control cycle... If the target heating power before the update is changed, it may rise to a higher level; if substituted, the target heating power after the update may rise to a higher level. Exceed It will automatically limit the output to the rated maximum heating power to avoid overloading the equipment.

[0097] In actual operation, the execution order of the above scheme is clear. At the beginning of the control cycle, the controller first receives the current control strategy from the previous step. Then read the previous heating power from the local cache. Calculate the target heating power before the update. Then the updated heating power was calculated. The controller puts After the heating drive module is written, the heating unit operates at this power within the current control cycle, and the temperature of the reaction zone subsequently changes. Temperature sampling in the next cycle then returns to step S101 to form a new activation process state. This completes the closed-loop implementation from "strategy result" to "physical execution": the current control strategy... The target heating power is converted into an updated target heating power with boundary constraints. This ultimately acts on the fly ash activation treatment equipment, making the entire solution a truly executable, continuous control process. Updated target heating power. It inherits the decision-making direction of the safety constraint reinforcement learning strategy, and maintains consistency with the physical capabilities of the equipment through incremental execution and interval projection. Therefore, it can form a stable, executable heating power regulation control that is consistent with the safety boundary in the fly ash activation treatment.

[0098] Steps S101 to S105, as illustrated in this embodiment, involve acquiring the current activation process state within the current control cycle and the previous activation process state within the previous control cycle. A temperature increment index is obtained by calculating the difference between the current and previous activation process states. The current safe operating boundary is calculated based on a preset material safety temperature, the current activation process state, the temperature increment index, a preset state contraction coefficient, and a preset trend suppression coefficient. The current activation process state and the current safe operating boundary are input into a preset reinforcement learning strategy model to obtain the current control strategy. The heating power of the activation reaction zone is controlled according to the current control strategy. This improves the activation efficiency of fly ash activation treatment and reduces unit energy consumption.

[0099] Please see Figure 2 This application also provides an intelligent control system for fly ash activation treatment based on safety constraint reinforcement learning, which can realize the above-mentioned intelligent control method for fly ash activation treatment based on safety constraint reinforcement learning. The system includes:

[0100] The acquisition unit 201 is used to acquire the current activation process status in the current control cycle and the previous activation process status in the previous control cycle;

[0101] The first calculation unit 202 is used to calculate the difference between the current activation process state and the previous activation process state to obtain the temperature increment index.

[0102] The second calculation unit 203 is used to calculate the current safe operating boundary based on the preset material safety temperature, the current activation process state, the temperature increment index, the preset state shrinkage coefficient, and the preset trend suppression coefficient.

[0103] The input unit 204 is used to input the current activation process state and the current safe operation boundary into the preset reinforcement learning strategy model to obtain the current control strategy;

[0104] Control unit 205 is used to control the heating power of the activation reaction zone according to the current control strategy.

[0105] The specific implementation of the intelligent control system for fly ash activation treatment based on safety constraint reinforcement learning is basically the same as the specific embodiment of the intelligent control method for fly ash activation treatment based on safety constraint reinforcement learning described above, and will not be repeated here.

[0106] The above description represents the preferred embodiments of the present invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of the present invention, and these improvements and modifications are also considered to be within the scope of protection of the present invention.

Claims

1. A smart control method for fly ash activation treatment based on safety constraint reinforcement learning, characterized in that, The method includes: Obtain the current activation process status within the current control cycle and the previous activation process status within the previous control cycle; The temperature increment index is obtained by calculating the difference between the current activation process state and the previous activation process state. The current safe operating boundary is calculated based on the preset material safety temperature, the current activation process state, the temperature increment index, the preset state shrinkage coefficient, and the preset trend suppression coefficient. The current activation process state and the current safe operation boundary are input into a preset reinforcement learning strategy model to obtain the current control strategy; The heating power of the activation reaction zone is controlled according to the current control strategy.

2. The intelligent control method for fly ash activation treatment based on safety constraint reinforcement learning according to claim 1, characterized in that, The step of controlling the heating power of the activation reaction region according to the current control strategy includes: If the current control strategy is greater than or equal to the first preset value, then the heating power of the activation reaction region is increased; If the current control strategy is less than or equal to the second preset value, then the heating power of the activation reaction zone is reduced; wherein the second preset value is less than the first preset value; If the current control strategy is less than the first preset value and greater than the second preset value, then the current heating power of the activation reaction region is maintained.

3. The intelligent control method for fly ash activation treatment based on safety constraint reinforcement learning according to claim 1, characterized in that, The step of controlling the heating power of the activation reaction region according to the current control strategy includes: Obtain the heating power from the previous control cycle; The target heating power is obtained by multiplying the preset power mapping coefficient by the current control strategy and adding the previous heating power. The heating power of the activation reaction region is controlled according to the target heating power.

4. The intelligent control method for fly ash activation treatment based on safety constraint reinforcement learning according to claim 3, characterized in that, After multiplying the preset power mapping coefficient by the current control strategy and adding the previous heating power to obtain the target heating power, the method further includes: Obtain the rated maximum heating power and the rated minimum heating power; The first heating power is obtained by selecting the maximum heating power from the rated minimum heating power and the target heating power; The minimum heating power is selected from the first heating power and the rated maximum heating power to obtain the updated target heating power.

5. The intelligent control method for fly ash activation treatment based on safety constraint reinforcement learning according to claim 1, characterized in that, The calculation of the current safe operating boundary based on the preset material safety temperature, the current activation process state, the temperature increment index, the preset state shrinkage coefficient, and the preset trend suppression coefficient includes: The difference between the material's safe temperature and the current activation process state is calculated, and then multiplied by the state shrinkage coefficient to obtain the first data; A one-sided trend term is constructed based on the temperature increment index, and then multiplied by the trend suppression coefficient to obtain the second data; The current safe operating boundary is obtained by subtracting the first data and the second data from the material's safe temperature.

6. The intelligent control method for fly ash activation treatment based on safety constraint reinforcement learning according to claim 1, characterized in that, The reinforcement learning strategy model includes a normalization module and a strategy output module. The step of inputting the current activation process state and the current safe operating boundary into the preset reinforcement learning strategy model to obtain the current control strategy includes: The current activation process status and the current safe operating boundary are input into the normalization module to obtain the normalized status index; The normalized state index is input to the strategy output module to obtain the current control strategy.

7. The intelligent control method for fly ash activation treatment based on safety constraint reinforcement learning according to claim 1, characterized in that, The step of obtaining the current activation process status within the current control cycle includes: Acquire multiple temperature sample values ​​within the current control cycle; Anomaly removal is performed on the multiple temperature sample values ​​to obtain updated multiple temperature sample values; The current activation process state is obtained by averaging multiple temperature sampling values.

8. An intelligent control system for fly ash activation treatment based on safety constraint reinforcement learning, characterized in that, The system includes: The acquisition unit is used to acquire the current activation process status within the current control cycle and the previous activation process status within the previous control cycle. The first calculation unit is used to calculate the difference between the current activation process state and the previous activation process state to obtain the temperature increment index. The second calculation unit is used to calculate the current safe operating boundary based on the preset material safety temperature, the current activation process state, the temperature increment index, the preset state shrinkage coefficient, and the preset trend suppression coefficient. The input unit is used to input the current activation process state and the current safe operation boundary into a preset reinforcement learning strategy model to obtain the current control strategy; The control unit is used to control the heating power of the activation reaction zone according to the current control strategy.

9. An electronic device, characterized in that, The electronic device includes a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, it implements the intelligent control method for fly ash activation treatment based on security constraint reinforcement learning as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the intelligent control method for fly ash activation treatment based on security constraint reinforcement learning as described in any one of claims 1 to 7.

Images