A method for self-adaptive optimization of graphene water treatment process parameters based on reinforcement learning

Abstract

This invention discloses an adaptive optimization method for graphene water treatment process parameters based on reinforcement learning, belonging to the field of intelligent water treatment control technology. The method involves acquiring operating signals and action records from the previous control cycle, mapping them into an enhanced process state table containing information on pollution accumulation, water quality disturbance, and effluent deviation; inputting this table into a strategy network to generate an original action set, mapping it to equipment setpoints, and performing action masking based on equipment physical boundaries and process constraints to obtain executable process actions; evaluating the action effects according to a comprehensive benefit rule based on feedback data from the next control cycle to generate long-term rewards; and converting the executable process actions into a set of field control instructions and sending them to the executing equipment. This improves the continuity, stability, and adaptive operation capability of the graphene water treatment process.

Description

Technical Field

[0001] This invention belongs to the field of intelligent control technology for water treatment, specifically relating to an adaptive optimization method for graphene water treatment process parameters based on reinforcement learning. Background Technology

[0002] The parameters of graphene water treatment processes exhibit nonlinearity, time lag, and sensitivity to influent disturbances. Existing technologies mainly employ fixed empirical settings, local feedback regulation, or static threshold triggering. Furthermore, material performance optimization and operational parameter adjustment are disconnected, and device control relies primarily on static thresholds and discrete mode switching. This can easily lead to problems such as short-term water production, long-term contamination, and difficulty in coordinating cleaning frequency and energy consumption, resulting in excessive effluent, frequent backwashing, increased manual verification costs, and high operation and maintenance costs. Summary of the Invention

[0003] This invention provides an adaptive optimization method for graphene water treatment process parameters based on reinforcement learning, comprising: Step S1, acquiring the operating signals and action records of the previous control cycle of the graphene water treatment device, and mapping the operating signals and the action records of the previous control cycle into an enhanced process state table containing pollution accumulation information, water quality disturbance information, and effluent deviation information; Step S2, inputting the enhanced process state table into a strategy network to generate an original action set, mapping the original action set to equipment setpoints, and performing action masking on the equipment setpoints according to the equipment physical boundaries and process constraints to obtain executable process actions; Step S3, evaluating the action effect according to the comprehensive benefit rule that simultaneously constrains water production effect, effluent quality, operating energy consumption, backwash cost, and action stability based on the feedback data of the next control cycle corresponding to the executable process action, and forming a long-term reward; Step S4, converting the executable process action into a field control instruction set for the graphene water treatment device and sending it to the execution device, while writing the executed action value into the action field corresponding to the next control cycle.

[0004] Preferably, mapping the operating signals and the action records of the previous control cycle into an enhanced process status table containing pollution accumulation information, water quality disturbance information, and effluent deviation information includes: acquiring influent flow rate, product flow rate, pressure difference across the graphene treatment layer, target pollutant concentration in the effluent, water temperature, influent conductivity, electrolytic cell operating current, and the influent pump frequency setpoint, circulation pump frequency setpoint, reagent dosing acceleration rate setpoint, backwash execution status, and backwash duration of the previous control cycle; based on the influent flow rate, product flow rate, pressure difference across the graphene treatment layer, target pollutant concentration in the effluent, water temperature, influent conductivity, and electrolytic cell operating current, the table is prepared. The process status table is constructed using the operating current of the decontamination tank, the setpoints of the influent pump frequency, the circulating pump frequency, the reagent dosing acceleration rate, the backwash execution status, and the backwash duration of the previous control cycle. Based on the process status table, the pollution accumulation information and the water quality disturbance information are generated, and the effluent quality deviation and product water flow deviation are generated as the effluent deviation information to obtain the enhanced process status table. The effluent quality deviation is the signed difference between the target effluent pollutant concentration and the preset effluent concentration target value, and the product water flow deviation is the signed difference between the preset product water target value and the product water flow rate.

[0005] Preferably, generating the contamination accumulation information includes: obtaining the cross-graphene treatment layer pressure difference, permeate flow rate, and continuous operating time since the most recent backwash in the process status table; calculating the degree of increase of the cross-graphene treatment layer pressure difference relative to the preset clean reference pressure difference, the degree of decrease of the permeate flow rate relative to the preset clean reference permeate flow rate, and the degree of accumulation of the continuous operating time relative to the preset clean reference operating time; and weighting and aggregating the degree of increase, the degree of decrease, and the degree of accumulation with the corresponding preset contamination weight coefficients to obtain the contamination accumulation information reflecting the combined effects of clogging, scaling, and flux reduction.

[0006] Preferably, generating the water quality disturbance information includes: obtaining the influent conductivity, influent flow rate, and water temperature from the process status table, and obtaining the moving average values ​​of the influent conductivity, influent flow rate, and water temperature within a preset pre-time window; calculating the degree of deviation of the influent conductivity, influent flow rate, and water temperature from their respective moving average values; and weighting and aggregating the degree of deviation with corresponding preset disturbance weight coefficients to obtain the water quality disturbance information reflecting the short-term fluctuation intensity of the incoming water conditions.

[0007] Preferably, obtaining the enhanced process state table includes: generating an adsorption site attenuation characterization quantity based on the effluent target pollutant concentration and reagent dosing acceleration setpoint in the process state table; generating a conductive interface reaction intensity characterization quantity based on the electrolytic cell operating current and circulating pump frequency setpoint in the process state table; grouping the fields in the process state table according to the filtration resistance related group, adsorption saturation related group, electrochemical reaction related group, and backwash recovery related group; and combining the adsorption site attenuation characterization quantity and the conductive interface reaction intensity characterization quantity as operating condition labels before participating in the operating condition feature combination before inputting into the strategy network to obtain the enhanced process state table.

[0008] Preferably, mapping the original action set to equipment settings includes: inputting the enhanced process status table into the strategy network to obtain the original action set corresponding to the influent pump, circulation pump, chemical dosing, and backwash triggering, respectively; incrementally mapping the adjustment amounts corresponding to the influent pump, circulation pump, and chemical dosing in the original action set according to the preset action step size of the corresponding actuator to obtain candidate settings for each actuator; firstly, applying the maximum change constraint between adjacent control cycles to the candidate settings, and then applying the upper and lower limit constraints of the equipment to obtain the influent pump frequency setting, circulation pump frequency setting, and chemical dosing acceleration rate setting; and coupling and mapping the adjustment amount corresponding to backwash triggering in the original action set with the pollution accumulation information and the water quality disturbance information to obtain the backwash triggering result.

[0009] Preferably, the adjustment amount corresponding to the backwash trigger in the original action set is coupled and mapped with the pollution accumulation information and the water quality disturbance information to obtain the backwash trigger result, including: generating a backwash start command when the backwash is not currently in the backwash lockout period based on the comparison relationship between the backwash trigger result and the preset trigger probability threshold; maintaining the backwash state after backwashing is started until the preset minimum backwash duration; determining whether to end the backwash based on the comparison result between the cross-graphene treatment layer pressure difference and the preset recovery pressure difference threshold, and the comparison result between the permeate flow rate and the preset recovery permeate threshold; and forcibly ending the backwash and generating a maintenance alarm when the preset maximum backwash duration is reached but the end condition is not met.

[0010] Preferably, the executable process actions are obtained by shielding the equipment setpoints based on the equipment's physical boundaries and process constraints, including: generating a shielding flag for further increasing the influent pump frequency based on the comparison result of the cross-graphene treatment layer pressure difference and a preset high pressure difference threshold; generating a shielding flag for reducing the circulation pump frequency and reducing the reagent dosage based on the comparison result of the effluent target pollutant concentration and a preset excessive concentration threshold; generating a linkage shielding flag for increasing the influent pump frequency and shutting down the backwash action based on the comparison result of the product water flow rate and a preset low flux threshold, and the comparison result of the pollution accumulation information and a preset high pollution threshold; organizing the shielding flags into shielding flag vectors corresponding to the influent pump, circulation pump, reagent dosage, and backwash actions respectively, and determining the final shielding result according to a preset priority order when multiple shielding conditions are met simultaneously; and performing action corrections on the equipment setpoints based on the final shielding result, including retaining, truncating to the execution value of the previous control cycle, and replacing with a safe preset value, to obtain the executable process actions.

[0011] Preferably, the evaluation of the action effect is based on a comprehensive benefit rule that simultaneously constrains water production effect, effluent quality, operating energy consumption, backwash cost, and action stability, and forms a long-term return including: calculating a single-cycle comprehensive benefit based on the feedback data of the next control cycle, with the water production flow rate maintenance level as a positive benefit and the increase of pollution accumulation information, the effluent target pollutant concentration exceeding the preset effluent concentration target value, the increase of total system power, the occurrence of backwash events, and the increase of action change amplitude in adjacent control cycles as negative constraints; generating a pressure difference safety boundary penalty term based on the comparison result of the cross-graphene treatment layer pressure difference in the feedback data of the next control cycle with the preset pressure difference safety threshold, and generating a recovery window cost term based on the pressure difference recovery and water production recovery process after backwashing. The changes in influent pump frequency, circulating pump frequency, chemical dosage, and backwash status are weighted and summed according to their respective single-cycle allowable variation ranges, and coupled with the water quality disturbance information to generate an anti-interference stability constraint term. A long-term return is formed based on the single-cycle comprehensive benefit, differential pressure safety boundary penalty term, recovery window cost term, and anti-interference stability constraint term. Offline training samples are constructed based on historical operation logs, semi-physical test data, and digital twin simulation samples. During the offline training phase, the strategy network and evaluation network are trained offline based on the long-term return. The digital twin simulation samples include at least the state transition results of the filter resistance growth sub-model, the effluent concentration response sub-model, and the backwash recovery sub-model.

[0012] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and running on the processor, wherein the processor executes the computer program to implement the steps of the above method.

[0013] This invention generates an original action set by inputting an enhanced process status table into a strategy network, maps it to equipment setpoints, and performs action masking to obtain executable process actions. It evaluates the action effects based on feedback data from the next control cycle corresponding to the executable process actions according to a comprehensive benefit rule, generating long-term returns. By converting the executable process actions into a set of field control instructions and sending them to the execution equipment, it improves the continuity, stability, and adaptive operation capability of the graphene water treatment process, thereby increasing treatment efficiency, reducing energy consumption, saving reagents, and lowering operation and maintenance costs. Attached Figure Description

[0014] Figure 1 The present invention provides an overall flowchart of an adaptive optimization method for graphene water treatment process parameters based on reinforcement learning.

[0015] Figure 2 This invention provides an environmental structure diagram applicable to an adaptive optimization method for graphene water treatment process parameters based on reinforcement learning.

[0016] Figure 3 This is a flowchart of the process state mapping unit in an embodiment of the present invention.

[0017] Figure 4 This is a flowchart of the action generation, physical mapping, and action masking unit in an embodiment of the present invention.

[0018] Figure 5 This is a flowchart of the comprehensive benefit assessment and long-term return formation unit in an embodiment of the present invention.

[0019] Figure 6 This is a flowchart of the field control command output unit in an embodiment of the present invention. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will now be described in further detail with reference to the accompanying drawings.

[0021] Example 1: Overall Flowchart of an Adaptive Optimization Method for Graphene Water Treatment Process Parameters Based on Reinforcement Learning

[0022] like Figures 1 to 2As shown in the figure, the adaptive optimization method for graphene water treatment process parameters based on reinforcement learning provided in this embodiment is executed by a water treatment control device. The water treatment control device includes a processor, a memory, a sensing interface, and an execution interface. The sensing interface is connected to an influent flow meter, a product water flow meter, a differential pressure transmitter, an online effluent analyzer, a conductivity meter, a thermometer, and an electrolysis power supply feedback interface. The execution interface is connected to an influent pump frequency converter, a circulating pump frequency converter, a reagent metering pump driver, a backwash valve assembly, and an electrolysis power supply. The processor reads the process control program from the memory, receives operating data according to the control cycle, and outputs control commands.

[0023] In this embodiment, the controlled object is a graphene filtration-electrochemical synergistic purification production line consisting of a graphene composite filtration component, a circulating shearing unit, and an electrochemical oxidation unit connected in series, with a designed water production capacity of [missing information]. The industrial control computer serves as the control device, equipped with a quad-core 2.5GHz CPU and 16GB of memory. The typical sensor sampling period is 10 seconds, based on the fact that the changes in flow rate, pressure difference, and effluent concentration in this production line are significantly slower than second-level control; therefore, a 10-second sampling period balances signal integrity and acquisition load. The typical control period is 30 seconds, which can be determined through on-site pump frequency step response tests. In practical engineering applications, it is typically set to 20 to 60 seconds based on the device's inertia. In this embodiment, each control cycle aggregates three consecutive sampling results before executing an action decision.

[0024] This embodiment includes the following steps:

[0025] like Figure 3 As shown, in step S1, the operating signal of the graphene water treatment device and the action record of the previous control cycle are obtained, and the operating signal and the action record of the previous control cycle are mapped into an enhanced process status table containing pollution accumulation information, water quality disturbance information and effluent deviation information.

[0026] In this embodiment, the operating signals include influent flow rate, product water flow rate, pressure difference across the graphene treatment layer, target pollutant concentration in the effluent, water temperature, influent conductivity, and electrolyzer operating current. The previous control cycle's action record includes the influent pump frequency setpoint, circulation pump frequency setpoint, reagent dosing acceleration setpoint, backwash execution status, and backwash duration. The execution entity is the processor, and the output is an enhanced process status table, which serves as input to the subsequent strategy network.

[0027] Step S101: Construct the process status table

[0028] Acquire the following parameters: influent flow rate, product flow rate, pressure difference across the graphene treatment layer, target pollutant concentration in the effluent, water temperature, influent conductivity, electrolytic cell operating current, and the influent pump frequency setpoint, circulation pump frequency setpoint, reagent dosing acceleration rate setpoint, backwash execution status, and backwash duration from the previous control cycle. Based on these parameters, construct a process status table.

[0029] In this embodiment, the processor in the first... Each control cycle reads and reassembles the process status table to obtain:

[0030] ,

[0031] The above formula corresponds to the state construction process in this step. Input data comes from the influent flow meter, product water flow meter, differential pressure transmitter, online effluent analyzer, thermometer, conductivity meter, electrolysis power supply feedback channel, and the execution record of the previous cycle. This formula addresses the problem of scattered original signals that are difficult to directly support reinforcement learning decisions. The output is a process state table with a unified field order. Execution occurs at the beginning of each control cycle; the processor is the execution entity. In the formula, This refers to the influent flow rate, in units of... ; The water production flow rate is expressed in units of... ; The pressure difference across the graphene treatment layer is expressed in units of... ; The target pollutant concentration in the effluent, in units of ; Water temperature, unit: ; The conductivity of the influent is expressed in units of... ; This refers to the operating current of the electrolytic cell, measured in units of... ; , , These are the setpoints for the inlet pump frequency, circulation pump frequency, and chemical dosing rate in the previous control cycle, respectively. This indicates the backwash execution status of the previous control cycle; This represents the duration of backwashing since the previous control cycle, in units of... .

[0032] In practical engineering applications, the processor further groups the fields according to the process mechanism: the filtration resistance-related group includes the pressure difference across the graphene treatment layer, the permeate flow rate, and the previous cycle's influent pump frequency setting; the adsorption saturation-related group includes the effluent target pollutant concentration and the previous cycle's reagent dosing acceleration rate setting; the electrochemical reaction-related group includes the electrolyzer operating current, the previous cycle's circulation pump frequency setting, and the water temperature; and the backwash recovery-related group includes the backwash execution status and backwash duration. These groupings constitute the internal field organization method of the aforementioned process status table.

[0033] To distinguish between short-term communication jitter and continuous signal loss, this step introduces a preset field threshold, which represents the upper limit of the number of missing state fields allowed within a control cycle. This threshold is typically determined offline based on field sensor reliability statistics. In this embodiment, the preset field threshold typically has three fields, and this value can be calibrated based on the distribution of single-cycle signal interruptions in historical logs. When the number of missing fields does not exceed this threshold, the processor uses substitute values ​​to fill in the missing fields; when it exceeds the threshold, subsequent steps pause new action calculations and maintain the executable process actions from the previous cycle for one control cycle.

[0034] In this embodiment, if the conductivity sensor momentarily misses a measurement, the effective value of the previous cycle is used as a substitute; if two consecutive control cycles are missing a measurement, the initial calibration conductivity of the device is written. If backwashing has not yet started, the backwashing execution status is written as off, and the backwashing duration is written as zero. To suppress abnormal sampling spikes, the aforementioned preset lead time window is defined as the historical observation period used to calculate the median benchmark and moving average. This parameter can be obtained through offline testing that takes into account both water quality fluctuation frequency and control lag requirements. In this embodiment, the typical value of the preset lead time window is 10 minutes, but in actual engineering applications, it can usually be configured between 5 minutes and 20 minutes. If a single sampled value deviates from the median value of the most recent time window by more than a preset proportion, it is determined to be an anomaly.

[0035] The preset ratio is used to measure the upper limit of the deviation of the instantaneous sampled value from the recent stable level, and can be obtained offline through statistical analysis of historical anomalies; its typical value is... When a single sampled value exceeds this proportion, it is replaced by the average of the other two samples in the same control cycle; if all three samples are abnormal, the valid aggregated value from the previous cycle is used. The above-mentioned abnormal replacement logic achieves reliable preprocessing of the aforementioned operating signals.

[0036] Within one control cycle of this embodiment, the aggregated result of three samplings is as follows:

[0037] , , , , , , The action value in the previous cycle was: , , , , .

[0038] This forms the basis for the subsequent enhanced process status table.

[0039] Step S102: Generate pollution accumulation information and water quality disturbance information

[0040] Obtain the cross-graphene treatment layer pressure difference, permeate flow rate, and continuous operating time since the most recent backwash from the process status table; calculate the degree of increase of the cross-graphene treatment layer pressure difference relative to the preset clean reference pressure difference, the degree of decrease of the permeate flow rate relative to the preset clean reference permeate flow rate, and the cumulative degree of the continuous operating time relative to the preset clean reference operating time; weight and aggregate the degree of increase, decrease, and cumulative degree with the corresponding preset contamination weight coefficients to obtain the contamination accumulation information reflecting the combined effects of clogging, scaling, and flux reduction.

[0041] Specifically, the process involves obtaining the influent conductivity, influent flow rate, and water temperature from the process status table, and obtaining the moving average values ​​of the influent conductivity, influent flow rate, and water temperature within a preset pre-set time window; calculating the deviation of the influent conductivity, influent flow rate, and water temperature from their respective moving average values; and weighting and aggregating the deviation values ​​with corresponding preset disturbance weighting coefficients to obtain the water quality disturbance information reflecting the short-term fluctuation intensity of the incoming water conditions.

[0042] In this embodiment, in order to obtain contamination accumulation information that can characterize the combined effects of clogging, scaling, and flux decline, the processor calculates a contamination accumulation factor:

[0043] ,

[0044] This formula corresponds to the pollution accumulation information generation and processing in this step. The input data comes from the differential pressure, permeate flow rate, and continuous operating time after backwashing in the current process status table; it addresses the problem that relying solely on a single differential pressure threshold cannot reflect the pollution evolution stage; the output is the pollution accumulation factor. The state enhancement phase of each control cycle is invoked by the processor. Where, , , Preset pollution weighting coefficients; Preset cleaning baseline pressure difference, unit: ; The upper limit of the allowable differential pressure, in units of ; To prevent smooth terms with a denominator of zero; The preset clean baseline permeable flow rate is expressed in units of... ; For smoothing the flow term; The continuous operating time since the most recent backwashing ended, in units of ; The preset cleaning reference runtime, in units of ; This represents the smoothing amount for the duration term. All of the above smoothing terms are numerically stable parameters and can be preset through program safety design. In this embodiment, typical values ​​are... .

[0045] In this embodiment, the preset pollution weighting coefficient is defined as the allocation coefficient of the contribution of pressure rise, flux decline, and continuous operation accumulation to the degree of pollution. This parameter can be obtained through regression analysis of historical operating samples and cleaning records; its typical value is... , , The preset cleaning baseline pressure difference is used to represent the baseline pressure difference under stable cleaning conditions after backwashing, and can be obtained by calibration of the average pressure difference during continuous stable operation after backwashing; in this embodiment, its typical value is 18 kPa, and in actual engineering applications, it can usually be configured in the range of 10 kPa to 30 kPa depending on the component specifications. The upper limit of the allowable operating pressure difference represents the safe pressure difference boundary that the filter component can operate for a long time, and can be determined based on the equipment supplier parameters and pressure resistance tests; in this embodiment, its typical value is 70 kPa, and it can usually be configured in the range of 50 kPa to 90 kPa. The preset cleaning baseline permeable flow rate is used to represent the target flux baseline under cleaning conditions, and can be obtained through steady-state operation tests after backwashing; in this embodiment, its typical value is The preset cleaning reference runtime is used to measure the time scale of contamination accumulation and can be calibrated by statistically analyzing the time taken from multiple batches of operation to the point where cleaning is required; its typical value is 360 minutes.

[0046] To further distinguish between adsorption site occupancy and changes in conductive interface reaction intensity, the processor generates adsorption site attenuation characterization and conductive interface reaction intensity characterization:

[0047] ;

[0048] .

[0049] The two formulas above correspond to the steps involved in the generation of graphene's mechanistic characteristics. The former takes the target pollutant concentration in the effluent and the previous cycle's reagent dosing acceleration rate as input, characterizing the adsorption site occupancy trend; the latter takes the electrolytic cell operating current and the previous cycle's circulating pump frequency as input, characterizing the change in the conductive interface reaction intensity. The outputs are respectively... and The timing of the call is synchronized with the pollution accumulation factor. In the formula, , These are the adsorption-related weighting coefficients; The preset target concentration value for effluent is given in units of... ; This is the upper limit for the pharmaceutical metering pump, in units of ; , These are the electrochemical-related weighting coefficients; The reference operating current is expressed in units of 1000 ppm. ; The reference circulating pump frequency, in units of ; , , , This is a smoothing term. In this embodiment, the typical values ​​for the adsorption-related weighting coefficients are 0.7 and 0.3, and the typical values ​​for the electrochemical-related weighting coefficients are 0.6 and 0.4, all obtained through on-site step tests and regression calibration of historical effluent quality.

[0050] The processor then calculates the water quality disturbance factor:

[0051] ,

[0052] This formula corresponds to the water quality disturbance information generation step. The inputs are the current conductivity, influent flow rate, and water temperature, as well as their moving averages within a preset pre-defined time window; it addresses the problem that short-term influent fluctuations are difficult to reflect using traditional single-point sampling; the output is the water quality disturbance factor. ; updated by the processor in each control cycle. Where, , , Preset perturbation weighting coefficients; , , These are the conductivity, influent flow rate, and moving average water temperature within the preceding time window, respectively. , , This is a smoothing term.

[0053] In this embodiment, the preset disturbance weighting coefficient is used to measure the relative impact of conductivity fluctuations, influent flow rate fluctuations, and temperature fluctuations on the intensity of influent disturbance. It can be determined offline through historical operating samples and effluent fluctuation sensitivity analysis; its typical value is... , , When there is only one sample in the moving average window, that sample is directly used as the corresponding average value; when the upper limit of the pressure difference is the same as the cleaning baseline pressure difference under extreme calibration conditions, the processor replaces the denominator with a smoothing term; when the current current is fixed due to maintenance of the electrolysis unit, only the circulation pump-related terms in the conductivity interface reaction intensity characterization quantity are retained.

[0054] In this embodiment, the moving average value within the preceding time window during the control period is: , , ; Continuous runtime is Based on the above calculation relationships, we obtain... , , , This result indicates that while the area has not yet entered a severe clogging stage, a moderate level of contamination has already accumulated, and the occupancy trend of adsorption sites is beginning to increase. , , and The calculation process constitutes the generation process of the aforementioned pollution accumulation information, water quality disturbance information, and operating condition labels.

[0055] Step S103: Generate an enhanced process status table

[0056] Based on the process status table, the pollution accumulation information and water quality disturbance information are generated, and the effluent quality deviation and product flow rate deviation are generated as the effluent deviation information to obtain the enhanced process status table. The effluent quality deviation is the signed difference between the target effluent pollutant concentration and the preset effluent concentration target value, and the product flow rate deviation is the signed difference between the preset product flow rate target value and the product flow rate. Based on the effluent target pollutant concentration and reagent dosing acceleration rate setting value in the process status table, an adsorption site attenuation characterization quantity is generated; based on the electrolytic cell operating current and circulating pump frequency setting value in the process status table, a conductive interface reaction intensity characterization quantity is generated; the fields in the process status table are grouped according to the filtration resistance related group, adsorption saturation related group, electrochemical reaction related group, and backwash recovery related group, and the adsorption site attenuation characterization quantity and the conductive interface reaction intensity characterization quantity are used as operating condition labels to participate in the operating condition feature combination before input to the strategy network to obtain the enhanced process status table.

[0057] In this embodiment, the processor concatenates the process state table with deviation, contamination characterization, and disturbance characterization to form an enhanced process state table: This formula corresponds to the enhanced state construction process in this step. The inputs are the aforementioned process state table, pollution accumulation factor, water quality disturbance factor, and deviation field; this addresses the problem that the original state cannot directly represent the degree of deviation from the control target; the output is the enhanced process state table input to the policy network; this is called by the processor before action generation. In the formula, The effluent quality deviation is defined as the signed difference between the target pollutant concentration in the effluent and the preset target value for effluent concentration. The deviation in water production flow rate is defined as the signed difference between the preset target value for water production and the current water production flow rate.

[0058] In this embodiment, the preset effluent concentration target value is defined as the baseline for effluent quality control, which can be determined based on reclaimed water standards or discharge standards. This parameter can be obtained through joint calibration using process design targets and on-site compliance rate statistics; its typical value is 22 mg / L. The preset permeable water target value is used to measure the deviation of the current production capacity from the designed stable production capacity, and can be determined based on the designed production capacity and safety margin; its typical value is... During this control cycle, the following is obtained: , .

[0059] During the initial power-on initialization phase, if there are no actions from the previous cycle, the processor reads the initial settings from memory and fills each action field. If the number of missing fields exceeds the aforementioned preset field threshold, the calculation of new actions is paused, and the executable process actions from the previous cycle are maintained for one control cycle to wait for signal recovery. After this processing, operational data from different sources are mapped into a unified engineering field space, enabling the subsequent policy network to make decisions under the same state table structure. The combination of the aforementioned process state table, pollution accumulation information, water quality disturbance information, effluent quality deviation, and product water flow deviation constitutes the aforementioned enhanced process state table.

[0060] like Figure 4 As shown, in step S2, the enhanced process status table is input into the strategy network to generate an original action set. The original action set is mapped to the equipment setting value, and the action masking is performed on the equipment setting value according to the equipment physical boundary and process constraints to obtain executable process actions.

[0061] In this embodiment, the processor remains the execution entity. The input to this step is an enhanced process status table, and the output is executable process actions. These executable process actions correspond to the states of the influent pump, circulation pump, chemical dosing, and backwashing, respectively, and are used for the generation of subsequent on-site control commands.

[0062] Step S201: Generate the original action set

[0063] The enhanced process status table is input into the strategy network to obtain the original action set corresponding to the influent pump, circulation pump, chemical dosing and backwash triggering, respectively.

[0064] In this embodiment, the processor inputs the enhanced process state table into the policy network to obtain the original action set: This formula corresponds to the original motion generation steps. The input is the enhanced process state table; it is used to solve the problem of unified output of coordinated adjustment actions among multiple actuators; the output is a four-dimensional original motion set; it is called by the processor after the state construction is completed in each control cycle. In the formula, For the inlet pump regulating factor, For the circulating pump adjustment factor, Add regulatory factors to the drug. This is the backwash trigger adjustment factor. All four outputs are compressed using hyperbolic tangent activation. .

[0065] In this embodiment, the input dimension of the strategy network is consistent with the number of fields in the enhanced process state table, typically 17 dimensions; the hidden layer adopts a three-layer fully connected structure, with typical node configurations of 128, 128, and 64; the output dimension is 4. To improve the coupling degree of the process mechanism, three feature extraction branches are set before the first hidden layer, divided according to process units, corresponding to the filter resistance related group, the adsorption saturation related group, and the electrochemical reaction related group, respectively. Each branch completes the local feature extraction of the variables in its group before merging; the backwash trigger adjustment factor also participates in the determination with the backwash recovery related group, so that the backwash action has an independent constraint channel. The evaluation network receives the input after splicing the enhanced process state table and the action set, and is used to evaluate the comprehensive operational value of the current action combination in several subsequent control cycles.

[0066] During this control cycle in this embodiment, the policy network outputs the original action set as follows: , , , When the number of missing fields in the enhanced process status table exceeds the aforementioned preset field threshold, the processor pauses the calculation of new actions and maintains the previous cycle's actions for one control cycle. In this embodiment, the specific value of the preset field threshold is three fields. The above four outputs constitute the aforementioned original action set.

[0067] Step S202: Map the original action set to the device settings and generate the backwash trigger result.

[0068] According to the preset action step size of the corresponding actuator, the adjustment amounts corresponding to the inlet pump, circulation pump and chemical dosing in the original action set are incrementally mapped to obtain candidate set values ​​for each actuator; the candidate set values ​​are first constrained by the maximum change amount in adjacent control cycles, and then constrained by the upper and lower limits of the equipment to obtain the inlet pump frequency set value, circulation pump frequency set value and chemical dosing acceleration rate set value; the adjustment amount corresponding to the backwash trigger in the original action set is coupled and mapped with the pollution accumulation information and the water quality disturbance information to obtain the backwash trigger result.

[0069] Specifically, based on the comparison between the backwash triggering result and the preset triggering probability threshold, a backwash start command is generated when the backwash lockout period is not currently in progress; after backwashing is started, the backwashing state is maintained until the preset minimum backwashing duration is reached; after the preset minimum backwashing duration is reached, a determination is made on whether to end the backwashing based on the comparison between the cross-graphene treatment layer pressure difference and the preset recovery pressure difference threshold, as well as the comparison between the permeate flow rate and the preset recovery permeate threshold; if the preset maximum backwashing duration is reached but the end condition is not met, the backwashing is forcibly ended and a maintenance alarm is generated.

[0070] In this embodiment, the three continuous actions of the inlet pump, circulation pump, and chemical dosing are first incrementally mapped according to their respective step sizes:

[0071] ;

[0072] ;

[0073] .

[0074] The three formulas above correspond to the physical mapping steps from continuous actions to device settings. The inputs are the execution value of the previous cycle, the current original action factor, and the action step size of each actuator; this addresses the problem that reinforcement learning actions lack physical dimensions and cannot directly drive the device; the output is the mapped candidate device settings; this is executed immediately by the processor after obtaining the original action. In the formulas, , , These are the setpoints for the inlet pump frequency, circulation pump frequency, and chemical dosing acceleration rate after the current cycle is mapped. , , Preset step length for the action; , , This is the projection function for the corresponding executor.

[0075] In this embodiment, the preset action step size is defined as the reference step size within each control cycle that allows the original action to be converted into an actual equipment adjustment amount. It can be obtained through step response tests during the equipment commissioning phase; its typical value is... , , When the pump frequency change in a single cycle exceeds this level, field tests show that the pressure difference and effluent concentration are more prone to significant oscillations; therefore, the above typical values ​​are adopted.

[0076] The specific execution logic of the above projection function is as follows: first, incremental superposition is performed; then, the maximum change in a single cycle is constrained; and finally, upper and lower limit constraints are applied to the equipment. In this embodiment, the allowable frequency range of the influent pump is 25Hz to 50Hz, the allowable frequency range of the circulation pump is 20Hz to 45Hz, and the allowable dosing rate of the reagent is 0.5mL / min to 8.0mL / min. The engineering meanings of the above ranges correspond to the rated range of the strain gauge and the nominal range of the metering pump, respectively, and can be determined according to the equipment nameplate parameters and on-site safety regulations. The maximum change in adjacent control cycles is used to limit the action slope, and its typical values ​​are 2.0Hz / cycle, 2.0Hz / cycle, and 0.8mL / min / cycle, respectively.

[0077] For the anti-washing action, the processor performs a coupling mapping to obtain the anti-washing trigger probability: This formula corresponds to the backwash trigger result generation step. The inputs are the original backwash action factor, the contamination accumulation factor, and the water quality disturbance factor; it addresses the issue that the backwash action cannot be directly determined by the output of a single neural network; the output is the backwash trigger probability. This is called by the processor after the continuous action mapping is completed. In the formula, It is an S-shaped compression function; This is the amplification factor for the original backwashing action; , The coupling weights for pollution accumulation factors and water quality disturbance factors; This is the preset backwash trigger threshold.

[0078] In this embodiment, the preset backwash trigger threshold is defined as the bias threshold used to map the neural network output and the pollution and disturbance characterization quantities to the trigger probability. It can be determined offline through multi-target scanning; its typical value is 1.1. The preset trigger probability threshold is used to determine whether a backwash start command is generated in the current cycle. It can be determined through a trade-off analysis of historical backwash records and compliance rates; its typical value is 0.60. The original backwash action amplification coefficient and coupling weight are used to adjust the influence of the neural network output, pollution accumulation information, and water quality disturbance information on the backwash decision. In this embodiment, the typical values ​​are 2.5, 1.2, and 0.8, respectively. These values ​​are derived from a comprehensive trade-off of the proportion of effluent exceeding limits, peak pressure difference, number of backwashes, and unit water energy consumption in multi-target scanning.

[0079] To limit the backwash time window, this step also introduces preset minimum backwash duration, preset maximum backwash duration, preset recovery differential pressure threshold, and preset recovery permeable water threshold. The aforementioned preset minimum backwash duration ensures that the backwash fluid switching and flushing process reaches an effective duration, and can be calibrated by the time required for the backwash flow rate to stabilize on-site; its typical value is 90s, and in practical engineering applications, it can usually be configured between 60s and 180s. The preset maximum backwash duration limits the production capacity loss caused by prolonged backwashing, and can be determined based on maintenance records and recovery benefit curves; its typical value is 180s. The preset recovery differential pressure threshold represents the upper limit of differential pressure recovery allowed when backwashing ends, and can be determined by statistical analysis of differential pressure recovery after backwashing; its typical value is [missing value]. In Example 2, the specific value is 22.5 kPa. The preset permeate recovery threshold represents the minimum permeate recovery level allowed when backwashing ends, and can be determined statistically through permeate recovery after backwashing; its typical value is... In Example 2, specifically... .

[0080] In this embodiment, if backwashing is initiated, the inlet pump frequency is frozen to the previous cycle value or operates according to the shutdown cleaning preset value, the chemical dosing remains at the previous cycle value, and the circulation pump operates at the backwash preset frequency. The backwash preset frequency is used to maintain the flushing intensity during the backwashing phase and can be calibrated through a backwash flow stability test; its typical value is 36Hz. If the chemical metering pump is in a maintenance shutdown state, the chemical dosing setting value is fixed to the pre-maintenance setting value.

[0081] Within this control cycle of this embodiment, based on the aforementioned original action and the execution value of the previous cycle, the mapped device setting value is obtained as follows: , , The probability of backwashing being triggered is Since this value is lower than the aforementioned preset trigger probability threshold, backwashing will not be initiated in this cycle. This process prioritizes inhibiting further contamination buildup by increasing the cycle intensity and appropriately dosing chemicals, rather than immediately initiating backwashing before the pressure differential reaches a high-risk state. The aforementioned frequency setpoint, dosing acceleration setpoint, and backwashing trigger result constitute the aforementioned equipment setpoints and backwashing trigger result.

[0082] Step S203: Perform action masking and obtain executable process actions.

[0083] Based on the comparison between the pressure difference across the graphene treatment layer and a preset high pressure difference threshold, a shielding flag is generated to further increase the frequency of the influent pump. Based on the comparison between the target pollutant concentration in the effluent and a preset excessive concentration threshold, a shielding flag is generated to reduce the frequency of the circulating pump and reduce the dosage of chemicals. Based on the comparison between the permeate flow rate and a preset low flux threshold, and the comparison between the pollution accumulation information and a preset high pollution threshold, a linked shielding flag is generated to increase the frequency of the influent pump and shut down the backwashing action. The shielding flags are organized into shielding flag vectors corresponding to the influent pump, circulating pump, chemical dosage, and backwashing actions, respectively. When multiple shielding conditions are met simultaneously, the final shielding result is determined according to a preset priority order. Based on the final shielding result, the equipment setpoint is modified by retaining, truncating to the execution value of the previous control cycle, and replacing it with a safe preset value, thereby obtaining the executable process actions.

[0084] In this embodiment, the processor organizes the masking results into a quaternion masking tag vector: This formula corresponds to the encoding steps of the action shielding result. The inputs are differential pressure, effluent concentration, product water flow rate, contaminant accumulation information, and candidate actions; it addresses the lack of a unified decision-making structure when multiple shielding conditions occur concurrently; the outputs are shielding marker vectors corresponding to the influent pump, circulation pump, chemical dosing, and backwashing actions, respectively; these are generated by the processor after the equipment settings are set. In the formula, , , , The values ​​include 0, 1, and 2, which respectively represent retaining the candidate action, truncating to the execution value of the previous cycle, and replacing it with a safe preset value.

[0085] In this embodiment, the preset high pressure differential threshold is used to identify the upper limit of the pressure differential when entering the high pressure differential risk zone, and can be determined based on the historical peak pressure differential and the component's tolerance margin; its typical value is 60 kPa. The preset excess concentration threshold is used to identify that the effluent quality has entered the excess range where further reduction in treatment intensity is not allowed, and can be determined based on the upper limit of the effluent standard and the operational fluctuation margin; its typical value is 25 mg / L. The preset low flux threshold is used to identify a state where the permeate is significantly low and may be accompanied by worsening clogging, and can be determined offline through the lower limit requirement of production capacity; its typical value is... The preset high pollution threshold is used to determine that the pollution accumulation information has reached a high risk level, and it can be determined through multi-target scanning and cleaning record statistical calibration; its typical value is 0.62.

[0086] When multiple shielding conditions are met simultaneously, the processor determines the final shielding result according to a preset priority order: "backwash lockout takes precedence over differential pressure safety, differential pressure safety takes precedence over effluent quality, and effluent quality takes precedence over low-flux-high-pollution linkage." The engineering significance of this preset priority order is to prioritize ensuring the continuity of backwashing that has already been initiated, then control high differential pressure risks, then ensure effluent quality constraints, and finally handle the linkage scenario of low-flux and high-pollution. This priority can be pre-configured based on safety risk levels and on-site experience rules.

[0087] Subsequently, the processor performs retention, truncation, or replacement based on the masking flags to form executable process actions. If all gain-type actions are masked, maintenance-type actions are retained, i.e., maintaining the influent pump frequency and circulation pump frequency of the previous cycle, while determining whether to initiate backwashing based on the backwash trigger result; when the device is in a shutdown cleaning window, the cleaning process preset table is directly written without calling the strategy output. Through this processing, the strategy output is always constrained within the execution domain allowed by the equipment's physical boundaries and process constraints, thereby reducing operational oscillations caused by action conflicts.

[0088] Within this control cycle of this embodiment, the current pressure difference of 46 kPa does not exceed the aforementioned preset high pressure difference threshold, and the effluent concentration of 24.1 mg / L does not exceed the aforementioned preset excessive concentration threshold. Therefore, the shielding flag does not trigger high-risk constraints, and the final executable process action is consistent with the mapped equipment settings. The aforementioned shielding flag vector and action correction rules thus realize the generation of the aforementioned executable process action.

[0089] like Figure 5 As shown, in step S3, based on the feedback data of the next control cycle corresponding to the executable process action, the action effect is evaluated according to the comprehensive benefit rule that simultaneously constrains water production effect, effluent quality, operating energy consumption, backwashing cost and action stability, and a long-term return is formed.

[0090] This step is executed by the processor, with inputs including feedback data for the next control cycle and current and historical action records, and outputs including the single-cycle comprehensive gain and long-term reward. This step can be used online to evaluate performance or as a policy optimization target during offline training.

[0091] Step S301: Calculate the comprehensive return for a single period

[0092] Based on the feedback data of the next control cycle, a single-cycle comprehensive benefit is calculated, with the maintenance of permeate flow rate as a positive benefit and the increase of pollution accumulation information, the target pollutant concentration in the effluent exceeding the preset target value, the increase of total system power, the occurrence of backwash events, and the increase of the magnitude of changes in actions in adjacent control cycles as negative constraints. Based on the comparison results between the cross-graphene treatment layer pressure difference and the preset pressure difference safety threshold in the feedback data of the next control cycle, a pressure difference safety boundary penalty term is generated, and a recovery window cost term is generated based on the pressure difference recovery and permeate recovery process after backwashing. The changes in influent pump frequency, circulation pump frequency, reagent dosage, and backwash status are weighted and summed according to their respective single-cycle allowable change ranges, and coupled with the water quality disturbance information to generate an anti-interference stability constraint term. Based on the single-cycle comprehensive benefit, pressure difference safety boundary penalty term, recovery window cost term, and anti-interference stability constraint term, a long-term return is formed.

[0093] As a specific implementation method, the execution logic of the above process is as follows: After the action is executed, the processor receives the feedback for the next cycle and constructs a single-cycle reward: This formula corresponds to the single-cycle comprehensive benefit aggregation step. The inputs are the feedback data for the next cycle, the current action, and historical actions; it addresses the lack of a unified evaluation standard for multi-objective operational effects; the output is the single-cycle reward value. ; used for performance evaluation during online runtime, and for sample reward calculation during offline training. Where, For water production revenue, For effluent quality constraints, As an energy consumption penalty item, As a cost item for anti-laundering, For interference resistance and stability, This is a penalty term for the pressure differential safety boundary. This is the cost of restoring the window after backwashing.

[0094] In this embodiment, the engineering meaning of the preset reward weight is the relative importance of multiple objectives such as water production, pollution, effluent quality, energy consumption, backwashing, and operational stability in the overall benefit, which can be determined through a combination of multi-objective simulation scanning and semi-physical experiments; its typical value is... , , , , , The weights corresponding to differential pressure safety and recovery window can be determined based on equipment safety and recovery cost sensitivity; in this embodiment, typical values ​​are 1.0 and 0.7, respectively. Typical values ​​for each smoothing term are... This is used to prevent the denominator from being zero.

[0095] To mitigate the risk of high differential pressure, this step introduces a preset differential pressure safety threshold. This parameter indicates the starting point for the differential pressure to enter the warning range and can be determined by multiplying the upper limit of the differential pressure by a safety factor; its typical value is [value missing]. To constrain slow recovery after backwashing, this step introduces a preset recovery time target value, which measures the target time for the pressure differential and permeate to recover to the set range after backwashing. It can be obtained from backwashing recovery statistics; its typical value is 300s, and it can usually be configured in the range of 180s to 600s.

[0096] In practical engineering applications, the processor calculates and weights the changes in influent pump frequency, circulating pump frequency, chemical dosage, and backwash status according to their respective allowable ranges for a single cycle, and then couples this weighted sum with the current water quality disturbance information to generate anti-interference stability constraints. The weights of each action component are used to measure the degree of impact of different actuator action jumps on the oscillation of the operating condition, and can be determined offline through correlation analysis between the number of historical action reversals and effluent fluctuations; typical values ​​are 0.30, 0.30, 0.20, and 0.20.

[0097] In this embodiment, when When there is no action from the previous cycle, the action difference in the stability term is set to zero. When the effluent concentration is lower than the target value, the effluent quality constraint term no longer provides additional rewards, only maintaining zero penalties, to avoid pursuing unnecessary redundant purification through excessive chemical dosing. When backwashing does not occur, the recovery window cost term is set to zero. Through the above evaluation mechanism, current cycle permeable water, long-term pollution, effluent exceeding limits, energy consumption, backwash cost, action fluctuation, differential pressure safety boundary, and recovery efficiency are all uniformly incorporated into the operational evaluation.

[0098] In the next control cycle of this embodiment, the feedback data is: , , Total system power Reference power Furthermore, no reverse washout occurred during this period. Based on the aforementioned comprehensive return rules, a single-period reward was obtained. This result indicates that the proposed solution does not simply increase the dosage of a single reagent based on effluent deviation, but simultaneously constrains flow rate, reagent consumption, pressure differential safety boundaries, and contamination accumulation. The aforementioned comprehensive benefits per cycle constitute the aforementioned operational effectiveness evaluation results.

[0099] Step S302: Develop long-term returns and perform offline training updates

[0100] Offline training samples are constructed based on historical operation logs, semi-physical test data, and digital twin simulation samples. During the offline training phase, the policy network and evaluation network are trained offline based on the long-term returns. The digital twin simulation samples include at least the state transition results of the filter resistance growth sub-model, the effluent concentration response sub-model, and the backwash recovery sub-model.

[0101] In this embodiment, during the offline training phase, the processor calculates the discounted reward for the state transition sequence: This formula corresponds to the steps involved in forming long-term returns. The input is from the first... A single-cycle reward sequence starting from each control cycle; used to address the issue in reinforcement learning where the reward-cost propagation relationship across multiple future cycles needs to be considered; the output is a discounted reward. This is called by the processor during the offline training phase. In the formula, In return for the truncation length, This represents the discount factor. In this embodiment, the typical value for the reward cutoff length is 64 control periods, and the typical value for the discount factor is 0.98. This parameter set can be determined by a trade-off between the average reward on the validation set and training stability.

[0102] Offline training samples include historical operation logs, semi-physical test data, and digital twin simulation samples. Historical operation logs record influent disturbance sequences, equipment action records, effluent quality records, differential pressure change records, and backwash records. Semi-physical test data is used to correct for differences in equipment coupling and actuator response. Digital twin simulation samples include at least the state transition results of the filter resistance growth sub-model, the effluent concentration response sub-model, and the backwash recovery sub-model. The aforementioned filter resistance growth sub-model provides the differential pressure growth trend based on influent conductivity, influent flow rate, and cumulative operating time; the effluent concentration response sub-model provides the concentration change trend based on reagent dosage, circulation intensity, and current settings; and the backwash recovery sub-model provides the differential pressure recovery and permeate recovery trends based on backwash duration and backwash flow rate. The initial values ​​of the parameters for each sub-model are derived from 30 days of historical operation logs and corrected by semi-physical test data.

[0103] In this embodiment, the typical learning rate of the policy network is [value to be filled in]. The typical value for evaluating the network learning rate is [value missing]. The typical batch size is 256, and the typical experience pool capacity is [value missing]. The typical length of each training trajectory is 64, the typical number of training rounds is 300, and the typical soft update coefficient is 0.005. All of these parameters can be obtained through joint calibration of the validation set average return, the proportion of outflow exceeding the limit, and training stability. The training effectiveness criterion is as follows: when the increase in the validation set average return after 20 consecutive training rounds does not exceed 1% of the absolute value of the average return of the previous 20 rounds, and the proportion of outflow exceeding the limit in the validation set no longer decreases, the training is considered converged. If an abnormal strategy occurs in the validation set where the average pressure difference peak exceeds the pressure difference upper limit for 3 consecutive rounds, the corresponding parameter update result is removed, and the system reverts to the previous effective parameters. If the training trajectory length is insufficient for the return truncation length, it is truncated according to the actual remaining length. If individual samples experience invalid cycles caused by equipment downtime, they are marked as terminated, and no further returns are accumulated. The above-described offline sample construction, long-term return formation, and strategy update process realizes the formation of the aforementioned long-term return and network training.

[0104] like Figure 6 As shown, in step S4, the executable process action is converted into a set of field control instructions for the graphene water treatment device and sent to the execution device. At the same time, the action value after execution is written into the action field corresponding to the next control cycle.

[0105] The processor is the main body responsible for executing this step. The inputs are the executable process actions and the current operating status of the equipment. The outputs are the field control instruction set and the results of writing the action field for the next cycle.

[0106] Step S401: Generate device-level control instruction set

[0107] Converting the executable process actions into a set of field control instructions for the graphene water treatment device and sending them to the execution equipment includes: converting the executable process actions into the frequency conversion setting values ​​of the inlet pump, the frequency conversion setting values ​​of the circulating pump, the drive setting values ​​of the reagent metering pump, the backwash valve opening and closing instructions, and the electrolysis power supply current following correction amount.

[0108] In this embodiment, the processor parses the executable process actions into a set of equipment-level control instructions, including the inlet pump frequency conversion setpoint, the circulating pump frequency conversion setpoint, the reagent metering pump pulse frequency setpoint, the backwash valve opening and closing instructions, and the electrolysis power supply current setpoint. To complete the closed-loop execution of the electrolysis cell's operating current, this step generates a current following correction amount based on the circulating pump frequency and the effluent quality deviation. This formula corresponds to the on-site current setpoint generation step. The inputs are the reference operating current, the current executable circulation pump frequency, the reference circulation pump frequency, and the current effluent quality deviation; it is used to address the mismatch between cyclic shear intensity changes and electrochemical reaction intensity; the output is the current cycle electrolysis power supply current setpoint; it is called by the processor when generating the control instruction set. In the formula, The current setting value for the current cycle, in units of ; It is the projection function of the current; This is the frequency-to-current tracking coefficient of the circulating pump, in units of... ; This is the correction factor for the current due to the water outlet deviation, in units of... ; This is the frequency at which the circulating pump can operate in the current cycle. The reference circulation pump frequency; This represents the current effluent quality deviation. The current projection function is preferably truncated first based on the maximum change in a single cycle, and then based on the upper and lower limits of the equipment current.

[0109] In this embodiment, the reference circulating pump frequency, in engineering terms, refers to the reference circulating intensity that ensures interfacial mass transfer under clean and stable operating conditions, and can be obtained through steady-state testing and calibration; its typical value is 34Hz. The reference operating current represents the stable current baseline under this reference circulating intensity, and can be obtained through steady-state testing after clean start-up; its typical value is 18A. Current follower coefficient. and The following can be determined through semi-physical linkage tests: When the circulation pump frequency increases by 1 Hz and other conditions remain approximately unchanged, the average current compensation requirement to maintain a similar effluent compliance rate is approximately 0.22A to 0.28A, therefore 0.25A / Hz is taken; when the effluent concentration exceeds the target by 1 mg / L, the average additional current compensation requirement is approximately 0.10A to 0.14A, therefore 0.12A / (mg / L) is taken.

[0110] To prevent excessively rapid current adjustment, this step introduces upper and lower current limits and a maximum change in current per cycle. These parameters define the rated operating range of the electrolytic power supply and the current change slope per cycle, respectively, and can be determined based on the power supply nameplate parameters and on-site testing; typical values ​​are a lower current limit of 8A, an upper current limit of 25A, and a maximum change in current per cycle of 1.5A / cycle. When backwashing is initiated, the current setting switches to the preset backwash current. In this embodiment, the specific value of the preset backwash current is 10A. The preset backwash current is used to maintain an auxiliary electrochemical state with low energy consumption during backwashing, and can be determined based on a trade-off between energy consumption and recovery effect during the backwashing stage; its typical value is 10A.

[0111] In one optional implementation, the field control command frame includes a command number, device number, generation timestamp, predicted effective timestamp, inlet pump frequency conversion setting, circulating pump frequency conversion setting, chemical metering pump drive setting, backwash valve open / closed status, electrolysis power supply current setting, and integrity check code. The executing device determines whether the field control command is within the allowed execution window based on the generation timestamp and predicted effective timestamp, and determines whether the field control command has been truncated or out of order during transmission based on the integrity check code. This command frame field design helps improve execution consistency in industrial bus communication scenarios.

[0112] Within this control cycle of this embodiment, the current setpoint is obtained as 18.46A based on the aforementioned relationship, and corresponding pump frequency, metering pump, backwash valve, and current commands are generated. The aforementioned pump frequency setpoint, backwash valve command, and current follow-up correction constitute the aforementioned field control command set.

[0113] Step S402: Send field control command and write the next cycle action field.

[0114] In this embodiment, the processor sends control instruction sets to the frequency converter, metering pump driver, solenoid valve group, and electrolysis power supply via the industrial bus. It then writes the executed action values ​​into the action field of the status table for the next cycle and updates the continuous runtime and backwash duration. If backwashing is initiated, the continuous runtime is reset to zero and the contamination evolution statistics window is reset; if backwashing is completed, the clean operating condition baseline pressure difference is re-recorded. This write-back mechanism allows the status table for the next cycle to directly inherit the actual execution results from the previous cycle, rather than inheriting unexecuted candidate actions.

[0115] In this control cycle of this embodiment, the processor writes 38.64Hz to the inlet pump inverter, 34.82Hz to the circulation pump inverter, and 5.74mL / min to the metering pump, keeps the backwash valve closed, writes 18.46A to the electrolysis power supply, and writes the above values ​​back to the status table for the next cycle. No mode transition occurs in this cycle, and the system smoothly enters the next round of decision-making.

[0116] The simulation verification object in this embodiment is The graphene filtration-electrochemical synergistic purification line simulation lasted 8 hours, with an initial pressure difference of 18 kPa and an influent conductivity of [missing information]. to The influent organic load fluctuated intermittently, varying randomly in segments, while the control scheme relied on manual, empirical control. Under the test conditions, the following results were obtained: the average interlayer pressure difference in the proposed scheme was 44 kPa, compared to 58 kPa in the control scheme; the average permeate flow rate in the proposed scheme was... The comparison scheme is The average concentration of the effluent from the proposed solution was 23.1 mg / L, while the control solution was 26.5 mg / L. Semi-physical verification was conducted on the same production line during three consecutive daytime shifts, with each shift having an effective statistical duration of 6 hours, for a total effective sample duration of 18 hours. Under the test conditions, the average interlayer pressure difference was 45.3 kPa, and the average permeate flow rate was [missing information]. The average concentration of the effluent was 23.4 mg / L; when controlled manually based on experience, the average interlayer pressure difference was 56.7 kPa, and the average permeate flow rate was... The average effluent concentration was 26.1 mg / L. This result indicates that by enhancing the synergistic effect of the process status table, action shielding, and comprehensive benefit design, it is possible to reduce differential pressure accumulation and maintain a higher permeable water level under the same influent load. Subsequently, based on the above calculation and processing results, the processor continues to execute steps S1 to S4 in subsequent control cycles to complete the adaptive optimization of parameters under continuous operation.

[0117] Example 2: Action shielding control process under high disturbance inflow conditions

[0118] This embodiment illustrates the specific execution method of the aforementioned action shielding mechanism under conditions of highly disturbed incoming water. Unless otherwise stated, its equipment configuration, control cycle, sampling cycle, strategy network structure, and basic training method are consistent with Embodiment 1.

[0119] This embodiment corresponds to a scenario of electroplating wastewater reuse, where the influent conductivity and flow rate briefly surge during shift changes. The control period is 30 minutes, during which the influent conductivity changes from... Rise to The influent flow rate is from Rise to In this embodiment, the aforementioned preset high pressure differential threshold remains 60 kPa, the preset over-limit concentration threshold remains 25 mg / L, the preset minimum backwash time remains 90 s, and the preset low throughput threshold remains... The preset high contamination threshold remains at 0.62, the preset trigger probability threshold remains at 0.60, the backwash recovery differential pressure threshold is set to 22.5 kPa, and the backwash recovery product water threshold is set to... The preset maximum backwash duration is 180 seconds. All of the above parameters are used to identify high-risk congestion zones and define backwash recovery boundaries.

[0120] During a critical control cycle, the processor collected the following data: , , , , , The action value in the previous cycle was: , , , The processor executes steps S101 to S103 in the aforementioned embodiment one accordingly, and obtains... , , , This indicates that there is currently both high pollution accumulation and strong water disturbance.

[0121] Subsequently, the processor executes the aforementioned step S201, and the policy network outputs the original action set as follows: , , , Then, perform the aforementioned step S202 to obtain the mapped setpoint: , , Backwashing trigger probability .

[0122] During step S203, the processor detects that the current pressure difference of 61 kPa is higher than the aforementioned preset high pressure difference threshold, and therefore generates a shielding flag for further increasing the influent pump frequency. Simultaneously, the effluent concentration of 26.8 mg / L is higher than the aforementioned preset excessive concentration threshold, therefore generating a shielding flag for reducing the circulation pump frequency and reducing the reagent dosage. Furthermore, due to the current product water flow rate... The flow rate is below the aforementioned preset low-flux threshold, and the contamination accumulation factor of 0.677 is higher than the aforementioned preset high-contamination threshold. Therefore, a linkage shielding flag is generated to increase the frequency of the influent pump and shut down the backwashing action. After processing according to priority, the final shielding flag vector is taken as... The resulting executable process actions are as follows: keep the inlet pump frequency constant at 41.0 Hz, increase the circulation pump frequency to 34.52 Hz, increase the reagent dosing rate to 5.49 mL / min, and open the backwash valve.

[0123] In this scenario, the action shielding mechanism does not simply initiate backwashing alone after the differential pressure exceeds the threshold; instead, it simultaneously blocks the flow-increasing action that would further worsen the blockage, ensuring that the backwashing action and the inlet flow-increasing action no longer conflict. Subsequently, the processor executes step S401. Since backwashing has been initiated, the current setting is switched to the aforementioned preset backwash current of 10A, the circulation pump switches to the preset backwash frequency of 36Hz, and the inlet pump remains frozen at 41.0Hz for one control cycle. After backwashing continues for the aforementioned preset minimum backwash duration, the processor continues to execute step S402, completing the duration update and status write-back. Five minutes after backwashing, the system recovers to: , , , .

[0124] The simulation verification object of this embodiment is a high-salt electroplating wastewater reuse line. The simulation duration is 24 hours, and three high-disturbance pulses are set. The control scheme is static threshold control, that is, backwashing when the pressure difference is greater than 55 kPa, and other parameters are fixed. Under the test conditions, the following results were obtained: the number of reversals per hour for the proposed scheme is 7.9, while that for the control scheme is 18.6; the percentage of time exceeding the limit for effluent concentration is 2.7% for the proposed scheme, while that for the control scheme is 9.8%; the average peak pressure difference is 58 kPa for the proposed scheme, while that for the control scheme is 67 kPa. A control loop joint debugging platform was built at the electroplating reuse site, and 12 hours of valid data from two high-disturbance shifts were selected for semi-physical verification. Under the test conditions, the following results were obtained: when using the proposed scheme, the number of reversals per hour is 8.4, the average peak pressure difference is 59.2 kPa, and the percentage of time exceeding the limit for effluent is 3.1%; when using static threshold control, the number of reversals per hour is 17.9, the average peak pressure difference is 65.8 kPa, and the percentage of time exceeding the limit for effluent is 10.4%. This result demonstrates that the aforementioned action shielding mechanism can effectively suppress conflicting control actions in highly disturbed scenarios. Subsequently, based on the above calculation and processing results, the processor continues to execute the subsequent control cycle steps as described in Example 1, completing the state update, benefit evaluation, and field command output under continuous operation. Its specific execution logic is consistent with Example 1 and will not be repeated here.

[0125] Example 3: Comparison of continuous operation with PID control scheme

[0126] This embodiment illustrates the overall operational performance of the present invention compared to a PID control scheme under continuous production conditions. Unless otherwise stated, the equipment configuration, process object, and basic control interface are consistent with Embodiment 1.

[0127] In this embodiment, the trained policy model is deployed on-site. The offline training samples contain 180,000 state transition records, of which 120,000 are from historical operation logs, 30,000 from semi-physical linkage experiments, and 30,000 from digital twin simulations. The policy network has 17-dimensional input and 4-dimensional output; the evaluation network has 21-dimensional input. The training batch size is 256, the discount factor is 0.98, and the policy network learning rate is... The network learning rate was evaluated. The soft update coefficient is 0.005. After 300 training rounds, convergence is determined by the criterion that "the average return of the validation set for 20 consecutive rounds does not exceed 1% and the proportion of water exceeding the limit no longer decreases".

[0128] During the continuous 24-hour simulation, the processor performed closed-loop optimization every 30 seconds according to steps S101 to S402 of Example 1. Specifically, a high organic load zone was set from the 6th to the 10th hour, an influent flow rate pulse was set at the 15th hour, and a conductivity pulse was set at the 18th hour. The control scheme used a three-loop PID controller: the flow rate loop adjusted the influent pump, the differential pressure loop adjusted the circulating pump, and the effluent concentration loop adjusted the chemical dosing; periodic backwashing was performed when the differential pressure exceeded 55 kPa. The PID parameters were taken from the stable on-site setpoints: influent pump loop... , , Circulating pump circuit , , Drug circuit , , .

[0129] In this embodiment, the difference in the present invention lies in the following: when the influent disturbance initially manifests as an amplified effluent deviation while the differential pressure has not yet exceeded the limit, the processor does not simply increase the dosage based on a single concentration deviation. Instead, it combines the pollution accumulation factor, the adsorption site attenuation characterization quantity, and the conductive interface reaction intensity characterization quantity to determine whether the current dominant contradiction is biased towards adsorption site occupancy, blockage accumulation, or insufficient electrochemical reaction, and then decides whether to increase the circulating shear, correct the current, or increase the dosage. When the differential pressure rises and low flux-high pollution linkage conditions occur, the processor directly blocks the flow-increasing action and retains backwashing through action shielding rules. When backwashing ends and the recovery window is entered, the processor suppresses action combinations that recover too slowly through the recovery window cost term. Thus, the actions that were originally dispersed in multiple local control loops are reconstructed into a unified optimization problem oriented towards continuous operation.

[0130] The simulation results of this embodiment show that, under the 24-hour mixed disturbance condition, the comprehensive energy consumption per ton of water of the present invention is: The PID scheme is The backwashing frequency of this invention is 7 times, while that of the PID method is 11 times; the average concentration of the target pollutant in the effluent of this invention is 21.8 mg / L, while that of the PID method is 27.6 mg / L; the average permeate flow rate of this invention is... The PID scheme is .

[0131] Furthermore, two rounds of 24-hour on-site comparative operation were conducted continuously on the same graphene filtration-electrochemical synergistic purification line. The first round used the PID scheme, and the second round used the scheme of this invention. The on-site influent water conditions were: influent conductivity range to Inlet flow rate range to The equivalent influent chemical oxygen demand (COD) fluctuated between 180 mg / L and 265 mg / L. Under the test conditions, the average comprehensive energy consumption per ton of water under the PID scheme was obtained as follows: The backwashing frequency was 10 times, the average concentration of the target pollutant in the effluent was 27.1 mg / L, and the average permeate flow rate was [missing information]. Under the scheme of this invention, the average comprehensive energy consumption per ton of water is The backwashing frequency was 7 times, the average concentration of the target pollutant in the effluent was 22.4 mg / L, and the average permeate flow rate was [missing information]. The results demonstrate that the synergistic design of the aforementioned enhanced process status table, physical mapping, action shielding, and comprehensive benefit rules can reduce unit permeable energy consumption, decrease backwashing frequency, and maintain a lower effluent pollutant concentration. Subsequently, based on the above calculations and processing results, the processor continues to execute the subsequent steps as described in Example 1, completing the status update, action generation, long-term return assessment, and on-site deployment and operation in the subsequent control cycle. The specific execution logic is consistent with Example 1 and will not be repeated here.

[0132] Example 4: An electronic device

[0133] This embodiment provides an electronic device, including a memory, a processor, a communication interface, and a computer program stored in the memory and running on the processor. The communication interface is used for data interaction with an influent flow meter, a product water flow meter, a differential pressure transmitter, an online effluent analyzer, a conductivity meter, a thermometer, an electrolysis power supply, an influent pump frequency converter, a circulating pump frequency converter, a reagent metering pump driver, and a backwash valve assembly. When the processor executes the computer program, it implements the steps of the graphene water treatment process parameter adaptive optimization method based on reinforcement learning described in any one of embodiments one to three, including operation signal acquisition, enhanced process state table construction, policy network inference, original action physical mapping, action masking, comprehensive benefit evaluation, long-term return formation, and on-site control command output.

[0134] In one specific implementation, the electronic device is any one of an industrial control computer, an edge controller, or a centralized control server. The memory can be one or more combinations of random access memory, read-only memory, flash memory, or solid-state drive, used to store control programs, historical operation logs, training samples, and network parameters. The processor can be a general-purpose CPU, an industrial-grade ARM processor, or an embedded processor with an AI acceleration unit. The communication interface can be at least one of an Ethernet interface, a serial port interface, a CAN bus interface, or an industrial fieldbus interface.

[0135] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for self-adaptive optimization of graphene water treatment process parameters based on reinforcement learning, characterized in that, include: Step S1: Obtain the operating signal and the action record of the previous control cycle of the graphene water treatment device, and map the operating signal and the action record of the previous control cycle into an enhanced process status table containing pollution accumulation information, water quality disturbance information and effluent deviation information. Step S2: Input the enhanced process status table into the strategy network to generate an original action set, map the original action set to the equipment setting value, and perform action masking on the equipment setting value according to the equipment physical boundary and process constraints to obtain executable process actions; Step S3: Based on the feedback data of the next control cycle corresponding to the executable process action, evaluate the action effect according to the comprehensive benefit rule that simultaneously constrains water production effect, effluent quality, operating energy consumption, backwashing cost and action stability, and form a long-term return. Step S4: Convert the executable process action into a set of field control instructions for the graphene water treatment device and send it to the execution device. At the same time, write the action value after execution into the action field corresponding to the next control cycle.

2. The method of claim 1, wherein the method is characterized by, The operation signal and the action record of the previous control cycle are mapped into an enhanced process status table that includes pollution accumulation information, water quality disturbance information, and effluent deviation information, including: The system acquires the following data: influent flow rate, product flow rate, pressure difference across the graphene treatment layer, target pollutant concentration in the effluent, water temperature, influent conductivity, electrolytic cell operating current, and the influent pump frequency setpoint, circulation pump frequency setpoint, reagent dosing acceleration setpoint, backwash execution status, and backwash duration from the previous control cycle. Based on the influent flow rate, product flow rate, pressure difference across the graphene treatment layer, target pollutant concentration in the effluent, water temperature, influent conductivity, electrolytic cell operating current, and the influent pump frequency setting, circulation pump frequency setting, reagent dosing acceleration setting, backwash execution status, and backwash duration of the previous control cycle, a process status table is constructed. Based on the process status table, the pollution accumulation information and the water quality disturbance information are generated, and the effluent quality deviation and product water flow deviation are generated as the effluent deviation information to obtain the enhanced process status table. The effluent quality deviation is the signed difference between the target effluent pollutant concentration and the preset effluent concentration target value, and the product water flow deviation is the signed difference between the preset product water target value and the product water flow.

3. The method of claim 2, wherein the method is characterized by, Generating the pollution accumulation information includes: Obtain the cross-graphene treatment layer pressure difference, permeate flow rate, and continuous operating time since the most recent backwashing from the process status table; The degree of increase of the cross-graphene treatment layer pressure difference relative to the preset clean reference pressure difference, the degree of decrease of the permeate flow rate relative to the preset clean reference permeate flow rate, and the cumulative degree of the continuous running time relative to the preset clean reference running time are calculated respectively. The degree of elevation, the degree of attenuation, and the degree of accumulation are weighted and aggregated with the corresponding preset pollution weight coefficients to obtain the pollution accumulation information that reflects the combined effects of blockage, scaling, and flux attenuation.

4. The method of claim 2, wherein the method is characterized by, Generating the water quality disturbance information includes: Obtain the influent conductivity, influent flow rate, and water temperature from the process status table, and obtain the moving average values ​​of the influent conductivity, influent flow rate, and water temperature within a preset preceding time window; Calculate the degree of deviation of the influent conductivity, influent flow rate, and water temperature from their respective moving average values; The deviation degree is weighted and aggregated with the corresponding preset disturbance weight coefficient to obtain the water quality disturbance information that reflects the short-term fluctuation intensity of the incoming water conditions.

5. The method for adaptive optimization of graphene water treatment process parameters based on reinforcement learning according to claim 2, characterized in that, Obtaining the enhanced process status table includes: Based on the target pollutant concentration in the effluent and the set value of the reagent dosing rate in the process status table, an adsorption site attenuation characterization quantity is generated. Based on the electrolytic cell operating current and circulating pump frequency settings in the process status table, a characterization quantity for the intensity of conductive interface reaction is generated. The fields in the process state table are grouped according to the filtration resistance related group, adsorption saturation related group, electrochemical reaction related group, and backwash recovery related group. The adsorption site attenuation characterization and the conductive interface reaction intensity characterization are used as operating condition labels to participate in the operating condition feature combination before the strategy network input, thus obtaining the enhanced process state table.

6. The method for adaptive optimization of graphene water treatment process parameters based on reinforcement learning according to claim 2, characterized in that, Mapping the original action set to device settings includes: The enhanced process status table is input into the strategy network to obtain the original action sets corresponding to the influent pump, circulation pump, chemical dosing and backwashing triggers, respectively; According to the preset action step size of the corresponding actuator, the adjustment amount of the water inlet pump, circulation pump and chemical dosing in the original action set is incrementally mapped to obtain the candidate setting value of each actuator; First, the maximum change in adjacent control cycles is constrained for the candidate setpoints, and then the upper and lower limits of the equipment are constrained to obtain the setpoints for the inlet pump frequency, the circulation pump frequency, and the chemical dosing rate. The adjustment amount corresponding to the backwash trigger in the original action set is coupled and mapped with the pollution accumulation information and the water quality disturbance information to obtain the backwash trigger result.

7. The method for adaptive optimization of graphene water treatment process parameters based on reinforcement learning according to claim 6, characterized in that, The adjustment amount corresponding to the backwash trigger in the original action set is coupled and mapped with the pollution accumulation information and the water quality disturbance information to obtain the backwash trigger result, including: Based on the comparison between the backwash triggering result and the preset triggering probability threshold, a backwash start command is generated when the current period is not in the backwash lockout period. After backwashing is started, it will remain in backwashing mode until the preset minimum backwashing time. After the preset minimum backwashing time is reached, the backwashing is terminated based on the comparison results of the pressure difference across the graphene treatment layer and the preset recovery pressure difference threshold, as well as the comparison results of the permeate flow rate and the preset recovery permeate threshold. If the preset maximum backwashing time is reached but the termination condition is not met, the backwashing will be forcibly terminated and a maintenance alarm will be generated.

8. The method for adaptive optimization of graphene water treatment process parameters based on reinforcement learning according to claim 6, characterized in that, Based on the physical boundaries and process constraints of the equipment, the execution actions of the set values ​​of the equipment are masked to obtain the executable process actions, including: Based on the comparison between the pressure difference across the graphene treatment layer and the preset high pressure difference threshold, a shielding flag is generated to prevent further increases in the frequency of the water inlet pump. Based on the comparison results between the target pollutant concentration in the effluent and the preset excessive concentration threshold, a shielding flag is generated to reduce the frequency of circulation pump operation and reduce the dosing of reagents. Based on the comparison results of the produced water flow rate and the preset low flux threshold, and the comparison results of the pollution accumulation information and the preset high pollution threshold, a linkage shielding flag is generated to increase the frequency of the influent pump and shut down the backwash action. The shielding marks are organized into shielding mark vectors corresponding to the water inlet pump, circulation pump, chemical dosing and backwashing actions, and the final shielding result is determined according to a preset priority order when multiple shielding conditions are met simultaneously. Based on the final shielding result, the device setting value is modified by retaining, truncating to the execution value of the previous control cycle, and replacing it with a safety preset value, thereby obtaining the executable process action.

9. The method for adaptive optimization of graphene water treatment process parameters based on reinforcement learning according to claim 2, characterized in that, The effectiveness of the operation is evaluated according to a comprehensive benefit rule that simultaneously constrains water production, effluent quality, operating energy consumption, backwashing costs, and operational stability, and the long-term returns include: Based on the feedback data of the next control cycle, the comprehensive benefit of a single cycle is calculated with the degree of maintenance of the production water flow rate as the positive benefit and the increase of pollution accumulation information, the target pollutant concentration in the effluent exceeding the preset target value of the effluent concentration, the increase of the total power of the system, the occurrence of backwash events and the increase of the change in the action of adjacent control cycles as negative constraints. Based on the comparison results between the cross-graphene treatment layer pressure difference and the preset pressure difference safety threshold in the feedback data of the next control cycle, a pressure difference safety boundary penalty term is generated, and a recovery window cost term is generated based on the pressure difference recovery and product water recovery process after backwashing. The changes in the frequency of the influent pump, the frequency of the circulating pump, the amount of chemical dosage, and the amount of backwash status are weighted and summed according to their respective single-cycle allowable change ranges, and then coupled with the water quality disturbance information to generate an anti-interference stability constraint term. Based on the single-cycle comprehensive return, the differential pressure safety boundary penalty term, the recovery window cost term, and the anti-interference stability constraint term, a long-term return is formed; Offline training samples are constructed based on historical operation logs, semi-physical test data, and digital twin simulation samples. During the offline training phase, the policy network and evaluation network are trained offline based on the long-term returns. The digital twin simulation samples include at least the state transition results of the filter resistance growth sub-model, the effluent concentration response sub-model, and the backwash recovery sub-model.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the graphene water treatment process parameter adaptive optimization method based on reinforcement learning as described in any one of claims 1 to 9.

Images