Reinforcement learning-based electrolytic aluminum series current adaptive allocation and global power saving control system

Abstract

This invention provides a reinforcement learning-based adaptive current distribution and global energy-saving control system for electrolytic aluminum production, relating to the field of electrolytic aluminum production control technology. It includes a sensing module, a reinforcement learning decision-making module, a collaborative control module, an execution module, and a closed-loop feedback module. The sensing module collects real-time operating parameters and anode current distribution data for the electrolytic aluminum series. The reinforcement learning decision-making module generates current distribution decisions based on the collected data using an improved reinforcement learning algorithm. The improved PPO reinforcement learning decision-making model constructed in this invention integrates non-contact magnetic coil measurement technology, solving the problems of low accuracy and poor stability of traditional contact measurement. Simultaneously, it accelerates model convergence through the GAE algorithm, achieving autonomous learning and dynamic optimization of current distribution decisions. This enables precise adaptation to the dynamic operating conditions of multi-field coupling in the electrolytic cell, ensuring a high degree of matching between current distribution and real-time cell conditions, thus improving control accuracy.

Description

Technical Field

[0001] This invention relates to the field of electrolytic aluminum production control technology, and in particular to a current adaptive allocation and global energy-saving control system for electrolytic aluminum series based on reinforcement learning. Background Technology

[0002] In the electrolytic aluminum production process, the rationality of current distribution directly determines production efficiency, energy consumption level, and product quality, making it a core aspect of electrolytic aluminum production control. Currently, the industry generally adopts a fixed current distribution mode or a simple PID algorithm for current regulation. This involves installing sensors at key locations in the electrolytic cell to collect basic operating parameters such as cell temperature and voltage, and then adjusting the current distribution ratio based on manual experience or preset thresholds. Some companies have attempted to introduce simple machine learning algorithms to optimize current control, but these remain limited to independent control of a single cell and do not consider the coupling effect between different electrolytic cells in the electrolytic aluminum series. Regarding current measurement, contact measurement methods are commonly used, directly collecting current data by installing current sensors at the anode conductors for subsequent control decisions. Furthermore, existing control systems often focus on a single objective for optimization, failing to achieve a synergistic balance between energy saving, output, and quality. Additionally, the transmission and execution delays of control commands are relatively high, making it difficult to adapt to the dynamically changing operating conditions of the electrolytic cells. Specifically, existing technologies have the following limitations, making it difficult to meet the demands of high efficiency, energy saving, and high quality in modern electrolytic aluminum production: First, the control mode is rigid; fixed current distribution and simple algorithms cannot adapt to the dynamic operating conditions of multi-field coupling in electrolytic cells, resulting in a mismatch between current distribution and real-time operating conditions, and low control accuracy. Second, the measurement method has defects; contact current measurement is easily affected by the high temperature and corrosive environment of the electrolytic cell, resulting in low measurement accuracy, poor stability, and easy damage to sensors, increasing maintenance costs. Third, there is a lack of global optimization thinking; the use of a single-cell independent control mode fails to consider the inter-cell coupling effect among the electrolytic cells in the series. The existing technologies have several drawbacks. First, they are prone to causing system oscillations due to local optimization, making it impossible to achieve overall optimality. Second, they have a single optimization objective, focusing only on a single goal and ignoring the coordinated improvement of output and product purity, which can easily lead to contradictions such as "saving electricity without increasing production" and "increasing production while reducing quality." Third, they have poor real-time control, with high delays in sensor data transmission and command execution, which cannot meet the millisecond-level response requirements for electrolytic aluminum current adjustment, resulting in the inability to implement decision commands in a timely manner and affecting the control effect. Therefore, this invention proposes a reinforcement learning-based electrolytic aluminum series current adaptive allocation and global energy-saving control system to solve the problems existing in the prior art. Summary of the Invention

[0003] To address the aforementioned issues, this invention proposes a reinforcement learning-based adaptive current allocation and global energy-saving control system for electrolytic aluminum. This system utilizes an improved PPO reinforcement learning decision model, which integrates non-contact magnetic coil measurement technology. This solves the problems of low accuracy and poor stability associated with traditional contact-based measurements. Furthermore, the GAE algorithm accelerates model convergence, enabling autonomous learning and dynamic optimization of current allocation decisions. This allows for precise adaptation to the dynamic operating conditions of multi-field coupling in the electrolytic cell, ensuring a high degree of matching between current allocation and real-time cell conditions, thereby improving control accuracy.

[0004] To achieve the objectives of this invention, the invention is implemented through the following technical solution: a current adaptive allocation and global energy-saving control system for electrolytic aluminum series based on reinforcement learning, comprising a sensing module, a reinforcement learning decision module, a collaborative control module, an execution module, and a closed-loop feedback module, characterized in that: the sensing module is used to collect real-time operating parameters and anode current distribution data of the electrolytic aluminum series; the reinforcement learning decision module generates current allocation decisions based on the collected data through an improved reinforcement learning algorithm; The collaborative control module is used to realize intelligent agent collaborative decision-making and series-level global optimization of multiple electrolytic cells; the execution module is used to convert decision instructions into execution signals and drive the execution mechanism; the closed-loop feedback module is used to feed back the real-time data after execution to the reinforcement learning decision module to form closed-loop control, realize adaptive adjustment of current distribution and global power saving optimization.

[0005] Further improvements are made in that: the sensing module includes a non-contact magnetic induction coil assembly, a temperature sensor, a voltage sensor, and an electrolyte concentration sensor; the non-contact magnetic induction coil assembly is installed at the junction of the upper horizontal busbar of the electrolytic cell and each set of anode guide rods, and the axis of the non-contact magnetic induction coil assembly is perpendicular to the magnetic field direction, used to detect the magnetic flux in real time and infer the anode current distribution data; the temperature sensor, voltage sensor, and electrolyte concentration sensor are used to collect the cell temperature, cell voltage, and electrolyte concentration data of the electrolytic cell, respectively.

[0006] Further improvements include: the measurement accuracy of the non-contact magnetic coil assembly is ≥ ±0.5%, and its operating temperature range is -20℃ to 150℃, adapting to the working environment of the electrolytic cell; the inverse formula for the anode current distribution data is: , Where I is the anode current (A); Φ is the magnetic flux detected by the non-contact magnetic induction coil assembly (Wb); N is the number of turns of the non-contact magnetic induction coil assembly; B is the magnetic field strength (T); and L is the effective length of the non-contact magnetic induction coil assembly (m).

[0007] Further improvements are made in the following aspects: The reinforcement learning decision module employs an improved proximal policy optimization (PPO) algorithm to construct the decision model, and the model's state input S includes electrolytic cell operating parameters, namely, cell temperature T, electrolyte concentration C, anode current distribution I, and cell voltage U; the model's action output A is the current distribution ratio α of each electrolytic cell; the model introduces GAE (Generalized Advantage Estimation) to reduce variance and accelerate model convergence. The formula for calculating GAE is: , in, denoted as γ, which is the generalized advantage estimate at time t; γ is the discount factor, ranging from 0.9 to 0.99, representing the degree of decay of future rewards; λ is the bias coefficient, ranging from 0.9 to 0.95, used to balance bias and variance; and k is the iteration step size. Let be the time series difference error at time t+k. , The instant reward at time t+k. Let be the value function of state s at time t+k. Let be the value function of state s at time t+k+1.

[0008] Further improvements are made by constructing a multi-objective reward function R for the improved PPO model, based on the state input, action output, and GAE generalized advantage estimation mechanism of the adaptive decision model. This R serves as the core evaluation criterion for model training and decision optimization, guiding the model to learn a current allocation strategy that meets the requirements of multi-objective optimization. Its calculation formula is as follows: , Wherein, ω1, ω2, ω3, and ω4 are the weight coefficients of each objective, and ω1+ω2+ω3+ω4=1, which are dynamically adjusted in real time according to the electrolytic cell operating parameters; η is the single cell current efficiency (unit %), corresponding to the anode current distribution I and cell voltage U in the model state input, reflecting the rationality of current distribution; ε is the series energy saving rate (unit %), corresponding to the energy saving effect of the model action output A; μ is the anode current uniformity (unit %), corresponding to the anode current distribution I in the state input, reflecting the balance of current distribution; s is the cell safety index, with a value range of 0~1, where 1 indicates complete safety and 0 indicates potential safety hazards, corresponding to the cell temperature T and electrolyte concentration C in the state input, ensuring stable operation of the electrolytic cell; this reward function works synergistically with the GAE generalized advantage estimation mechanism to reduce model training variance, accelerate model convergence, and ensure that the current distribution decision output by the model accurately matches the real-time operating conditions by quantifying the benefits of each objective.

[0009] Further improvements are made in that: the reinforcement learning decision module adopts a Pareto optimal strategy to achieve multi-objective balance. The criteria for determining the Pareto optimal solution are: there are no other decision schemes that make at least one of the three objectives—energy saving rate, output, and purity—better than the current scheme, and the other objectives are not worse than the current scheme; the multi-objective weight adjustment formula is: , Where ωj is the weight of the j-th target, j=1 corresponds to the energy saving rate, j=2 corresponds to the output, and j=3 corresponds to the purity; fj is the actual value of the current target, fj,min is the minimum value of the target, and fj,max is the maximum value of the target. A further improvement lies in the following: The collaborative control module employs the multi-agent deep deterministic policy gradient (MADDPG) algorithm, treating each electrolytic cell in the aluminum electrolysis series as an independent agent. The policy update formula for each agent is as follows: in, Let be the policy gradient of the i-th agent. Let be the policy parameters for the i-th agent. Let represent the expectation, D be the experience replay buffer, s be the global state, a1~aN be the actions of N agents respectively, r be the immediate reward, and s' be the global state at the next time step. The strategy for the i-th agent to output action ai in state s. Let be the Critic network value function for the i-th agent. Network parameters.

[0010] Further improvements are made in the following: The collaborative control module constructs a global energy-saving collaborative optimization model for the electrolytic aluminum series. The global state space Stotal of the model has an inclusion and correspondence relationship with the local state space Si of a single electrolytic cell, specifically as follows: The global state space Stotal is composed of the integrated local state spaces Si of all electrolytic cells in the series, i.e., Stotal = {S1, S2, ..., SN}, where N is the total number of electrolytic cells in the electrolytic aluminum series, and Si is the independent local state space of the i-th electrolytic cell; the parameter sources, parameter types, and definitions of the local state space Si of each electrolytic cell are completely corresponding to the data collected by the sensing module and the state input parameters of the decision model, i.e., Si = {Ti, Ci, Ii, Ui}, where Ti is the cell temperature of the i-th electrolytic cell, Ci is the electrolyte concentration of the i-th electrolytic cell, Ii is the anode current distribution of the i-th electrolytic cell, and Ui is the cell voltage of the i-th electrolytic cell; the global energy-saving collaborative optimization model for the electrolytic aluminum series takes the overall optimization of the series as its core objective, and its global optimization objective is to maximize the total reward of the series. , where Ri is the single-cell reward of the i-th electrolytic cell, calculated by the multi-objective reward function; by integrating the local states Si and corresponding single-cell rewards Ri of all single cells into the global state space Stotal, real-time updates of the global state and global optimization decisions are achieved.

[0011] Further improvements are made in the following aspects: the execution module includes an industrial Ethernet, a PLC, and a rectifier; the industrial Ethernet is used to transmit data collected by the sensing module and instructions generated by the reinforcement learning decision module, with a transmission delay of ≤50ms; the PLC is used to convert current adjustment instructions into 4-20mA analog signals and transmit them to the current regulator of the rectifier, with an execution delay of ≤100ms; the rectifier is used to adjust the output current according to the analog signal to achieve precise execution of current distribution.

[0012] Further improvements are made in that the feedback delay of the closed-loop feedback module is ≤50ms, and the data of anode current distribution, cell temperature, cell voltage and electrolyte concentration after execution are fed back to the reinforcement learning decision module in real time to update the state space and value function of the model, so as to realize dynamic iterative optimization of decision.

[0013] The beneficial effects of this invention are as follows: 1. This invention adapts to dynamic operating conditions and improves control accuracy. The improved PPO reinforcement learning decision model integrates non-contact magnetic induction coil measurement technology, which solves the problems of low accuracy and poor stability of traditional contact measurement. At the same time, the model convergence is accelerated by the GAE algorithm, realizing autonomous learning and dynamic optimization of current distribution decision. It can accurately adapt to the dynamic operating conditions of multi-field coupling in the electrolytic cell, so that the current distribution is highly matched with the real-time cell conditions, thus improving control accuracy.

[0014] 2. This invention achieves multi-agent collaboration to solve local operational risks. Based on the distributed collaborative allocation architecture of the MADDPG algorithm, each electrolytic cell is treated as an independent intelligent agent to achieve differentiated allocation of current in each cell. This invention specifically solves problems such as local overvoltage and cold cells caused by differences in cell conditions in the prior art, reduces the failure rate of electrolytic cells, improves the stability of series operation, and reduces the failure rate.

[0015] 3. The present invention uses global optimization decision-making to improve energy saving effect. The series of global optimization models constructed solve the inter-cell coupling effect, break the limitation of independent control of a single cell, achieve overall optimization rather than local optimization, avoid system oscillation caused by local optimization, improve the global energy saving rate, which is far higher than the energy saving level of the existing technology, and significantly reduce the energy consumption of electrolytic aluminum production.

[0016] 4. This invention employs multi-objective collaborative optimization, filling a gap in the industry and achieving a synergistic balance among the three objectives of energy saving, output, and quality. By dynamically weighing each objective through the Pareto optimal strategy, it resolves the contradictions of "energy saving without increasing output" and "increasing output but decreasing quality" caused by traditional single-objective optimization. While saving energy, it increases aluminum output and reduces product purity volatility, filling the technological gap in multi-objective collaborative control in the electrolytic aluminum industry.

[0017] 5. The low-latency closed-loop control of this invention ensures real-time implementation of decisions. The designed industrial-grade low-latency closed-loop execution system, through the collaboration of hardware and software such as industrial Ethernet and PLC, reduces the control response time from the traditional 5-10 seconds to ≤0.5 seconds, meeting the millisecond-level response requirements for electrolytic aluminum current adjustment. This ensures that reinforcement learning decisions can be implemented in a timely manner, improves the real-time performance and effectiveness of regulation, and further guarantees production stability and optimization results. Attached Figure Description

[0018] Figure 1 This is a diagram illustrating the composition of the present invention. Detailed Implementation

[0019] To enhance understanding of the present invention, the present invention will be further described in detail below with reference to embodiments. These embodiments are only used to explain the present invention and do not constitute a limitation on the scope of protection of the present invention.

[0020] Example 1 according to Figure 1 As shown, this embodiment proposes a reinforcement learning-based adaptive current allocation and global energy-saving control system for electrolytic aluminum series. For a single electrolytic cell in a 100kA electrolytic aluminum series, an improved PPO reinforcement learning decision model is built. The specific implementation steps are as follows: Sensing module deployment: At the junction of the upper horizontal busbar and the 16 sets of anode guide rods of the electrolytic cell, non-contact magnetic induction coils (N=1000 turns, effective length L=0.5m, measurement accuracy ±0.4%, operating temperature -20℃~150℃) perpendicular to the magnetic field direction are installed. At the same time, K-type thermocouple temperature sensors (measurement range 0~1000℃, accuracy ±1℃), voltage sensors (measurement range 0~10V, accuracy ±0.1V) and electrolyte concentration sensors (measurement range 2~10wt%, accuracy ±0.05wt%) are installed to collect real-time operating parameters and magnetic flux data.

[0021] Model parameter settings: The state input of the improved PPO model is S = {T, C, I, U}, where T is the tank temperature (target range 940~960℃), C is the electrolyte concentration (target range 5~7wt%), I is the current distribution of 16 anodes (unit A), and U is the tank voltage (target range 4.2~4.5V); the action output A is the current distribution ratio α of 16 anodes (α1+α2+...+α16=1); the discount factor γ of the GAE algorithm is 0.95, and the deviation coefficient λ is 0.92; the initial weights of the multi-objective reward function are ω1=0.3 (current efficiency), ω2=0.3 (energy saving rate), ω3=0.2 (current uniformity), and ω4=0.2 (safety index).

[0022] Model Training and Optimization: Operating data of the electrolyzer was collected continuously for 72 hours (sampling frequency 1 time / second) as the training dataset, which was then divided into a training set (80%) and a test set (20%). During model training, the variance was reduced using the GAE algorithm, with 1000 iterations and an initial learning rate of 0.001. An adaptive learning rate decay strategy was adopted, decreasing the learning rate by 10% every 100 iterations. After training, the model's convergence speed was improved by 35% compared to the traditional PPO algorithm, and the response time for current allocation decisions was ≤0.1 seconds.

[0023] Implementation results: After one month of operation, the uniformity of anode current increased from 82% to 95%, the current efficiency of a single cell increased from 92% to 95.5%, the energy saving rate of a single cell reached 11.8%, the temperature fluctuation of the cell was controlled within ±5℃, and no safety hazards of the cell were found, which verified the adaptive control capability and accuracy of the model.

[0024] Example 2 according to Figure 1 As shown, this embodiment proposes a reinforcement learning-based adaptive current allocation and global power-saving control system for electrolytic aluminum series. For a certain 200kA electrolytic aluminum series (a total of 50 electrolytic cells), a multi-agent collaborative allocation architecture based on MADDPG is built. The specific implementation steps are as follows: Agent division: The 50 electrolytic cells are regarded as independent agents (Agent 1 to Agent 50). Each agent is equipped with an independent sensing unit (same as the sensing module in Example 1) and is responsible for collecting its own cell condition data. A central coordination unit is set up to receive the status data of each agent, transmit collaborative decision-making instructions, and realize information interaction between agents.

[0025] MADDPG algorithm parameter settings: The policy network of each agent uses a 3-layer neural network (128 neurons in the input layer, 256 neurons in the hidden layer, and 1 neuron in the output layer), and the Critic network uses a 3-layer neural network (256 neurons in the input layer, 512 neurons in the hidden layer, and 1 neuron in the output layer); the capacity of the experience replay buffer is set to 10. 6 The batch size is 256; the discount factor γ = 0.98; the exploration rate ε decreases linearly from 0.9 to 0.1, and the decay steps are 5000.

[0026] Collaborative decision-making process: The central coordination unit receives real-time tank condition data (tank temperature Ti, electrolyte concentration Ci, current distribution Ii, tank voltage Ui) uploaded by each agent. Based on the global state space Stotal composition rules, it integrates the local state spaces Si of all agents to construct the global state space Stotal = {S1, S2, ..., S50}. Each agent generates its own current allocation request based on its local state Si and global state Stotal using the MADDPG algorithm. The central coordination unit, based on the total current demand (200kA) and combined with the global optimization objective, collaboratively optimizes the requests of each agent, determines the final current allocation scheme, and distributes it to the execution units of each agent.

[0027] Implementation results: After two months of operation, the local overvoltage rate in the series decreased from 15% to 2.8%, the cold cell problem was resolved at a rate of 96%, the temperature and voltage fluctuations of each electrolytic cell were reduced by 40%, and the overall operational stability of the series was significantly improved, laying the foundation for global power saving optimization.

[0028] Example 3 according to Figure 1 As shown, this embodiment proposes a reinforcement learning-based adaptive current allocation and global energy-saving control system for electrolytic aluminum series. Based on the 200kA electrolytic aluminum series of Embodiment 2, a global energy-saving collaborative control framework is built. The specific implementation steps are as follows: Global State Space Construction: Based on the correspondence between the global state space Stotal and the local state spaces Si, the local state spaces Si of 50 electrolytic cells are integrated to construct the global state space Stotal = {S1, S2, ..., S50}, where the local state Si of each electrolytic cell is {Ti, Ci, Ii, Ui} (Ti is the cell temperature of the i-th electrolytic cell, Ci is its electrolyte concentration, Ii is its anode current distribution, and Ui is its cell voltage). All of the above parameters are collected in real time through the sensing module. At the same time, a series of global parameters such as total current, total energy consumption, and total output are added to form a complete global state dataset. The data update frequency is 1 time / second to ensure real-time synchronization between Stotal and each Si.

[0029] Global optimization model training: The core objective is to maximize the total series reward. , where Ri is the single-cell reward of the i-th electrolytic cell, calculated by the multi-objective reward function of the improved PPO model; the global optimization model is trained with 1500 iterations and a learning rate of 0.0008, and an early stopping strategy is adopted (training is stopped when the validation set loss does not decrease for 50 consecutive iterations) to ensure that the model can achieve a series of overall optimal decisions based on the global state space Stotal.

[0030] Solution to inter-cell coupling effect: Through a global optimization model, the current fluctuation transmission law of each electrolytic cell is analyzed in real time. Based on the real-time data of each local state Si in the global state space Stotal, when an abnormal current fluctuation occurs in a certain electrolytic cell, the model adjusts the current distribution ratio of adjacent electrolytic cells in a timely manner to suppress the transmission of fluctuations. At the same time, the distribution strategy of the total current of the series is optimized, and combined with the real-time operating conditions of each Si, coupling interference caused by excessively high or low current in local electrolytic cells is avoided. This fully meets the technical objective of "solving the inter-cell coupling effect and achieving overall optimization of the series".

[0031] Implementation results: After three months of operation, the overall power saving rate of the series reached 14.5%, which is 8.2 percentage points higher than the power saving rate of the traditional single-slot control mode; the system oscillation rate dropped from 8% to below 0.1%, and the total energy consumption of the series decreased by 14.2%, which verified the effectiveness of global optimization.

[0032] Example 4 according to Figure 1 As shown, this embodiment proposes a reinforcement learning-based adaptive current allocation and global energy-saving control system for electrolytic aluminum series. Based on the 200kA electrolytic aluminum series of Embodiment 3, a multi-objective constraint energy-saving-production-quality balance mechanism is implemented. The specific implementation steps are as follows: Multi-objective parameter definition: Define the evaluation criteria for the three major objectives: energy saving rate ε (target ≥ 12%), aluminum production P (target ≥ 1200t / month), and product purity (target ≥ 99.7%, volatility ≤ 0.05%); Set the value range for each objective: ε∈[10%, 15%], P∈[1150t / month, 1250t / month], Purity∈[99.6%, 99.8%].

[0033] Pareto optimal strategy implementation: The NSGA-III algorithm is used to solve the multi-objective optimization problem, generating a Pareto optimal solution set, from which the optimal decision scheme is selected; the weights of each objective are dynamically adjusted according to real-time operating conditions. For example, when the temperature of a certain electrolytic cell exceeds 970℃, the purity objective weight ω3 is increased from 0.2 to 0.4, reducing the current allocation of that electrolytic cell and prioritizing purity; when the series output is less than 1180t / month, the output objective weight ω2 is increased from 0.3 to 0.4, appropriately increasing the current allocation of key electrolytic cells.

[0034] Balance mechanism verification: Three sets of comparative experiments were set up, using traditional single-objective (power saving) optimization, fixed-weight multi-objective optimization, and dynamic weight Pareto optimal optimization of the present invention, respectively. Each set of experiments was run for 1 month, and the achievement of the three objectives was recorded.

[0035] Implementation Results: Experimental results show that the dynamic weight Pareto optimal optimization scheme of this invention achieves a power saving rate of 12%, an aluminum output of 1237 t / month (an increase of 3.1%), and a product purity of 99.75% (with a volatility of 0.04%), all meeting the target requirements. In contrast, the traditional single-objective optimization scheme achieves a power saving rate of 11.5%, but an output of only 1190 t / month and a purity volatility of 0.08%. The fixed-weight multi-objective optimization scheme achieves a power saving rate of 10.8%, an output of 1220 t / month, and a purity volatility of 0.06%, verifying the superiority of the multi-objective balance mechanism of this invention.

[0036] Example 5 according to Figure 1 As shown, this embodiment proposes a reinforcement learning-based adaptive current allocation and global power-saving control system for electrolytic aluminum series. For the aforementioned 200kA electrolytic aluminum series, a low-latency closed-loop feedback control execution system is built. The specific implementation steps are as follows: Hardware deployment: An industrial Ethernet (Ethernet / IP) network is used to build the data transmission network. A Siemens S7-1500 series PLC is selected as the control core, and an ABB ACS880 series rectifier is selected as the actuator. The sensors of the sensing module are connected to the PLC via shielded cables, and the PLC and rectifier are connected via 4-20mA analog signal transmission to ensure the stability and anti-interference of data transmission.

[0037] Delay control settings: Optimize the transmission parameters of industrial Ethernet, set the data transmission baud rate to 1000Mbps, adopt a priority scheduling mechanism to ensure the priority transmission of sensing data and decision instructions, and control the sensor data transmission delay within 50ms; optimize the program execution efficiency of PLC, simplify the instruction conversion process, and control the instruction execution delay within 80ms; optimize the response parameters of rectifier, and control the current adjustment response time within 20ms.

[0038] Closed-loop feedback process: The sensing module collects the operating condition data of the electrolytic cell and the current data after execution in real time, and transmits it to the PLC via industrial Ethernet. The PLC forwards the data to the reinforcement learning decision module. Based on the updated status data, the decision module generates a new current allocation instruction, which is converted into an analog signal by the PLC and transmitted to the rectifier for execution. After the rectifier executes the instruction, it feeds back the actual current data to the sensing module, forming a complete closed-loop control link.

[0039] Implementation Results: The system's latency performance was tested, and the results showed that the average sensor data transmission latency was 38ms, the average PLC instruction execution latency was 72ms, the average rectifier response latency was 15ms, and the average overall system control response time was 0.42 seconds, meeting the design requirement of ≤0.5 seconds. Compared with the traditional control system (average response time 7.5 seconds), the response speed was improved by 94.4%, ensuring the real-time implementation of reinforcement learning decisions and further improving the control effect.

[0040] Validation data: The present invention has been verified through the above five embodiments. It was continuously operated for six months in a 200kA electrolytic aluminum series (50 electrolytic cells), and all performance indicators met the design requirements. The specific data comparison with the traditional control system is shown in the table below: In summary, this invention effectively solves many technical defects in existing electrolytic aluminum current control, and realizes adaptive current distribution, global power saving, multi-objective collaborative optimization and low-delay control, which significantly improves the efficiency, energy saving level and product quality of electrolytic aluminum production.

[0041] This invention relates to a reinforcement learning-based adaptive current allocation and global energy-saving control system for electrolytic aluminum, adapting to dynamic operating conditions and improving control accuracy. The improved PPO reinforcement learning decision model integrates non-contact magnetic coil measurement technology, solving the problems of low accuracy and poor stability associated with traditional contact measurements. Simultaneously, the GAE algorithm accelerates model convergence, enabling autonomous learning and dynamic optimization of current allocation decisions. This allows for precise adaptation to the dynamic operating conditions of multi-field coupling in the electrolytic cell, ensuring a high degree of matching between current allocation and real-time cell conditions, thus improving control accuracy. Furthermore, this invention achieves multi-agent collaboration, resolving potential localized operational risks. Based on the MADDPG algorithm, a distributed collaborative allocation architecture treats each electrolytic cell as an independent agent, achieving differentiated current allocation for each cell. This specifically addresses issues such as localized overvoltage and cold cells caused by differences in cell conditions in existing technologies, reducing the electrolytic cell failure rate and improving the overall operational stability of the series. This invention employs global optimization decision-making to enhance energy-saving performance. The constructed series of global optimization models resolves inter-cell coupling effects, overcomes the limitations of independent single-cell control, and achieves overall optimization rather than local optimization. This avoids system oscillations caused by local optimization, significantly improving the overall energy-saving rate, far exceeding the energy-saving levels of existing technologies, and substantially reducing energy consumption in electrolytic aluminum production. Furthermore, this invention's multi-objective collaborative optimization fills a gap in the industry, achieving a synergistic balance between energy saving, output, and quality. Through a Pareto optimality strategy, it dynamically weighs each objective, resolving the contradictions of "energy saving without increased production" and "increased production with decreased quality" caused by traditional single-objective optimization. While saving energy, it increases aluminum output and reduces product purity volatility, filling the technological gap in multi-objective collaborative control within the electrolytic aluminum industry. This invention features low-latency closed-loop control, ensuring real-time implementation of decisions. The designed industrial-grade low-latency closed-loop execution system, through the collaboration of hardware and software such as industrial Ethernet and PLC, reduces the control response time from the traditional 5-10 seconds to ≤0.5 seconds, meeting the millisecond-level response requirements for electrolytic aluminum current adjustment. This ensures that reinforcement learning decisions can be implemented in a timely manner, improving the real-time performance and effectiveness of regulation, and further guaranteeing production stability and optimization results.

[0042] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely illustrative of the principles of the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the present invention as claimed. The scope of protection of the present invention is defined by the appended claims and their equivalents.

Claims

1. A current adaptive allocation and global energy-saving control system for electrolytic aluminum series based on reinforcement learning, comprising a sensing module, a reinforcement learning decision-making module, a cooperative control module, an execution module, and a closed-loop feedback module, characterized in that: The sensing module is used to collect real-time operating parameters and anode current distribution data of the electrolytic aluminum series; the reinforcement learning decision module generates current allocation decisions based on the collected data through an improved reinforcement learning algorithm. The collaborative control module is used to realize intelligent agent collaborative decision-making and series-level global optimization of multiple electrolytic cells; the execution module is used to convert decision instructions into execution signals and drive the execution mechanism; the closed-loop feedback module is used to feed back the real-time data after execution to the reinforcement learning decision module to form closed-loop control, realize adaptive adjustment of current distribution and global power saving optimization.

2. The current adaptive allocation and global power-saving control system for electrolytic aluminum series based on reinforcement learning according to claim 1, characterized in that: The sensing module includes a non-contact magnetic induction coil assembly, a temperature sensor, a voltage sensor, and an electrolyte concentration sensor. The non-contact magnetic induction coil assembly is installed at the junction of the upper horizontal busbar of the electrolytic cell and each set of anode guide rods, and the axis of the non-contact magnetic induction coil assembly is perpendicular to the magnetic field direction. It is used to detect the magnetic flux in real time and infer the anode current distribution data. The temperature sensor, voltage sensor, and electrolyte concentration sensor are used to collect the cell temperature, cell voltage, and electrolyte concentration data of the electrolytic cell, respectively.

3. The current adaptive allocation and global power-saving control system for electrolytic aluminum series based on reinforcement learning according to claim 2, characterized in that: The non-contact magnetic induction coil assembly has a measurement accuracy of ≥±0.5% and an operating temperature range of -20℃ to 150℃, making it suitable for the working environment of the electrolytic cell; the inverse formula for the anode current distribution data is: ， Where I is the anode current (A); Φ is the magnetic flux detected by the non-contact magnetic induction coil assembly (Wb); N is the number of turns of the non-contact magnetic induction coil assembly; B is the magnetic field strength (T); and L is the effective length of the non-contact magnetic induction coil assembly (m).

4. The current adaptive allocation and global power-saving control system for electrolytic aluminum series based on reinforcement learning according to claim 1, characterized in that: The reinforcement learning decision-making module employs an improved proximal policy optimization (PPO) algorithm to construct the decision model. The model's state input S includes electrolytic cell operating parameters: cell temperature T, electrolyte concentration C, anode current distribution I, and cell voltage U. The model's action output A is the current distribution ratio α of each electrolytic cell. The model introduces Generalized Advantage (GAE) estimation to reduce variance and accelerate model convergence. The formula for calculating GAE is: ， in, denoted as γ, which is the generalized advantage estimate at time t; γ is the discount factor, ranging from 0.9 to 0.99, representing the degree of decay of future rewards; λ is the bias coefficient, ranging from 0.9 to 0.95, used to balance bias and variance; and k is the iteration step size. Let be the time series difference error at time t+k. , The instant reward at time t+k. Let be the value function of state s at time t+k. Let be the value function of state s at time t+k+1.

5. The current adaptive allocation and global power-saving control system for electrolytic aluminum series based on reinforcement learning according to claim 4, characterized in that: Based on the state input, action output and GAE generalized advantage estimation mechanism of the adaptive decision model, a multi-objective reward function R of the improved PPO model is constructed as the core evaluation criterion for model training and decision optimization, which is used to guide the model to learn a current allocation strategy that meets the needs of multi-objective optimization. The calculation formula is as follows: ， Wherein, ω1, ω2, ω3, and ω4 are the weight coefficients of each objective, and ω1+ω2+ω3+ω4=1, which are dynamically adjusted in real time according to the electrolytic cell operating parameters; η is the single cell current efficiency (unit %), corresponding to the anode current distribution I and cell voltage U in the model state input, reflecting the rationality of current distribution; ε is the series energy saving rate (unit %), corresponding to the energy saving effect of the model action output A; μ is the anode current uniformity (unit %), corresponding to the anode current distribution I in the state input, reflecting the balance of current distribution; s is the tank safety index, with a value range of 0 to 1, where 1 indicates complete safety and 0 indicates potential safety hazards. It corresponds to the tank temperature T and electrolyte concentration C in the state input, ensuring the stable operation of the electrolyzer. This reward function works in synergy with the GAE generalized advantage estimation mechanism to reduce the model training variance and accelerate model convergence by quantifying the benefits of each objective, ensuring that the current allocation decision output by the model is accurately matched with the real-time operating conditions.

6. The current adaptive allocation and global power-saving control system for electrolytic aluminum series based on reinforcement learning according to claim 5, characterized in that: The reinforcement learning decision module employs a Pareto optimal strategy to achieve multi-objective balance. The criteria for determining the Pareto optimal solution are: there are no other decision schemes that make at least one of the three objectives—energy saving rate, output, and purity—better than the current scheme, and the other objectives are not worse than the current scheme; the multi-objective weight adjustment formula is: ， Where ωj is the weight of the j-th target, j=1 corresponds to the energy saving rate, j=2 corresponds to the output, and j=3 corresponds to the purity; fj is the actual value of the current target, fj,min is the minimum value of the target, and fj,max is the maximum value of the target.

7. The current adaptive allocation and global power-saving control system for electrolytic aluminum series based on reinforcement learning according to claim 6, characterized in that: The collaborative control module employs the multi-agent deep deterministic policy gradient (MADDPG) algorithm, treating each electrolytic cell in the aluminum electrolysis series as an independent agent. The policy update formula for each agent is as follows: in, Let be the policy gradient of the i-th agent. Let be the policy parameters for the i-th agent. Let represent the expectation, D be the experience replay buffer, s be the global state, a1~aN be the actions of N agents respectively, r be the immediate reward, and s' be the global state at the next time step. The strategy for the i-th agent to output action ai in state s. Let be the Critic network value function for the i-th agent. Network parameters.

8. The current adaptive allocation and global power-saving control system for electrolytic aluminum series based on reinforcement learning according to claim 7, characterized in that: The collaborative control module constructs a global energy-saving collaborative optimization model for the electrolytic aluminum series. The global state space Stotal of the model has an inclusion and correspondence relationship with the local state space Si of a single electrolytic cell, specifically as follows: The global state space Stotal is composed of the integrated local state spaces Si of all electrolytic cells within the series, i.e., Stotal = {S1, S2, ..., SN}, where N is the total number of electrolytic cells in the electrolytic aluminum series, and Si is the independent local state space of the i-th electrolytic cell. The parameter sources, parameter types, and definitions of the local state space Si of each electrolytic cell completely correspond to the data collected by the sensing module and the state input parameters of the decision model, i.e., Si = {Ti, Ci, Ii, Ui}, where Ti is the cell temperature of the i-th electrolytic cell, Ci is the electrolyte concentration of the i-th electrolytic cell, Ii is the anode current distribution of the i-th electrolytic cell, and Ui is the cell voltage of the i-th electrolytic cell. The global energy-saving collaborative optimization model for the electrolytic aluminum series takes the overall optimization of the series as its core objective, and its global optimization objective is to maximize the total reward of the series. , where Ri is the single-cell reward of the i-th electrolytic cell, calculated by the multi-objective reward function; by integrating the local states Si and corresponding single-cell rewards Ri of all single cells into the global state space Stotal, real-time updates of the global state and global optimization decisions are achieved.

9. The current adaptive allocation and global power-saving control system for electrolytic aluminum series based on reinforcement learning according to claim 1, characterized in that: The execution module includes an industrial Ethernet, a PLC, and a rectifier; the industrial Ethernet is used to transmit data collected by the sensing module and instructions generated by the reinforcement learning decision module, with a transmission delay of ≤50ms; the PLC is used to convert current adjustment instructions into 4-20mA analog signals and transmit them to the current regulator of the rectifier, with an execution delay of ≤100ms; the rectifier is used to adjust the output current according to the analog signal to achieve precise execution of current distribution.

10. The current adaptive allocation and global power-saving control system for electrolytic aluminum series based on reinforcement learning according to claim 1, characterized in that: The feedback delay of the closed-loop feedback module is ≤50ms. It feeds back the data of anode current distribution, cell temperature, cell voltage and electrolyte concentration after execution to the reinforcement learning decision module in real time, so as to update the state space and value function of the model and realize dynamic iterative optimization of decision.

Images