Energy AI agent decision-making method and system based on adaptive learning and loopback control

Abstract

The invention relates to the technical field of artificial intelligence, in particular to an energy AI agent decision-making method and system based on adaptive learning and loopback control. The method comprises the following specific steps: collecting energy data and environment data in real time, and performing data fusion processing; generating an original strategy of the energy AI intelligent agent according to the fused data based on an adaptive learning algorithm; correcting the original strategy in real time based on a preset loopback loss function, and outputting a preliminary strategy instruction; receiving the preliminary strategy instruction, and optimizing the preliminary strategy instruction to generate a final execution instruction under the condition of meeting physical constraints and security constraints of the system; and executing the final execution instruction. By adopting the method, the self-adaptive optimization capability can be realized, the execution safety can be ensured, and the method has the characteristic of supporting continuous optimization.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to an energy AI agent decision-making method and system based on adaptive learning and loop control. Background Technology

[0002] With the rapid development of energy internet technology, the management of complex energy systems such as microgrids, smart buildings, and zero-carbon parks is becoming increasingly difficult. These systems typically include a variety of heterogeneous devices such as distributed photovoltaics, energy storage systems, and flexible loads (such as electric vehicle charging stations and HVAC systems), exhibiting high dynamism, uncertainty, and nonlinearity.

[0003] Existing energy management systems (EMS) typically employ the following control strategies, but all of them have significant limitations: Rule-based control relies on manually preset logic (such as "charge when there is surplus power in the photovoltaic system"). While this method is stable and reliable, it cannot achieve global optimization in the face of complex and ever-changing environments (such as electricity price fluctuations and sudden load changes), resulting in poor economic efficiency and energy performance.

[0004] Traditional model predictive control (MMC) relies on rolling optimization based on physical models. However, constructing accurate physical models is extremely difficult, and solving large-scale non-convex optimization problems online involves a large amount of computation, making it difficult to meet the millisecond-level real-time control requirements.

[0005] Deep reinforcement learning (DRL) employs autonomous learning strategies through interaction with the environment, possessing strong adaptive capabilities. However, the output of end-to-end neural networks is a "black box," lacking interpretability and safety guarantees. In practical industrial control, directly executing actions generated by DRL may lead to system violations of safety constraints (such as voltage overruns and equipment overloads), and it is difficult to guarantee the smoothness of the control process.

[0006] In addition, existing energy systems are often in a state of "data silos", and data from various subsystems cannot be efficiently integrated; and once the system is deployed, the model parameters are usually fixed, lacking the ability to continuously learn and evolve, and cannot adapt to environmental changes caused by equipment aging or expansion.

[0007] Therefore, there is an urgent need for an energy intelligent agent decision-making method and system that can both utilize the adaptive optimization capabilities of artificial intelligence and ensure the safety and stability of execution through control theory. Summary of the Invention

[0008] Therefore, it is necessary to provide an energy intelligent agent decision-making method and system that can simultaneously possess adaptive optimization capabilities and ensure execution safety, and which supports continuous optimization, in order to address the above-mentioned technical problems.

[0009] To achieve the above-mentioned objectives of this invention, the technical solution adopted is as follows: An adaptive learning and loop-loop control-based energy AI agent decision-making method includes the following specific steps: Real-time collection of energy and environmental data, followed by data fusion processing; Based on an adaptive learning algorithm, an initial strategy for the energy AI agent is generated from the fused data; the initial strategy is then corrected in real time based on a preset loop lag loss function, and a preliminary strategy instruction is output. Receive the preliminary strategy instruction, and optimize the preliminary strategy instruction under the condition of satisfying the system physical constraints and security constraints to generate the final execution instruction; Execute the final execution instruction.

[0010] Preferably, after executing the final execution instruction, a status receipt is also generated for federated learning; the specific steps of the federated learning are as follows: Local training: Utilizing the returned status feedback to train a local copy of the adaptive learning algorithm and execute instructions. Perform incremental training and calculate Compared to the previous global model The parameter difference; Secure aggregation: Differential privacy noise is added before uploading parameter differences, and the parameter differences are weighted and averaged according to the sample size weight of each node to update the global model. ; Model fusion: Receive the updated global model and combine it with the local model Perform weighted fusion.

[0011] An energy AI intelligent agent decision-making system based on adaptive learning and loop control includes: The multi-source sensing layer is used to collect energy and environmental data in real time and perform data fusion processing. The decision generation and loop control core DR-DCC is connected to the multi-source perception layer. It includes a decision generation module and a loop control module. The decision generation module is used to generate the original strategy of the energy AI agent based on the fused data using an adaptive learning algorithm. The loop control module is used to correct the original strategy in real time based on a preset loop loss function and output the initial strategy command. The dynamic optimization module is connected to the DR-DCC and is configured to receive the preliminary strategy instructions and optimize the preliminary strategy instructions under the condition of satisfying the system physical constraints and security constraints to generate the final execution instructions. A multimodal execution unit, connected to the dynamic optimization module, is configured to execute the final execution instruction.

[0012] Preferably, the decision generation module is based on an adaptive learning algorithm to generate the original strategy of the energy AI agent based on the fused data. The specific steps are as follows: Constructing the state space of an energy AI agent:

[0013] in, Represents the time step index. This represents the spatiotemporal feature embeddings from the multi-source perception layer, and also represents the latest dynamics of the system. Represents the space of all real numbers; Represents the global context vector; Representing the time-encoded vector, sine and cosine encoding is used to transform time into features that the model can understand, where:

[0014] Represents the projected change in net load in the near future; dimension c It is the total dimension of the last three vectors. ，c= + + ; Constructing the strategy space:

[0015] in, For strategy parameters, For policy networks, To explore noise, It is a continuous action vector representing the strategy, and its dimension m is determined by the total control dimension of all controllable devices; Constructing a reward function :

[0016] in: Represents the total power consumption of the system. Represents real-time electricity price. Represents real-time carbon emission intensity based on marginal emission factors. This represents the comfort deviation, calculated as the weighted sum of the squared differences between the indoor temperature and the set temperature. This represents a constraint reward item, which is a large negative reward when a safety constraint is violated, and 0 otherwise. All of these are adjustable weighting coefficients used to balance the priorities of different objectives; Introducing value network evaluation ,based on Update the value network; The decision generation module adopts a meta-learning mechanism based on the value network for adaptive updates. It updates the strategy on different tasks through an inner loop and searches for initialization parameters that have good performance on all tasks through an outer loop, thereby generating the original strategy of the energy AI agent.

[0017] Furthermore, a meta-learning mechanism is employed to perform adaptive updates based on the value network. The specific steps are as follows: Inner loop: Update and fine-tune the strategy on several different pre-defined tasks. This enables it to achieve higher yields in the current environment. Actions:

[0018]

[0019] in It is a task The loss function on the DDPG is the policy gradient loss. Represents the expected value. Represents the cumulative reward value. The learning rate for the outer loop. Represents the performance metric function. Represents the gradient operator; Outer Loop: After a set number of inner loop updates, the losses from different tasks are aggregated, a primary update is performed, and the most universal solution is found. ;

[0020]

[0021] The result As the policy parameter of the original policy, the corresponding output is... and .

[0022] Furthermore, the loop control module performs real-time correction on the original strategy based on a preset loop loss function and outputs preliminary strategy instructions. The specific steps are as follows: Based on the output of the decision generation module and Construct the laparosional loss function :

[0023] in, and They represent the first Intermediate states and actions in each iteration. Value network estimate With target value The squared temporal difference error characterizes the value deviation of the action; For the current policy distribution Distribution of prior security strategies of KL Divergence represents the degree to which an action deviates from the safety boundary; Represents the regularization coefficient; The initial strategy instructions can be obtained using any of the following methods. : Gradient descent method for direct correction:

[0024] Iterative update method:

[0025] in, It is the gradient of the cyclic control objective function with respect to the action, indicating the optimal adjustment direction. The learning rate of the loopback controller; Output .

[0026] Furthermore, the dynamic optimization module includes an inner-loop optimizer and an outer-loop optimizer; The inner loop optimizer is configured to operate in short cycles, employing the online convex optimization OCP algorithm. Using the system as the reference center, an objective function is constructed and the optimal feasible solution satisfying the system constraints is solved; the objective function is:

[0027]

[0028]

[0029]

[0030] in, This indicates the output action of convex optimization. For immediate cost function, Indicates the weighting coefficient; The objective function is constructed by constructing an augmented Lagrange function. The Alternating Direction Multiplier Method (ADMM) is used for iterative solution.

[0031] The iterative steps are as follows:

[0032] in, As an auxiliary variable, As dual variables, The indicator function for the constraint set; The final action is obtained after solving. ; The outer-loop optimizer is designed for long-term operation and is configured to use a Bayesian optimization (BO) algorithm to optimize the cost function parameters of the inner-loop optimizer and the weight parameters of the lap-loop control module. The specific steps are as follows: Gaussian process regression was used to evaluate system performance indicators. To perform modeling, x represents the vector of hyperparameters that need to be optimized:

[0033] The hyperparameter vectors mentioned include, but are not limited to, those mentioned above. , Any one or more of the sub-weights in the formula; where This represents the updated hyperparameter vector, with the kernel function using the Matern5 / 2 kernel to fit the physical process:

[0034] And utilize the desired improvement to the acquisition function Select the next set of hyperparameter evaluation points: in, For Euclidean distance, The best observation value at present Record and And modify the corresponding parameters.

[0035] Furthermore, it also includes a federated evolution module; the federated evolution module connects the multimodal execution unit and the DR-DCC, and is configured to perform the following operations: Local training: Locally utilizes the state receipts returned by the multimodal execution unit to train a local copy of the model for DR-DCC and the multimodal execution unit. Perform incremental training and calculate Compared to the previous global model The parameter difference; Secure aggregation: Differential privacy noise is added before uploading the parameter differences. The server performs a weighted average of the parameter differences based on the sample size weights of each node, and updates the global model. ; Model fusion: Nodes receive the updated global model and combine it with the local model Perform weighted fusion.

[0036] Furthermore, a local composite loss function is used during the local training phase. include:

[0037] in, The reinforcement learning loss includes the value network, policy network, and loopback correction term;

[0038] Value network loss is used to assess the accuracy of long-term value estimation for loop closure control.

[0039] The policy network loss is used to optimize the control performance of the decision generation module.

[0040] The lap-loop control correction loss is used to evaluate the correction effect of the lap-loop control module.

[0041] To control stability regularization, a term is used to penalize the output difference between the local and global policies:

[0042] The formula for updating local parameters is:

[0043] During the secure aggregation phase, the following computations are performed to generate the next-generation global model: compute the local model difference. And add Gaussian noise that satisfies differential privacy: Based on the data volume of each local node Calculate aggregate weights :

[0044] Global model updated based on weights and differences : During the model fusion phase, after receiving the global model, the local node uses a soft update strategy for fusion:

[0045] in, It is a fusion coefficient.

[0046] Furthermore, the multimodal execution unit includes: a mobile energy dispatching robot configured to perform path planning based on the A* algorithm and dynamic window method, and to perform battery handling and load switching tasks; a fixed smart socket configured to collect effective values ​​of voltage and current and calculate power, and to trigger power-off protection when the monitored value exceeds the rated threshold and the duration exceeds the delay setting; and an environmental regulation robot configured to dynamically adjust the temperature setpoint based on a predictive average voting model, and to adjust the lighting illuminance in conjunction with feedback from the personnel presence sensor.

[0047] The beneficial effects of this invention are as follows: This invention designs an energy AI intelligent agent decision-making system based on adaptive learning and loop control, including a multi-source perception layer, a decision generation and loop control core DR-DCC, a dynamic optimization module, and a multimodal execution unit. By introducing a loop control mechanism into the decision generation and loop control core, this invention ensures that the generated initial strategy actions must undergo real-time evaluation and correction by the loop control module. This mechanism utilizes gradient information to perform instantaneous fine-tuning of the strategy, effectively correcting the output deviation of the AI ​​model and ensuring that the output action is optimal in value assessment and controllable in risk, solving the problem of the lack of safety boundaries in traditional AI decision-making in industrial applications. This invention, through the setting of a dynamic optimization module, uses the corrected actions generated by the AI ​​as reference values ​​and performs online optimization solutions combined with real-time physical constraints, combining the global optimization capability of AI for long-term economic efficiency with the strict constraints of physical safety in numerical optimization. Simultaneously, this invention, through the dynamic optimization module, establishes a complete information flow from "perception-decision-execution-feedback." This collaborative mechanism enables the system to uniformly schedule heterogeneous devices such as photovoltaics, energy storage, air conditioning, and robots, achieving multi-objective collaborative optimization of energy efficiency, economy, and carbon emissions. Attached Figure Description

[0048] Figure 1 A flowchart illustrating an energy AI agent decision-making method based on adaptive learning and loop control; Figure 2 This is a system block diagram of an energy AI intelligent agent decision-making system based on adaptive learning and loop control in one embodiment. Figure 3 Here is a flowchart of the DR-DCC core algorithm in one embodiment; Figure 4 Here is a block diagram of the dynamic optimization module in another embodiment; Figure 5 Here is a flowchart of the federated evolution module in one embodiment; Figure 6 This is a graph showing the convergence of the reward function during training in one embodiment. Detailed Implementation

[0049] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0050] Example 1 like Figure 1 As shown, an energy AI agent decision-making method based on adaptive learning and loop closure control includes the following specific steps: Real-time collection of energy and environmental data, followed by data fusion processing; Based on an adaptive learning algorithm, an initial strategy for the energy AI agent is generated from the fused data; the initial strategy is then corrected in real time based on a preset loop lag loss function, and a preliminary strategy instruction is output. Receive the preliminary strategy instruction, and optimize the preliminary strategy instruction under the condition of satisfying the system physical constraints and security constraints to generate the final execution instruction; Execute the final execution instruction.

[0051] Example 2 like Figure 2 As shown, an energy AI intelligent agent decision-making system based on adaptive learning and loop control includes: The multi-source sensing layer is used to collect energy and environmental data in real time and perform data fusion processing. The decision generation and loop control core DR-DCC is connected to the multi-source perception layer. It includes a decision generation module and a loop control module. The decision generation module is used to generate the original strategy of the energy AI agent based on the fused data using an adaptive learning algorithm. The loop control module is used to correct the original strategy in real time based on a preset loop loss function and output the initial strategy command. The dynamic optimization module is connected to the DR-DCC and is configured to receive the preliminary strategy instructions and optimize the preliminary strategy instructions under the condition of satisfying the system physical constraints and security constraints to generate the final execution instructions. A multimodal execution unit, connected to the dynamic optimization module, is configured to execute the final execution instruction and output a status receipt; The federated evolution module connects the multimodal execution unit and the DR-DCC, and is used to receive the status feedback returned by the multimodal execution unit to perform federated learning on the DR-DCC and the multimodal execution unit.

[0052] In this embodiment, the multi-source sensing layer is specifically configured as follows: Physical forms: multi-protocol sensors, edge computing gateways, IoT terminals Functional Boundaries: Collect various energy and environmental data, and perform cleaning and fusion. Inputs: electrical quantities, environmental quantities, occupancy, price signals, etc. Output: Clean, standardized data stream.

[0053] The specific configuration of the DR-DCC core is as follows: Physical form: Industrial-grade GPU servers, high-end edge computing devices Functional Boundaries: Generate intelligent decisions and provide self-feedback and evaluation of those decisions. Inputs: Standardized data, historical experience, external objectives Output: Preliminary strategy instructions, strategy value and risk assessment.

[0054] The specific configuration of the dynamic optimization module is as follows: Physical form: Embedded FPGA accelerator card, optimized solver container deployed on a server. Functional Boundaries: Ensure the safety and feasibility of instructions, and fine-tune policy parameters in real time through an adaptive learning mechanism. Inputs: Initial strategy, real-time system constraints Output: Safe, reliable, and refined final execution instructions.

[0055] The specific configuration of the multimodal execution unit is as follows: Physical forms: mobile robots, smart sockets, environmental control robots, dimmable LED drivers Functional Boundaries: Precisely translating digital instructions into physical actions. Input: Final execution command Output: Physical action and a receipt of the action execution result.

[0056] The specific configuration of the Federation Evolution Module is as follows: Physical form: cloud server cluster, or regional data center server acting as an aggregation node. Functional Boundaries: Through an adaptive learning framework, all terminals can share knowledge and evolve together, achieving continuous system optimization. Input: Learning notes from the local models on each terminal.

[0057] In one specific embodiment, the multi-source sensing layer further includes a data fusion unit and a data uploading unit: The data fusion unit is used to perform multi-source data fusion of the energy data E(t) and environmental data C(t) using Kalman filtering and attention mechanisms. ; The data upload unit is used to compress the fused data through edge nodes and upload it to the DR-DCC.

[0058] In a specific embodiment, the core algorithm flow of DR-DCC is as follows: Figure 3 As shown, the decision generation module, based on an adaptive learning algorithm, generates the original strategy of the energy AI agent according to the fused data. The specific steps are as follows: Constructing the state space of an energy AI agent:

[0059] in, Represents the time step index. This represents the spatiotemporal feature embeddings from the multi-source perception layer, and also represents the latest dynamics of the system. Represents the space of all real numbers; Represents the global context vector; Representing the time-encoded vector, sine and cosine encoding is used to transform time into features that the model can understand, where:

[0060] Represents the projected change in net load in the near future; dimension c It is the total dimension of the last three vectors. ，c= + + ; Constructing the strategy space:

[0061] in, For strategy parameters, For policy networks, To explore noise, It is a continuous action vector representing the strategy, and its dimension m is determined by the total control dimension of all controllable devices; Constructing a reward function :

[0062] in: Represents the total power consumption of the system. Represents real-time electricity price. Represents real-time carbon emission intensity based on marginal emission factors. This represents the comfort deviation, calculated as the weighted sum of the squared differences between the indoor temperature and the set temperature. This represents a constraint reward item, which is a large negative reward when a safety constraint is violated, and 0 otherwise. All of these are adjustable weighting coefficients used to balance the priorities of different objectives; Introducing value network evaluation ,based on Update the value network; The decision generation module adopts a meta-learning mechanism based on the value network for adaptive updates. It updates the strategy on different tasks through an inner loop and searches for initialization parameters that have good performance on all tasks through an outer loop, thereby generating the original strategy of the energy AI agent.

[0063] In one specific embodiment, such as Figure 4 As shown, an adaptive update based on the value network is performed using a meta-learning mechanism. The specific steps are as follows: Inner loop: Update and fine-tune the strategy on several different pre-defined tasks. This enables it to achieve higher yields in the current environment. Actions:

[0064]

[0065] in It is a task The loss function on the DDPG is the policy gradient loss. Represents the expected value. Represents the cumulative reward value. The learning rate for the outer loop. Represents the performance metric function. Represents the gradient operator; Outer Loop: After a set number of inner loop updates, the losses from different tasks are aggregated, a primary update is performed, and the most universal solution is found. ;

[0066]

[0067] The result As the policy parameter of the original policy, the corresponding output is... and .

[0068] In this embodiment, after internal and external circulation, In one specific embodiment, the loop closure control module performs real-time correction on the original strategy based on a preset loop closure loss function and outputs preliminary strategy instructions. The specific steps are as follows: Based on the output of the decision generation module and Construct the laparosional loss function :

[0069] in, and They represent the first Intermediate states and actions in each iteration. Value network estimate With target value The squared temporal difference error characterizes the value deviation of the action; For the current policy distribution Distribution of prior security strategies of KL Divergence represents the degree to which an action deviates from the safety boundary; Represents the regularization coefficient; The initial strategy instructions can be obtained using any of the following methods. : Gradient descent method for direct correction:

[0070] Iterative update method:

[0071] in, It is the gradient of the cyclic control objective function with respect to the action, indicating the optimal adjustment direction. The learning rate of the loopback controller; Output .

[0072] In one specific embodiment, such as Figure 4 As shown, the dynamic optimization module includes an inner loop optimizer and an outer loop optimizer; The inner loop optimizer is configured to operate in short cycles, employing the online convex optimization OCP algorithm. Using the system as the reference center, an objective function is constructed and the optimal feasible solution satisfying the system constraints is solved; the objective function is:

[0073]

[0074]

[0075]

[0076] in, This indicates the output action of convex optimization. For immediate cost function, Indicates the weighting coefficient; This embodiment includes constraints, primarily device-level constraints (mainly inequality constraints): Power upper and lower limits: (Energy storage charging and discharging power limitations) Capacity constraints: (State of charge limitation for energy storage) Slope rate constraint: (Energy storage power change rate limit) Temperature setting range:

[0077] Including system-level constraints (equality and inequality constraints): Power balance equation (crucial):

[0078] Grid interaction power limits:

[0079] Bus voltage constraint: .

[0080] In this embodiment, the global problem is decomposed into multiple subproblems that can be solved in parallel:

[0081]

[0082]

[0083] in, It is the indicator function of the constraint set (its value is 0 when the constraint is satisfied, otherwise it is infinity), and z is the introduced auxiliary variable; The objective function is constructed by constructing an augmented Lagrange function. The Alternating Direction Multiplier Method (ADMM) is used for iterative solution.

[0084] The iterative steps are as follows:

[0085] in, As an auxiliary variable, As dual variables, The indicator function for the constraint set; The final action is obtained after solving. ; The outer-loop optimizer is designed for long-term operation and is configured to use a Bayesian optimization (BO) algorithm to optimize the cost function parameters of the inner-loop optimizer and the weight parameters of the lap-loop control module. The specific steps are as follows: Gaussian process regression was used to evaluate system performance indicators. To perform modeling, x represents the vector of hyperparameters that need to be optimized:

[0086] The hyperparameter vectors mentioned include, but are not limited to, those mentioned above. , Any one or more of the sub-weights in the formula; where This represents the updated hyperparameter vector, with the kernel function using the Matern5 / 2 kernel to fit the physical process:

[0087] And utilize the desired improvement to the acquisition function Select the next set of hyperparameter evaluation points: in, For Euclidean distance, The best observation value at present Record and And modify the corresponding parameters.

[0088] In this embodiment, the iteration process of BO is as follows: 1. Initialization: Randomly select a small number of hyperparameter combinations x1, x2, ..., xn and run them on the system for a period of time, collecting performance data f(x1), ..., f(xn).

[0089] 2. Iterate until convergence: Fit Gaussian process model using all historical data {xi, f(xi)}. Maximize acquisition function: Find the point xnext that maximizes EI(x). Assessment: Configure the system as After running for a period of time, performance observations were obtained. Update the dataset: ( Add to historical datasets.

[0090] In one specific embodiment, such as Figure 5 As shown, it also includes a federated evolution module; the federated evolution module connects the multimodal execution unit and the DR-DCC, and is configured to perform the following operations: Local training: Locally utilizes the state receipts returned by the multimodal execution unit to train a local copy of the model for DR-DCC and the multimodal execution unit. Perform incremental training and calculate Compared to the previous global model The parameter difference; Secure aggregation: Differential privacy noise is added before uploading the parameter differences. The server performs a weighted average of the parameter differences based on the sample size weights of each node, and updates the global model. ; Model fusion: Nodes receive the updated global model and combine it with the local model Perform weighted fusion.

[0091] In one specific embodiment, a local composite loss function is used during the local training phase. include:

[0092] in, The reinforcement learning loss includes the value network, policy network, and loopback correction term;

[0093] Value network loss is used to assess the accuracy of long-term value estimation for loop closure control.

[0094] The policy network loss is used to optimize the control performance of the decision generation module.

[0095] The lap-loop control correction loss is used to evaluate the correction effect of the lap-loop control module.

[0096] To control stability regularization, a term is used to penalize the output difference between the local and global policies:

[0097] The formula for updating local parameters is:

[0098] During the secure aggregation phase, the following computations are performed to generate the next-generation global model: compute the local model difference. And add Gaussian noise that satisfies differential privacy: Based on the data volume of each local node Calculate aggregate weights :

[0099] Global model updated based on weights and differences : In this embodiment, to further prevent the original data from being derived from the model difference, the original data can be extracted before uploading. Add differential privacy noise.

[0100] Gaussian mechanism: The client uploads a model update with added noise.

[0101]

[0102] The standard deviation of noise With privacy budget Related. Noise is partially canceled out during aggregation, but strong privacy protection is still provided.

[0103] During the model fusion phase, after receiving the global model, the local node uses a soft update strategy for fusion:

[0104] in, It is a fusion coefficient.

[0105] In this embodiment, federated evolution is not a continuous process, but is initiated by a triggering mechanism: 1. Performance evaluation trigger: When the client detects that the performance of the local model (such as average reward) has plateaued or significantly declined over a period of time, it actively sends an evolution request to the server.

[0106] 2. Periodic triggering: Federated aggregation is automatically performed every T hours (e.g., 24 hours).

[0107] 3. Server Coordination: When the server receives enough requests (e.g., more than 50% of clients) or when a timer is reached, it initiates a new round of federated evolution.

[0108] In one specific embodiment, the multimodal execution unit includes: a mobile energy dispatching robot configured to perform path planning based on the A* algorithm and dynamic window method, and to perform battery handling and load switching tasks; a fixed smart socket configured to collect effective values ​​of voltage and current and calculate power, and to trigger power-off protection when the monitored value exceeds the rated threshold and the duration exceeds the delay setting; and an environmental regulation robot configured to dynamically adjust the temperature setpoint based on a predictive average voting model, and to adjust the lighting illuminance in conjunction with feedback from the personnel presence sensor.

[0109] In this embodiment, the mobile energy dispatching robot is specifically: ①Core Functions: Navigation and positioning: Autonomously move to the target location.

[0110] Battery handling: Using robotic arms to pick up, place, and exchange batteries.

[0111] Load switching: Physically connecting unplanned loads (such as standby generators).

[0112] ② Calculation method: 1. Route planning and navigation Global path planning (A* algorithm): Cost function:

[0113] : The actual cost (e.g., travel distance) from the starting point to node n.

[0114] : The estimated cost from node n to the destination, usually expressed as Euclidean distance or Manhattan distance.

[0115]

[0116] The penalty item is used to avoid energy hotspots or high-interference areas (dynamically updated based on real-time map information provided by the communication module).

[0117] Local Motion Planning and Obstacle Avoidance (Dynamic Window Method DWA): Sample multiple sets of velocities in the velocity space (v,ω) (linear velocity, angular velocity), simulate the trajectory over a short period of time, and select an optimal velocity.

[0118] Objective function:

[0119] heading(v,ω): The alignment of the trajectory endpoint with the target point.

[0120] dist(v,ω): Distance on the trajectory from the nearest obstacle.

[0121] velocity(v,ω): An evaluation value of the current sampling speed, encouraging rapid movement.

[0122] σ: Normalization function.

[0123] α,β,γ: weighting coefficients.

[0124] In this embodiment, the battery scheduling optimization model is specifically as follows: The instructions received by AMR-ES are essentially a scheduling problem: moving a battery from point A to point B. Its local decision point lies in selecting the optimal execution sequence (if multiple tasks are queued). Its objective function is to minimize the total task completion time. in This is the completion time of task i. It is limited by: Battery power constraints: (The schedulable battery must be above the minimum charge level) Workspace constraints for robotic arms.

[0125] Timing constraints between tasks.

[0126] Real-time scheduling is typically achieved using heuristic rules (such as Early Deadline First, EDF) or small-scale online integer programming.

[0127] In this embodiment, the fixed smart socket robot is specifically: ①Core Functions: Millisecond-level on / off control: Executes the power supply and power off of the device.

[0128] High-precision metering: Real-time monitoring of voltage, current, power, and electricity consumption of connected devices.

[0129] Local protection: overcurrent, overload, and short circuit protection.

[0130] ② Calculation method: Power calculation Real-time digital signal processing is employed. RMS values ​​of voltage / current:

[0131] Where N is the number of sampling points within a period.

[0132] Active power:

[0133] Energy accumulation:

[0134] Overload protection logic Judgment conditions: If ( ) and ( then trip Socket rated current.

[0135] Overload factor.

[0136] Overload duration.

[0137] Inverse time delay or fixed delay.

[0138] Status feedback S-Plug encapsulates metering data into status information using the following formula and reports it periodically:

[0139] In this embodiment, the environmental regulation robot is specifically: ①Core Functions: Multivariable control: Closed-loop regulation of temperature, humidity, illuminance, air quality, etc.

[0140] Human-based optimization: Based on human presence sensors, it enables the linkage of "powering on when people arrive and saving energy when people leave".

[0141] ② Calculation method: Multivariable PID control Taking temperature control as an example, an incremental digital PID algorithm is used: in: : This is the output increment (such as the change in compressor frequency).

[0142] Temperature difference between the current moment, the previous moment, and the moment before that ( ).

[0143] , , The proportional, integral, and differential coefficients are usually initialized by a self-tuning algorithm.

[0144] Comfort Model and Energy Efficiency Trade-off EnvBot's setpoint is not fixed, but dynamically fine-tuned based on a Predicted Average Voting (PMV) model to strike a balance between comfort and energy efficiency. Simplified PMV calculation formula: Where M is the human metabolic rate, W is the work done, and H is the net heat loss of the human body (a complex function of temperature, humidity, wind speed, and clothing temperature). Control strategy: Within the range of |PMV| < 0.5 (slightly insufficient), slightly fluctuating the set temperature (e.g., increasing it from 24℃ to 25℃ in summer) can achieve significant energy-saving effects.

[0145] Personnel-activated lighting control Illumination control based on sensor fusion (PIR infrared motion + millimeter-wave radar presence sensing):

[0146] if If there is someone, then f(P) = 1, and the output is the set illuminance.

[0147] if (If no one is present), then f(P) = 0 or f(P) = 0.2 (maintain minimum illumination).

[0148] In this embodiment, all execution units feed back the execution results to the system through a unified state estimator.

[0149] Due to delays and noise in execution and sensing, the state estimator needs to use KF for state fusion estimation.

[0150] Equations of state:

[0151] Observation equation: in: : State vector (e.g., robot position, socket power, room temperature).

[0152] : Control input.

[0153] : Observation vector (sensor reading).

[0154] Process noise and observation noise.

[0155] KF outputs the optimal state estimate through a two-step prediction-update loop. This forms a loop, serving as the final basis for judging the success of the execution and providing feedback data.

[0156] Example 3 This embodiment is applied to a zero-carbon smart industrial park in Jiangsu Province. The park includes a distributed photovoltaic system (1.8MWp), a lithium iron phosphate energy storage system (2.4 MWh), office buildings (air conditioning and lighting loads), and electric vehicle charging stations.

[0157] The energy AI intelligent agent decision-making system based on adaptive learning and loopback control described in this embodiment is physically deployed as follows: Multi-source sensing layer: Deploys 185 multi-protocol sensor nodes to collect photovoltaic power generation, energy storage SOC, building temperature and grid PCC point switching power; data is aggregated through edge computing gateway (based on Jetson Orin NX).

[0158] DR-DCC Core and Dynamic Optimization Module: Deployed in the campus data center on a GPU server (configured with NVIDIA A100) to run deep reinforcement learning models and convex optimization solvers.

[0159] Multimodal execution unit: including 6 mobile energy dispatching robots (AMR-ES, used to transport mobile energy storage modules), 120 smart sockets and 45 environmental control robots (interfacing with the central air conditioning BA system).

[0160] Federation Evolution Module: Connects to the cloud aggregation server and forms a federated learning network with nodes in other campuses.

[0161] In this embodiment, the system operation process is divided into five stages: data perception, decision generation and loopback correction, dynamic optimization, execution and evolution. The following example, taken at 14:00 on a summer day during the peak electricity price period, illustrates how the system performs peak shaving and valley filling and energy efficiency optimization.

[0162] Step 1: Multi-source sensing and data fusion. The multi-source sensing layer collects data in real time at the current moment. The data.

[0163] Data collection: Photovoltaic output dropped sharply to 800kW (cloudy), total load of the park was 3200kW, real-time electricity price was at its peak (1.2 yuan / kWh), and average indoor temperature was 26.5℃.

[0164] Data fusion: The edge gateway uses Kalman filtering to remove sensor noise and attention mechanisms to align data of different frequencies, generating a standardized state data stream.

[0165] Data Upload: The merged data is serialized into Protobuf format and uploaded to the DR-DCC core.

[0166] Step 2: Decision Generation and Loop Correction (DR-DCC) State Construction: The decision generation module constructs state vectors.

[0167] Time encoding According to the formula Calculate and accurately capture the time characteristic of "2 PM".

[0168] Preliminary Action Generation: Policy Network Based on DDPG Algorithm Generate preliminary strategy .

[0169] Preliminary strategy: Discharge energy storage at full power (-500kW), and raise the air conditioning set temperature to 27℃.

[0170] At this time, the reward function A comprehensive calculation of electricity costs, carbon emissions, and comfort penalties: .

[0171] Loopback control correction: The loopback control module starts immediately and calculates the loopback loss function. : The system detected that "full power discharge of energy storage" may cause the battery temperature to overheat (violating safety priors). ),lead to The divergence term increases. The module uses gradient descent. Fine-tune the movements. Corrected movements. The energy storage discharge power is adjusted to -420kW, and the air conditioner temperature is set to 27℃.

[0172] In this embodiment, the convergence process of the reward function is as follows: Figure 6 As shown: Training phase: 0-24 days Initial reward value: -18.6 (Day 1) Convergence reward: -4.2 ± 0.15 (after day 24) Convergence episodes: 22,500 The convergence curve follows an exponential decay law:

[0173] Step 3: Dynamic optimization (OCP + BO) of the corrected actions It is sent to the dynamic optimization module to ensure physical feasibility.

[0174] Inner loop online convex optimization (OCP): Using the reference center, construct the objective function: Constraints include power balance equations The ADMM algorithm is used to iteratively solve for the augmented Lagrangian function. [cite: 296], converges to obtain the final instruction within 20ms. Final directive Energy storage discharge: 415kW; the shortfall is supplemented by a mobile robot connected to a backup battery. Outer loop Bayesian optimization (BO): Every 15 minutes, the outer loop uses Gaussian process regression (GPR) to model the system's long-term energy efficiency index. The kernel function uses Matern 5 / 2: BO automatically adjusts the weight parameters in the inner loop optimization based on the desired improvement (EI) acquisition function. To adapt to the fluctuating characteristics of today's cloudy weather.

[0175] Step 4: Multimodal Execution The multimodal execution unit receives instructions. : Energy storage system: Performs 415kW constant power discharge.

[0176] Environmental robot: smoothly transitions the office area's air conditioning setpoint to 27°C and turns off the lighting in the unmanned meeting room via infrared sensing.

[0177] Mobile robot: Automatically navigates to area B and connects a fully charged battery module to the microgrid interface to fill the power gap.

[0178] Step 5: During the nighttime off-peak hours of Federation Evolution, the system enters Federation Evolution mode.

[0179] Local training: Update the local model using data from the entire day's operation and calculate the loss function. The regularization term Ensure that local policies do not deviate from global security baselines.

[0180] Secure aggregation: Computational model difference Add differential privacy Gaussian noise Upload later.

[0181] Model fusion: Receives a new global model aggregated from the cloud. and with coefficient Perform a soft update and integration to complete the system evolution.

[0182] Table 1 shows a comparison of the energy efficiency indicators of this intelligent agent decision-making system with those of traditional rule-based control strategies. Table 1

[0183] The system performance indicators of this intelligent agent decision-making system are shown in Table 2: Table 2

[0184] Table 3 shows a comparison of the reflection optimization performance of this intelligent agent decision-making system with that of traditional rule-based control strategies. Table 3

[0185] The economic and environmental benefits of the intelligent agent decision-making system are shown in Table 4: Table 4

[0186] In summary, after six months of operation in the park, the system demonstrated significant improvements in energy efficiency: a 19.3% reduction in overall energy consumption and a 25.4% reduction in peak load; a marked reduction in carbon emissions: a 22.8% reduction, exceeding the demonstration zone's emission reduction target; stable and reliable operation: 99.97% system availability and 99.88% decision accuracy; outstanding economic benefits: a 1.62-year payback period and an annualized return of 61.7%; and excellent user experience: an employee satisfaction survey showed a 91% satisfaction rate, with no voltage overruns or equipment overloads, validating the effectiveness of loop control and dynamic optimization.

Claims

1. An energy AI agent decision-making method of adaptive learning and loop control, characterized in that, The method comprises the following specific steps: Real-time acquisition of energy data and environmental data, and data fusion processing; Based on the adaptive learning algorithm, the original strategy of the energy AI agent is generated according to the data after fusion processing; the original strategy is corrected in real time based on the preset loop loss function, and a preliminary strategy instruction is output; Receiving the preliminary strategy instruction, and optimizing the preliminary strategy instruction under the condition of meeting the system physical constraints and safety constraints to generate a final execution instruction; Executing the final execution instruction.

2. The method of claim 1, wherein, After executing the final execution instruction, a state reply is also generated, and federated learning is performed; the specific steps of the federated learning are: Local training: locally utilizing the returned status receipts to adapt the learning algorithm and the local model copy of the execution instructions Incremental training, computing the parameter difference from the last round of global model ​ Secure aggregation: add differential privacy noise before uploading parameter difference, weight the parameter difference according to the sample size of each node, and update the global model ; Model fusion: receive updated global model and weight it with the local model fusion.

3. An energy AI agent decision system based on adaptive learning and loop control, characterized in that, Comprising: A multi-source perception layer for real-time acquisition of energy data and environmental data, and data fusion processing; A decision generation and loop control core DR-DCC connected to the multi-source perception layer, which internally comprises a decision generation module and a loop control module; the decision generation module is used to generate the original strategy of the energy AI agent based on the adaptive learning algorithm according to the data after fusion processing; the loop control module is used to correct the original strategy in real time based on the preset loop loss function, and output a preliminary strategy instruction; A dynamic optimization module connected to the DR-DCC, configured to receive the preliminary strategy instruction, and optimize the preliminary strategy instruction under the condition of meeting the system physical constraints and safety constraints to generate a final execution instruction; A multi-modal execution unit connected to the dynamic optimization module, configured to execute the final execution instruction.

4. The system of claim 3, wherein, The decision generation module generates the original strategy of the energy AI agent based on the adaptive learning algorithm according to the data after fusion processing, and the specific steps are: Constructing the state space of the energy AI agent: wherein, represents a time step index, represents a spatio-temporal feature embedding from the multi-source perception layer, also representing the latest dynamics of the system, represents the entire real space; represents the global environmental context vector; represents a time encoding vector, using a sine-cosine encoding to transform time into a model understandable feature, wherein: net load prediction change amount representing a future short-term; dimension c is the total dimension of the last three vectors ，c= + + ; Constructing the strategy space: wherein, is a policy parameter, is a policy network, is an exploration noise, is a continuous action vector representing a policy, whose dimension m is determined by the total control dimension of all controllable devices; Constructing a reward function : wherein: represents the total electricity power consumption of the system, represents the real-time electricity price, represents the real-time carbon emission intensity based on marginal emission factor, represents the comfort deviation, calculated as the weighted sum of squared differences between indoor temperature and set temperature, represents the constraint reward term, which is a large negative reward when the safety constraint is violated, otherwise 0; are all adjustable weight coefficients to balance the priority of different objectives; Introducing a value network evaluation , based on updating the value network; The decision generation module adopts a meta-learning mechanism to perform adaptive update based on a value network, updates the strategy on different tasks through inner loop, and finds initialization parameters that have good performance on all tasks through outer loop, thereby generating the original strategy of the energy AI agent.

5. The system of claim 4, wherein, Adopting a meta-learning mechanism to perform adaptive update based on a value network, the specific steps are: Inner loop: update policy on a number of different tasks separately, fine-tuning so that it can produce higher action: wherein is a loss function on the task , i.e. the policy gradient loss of DDPG, represents an expected value, represents a cumulative reward value, is a learning rate for the outer loop, denotes a performance metric function, denotes a gradient operator; Outer loop: After a certain number of inner loop updates, the losses on different tasks are aggregated and a meta update is performed to find the most general ; The obtained As the strategy parameter of the original strategy, output the corresponding and .

6. The system of claim 5, wherein, The loop control module corrects the original strategy in real time based on the preset loop loss function, and outputs a preliminary strategy instruction, and the specific steps are: based on the output of the decision generation module and , a loopback loss function is constructed : wherein, and respectively represent the intermediate state and action in the value network estimate value time series difference error square of the target value , which characterizes the value deviation of the action; is the current policy distribution the divergence of the prior safe policy distribution KL , which characterizes the degree of action deviation from the safe boundary; denotes the regularization coefficient;​ The preliminary policy instructions are obtained using either of the following methods : Gradient descent method directly corrects: Iterative update method: wherein, is the gradient of the loop control objective function with respect to the action, indicating the optimal adjustment direction, is the learning rate of the loop controller; Output .

7. The system of claim 6, wherein, The dynamic optimization module includes an inner loop optimizer and an outer loop optimizer; The inner loop optimizer is configured to run in a short cycle, using an online convex optimization (OCP) algorithm to As a reference center, construct a target function and solve the optimal feasible solution that satisfies the system constraints; the target function is: wherein, represents the output action of the convex optimization, is the instantaneous cost function, represents the weight coefficient; The objective function is constructed by building an augmented Lagrangian function and is solved iteratively using the alternating direction method of multipliers (ADMM): The iterative steps are: wherein, is an auxiliary variable, is a dual variable, is an indicator function of the constraint set; Solving the equation results in the final execution action ; The outer loop optimizer is used to work in a long cycle, and is configured to use a Bayesian optimization BO algorithm to optimize the cost function parameters of the inner loop optimizer and the weight parameters of the loop control module, and the specific steps are: Modeling system performance indicators with Gaussian process regression x representing the vector of hyperparameters to optimize: The hyperparameter vector includes but is not limited to , Any one or more of the sub-weights in ; wherein represents the updated hyperparameter vector, and the kernel function employs a Matern 5 / 2 kernel to fit the physical process: and using the desired improvement acquisition function select the next set of hyperparameter evaluation points: wherein, is the Euclidean distance, is the current best observation record and and modify the corresponding parameters.

8. The system of any one of claims 3 to 7, wherein, It also includes a federated evolution module; the federated evolution module is connected to the multi-modal execution unit and the DR-DCC, and is configured to perform the following operations: Local training: locally using the state receipt returned by the multi-modal execution unit to update the local model copies of the DR-DCC and the multi-modal execution unit Incremental training, computing the parameter difference with the global model of the previous round Secure aggregation: add differential privacy noise before uploading parameter difference, and the server side weights the parameter difference according to the sample size of each node to update the global model ; Model fusion: the node receives the updated global model and weights it with the local model for a weighted fusion.

9. The system of claim 8, wherein, The local composite loss function employed in the local training phase comprises: wherein, is a reinforcement learning loss comprising a value network, a policy network, and a loop correction term; Loss for the value network to assess the accuracy of the recurrent control long-term value estimate: For the policy network loss, the control performance of the decision generation module is optimized: To correct the loss of the loop control, a correction effect evaluation module is used to evaluate the correction effect of the loop control module: To control the stability regularizer term, which is used to penalize the output difference between the local policy and the global policy: The local parameter update formula is: At the secure aggregation phase, the following computation is performed to generate the new generation global model: compute local model difference and add Gaussian noise that satisfies differential privacy: According to the amount of data of each local node Computing aggregation weights : Updating a global model based on weights and differential updates : In the model fusion stage, after the local node receives the global model, a soft update strategy is adopted for fusion: wherein is a fusion coefficient.

10. The system of claim 3, wherein, The multi-modal execution unit includes: a mobile energy scheduling robot configured to perform battery transportation and load switching tasks based on path planning using the A* algorithm and the dynamic window method; a fixed intelligent socket configured to collect voltage and current effective values and calculate power, and trigger power-off protection when the monitored values exceed the rated threshold and the duration exceeds the delay setting; an environmental regulation robot configured to dynamically adjust the temperature set value based on the predictive average voting model, and adjust the lighting illuminance in combination with the feedback of the personnel presence sensor.

Images