Large-scale public building demand response intelligent regulation and control method based on multi-mode reinforcement learning agent system

Abstract

The invention discloses a large-scale public building demand response intelligent regulation and control method based on a multi-mode reinforcement learning agent system. According to the method, a multi-modal reinforcement learning agent system is constructed by introducing a multi-modal data processing and multi-agent cooperation mechanism; the multi-modal reinforcement learning agent system is deployed in the large-scale public building, and cooperative control of the multi-modal reinforcement learning agent system is realized through a distributed execution mechanism; and cooperatively solving the demand response double-layer regulation and control model of the large-scale public building in the electricity-carbon-green certificate market based on a multi-mode reinforcement learning agent system, thereby regulating and controlling key adjustable and controllable equipment, and realizing participation of the large-scale public building in demand response regulation and control of the electricity market, the carbon market and the green certificate market. The method has the advantages that through multi-modal data fusion, the operation and maintenance state sensing capability of the large-scale public building is improved, and the adaptability of the large-scale public building to a dynamic market and a complex environment is enhanced.

Description

Technical Field

[0001] The present invention belongs to the intersection of large-scale public building energy systems and artificial intelligence, and specifically relates to a large-scale public building demand response intelligent control method based on a multimodal reinforcement learning intelligent agent system. Background Art

[0003] In recent years, the rise of artificial intelligence technology, especially reinforcement learning, has provided a new path for the intelligent optimization of complex systems. Reinforcement learning can adaptively optimize control strategies through interactive learning, and multimodal reinforcement learning further combines the characteristics of multi-source heterogeneous data in buildings (such as environmental status, equipment operating status, and market price dynamics), providing an efficient solution for demand response in large public buildings. In addition, through multi-agent collaborative reinforcement learning technology, unified optimization of global and local goals can be achieved in the multi-level structure of the building (such as the market layer and the equipment layer). While supporting buildings to participate in electricity market transactions, carbon markets, and green certificate markets, this method can dynamically control internal building equipment (such as air conditioning, heating, lighting systems, etc.) to maximize energy efficiency, minimize operating costs, and improve environmental benefits.

[0004] In response to the above challenges, how to use artificial intelligence to collaboratively optimize the operation and maintenance control strategy of large-scale building integrated energy systems, thereby improving the operation and maintenance efficiency and safety and reliability of large-scale building integrated energy systems, achieving green energy saving, and at the same time, ensuring the normal operation and maintenance safety and comfort of large buildings in a variety of scenarios, is an urgent problem to be solved. Summary of the invention

[0005] The present invention proposes an intelligent control method for demand response of large public buildings based on a multimodal reinforcement learning intelligent agent system, aiming to overcome the limitations of traditional methods. By introducing multimodal data processing and multi-agent collaborative mechanism, an intelligent agent system that can adapt to the dynamic market and complex operating environment of electricity-carbon-green certificate is constructed, providing an efficient and intelligent solution for large public buildings to participate in demand response in the electricity-carbon-green certificate market.

[0006] In order to achieve the above object, the present invention provides the following technical solutions:

[0007] A method for intelligent control of demand response of large public buildings based on a multimodal reinforcement learning agent system comprises the following steps:

[0008] S1, according to the operation and maintenance characteristics of large public buildings, determine the scope of demand response regulation of large public buildings and identify key adjustable equipment; according to the demand response regulation scope and key adjustable equipment, determine the types of intelligent agents involved and their respective functions, and obtain a multimodal reinforcement learning intelligent agent system;

[0009] S2, using intelligent sensors to realize the perception and collection of multimodal data of large public buildings, constructing a multimodal data set D of large public buildings, and extracting features from the set D to obtain a multimodal data feature set F of large public buildings; combining the existing large language model and visual language model to preliminarily construct each agent of the multimodal reinforcement learning agent system in step S1, and using a deep reinforcement learning algorithm to preliminarily train each agent;

[0010] S3, according to the multimodal data feature set F and the agents constructed in step S2, a deep learning reinforcement algorithm is used to centrally train the multimodal reinforcement learning agent system, and the coordinated control of the agents constructed in step S2 is realized based on a distributed execution mechanism;

[0011] S4, deploying the multimodal reinforcement learning agent system into the large public building;

[0012] S5, the multimodal reinforcement learning intelligent agent system participates in the intelligent regulation of demand response of large public buildings. The specific method is: establish a two-layer regulation model of demand response of large public buildings in the electricity-carbon-green certificate market, collaboratively solve the two-layer regulation model of demand response of large public buildings in the electricity-carbon-green certificate market based on the multimodal reinforcement learning intelligent agent system, and regulate the key adjustable equipment in step S1 based on the model solution results; and maintain the multimodal reinforcement learning intelligent agent system.

[0013] Furthermore, in step S1:

[0014] The scope of demand response regulation for large public buildings includes electricity market demand response regulation, carbon market demand response regulation, and green certificate market demand response regulation. The key controllable equipment includes cold and heat source power equipment, central air-conditioning equipment, energy storage equipment, lighting equipment, and renewable energy equipment.

[0015] The multimodal reinforcement learning agent system includes an upper-layer agent system, a lower-layer agent system and a communication and coordination agent system; the upper-layer agent system and the lower-layer agent system perform global collaborative operation through the communication and coordination agent system;

[0016] The upper-level intelligent agent system includes the power market demand response agent, the carbon market demand response agent, and the green certificate market demand response agent;

[0017] The lower-level intelligent agent system includes environmental perception agent, energy management agent, user behavior agent, fault detection and maintenance agent, strategy generation agent, and multimodal data fusion agent.

[0018] Each intelligent agent has different functional characteristics, and the corresponding intelligent agent can be expanded and constructed according to the needs of different scenarios.

[0019] The functions of the upper-level agent system are:

[0020] 1) The power market demand response agent minimizes electricity costs by optimizing power procurement and power control strategies, supporting grid stability and improving building economic benefits;

[0021] 2) The carbon market demand response agent is responsible for monitoring and optimizing the carbon emissions of buildings, reducing carbon emission costs and promoting the realization of green and low-carbon goals by participating in carbon quota trading and emission reduction strategy formulation;

[0022] 3) The green certificate market demand response agent improves the utilization rate of renewable energy by rationally planning the purchase and sale of green certificates, while ensuring policy compliance and optimization of transaction costs.

[0023] Functions of the lower-level agent system:

[0024] 4) The environment-aware agent is responsible for real-time collection and analysis of environmental data, including temperature, humidity, light intensity and other information, to provide accurate basic data support for demand response regulation;

[0025] 5) User behavior agents analyze user behavior patterns (such as working hours, activity areas, etc.), intelligently control the indoor environment, and dynamically adjust energy distribution and equipment operation to meet user needs;

[0026] 6) The multimodal data fusion agent is responsible for processing multi-source heterogeneous data (images, audio, video, etc.) from large public buildings, including environmental data, equipment operation status data, user behavior data, and external market dynamic information. Through efficient data fusion and feature extraction methods, it provides comprehensive and accurate decision support for demand response intelligent regulation;

[0027] 7) Energy management agent is responsible for monitoring and optimizing the energy consumption of the building, such as electricity, heat, etc., and formulating efficient energy use strategies to reduce energy consumption;

[0028] 8) The strategy generation agent uses deep reinforcement learning algorithms to optimize intelligent decision-making based on multimodal data and preset goals, and realizes dynamic adjustment and execution of demand response strategies;

[0029] 9) The fault detection and maintenance agent is responsible for monitoring the operating status of building equipment, promptly detecting and responding to potential faults, reducing energy waste, improving equipment operating efficiency and extending its service life.

[0030] Functions of the communication coordination agent system:

[0031] 10) The communication and coordination agent system is responsible for information interaction and collaboration in the multi-agent system by coordinating the tasks and data flows of the upper and lower-level agents, ensuring the consistency and coordination of the behaviors of each agent.

[0032] Through the global collaboration of these intelligent agents, the entire system can achieve accurate, efficient and intelligent management of building demand response under complex and changing environmental and market conditions.

[0033] It should be pointed out that the above outlines the general composition of the multimodal enhanced intelligent agent system for large public buildings, and corresponding intelligent agents can be constructed according to the requirements of different scenarios.

[0034] Furthermore, the step S2 specifically includes the following steps:

[0035] Step S21, using intelligent sensors to realize the perception and collection of multimodal data of large public buildings, the intelligent sensors include temperature and humidity sensors, gas sensors, infrared sensors, intelligent cameras, and radio frequency identification sensors. Multimodal data of large public buildings refers to diversified types of data reflecting the operation and maintenance status of various links of large public buildings; the operation and maintenance status of various links of large public buildings includes environmental data, energy consumption data, indoor personnel behavior data, electricity market data, and carbon market data; diversified types of data include text, audio, video, image and other multimodal data, which are uploaded to the existing comprehensive energy intelligent management and control platform of large public buildings through wireless communication technology.

[0036] Among them, environmental data refers to indoor temperature and humidity, indoor CO 2 concentration, outdoor meteorological data (temperature, humidity, wind speed, etc.); energy consumption data refers to the real-time energy consumption of electrical equipment in various areas of large public buildings, and real-time energy consumption statistics of various cold and heat source equipment; indoor personnel behavior data refers to the real-time distribution characteristics of indoor personnel in large public buildings, including the number of personnel in each functional area, personnel flow, and real-time personnel load; electricity market data includes real-time information such as grid load, electricity price, load fluctuation, grid stability, etc.; carbon market data includes real-time carbon trading volume, real-time carbon tax, carbon emissions and real-time carbon emission factors.

[0037] Step S22, constructing a multimodal data set D for large public buildings, wherein the multimodal data set D is composed of a historical data set summarized by an existing integrated energy intelligent management and control platform for large public buildings and a previous knowledge set for large public buildings; the previous knowledge set for large public buildings includes electricity-carbon coupling demand response, green building energy efficiency optimization, energy system integration and management, covering renewable energy utilization, energy conversion and storage technology, and existing knowledge of intelligent energy monitoring and scheduling systems. The carriers of this existing knowledge can be e-books, existing standards, audio, video, or case collection materials formed during the operation and maintenance of large public buildings, or even voice conversations between operation and maintenance personnel.

[0038] The large public building multimodal data set D is expressed in the form of a multi-tuple as follows:

[0039] D={D 1 ,D 2 ,…,D i ,…,D n}

[0040] Among them, D i represents the data set of the i-th mode, and n is the total number of modes. The data set D of each mode i It can be further expressed as:

[0041] D i ={d i1 (t),d i2 (t),…,d ik (t),…,d imi (t)}

[0042] Among them, d ik (t) is the value of the kth data point of the ith mode at time t, m i is the total number of data points of the ith mode.

[0043] Step S23, using an intelligent algorithm to perform feature processing and fusion on the multimodal data set D to obtain a multimodal data feature set F of large public buildings. The feature processing and fusion specifically include denoising, time alignment, standardization and normalization, feature extraction, and feature fusion.

[0044] Among them, denoising refers to the use of algorithms such as mean filtering, median filtering and Gaussian filtering to clean up noise and interference in various types of intelligent sensor data, thereby improving the quality of multimodal data; time alignment refers to the use of technologies such as timestamp synchronization, interpolation methods or dynamic time warping to make data from different modes consistent in time, that is, to ensure that the readings of different sensors are at the same time point, such as in video and audio data, to ensure the temporal correspondence of visual information and sound information; standardization and normalization refers to the standardization of the multimodal data to make the data dimensions consistent; feature extraction refers to the use of wavelet transform and Fourier transform to extract time-frequency domain feature information from the original data, such as power consumption patterns, temperature changes, personnel activity patterns, etc.; feature fusion is the process of integrating the features obtained from the above feature extraction using technologies such as weighted averaging, splicing, and deep learning fusion, thereby forming a more comprehensive and robust feature representation.

[0045] The multimodal data feature set F of large public buildings is expressed in the form of a multi-tuple as follows:

[0046] F={F 1 ,F 2 ,…,F i ,…,F n}

[0047] Among them, F i represents the data feature set of the i-th modality, and n is the total number of modalities. The data feature set of each modality can be further expressed as:

[0048]

[0049] Among them, f ik (t) is the value of the k-th feature of the i-th modality at time t, and M i is the total number of data features of the i-th modality. It should be noted that the above steps outline the general process of multi-modal data feature processing and fusion for large public buildings, and corresponding algorithms and technologies need to be selected and adjusted according to the characteristics of different modal data and application scenarios.

[0050] Step S24, preliminarily construct each single agent of large public buildings based on existing large language models, vision-language models, and deep reinforcement learning algorithms. The specific steps include:

[0051] Step S241, use the multi-modal data feature set F of large public buildings as input features and provide them to existing large language models and vision-language models to preliminarily construct each agent in the multi-modal reinforcement learning agent system (specifically including power market demand response agents, carbon market demand response agents, green certificate market demand response agents, environmental perception agents, energy management agents, multi-modal data fusion agents, user behavior agents, fault detection and maintenance agents, policy generation agents). Existing large language models include GPT-4, Gemini, etc., which have decision-making and reasoning abilities similar to humans; vision-language models such as CLIP, Qwen2-VL, PaliGemma, etc., combine visual perception of images and videos with text-based reasoning to process and mine complex multi-modal data.

[0052] Step S242, use the deep reinforcement learning algorithm to preliminarily train each agent;

[0053] Parameter definition, including defining the global state S, the local state space O i of the i-th single agent, the action space A i of the i-th single agent, and the reward function r i of the i-th single agent;

[0054] Global state is used to describe the operating state of the entire large public building, specifically including environmental temperature, environmental humidity, equipment operating state, personnel activity level, power market, carbon market, and real-time status information of the green certificate market.

[0055] The local state of the i-th single agent is o i , and the local state space O it observesi is a subset of the global state, namely:

[0056]

[0057] Among them, T is the ambient temperature, H represents the ambient humidity, E represents the equipment operating status, P represents the personnel activity level, Gr represents the electricity market transaction volume, Ca represents the carbon market transaction volume, and El represents the green certificate market transaction volume;

[0058] All possible actions a performed by the i-th single agent i The set of is represented as the action space A i ,Right now:

[0059] A i ={a i ∣a i =[ΔT AC ,ΔL,ΔH heating ,ΔGr,ΔCa,ΔEl]}

[0060] In the formula, a i is the joint action of the ith single agent, ΔT AC represents the ambient temperature adjustment of large public buildings, ΔL represents the lighting adjustment of large public buildings, ΔH heating represents the heating regulation volume of large public buildings, ΔGr is the trading volume of large public buildings in the electricity market, ΔCa is the trading volume of large public buildings in the carbon market, and ΔEl is the trading volume of large public buildings in the green certificate market.

[0061] Reward function r i It is defined based on the current state and actions performed by the i-th single agent to guide the learning process of the agent.

[0062] r i =r i,energy +r i,comfort +r i,grid +r i,carbon +r i,green

[0063] In the formula, r i,energy is the reward based on energy consumption, r i,comfort is a reward based on indoor comfort, r i,grid is the reward for participating in the electricity market demand response, r i,carbon is the reward for participating in carbon market transactions, r i,green It is a reward for participating in green certificate market transactions.

[0064] The essence of using deep reinforcement learning algorithm to train the i-th agent is to learn O i →A iThe strategy π corresponding to maximizing the expected cumulative reward under i , which is specifically expressed by the following mathematical model:

[0065]

[0066] In the formula, J i (π i ) is the learning strategy π of the i-th agent i performance indicators; Represents the expected value, from the initial state s 0 and action a 0 The distribution probability of ρ(·|π i ) is in strategy π i The state distribution under is the discount factor of the ith agent at time t, which is used to balance the importance of immediate rewards and future rewards,

[0067] The agent operates in a simulated environment and adjusts its behavior strategy based on feedback. At each step, the agent chooses an action and is rewarded based on the effect of the action (such as energy consumption, comfort, etc.). The simulated environment refers to a virtual or computational model used to test and train the behavior of the agent. This environment maps certain physical features of the real world as much as possible, allowing the agent to interact in it and learn how to achieve specific goals.

[0068] Furthermore, in step S3:

[0069] According to the complexity of large public buildings, a global reward function R is constructed using a multimodal data feature set F through a distributed execution mechanism. global , to achieve the collaborative control of the multimodal reinforcement learning agent system. Specifically, the following steps are included:

[0070] Step S31, establishing a global joint action a of the multimodal reinforcement learning agent system;

[0071] The global joint action a of the multimodal reinforcement learning agent system is expressed as:

[0072] a=[a 1 ,a 2 ,…,a i ,…,a N ]

[0073] Among them, a i represents the joint action of the ith agent;

[0074] Construct the global action space A of the multimodal reinforcement learning agent system. The global action space A of the multimodal reinforcement learning agent system is the Cartesian product of the action spaces of all agents, which is specifically expressed as:

[0075] A=A 1 ×A 2 ×…×A i ×…×A N

[0076] Among them, A i represents the action space of the ith agent.

[0077] Step S32, define the joint Q function represents the reward of the i-th agent under the global state S and the global joint action a;

[0078]

[0079] Among them, φ i represents the Q-network parameters (such as weights and biases) of the ith agent.

[0080] The policy network μ of the i-th agent i (o i θ i ) by optimizing the policy network gradient Make the objective function J i (π i ) maximizes the policy gradient It is expressed as:

[0081]

[0082] in, represents the gradient, is the gradient of the joint Q function with respect to the action of the ith agent; θ i are the parameters of the policy network; is the policy network μ i (o i θ i ) about the parameter θ i gradient.

[0083] Joint Q function Through the mean square error function L(φ i )renew:

[0084]

[0085] The target value y i for:

[0086]

[0087] a′=[a′ 1 ,a′ 2 ,…,a i i,…,a′ N ,] represents the global joint action of the multimodal reinforcement learning agent system in the next state; S′ is the global state of the multimodal reinforcement learning agent system in the next state; φ i ′ is the Q network parameter of the i-th agent in the next state.

[0088] Cooperative control of multimodal reinforcement learning agent systems through global objective J global It is expressed as:

[0089]

[0090] Among them, E global,π represents the global expected value, R global is the global reward function, defined as the weighted sum of all agents’ rewards:

[0091]

[0092] w i is the weight of the ith agent.

[0093] In the training phase, we focus on training and optimizing the agent strategy and Q function. The distributed update rule of the strategy for each agent i is:

[0094]

[0095] The parameter update rule of the Q function is:

[0096]

[0097] Among them, η is the learning rate.

[0098] Step S33, in distributed execution, the agent observes the local i and strategy μ i (o i θ i ) decision-making actions, and jointly realize the coordinated operation of each agent in the multimodal reinforcement learning agent system.

[0099] Based on the multimodal reinforcement learning agent system, multi-scenario demand response control strategies can be designed, including dynamic equipment scheduling, energy consumption optimization, and user comfort assurance.

[0100] Furthermore, in step S4:

[0101] Deploy the trained multi-modal reinforcement learning agent system to the actual integrated energy intelligent management and control platform and integrate it with the control systems in the building (such as air conditioners, lighting, heating systems, etc.). This includes deploying the upper-layer agent system in the integrated energy intelligent management and control platform to handle electricity-carbon-green certificate market transactions; embedding the lower-layer agent in the device controller to dynamically regulate each device. Specifically, it includes the following steps:

[0102] Step S41: Integrate the upper-layer agent system with the integrated energy intelligent management and control platform. The data interface can be docked with the building management system (BMS) and the energy market platform through protocols (such as OPC UA, Modbus, or RESTful API) to obtain market prices (electricity, carbon, green certificates), building energy consumption prediction data, and historical transaction record information. On the other hand, submit transaction requests to the energy market through the integrated energy intelligent management and control platform, such as green certificate purchase, carbon emission quota trading, electricity procurement, etc. Then, dynamically adjust the operation of the lower-layer agent system according to the transaction results.

[0103] Step S42: In the integration of the lower-layer agent system with the device controller, perform embedded deployment: Embed the trained lower-layer agent in the device controller, including PLC (programmable logic controller) and IoT devices.

[0104] Furthermore, step S5 specifically includes the following steps:

[0105] Step S51: Establish a two-layer regulation model for the demand response of large public buildings participating in the electricity-carbon-green certificate market, specifically including an upper-layer economic model and a lower-layer day-ahead scheduling model.

[0106] The upper-layer economic model aims to maximize the profit obj obtained by large public buildings participating in the demand response of the electricity market, carbon market, and green certificate market. The upper-layer economic model is specifically as follows: up For the purpose. The upper-layer economic model is specifically:

[0107]

[0108] In the formula, is the electricity purchase price of large public building b at time t, is the marginal cost of large public building b at time t, is the electricity consumption of large public building b at time t, is the green certificate price of large public building b at time t, is the green certificate trading volume purchased by large public building b at time t, is the carbon price of large public building b at time t, is the carbon emission of large public building b at time t.

[0109] The lower-level day-ahead dispatch model is based on the minimization of the cost of each energy device in the demand response of large public buildings participating in the electricity market, carbon market and green certificate market. low This involves determining the market clearing price and trading volume of each market participant to meet the supply and demand balance of the electricity carbon market and the green certificate market. The lower-level day-ahead scheduling model is as follows:

[0110]

[0111] in, represents the day-ahead scheduling cost of a large public building b at time t, is the day-ahead scheduling decision variable of each key controllable equipment in large public building b within time t; represents the total cost of the electricity market for a large public building b in time t, including the electricity purchase cost and demand response cost; represents the total carbon market cost of large public building b in time t, including the cost of purchasing carbon quotas; It represents the total cost or benefit of the green certificate market of a large public building b in time t, depending on whether it buys or sells green certificates.

[0112] It can be expressed as:

[0113]

[0114] Among them, eq 1 (t) refers to the day-ahead dispatch decision of the cold and heat source power equipment in the large public building b; eq 2 (t) refers to the day-ahead scheduling decision of air-conditioning equipment in large public building b; eq 3 (t) refers to the day-ahead dispatch decision of the energy storage equipment in the large public building b; eq 4 (t) refers to the day-ahead scheduling decision of lighting equipment in a large public building b; eq 5 (t) refers to the day-ahead dispatch decision of renewable energy equipment in the large public building b.

[0115] The constraints of the lower-level day-ahead dispatch model include electricity supply and demand balance constraints, carbon emission limit constraints, green certificate trading limit constraints, and energy equipment output constraints.

[0116] Electricity supply and demand balance constraints:

[0117]

[0118] Among them, q b,buy (t) is the electricity purchase amount of large public building b at time t, q b,sell (t) is the electricity sales of large public building b at time t, qgrid (t) is the amount of electricity purchased or sold by large public building b, This holds true for all times t.

[0119] Carbon emission limit constraints:

[0120]

[0121] Among them, e b (t) is the carbon emission of large public building b at time t, E b is the carbon quota of large public building b.

[0122] Green Certificate Trading Restrictions:

[0123]

[0124] Among them, z b (t) is the green certificate transaction volume of large public building b at time t, G b (t) is the renewable energy power generation of large public building b at time t.

[0125] Energy equipment output constraints:

[0126]

[0127] P upper With P lower They respectively refer to the lower and upper output limits of the i-th energy equipment in a large public building.

[0128] The above constraints ensure that when large public buildings participate in electricity-carbon market transactions, their electricity purchases meet electricity demand, carbon emissions do not exceed the prescribed quota, and green certificate trading volume matches building renewable energy trading. These constraints are further integrated into the objective function of the lower-level model to ensure that large public buildings reasonably participate in the electricity-carbon-green certificate market demand response.

[0129] Step S52: Use the upper-layer agent system and the lower-layer agent system to solve the upper-layer economic model and the lower-layer day-ahead scheduling model respectively, and realize information interaction and collaborative solution between the upper-layer and lower-layer agent systems through the communication coordination agent system to solve the upper-layer economic model and obtain Solve the lower-level day-ahead scheduling model to obtain

[0130] By establishing the upper mean square error function L(φ up ), the objective function of the upper agent system J up , the policy network gradient of the upper agent system Then solve the upper economic model to get Specifically:

[0131] Establish the upper-level mean square error function L(φ up ),

[0132] L(φ up )=E up [(Q up (S up ,a up ; φ up )-y up ) 2 ]

[0133] y up =R up +γ up Q up′

[0134] In the formula, L(φ up ) is the upper mean square error function of the upper agent system; E up is the expected value of the upper-level intelligent system; Q up (S up ,a up ; φ up ) is the joint Q function of the upper agent system in the multimodal reinforcement learning agent system; S up is the global state of the upper-level agent system; a up is the joint action of the upper-level agent system, expressed as φ up For adjusting Q up (S up ,a up ; φ up ) learning parameters of the function; y up is the target value of the upper-level intelligent system; R up is the reward function established by the upper-level intelligent system based on the upper-level economic model, that is, R up =obj up ; γ up The discount factor of the upper-level agent system is used to balance the rewards of the upper-level agent system; Q up′ It is the joint Q function of the upper-level agent system in the multimodal reinforcement learning agent system in the next state.

[0135] The objective function of the upper agent system J up for:

[0136]

[0137] Furthermore, the policy update process of the upper-level agent system is based on the policy network gradient of the upper-level agent system. Implementation, specifically:

[0138]

[0139] In the formula, is the objective function of the upper agent system J up About its policy network parameters θ up The policy network gradient is used to guide the policy update to maximize J up ; E θup is the upper-level agent system’s policy network parameter θ up The expectation of up is the policy network parameter of the upper-layer intelligent system; μ up (S up θ up ) is the upper-level agent system strategy network; is the upper-level agent system strategy network μ up (S up θ up )’s gradient; is the gradient of the joint Q function of the upper agent system.

[0140] By establishing the lower-level mean square error function L(φ low ), the objective function of the lower-level agent system J low , the policy network gradient of the lower-level agent system Solving the lower-level day-ahead scheduling model yields Specifically:

[0141] The upper mean square error function L(φ up )for,

[0142] L(φ low )=E low [(Q low (S low ,a low ; φ low )-y low ) 2 ]

[0143] y low =R low +γ low Q low′

[0144] In the formula, L(φ low ) is the lower-level mean square error function of the lower-level agent system; E low is the expected value of the lower-level intelligent system; Q low (S low ,a low ; φ low) is the joint Q function of the lower-level agent system in the multimodal reinforcement learning agent system; S low is the global state of the lower-level agent system; a low is the joint action of the lower-level agent system, expressed as φ low For adjusting Q low (S low ,a low ; φ low ) learning parameters of the function; y low is the target value of the lower-level intelligent system; R low is the reward function established by the lower-level agent system according to the lower-level day-ahead scheduling model, that is, R low =obj low ; γ low The discount factor of the lower-level agent system is used to balance the rewards of the lower-level agent system; Q low′ It is the joint Q function of the lower-level agent system in the multimodal reinforcement learning agent system in the next state.

[0145] The objective function of the lower-level agent system J low for:

[0146]

[0147] Furthermore, the policy update process of the lower-level agent system is based on the policy network gradient of the lower-level agent system Implementation, specifically:

[0148]

[0149] In the formula, is the objective function of the upper agent system J low About its policy network parameters θ low The policy network gradient is used to guide the policy update to maximize J low ; E θlow is the lower-level agent system’s policy network parameter θ low The expectation of low is the policy network parameter of the lower-level intelligent system; μ low (S low θ low ) is the lower-level agent system strategy network; is the lower-level agent system policy network μ low (S low θ low )’s gradient; is the gradient of the joint Q function of the lower-level agent system.

[0150] Based on the model solution results Regulating the key adjustable equipment;

[0151] The upper-layer intelligent agent outputs market trading strategies (such as electricity purchase and carbon emission quotas); the lower-layer intelligent agent feedbacks equipment operation results and environmental status (such as actual energy consumption, carbon emissions, and indoor comfort). According to real-time market fluctuations and equipment operation status, the upper and lower-layer intelligent agents adjust their respective strategies.

[0152] In practical applications, the upper-level intelligent system responds to and regulates real-time demand based on real-time monitored environmental conditions (including grid load, electricity prices, and internal building energy consumption). For example, during peak hours of the grid, the lower-level intelligent system can reduce building energy consumption by reducing air conditioning loads and delaying the operation of non-critical equipment.

[0153] The system dynamically feeds back and optimizes the behavior of the lower-level intelligent system based on real-time feedback: If a control strategy leads to reduced user comfort or unoptimized energy efficiency, the multimodal reinforcement learning intelligent system will adjust itself to obtain the best

[0154] Collect environmental data (temperature and humidity, light intensity, CO 2 concentration, etc.); collaboratively solve the large-scale public building-electricity-carbon-green certificate market demand response two-layer regulation model based on the multimodal reinforcement learning intelligent system and send control signals to the equipment according to the solution results.

[0155] And the key adjustable devices are regulated based on the model solution results.

[0156] Step S53, maintaining the multimodal reinforcement learning agent system, specifically by: monitoring the energy usage in the building in real time through intelligent sensors, and further optimizing and maintaining the behavior of the multimodal reinforcement learning agent system based on feedback information. Including but not limited to: regularly checking and maintaining the system to ensure that the agent maintains optimal performance during long-term use; regularly evaluating the performance of the system, checking indicators such as the building's energy efficiency, user comfort, and grid load balance; and regularly analyzing the system operation log to evaluate the agent's decision-making effect and response speed.

[0157] The beneficial effects of the present invention are:

[0158] Optimizing building energy management, especially demand response systems for buildings, by leveraging multimodal data and reinforcement learning.

[0159] (1) The present invention combines multimodal data fusion with reinforcement learning to improve the accuracy of demand response of large public buildings in the electricity-carbon-green certificate market. The method of the present invention can make full use of the multimodal data inside the building (such as indoor temperature and humidity, CO 2The intelligent system based on multimodal reinforcement learning can adapt to complex and changing environments, accurately optimize demand response strategies, and significantly improve the accuracy and flexibility of energy regulation.

[0160] (2) The present invention uses a multimodal reinforcement learning agent system coordination mechanism. The upper-level agent system optimizes the trading strategy of the building in the electricity market, carbon trading market and green certificate market in real time, and the lower-level agent system dynamically regulates the internal equipment of the building (such as air conditioning, lighting, heating, etc.). The coordinated work of the upper-level agent system and the lower-level agent system enables large public buildings to minimize operating costs and maximize economic benefits while meeting indoor comfort requirements.

[0161] (3) The present invention deeply integrates the energy regulation of large public buildings with the electricity, carbon emissions, and green certificate markets, significantly reducing carbon emissions while optimizing the energy efficiency of large public buildings, and providing intelligent solutions for green and low-carbon buildings. The method can effectively support large public buildings to participate in the intelligent upgrade of electricity-carbon-green certificate market demand response, and promote the realization of the "dual carbon" goal and sustainable development. BRIEF DESCRIPTION OF THE DRAWINGS

[0162] Figure 1 This is a flow chart of the intelligent control method for demand response of large public buildings based on a multimodal reinforcement learning agent system according to the present invention.

[0163] Figure 2 This is a schematic diagram of the composition of multiple agents in the intelligent management and control optimization method for a large-scale building integrated energy system based on a large language model and multiple agents described in the present invention.

[0164] Figure 3 This is a schematic diagram of the single-agent construction process of the large-scale public building demand response intelligent control method based on a multimodal reinforcement learning agent system described in the present invention.

[0165] Figure 4 This is a schematic diagram of a large-scale public building demand response intelligent control method based on a multimodal reinforcement learning agent system described in the present invention. DETAILED DESCRIPTION

[0166] like Figure 1 The present invention is a flow chart of the intelligent control method for large-scale airport building demand response based on a multimodal reinforcement learning agent system.

[0167] Airports are typical public buildings. The present invention takes a large-scale airport building integrated energy system located in a severe cold region as an example to explain the method of the present invention in detail, including the following steps:

[0168] S1, according to the operation and maintenance characteristics of large airport public buildings, determine the scope of demand response regulation of large airport public buildings and identify key adjustable equipment; according to the demand response regulation scope and key adjustable equipment, determine the types of intelligent agents involved and their respective functions, and obtain a multimodal reinforcement learning intelligent agent system;

[0169] S2, using intelligent sensors to realize multimodal data perception and collection of large airport public buildings, constructing a multimodal data set D of large airport public buildings, and extracting features from the set D to obtain a multimodal data feature set F of large airport public buildings, preliminarily constructing each agent of the multimodal reinforcement learning agent system in step S1 in combination with the existing large language model and visual language model, and preliminarily training each preliminarily constructed agent using a deep reinforcement learning algorithm;

[0170] S3, according to the multimodal data feature set F and the agents constructed in step S2, a deep learning reinforcement algorithm is used to centrally train the multimodal reinforcement learning agent system, and the coordinated control of the agents constructed in step S2 is realized based on a distributed execution mechanism;

[0171] S4, deploying the multimodal reinforcement learning agent system into the public buildings of the large airport;

[0172] S5, the multimodal reinforcement learning intelligent agent system participates in the intelligent regulation of demand response of public buildings in large airports. The specific method is: establish a two-layer regulation model of demand response of public buildings in large airports in the electricity-carbon-green certificate market, collaboratively solve the two-layer regulation model of demand response of public buildings in large airports in the electricity-carbon-green certificate market based on the multimodal reinforcement learning intelligent agent system, and regulate the key adjustable equipment in step S1 based on the model solution results; and maintain the multimodal reinforcement learning intelligent agent system.

[0173] In the step S1, the scope of demand response regulation of public buildings in large airports includes electricity market demand response regulation, carbon market demand response, and green certificate market demand response regulation. The key adjustable equipment includes cold and heat source power equipment, central air-conditioning equipment, energy storage equipment, lighting equipment, and renewable energy equipment.

[0174] like Figure 2 As shown, the multimodal reinforcement learning agent system includes an upper-layer agent system, a lower-layer agent system and a communication and coordination agent system; the upper-layer agent system and the lower-layer agent system perform global collaborative operation through the communication and coordination agent system;

[0175] The upper-level intelligent agent system includes the power market demand response agent, the carbon market demand response agent, and the green certificate market demand response agent;

[0176] The lower-level intelligent agent system includes environmental perception agent, energy management agent, user behavior agent, fault detection and maintenance agent, strategy generation agent, and multimodal data fusion agent.

[0177] Each intelligent agent has different functional characteristics, and the corresponding intelligent agent can be expanded and constructed according to the needs of different scenarios.

[0178] The functions of the upper-level agent system are:

[0179] 1) The power market demand response agent minimizes electricity costs by optimizing power procurement and power control strategies, supporting grid stability and improving building economic benefits;

[0180] 2) The carbon market demand response agent is responsible for monitoring and optimizing the carbon emissions of buildings, reducing carbon emission costs and promoting the realization of green and low-carbon goals by participating in carbon quota trading and emission reduction strategy formulation;

[0181] 3) The green certificate market demand response agent improves the utilization rate of renewable energy by rationally planning the purchase and sale of green certificates, while ensuring policy compliance and optimization of transaction costs.

[0182] The functions of the lower-level agent system are:

[0183] 4) The environment-aware agent is responsible for real-time collection and analysis of environmental data, including temperature, humidity, light intensity and other information, to provide accurate basic data support for demand response regulation;

[0184] 5) User behavior agents analyze user behavior patterns (such as working hours, activity areas, etc.), intelligently control the indoor environment, and dynamically adjust energy distribution and equipment operation to meet user needs;

[0185] 6) The multimodal data fusion agent is responsible for processing multi-source heterogeneous data (images, audio, video, etc.) from large airport public buildings, including environmental data, equipment operation status data, user behavior data, and external market dynamic information. Through efficient data fusion and feature extraction methods, it provides comprehensive and accurate decision support for demand response intelligent regulation;

[0186] 7) Energy management agent is responsible for monitoring and optimizing the energy consumption of the building, such as electricity, heat, etc., and formulating efficient energy use strategies to reduce energy consumption;

[0187] 8) The strategy generation agent uses deep reinforcement learning algorithms to optimize intelligent decision-making based on multimodal data and preset goals, and realizes dynamic adjustment and execution of demand response strategies;

[0188] 9) The fault detection and maintenance agent is responsible for monitoring the operating status of building equipment, promptly detecting and responding to potential faults, reducing energy waste, improving equipment operating efficiency and extending its service life.

[0189] Functions of the communication coordination agent system:

[0190] 10) The communication and coordination agent system is responsible for information interaction and collaboration in the multi-agent system by coordinating the tasks and data flows of the upper and lower-level agents, ensuring the consistency and coordination of the behaviors of each agent.

[0191] Through the global collaboration of these intelligent agents, the entire system can achieve accurate, efficient and intelligent management of building demand response under complex and changing environmental and market conditions.

[0192] It should be pointed out that the above outlines the general composition of the multimodal enhanced intelligent agent system for large airport public buildings, and corresponding intelligent agents can be constructed according to the requirements of different scenarios.

[0193] like Figure 3 As shown, the step S2 specifically includes the following steps:

[0194] Step S21, using intelligent sensors to realize the perception and collection of multimodal data of public buildings in large airports, the intelligent sensors include temperature and humidity sensors, gas sensors, infrared sensors, intelligent cameras, and radio frequency identification sensors. Multimodal data of public buildings in large airports refers to diversified types of data reflecting the operation and maintenance status of various links of public buildings in large airports; the operation and maintenance status of various links of public buildings in large airports includes environmental data, energy consumption data, indoor personnel behavior data, electricity market data, and carbon market data; diversified types of data include multimodal data such as text, audio, video, and images, which are uploaded to the existing comprehensive energy intelligent management and control platform of public buildings in large airports through wireless communication technology.

[0195] Among them, environmental data refers to indoor temperature and humidity, indoor CO 2 concentration, outdoor meteorological data (temperature, humidity, wind speed, etc.); energy consumption data refers to the real-time energy consumption of electrical equipment in various areas of public buildings in large airports, and real-time energy consumption statistics of various cold and heat source equipment; indoor personnel behavior data refers to the real-time distribution characteristics of indoor personnel in public buildings in large airports, including the number of personnel in each functional area, personnel flow, and real-time personnel load; electricity market data includes real-time information such as grid load, electricity price, load fluctuation, and grid stability; carbon market data includes real-time carbon trading volume, real-time carbon tax, carbon emissions, and real-time carbon emission factors.

[0196] Step S22, constructing a multimodal data set D of large airport public buildings, wherein the multimodal data set D is composed of a historical data set summarized by an existing integrated energy intelligent management and control platform for large airport public buildings and a previous knowledge set of large airport public buildings; the previous knowledge set of large airport public buildings includes electricity-carbon coupling demand response, green building energy efficiency optimization, energy system integration and management, covering renewable energy utilization, energy conversion and storage technology, and intelligent energy monitoring and scheduling system existing knowledge, and the carriers of this existing knowledge can be e-books, existing standards, audio, video, or case collection materials formed during the operation and maintenance of large airport public buildings, or even voice conversations between operation and maintenance personnel.

[0197] The multimodal data set D of the large airport public buildings is expressed in the form of a multi-tuple as follows:

[0198] D={D 1 ,D 2 ,…,D i ,…,D n}

[0199] Among them, D i represents the data set of the i-th mode, and n is the total number of modes. The data set D of each mode i It can be further expressed as:

[0200] D i ={d i1 (t),d i2 (t),…,d ik (t),…,d imi (t)}

[0201] Among them, d ik (t) is the value of the kth data point of the ith mode at time t, m i is the total number of data points of the ith mode.

[0202] Step S23, using an intelligent algorithm to perform feature processing and fusion on the multimodal data set D to obtain a multimodal data feature set F of large airport public buildings. The feature processing and fusion specifically include denoising, time alignment, standardization and normalization, feature extraction, and feature fusion.

[0203] Among them, denoising refers to the use of algorithms such as mean filtering, median filtering and Gaussian filtering to clean up noise and interference in various types of intelligent sensor data, thereby improving the quality of multimodal data; time alignment refers to the use of technologies such as timestamp synchronization, interpolation methods or dynamic time warping to make data from different modes consistent in time, that is, to ensure that the readings of different sensors are at the same time point, such as in video and audio data, to ensure the temporal correspondence of visual information and sound information; standardization and normalization refers to the standardization of the multimodal data to make the data dimensions consistent; feature extraction refers to the use of wavelet transform and Fourier transform to extract time-frequency domain feature information from the original data, such as power consumption patterns, temperature changes, personnel activity patterns, etc.; feature fusion is the process of integrating the features obtained from the above feature extraction using technologies such as weighted averaging, splicing, and deep learning fusion, thereby forming a more comprehensive and robust feature representation.

[0204] The multimodal data feature set F of the large airport public buildings is expressed in the form of a multi-tuple as follows:

[0205] F={F 1 ,F 2 ,…,F i ,…,F n}

[0206] Among them, F i represents the data feature set of the i-th mode, and n is the total number of modes. The data feature set of each mode can be further expressed as:

[0207]

[0208] Among them, f ik (t) is the value of the kth feature of the ith mode at time t, M i is the total number of data features of the i-th mode. It should be pointed out that the above steps outline the general process of multimodal data feature processing and fusion of large airport public buildings. It is necessary to select and adjust the corresponding algorithms and technologies according to the characteristics of different modal data and application scenarios.

[0209] Step S24, based on the existing large language model, visual language model, and deep reinforcement learning algorithm, the steps of preliminarily constructing each single agent of a large airport public building specifically include:

[0210] Step S241 refers to providing the multimodal data feature set F of large airport public buildings as input features to the existing large language model and visual language model, and preliminarily constructing each agent in the multimodal reinforcement learning agent system (specifically including power market demand response agent, carbon market demand response agent, green certificate market demand response field agent, environmental perception agent, energy management agent, multimodal data fusion agent, user behavior agent, fault detection and maintenance agent, strategy generation agent). Existing large language models include GPT-4, Gemini, etc., which have decision-making and reasoning capabilities similar to humans; visual language models such as CLIP, Qwen2-VL, PaliGemma, etc., combine visual perception of images and videos with text-based reasoning to process and mine complex multimodal data.

[0211] Step S242, using a deep reinforcement learning algorithm to perform preliminary training on each intelligent agent;

[0212] Parameter definition, including definition of global state S, local state space O of the i-th single agent i , the action space A of the i-th single agent i and the reward function r of the i-th single agent i .

[0213] Global State Used to describe the operating status of the entire large airport public buildings, including ambient temperature, ambient humidity, equipment operating status, personnel activity level, electricity market, carbon market, and green certificate market real-time status information.

[0214] The local state of the i-th single agent is o i , the observed local state space o i is a subset of the global state, namely:

[0215]

[0216] Among them, T is the ambient temperature, H represents the ambient humidity, E represents the equipment operating status, P represents the personnel activity level, Gr represents the electricity market transaction volume, Ca represents the carbon market transaction volume, and El represents the green certificate market transaction volume;

[0217] All possible actions a performed by the i-th single agent i The set of is represented as the action space A i ,Right now:

[0218] A i ={a i ∣a i =[ΔT AC ,ΔL,ΔH heating ,ΔGr,ΔCa,ΔEl]}

[0219] In the formula, a i is the joint action of the ith agent, ΔT AC represents the temperature adjustment of public buildings in large airports, ΔL represents the lighting adjustment of public buildings in large airports, and ΔH heating represents the heating regulation amount, ΔGr is the trading volume of large airport public buildings in the electricity market, ΔCa is the trading volume of large airport public buildings in the carbon market, and ΔEl is the trading volume of large airport public buildings in the green certificate market.

[0220] Reward function r i It is defined based on the current state and actions performed by the i-th single agent to guide the learning process of the agent.

[0221] r i =r i,energy +r i,comfort +r i,grid +r i,carbon +r i,green

[0222] In the formula, r i,energy is the reward based on energy consumption, r i,comfort is a reward based on indoor comfort, r i,grid is the reward for participating in the electricity market demand response, r i,carbon is the reward for participating in carbon market transactions, r i,hreen It is a reward for participating in green certificate market transactions.

[0223] The essence of using deep reinforcement learning algorithm to train the i-th agent is to learn O i →a i The strategy π corresponding to maximizing the expected cumulative reward under i , which is specifically expressed by the following mathematical model:

[0224]

[0225] In the formula, J i (π i ) is the learning strategy π of the i-th agent i performance indicators; Represents the expected value, from the initial state s 0 and action a 0 The distribution probabilities of (these distributions are determined by the strategy π i Determine) is calculated; ρ(·|π i ) is in strategy π i The state distribution under is the discount factor of the ith agent at time t, which is used to balance the importance of immediate rewards and future rewards,

[0226] The agent operates in a simulated environment and adjusts its behavior strategy based on feedback. At each step, the agent chooses an action and is rewarded based on the effect of the action (such as energy consumption, comfort, etc.). The simulated environment refers to a virtual or computational model used to test and train the behavior of the agent. This environment maps certain physical features of the real world as much as possible, allowing the agent to interact in it and learn how to achieve specific goals.

[0227] In step S3, according to the complexity of large public buildings, a global reward function R is constructed using the multimodal data feature set F through a distributed execution mechanism. global , to achieve the collaborative control of the multimodal reinforcement learning agent system. Figure 4 As shown, step S3 specifically includes the following steps:

[0228] Step S31, establishing a global joint action a of the multimodal reinforcement learning agent system;

[0229] The global joint action a of the multimodal reinforcement learning agent system is expressed as:

[0230] a=[a 1 ,a 2 ,…,a i ,…,a N ]

[0231] Among them, a i represents the joint action of the ith agent;

[0232] Construct the global action space A of the multimodal reinforcement learning agent system. The global action space A of the multimodal reinforcement learning agent system is the Cartesian product of the action spaces of all agents, which is specifically expressed as:

[0233] A=A 1 ×A 2 ×…×A i ×…×A N

[0234] Among them, A i Represents the action space of the i-th single agent.

[0235] Step S32, define the joint Q function represents the reward of the ith agent under the global state S and the global joint action a:

[0236]

[0237] Among them, φ i are the Q network parameters (such as weights and biases) of the ith agent

[0238] The policy network μ of the i-th agent i (o i θ i ) by optimizing the policy network gradient Make the objective function J i (π i ) maximizes the policy gradient It is expressed as:

[0239]

[0240] in, represents the gradient, is the gradient of the joint Q function with respect to the action of the ith agent; θ i are the parameters of the policy network; is the policy network μ i (o i θ i ) about the parameter θ i gradient.

[0241] Joint Q function Through the mean square error function L(φ i )renew:

[0242]

[0243] Among them, the target value y i for:

[0244]

[0245] a′=[a′ 1 ,a′ 2 ,…,a i i,…,α′ N ,] represents the global joint action of the multimodal reinforcement learning agent system in the next state; S′ is the global state of the multimodal reinforcement learning agent system in the next state; φ i ′ is the Q network parameter of the i-th agent in the next state.

[0246] Cooperative control of multimodal reinforcement learning agent systems through global objective J global It is expressed as:

[0247]

[0248] Among them, E global,π represents the global expected value, R global is the global reward function, defined as the weighted sum of all agents’ rewards:

[0249]

[0250] w i is the weight of the ith agent.

[0251] In the training phase, the agent strategy and Q function are trained intensively. The distributed update rule of the strategy of each agent i is:

[0252]

[0253] The parameter update rule of the Q function is:

[0254]

[0255] Among them, η is the learning rate.

[0256] Step S33, in distributed execution, the agent observes the local i and strategy μ i (o i θ i ) decision-making actions, and jointly realize the coordinated operation of each agent in the multimodal reinforcement learning agent system.

[0257] Based on the multimodal reinforcement learning agent system, multi-scenario demand response control strategies can be designed, including dynamic equipment scheduling, energy consumption optimization, and user comfort assurance.

[0258] In step S4, the trained multimodal reinforcement learning agent system is deployed to the actual integrated energy intelligent management and control platform, and integrated with the control systems in the building (such as air conditioning, lighting, heating systems, etc.), including the upper-level agent system deployed in the integrated energy intelligent management and control platform to handle electricity-carbon-green certificate market transactions; the lower-level agent is embedded in the device controller to dynamically control each device. Specifically, the following steps are included:

[0259] Step S41, the upper-level intelligent system is integrated with the comprehensive energy intelligent management and control platform. The data interface can be connected to the building management system (BMS) and the energy market platform through a protocol (such as OPC UA, Modbus or RESTful API) to obtain market prices (electricity, carbon, green certificates), building energy consumption forecast data, and historical transaction record information; on the other hand, transaction requests are submitted to the energy market through the comprehensive energy intelligent management and control platform, such as green certificate purchase, carbon emission quota trading, electricity procurement, etc.; and then the operation of the lower-level intelligent system is dynamically adjusted according to the transaction results.

[0260] Step S42, in the integration of the lower-level intelligent agent system and the device controller, embedded deployment: the trained lower-level intelligent agent is embedded in the device controller, including PLC (programmable logic controller) and IoT devices.

[0261] The step S5 specifically includes the following steps:

[0262] Step S51, establish a two-layer regulation model for large airport public buildings to participate in the electricity-carbon-green certificate market demand response, specifically including an upper-layer economic model and a lower-layer day-ahead scheduling model.

[0263] The upper economic model is based on the maximum profit obtained by large airport public buildings participating in the electricity market, carbon market and green certificate market demand response. up The upper economic model is as follows:

[0264]

[0265] In the formula, is the electricity purchase price of the large airport public building b at time t, is the marginal cost of a large airport public building b at time t, is the power consumption of the public building b of a large airport at time t, is the green certificate price of large airport public building b at time t, is the transaction volume of green certificates purchased by large airport public building b at time t, is the carbon price of the large airport public building b at time t, is the carbon emission of a large airport public building b at time t;

[0266] The lower-level day-ahead dispatch model is based on the participation of large airport public buildings in the electricity market, carbon market and green certificate market demand response to minimize the cost of each energy device low This involves determining the market clearing price and trading volume of each market participant to meet the supply and demand balance of the electricity carbon market and the green certificate market. The lower-level day-ahead scheduling model is as follows:

[0267]

[0268] in, represents the day-ahead scheduling cost of a large airport public building b at time t, is the day-ahead scheduling decision variable of each key controllable equipment in the public building b of a large airport at time t; represents the total electricity market cost of a large airport public building at time t, including the electricity purchase cost and demand response cost; represents the total carbon market cost of a large airport public building b at time t, which may include the cost of purchasing carbon quotas; It represents the total cost or benefit of the green certificate market of a large airport public building b at time t, depending on whether it buys or sells green certificates.

[0269] It can be expressed as:

[0270]

[0271] eq 1 (t) refers to the day-ahead scheduling decision of the cold and heat source power equipment in the public building b of a large airport; eq 2 (e) refers to the day-ahead scheduling decision of air-conditioning equipment in public building b of a large airport; eq 3 (t) refers to the day-ahead scheduling decision of energy storage equipment in the public building b of a large airport; eq 4 (t) refers to the day-ahead scheduling decision of lighting equipment in public building b of a large airport; eq 5 (t) refers to the day-ahead scheduling decision of renewable energy equipment in public building b of a large airport.

[0272] The constraints of the lower-level day-ahead dispatch model include electricity supply and demand balance constraints, carbon emission limit constraints, green certificate trading limit constraints, and energy equipment output constraints.

[0273] Electricity supply and demand balance constraints:

[0274]

[0275] Among them, q b,buy (t) is the electricity purchase amount of the large airport public building b at time t, q b,sell (t) is the electricity sales of the large airport public building b at time t, q grid (t) is the amount of electricity purchased or sold by the public building b of the large airport, This holds true for all times t.

[0276] Carbon emission limit constraints:

[0277]

[0278] Among them, e b (t) is the carbon emission of the large airport public building b at time t, E b It is the carbon quota of large airport public building b.

[0279] Green Certificate Trading Restrictions:

[0280]

[0281] Among them, z b (t) is the green certificate transaction volume of large airport public building b at time t, G b (t) is the renewable energy power generation of the large airport public building b at time t.

[0282] Energy equipment output constraints:

[0283]

[0284] P upper With P lower They respectively refer to the lower and upper output limits of the i-th energy equipment in the public buildings of large airports.

[0285] The above constraints ensure that when large airport public buildings participate in electricity-carbon market transactions, their electricity purchases meet electricity demand, carbon emissions do not exceed the prescribed quota, and green certificate trading volume matches building renewable energy trading. These constraints are further integrated into the objective function of the lower-level model to ensure that large airport public buildings reasonably participate in the electricity-carbon-green certificate market demand response.

[0286] Step S52: Use the upper-layer agent system and the lower-layer agent system to solve the upper-layer economic model and the lower-layer day-ahead scheduling model respectively, and realize information interaction and collaborative solution between the upper and lower-layer agent systems through the communication coordination agent system to solve the upper-layer economic model and obtain Solve the lower-level day-ahead scheduling model to obtain

[0287] By establishing the upper mean square error function L(φ up ), the objective function of the upper agent system J up , the policy network gradient of the upper agent system Then solve the upper economic model to get Specifically:

[0288] The upper-level mean square error function L(φ up )for,

[0289] L(φ up )=E up [(Q up (S up ,a up ; φ up )-y up ) 2 ]

[0290] y up =R up +γ up Q up '

[0291] In the formula, L(φ up ) is the upper mean square error function of the upper agent system; E up is the expected value of the upper-level intelligent system; Q up (S up ,a up ; φ up) is the joint Q function of the upper agent system in the multimodal reinforcement learning agent system; S up is the global state of the upper-level agent system; a up is the joint action of the upper-level agent system, expressed as φ up For adjusting Q up (S up ,a up ; φ up ) learning parameters of the function; y up is the target value of the upper-level intelligent system; R up is the reward function established by the upper-level intelligent system based on the upper-level economic model, that is, R up =obj up ; γ up The discount factor of the upper-level agent system is used to balance the rewards of the upper-level agent system; Q up′ It is the joint Q function of the lower-level agent system in the multimodal reinforcement learning agent system in the next state.

[0292] The objective function of the upper agent system J up for:

[0293]

[0294] Furthermore, the policy update process of the upper-level agent system is based on the policy network gradient of the upper-level agent system. Implementation, specifically:

[0295]

[0296] In the formula, is the objective function of the upper agent system J up About its policy network parameters θ up The policy network gradient is used to guide the policy update to maximize J up ; E θup is the upper-level agent system’s policy network parameter θ up The expectation of up is the policy network parameter of the upper-layer intelligent system; μ up (S up θ up ) is the upper-level agent system strategy network; is the upper-level agent system strategy network μ up (S up θ up )’s gradient; is the gradient of the joint Q function of the upper agent system.

[0297] By establishing the lower-level mean square error function L(φ low ), the objective function of the lower-level agent system J low , the policy network gradient of the lower-level agent system Solving the lower-level day-ahead scheduling model yields Specifically:

[0298] The upper mean square error function L(φ up )for,

[0299] L(φ low )=E low [(Q low (S low ,a low ; φ low )-y low ) 2 ]

[0300] y low =R low +γ low Q low′

[0301] In the formula, L(φ low ) is the lower-level mean square error function of the lower-level agent system; E low is the expected value of the lower-level intelligent system; Q low (S low ,a low ; φ low ) is the joint Q function of the lower-level agent system in the multimodal reinforcement learning agent system; S low is the global state of the lower-level agent system; a low is the joint action of the lower-level agent system, expressed as φ low For adjusting Q low (S low ,a low ; φ low ) learning parameters of the function; y low is the target value of the lower-level intelligent system; R low is the reward function established by the lower-level agent system according to the lower-level day-ahead scheduling model, that is, R low =obj low ; γ low The discount factor of the lower-level agent system is used to balance the rewards of the lower-level agent system; Q low′ It is the joint Q function of the lower-level agent system in the multimodal reinforcement learning agent system in the next state.

[0302] The objective function of the lower-level agent system J low for:

[0303]

[0304] Furthermore, the policy update process of the upper-level agent system is based on the policy network gradient of the lower-level agent system. Implementation, specifically:

[0305]

[0306] In the formula, is the objective function of the upper agent system J low About its policy network parameters θ low The policy network gradient is used to guide the policy update to maximize J low ; is the lower-level agent system’s policy network parameter θ low The expectation of low is the policy network parameter of the lower-level intelligent system; μ low (S low θ low ) is the lower-level agent system strategy network; is the lower-level agent system strategy network μ low (S low θ low )’s gradient; is the gradient of the joint Q function of the lower-level agent system.

[0307] Based on the model solution results Regulating the key adjustable equipment;

[0308] The upper-layer intelligent agent outputs market trading strategies (such as electricity purchase and carbon emission quotas); the lower-layer intelligent agent feedbacks equipment operation results and environmental status (such as actual energy consumption, carbon emissions, and indoor comfort). According to real-time market fluctuations and equipment operation status, the upper and lower-layer intelligent agents adjust their respective strategies.

[0309] In practical applications, the upper-level intelligent system responds to and regulates real-time demand based on real-time monitored environmental conditions (including grid load, electricity prices, and internal building energy consumption). For example, during peak hours of the grid, the lower-level intelligent system can reduce building energy consumption by reducing air conditioning loads and delaying the operation of non-critical equipment.

[0310] The system dynamically feeds back and optimizes the behavior of the lower-level intelligent system based on real-time feedback: If a control strategy leads to reduced user comfort or unoptimized energy efficiency, the multimodal reinforcement learning intelligent system will adjust itself to obtain the best

[0311] Collect environmental data (temperature and humidity, light intensity, CO2 concentration, etc.); collaboratively solve the large-scale airport public building-electricity-carbon-green certificate market demand response two-layer control model based on the multimodal reinforcement learning intelligent system and send control signals to the equipment according to the solution results.

[0312] And the key adjustable devices are regulated based on the model solution results.

[0313] Step S53, maintaining the multimodal reinforcement learning agent system, specifically by: monitoring the energy usage in the building in real time through intelligent sensors, and further optimizing and maintaining the behavior of the multimodal reinforcement learning agent system based on feedback information. Including but not limited to: regularly checking and maintaining the system to ensure that the agent maintains optimal performance during long-term use; regularly evaluating the performance of the system, checking indicators such as the building's energy efficiency, user comfort, and grid load balance; and regularly analyzing the system operation log to evaluate the agent's decision-making effect and response speed.

Claims

1. A method for intelligent control of demand response of large public buildings based on a multimodal reinforcement learning agent system, characterized in that: The steps include: S1, according to the operation and maintenance characteristics of large public buildings, determine the scope of demand response regulation of large public buildings and identify key adjustable equipment; according to the demand response regulation scope and key adjustable equipment, determine the types of intelligent agents involved and their respective functions, and obtain a multimodal reinforcement learning intelligent agent system; S2, using intelligent sensors to realize the perception and collection of multimodal data of large public buildings, constructing a multimodal data set D of large public buildings, and extracting features from the set D to obtain a multimodal data feature set F of large public buildings; Preliminarily construct each agent of the multimodal reinforcement learning agent system in step S1 by combining the existing large language model and visual language model, and preliminarily train each agent using a deep reinforcement learning algorithm; S3, according to the multimodal data feature set F and the agents constructed in step S2, a deep learning reinforcement algorithm is used to centrally train the multimodal reinforcement learning agent system, and the coordinated control of the agents constructed in step S2 is realized based on a distributed execution mechanism; S4, deploying the multimodal reinforcement learning agent system into the large public building; S5, the multimodal reinforcement learning intelligent agent system participates in the intelligent regulation of demand response of large public buildings. The specific method is: establish a two-layer regulation model of demand response of large public buildings in the electricity-carbon-green certificate market, collaboratively solve the two-layer regulation model of demand response of large public buildings in the electricity-carbon-green certificate market based on the multimodal reinforcement learning intelligent agent system, and regulate the key adjustable equipment in step S1 based on the model solution results; and maintain the multimodal reinforcement learning intelligent agent system.

2. According to claim 1, a large-scale public building demand response intelligent control method based on a multimodal reinforcement learning agent system is characterized in that: In step S1: The scope of demand response regulation for large public buildings includes electricity market demand response regulation, carbon market demand response regulation, and green certificate market demand response regulation. The key controllable equipment includes cold and heat source power equipment, central air conditioning equipment, energy storage equipment, lighting equipment, and renewable energy equipment. The multimodal reinforcement learning agent system includes an upper-layer agent system, a lower-layer agent system and a communication and coordination agent system; the upper-layer agent system and the lower-layer agent system perform global collaborative operation through the communication and coordination agent system; The upper-level intelligent agent system includes an electricity market demand response intelligent agent, a carbon market demand response intelligent agent, and a green certificate market demand response intelligent agent; the lower-level intelligent agent system includes an environmental perception intelligent agent, an energy management intelligent agent, a user behavior intelligent agent, a fault detection and maintenance intelligent agent, a strategy generation intelligent agent, and a multimodal data fusion intelligent agent.

3. According to claim 2, a large-scale public building demand response intelligent control method based on a multimodal reinforcement learning agent system is characterized in that: The step S2 includes the following steps: Step S21, using intelligent sensors to realize the perception and collection of multimodal data of large public buildings, the intelligent sensors include temperature sensors, gas sensors, infrared sensors, intelligent cameras, and radio frequency identification sensors; multimodal data of large public buildings refers to diversified types of data reflecting the operation and maintenance status of various links of large public buildings; the operation and maintenance status of various links of large public buildings includes environmental data, energy data, indoor personnel behavior data, electricity market data, and carbon market data; diversified types of data include text, audio, video, and image multimodal data, which are uploaded to the existing comprehensive energy intelligent management and control platform of large public buildings through wireless communication technology; Step S22, constructing a multimodal data set D of large public buildings; the multimodal data set D is composed of a historical data set summarized by an existing integrated energy intelligent management and control platform for large public buildings and a previous knowledge set of large public buildings; the previous knowledge set of large public buildings includes electricity-carbon coupling demand response, green building energy efficiency optimization, energy system integration and management, covering renewable energy utilization, energy conversion and storage technology, and intelligent energy monitoring and scheduling system existing knowledge, the carriers of this existing knowledge can be electronic books, existing standards, audio, video, or case collection materials formed during the operation and maintenance of large public buildings, or even conversations between operation and maintenance personnel; The large public building multimodal data set D is represented in the form of a multi-tuple: D={D1,D2,…,D i ,…,D n } Among them, D i represents the data set of the i-th mode, n is the total number of modes; the data set D of each mode i It can be further expressed as: Among them, d ik (t) is the value of the kth data point of the ith mode at time t, m i is the total number of data points of the i-th mode; Step S23, use intelligent algorithms to perform feature processing and fusion on the multimodal data set D, specifically including denoising, time alignment, standardization and normalization, feature extraction, and feature fusion, to obtain a large public building multimodal data feature set F, which is expressed in the form of a multi-tuple: F={F1,F2,…,F i ,…,F n } Among them, F i represents the data feature set of the i-th mode, n is the total number of modes; the data feature set F of the i-th mode i It is expressed as: Among them, f ik (t) is the value of the kth feature of the ith mode at time t, M i is the total number of data features of the i-th mode; Step S24, based on the existing large language model, visual language model, and deep reinforcement learning algorithm, preliminarily constructs each single agent of a large public building, specifically: Firstly, the multimodal data feature set F of large public buildings is provided as input features to the existing large language model and visual language model, and each agent in the multimodal reinforcement learning agent system is preliminarily constructed; Then, the deep reinforcement learning algorithm is used to conduct preliminary training on each agent. The specific method is as follows: Define parameters: global state S, local state space O observed by the i-th single agent i , the action space A of the i-th single agent i and the reward function r of the i-th single agent i ; Global State Used to describe the operation and maintenance status of the entire large public building, including ambient temperature, ambient humidity, equipment operation status, personnel activity level, power market, carbon market, and green certificate market real-time status information; The local state of the i-th single agent is o i , the observed local state space O i is a subset of the global state, namely: Among them, T is the ambient temperature, H represents the ambient humidity, E represents the equipment operating status, P represents the personnel activity level, Gr represents the electricity market information, Ca represents the carbon market information, and El represents the green certificate market information; All possible actions a performed by the i-th single agent i The set of is represented as the action space A i ,Right now: A i ={a i ∣a i =[ΔT AC ,ΔL,ΔH heating ,ΔGr,ΔCa,ΔEl]} In the formula, a i is the joint action of the ith agent, ΔT AC represents the ambient temperature adjustment of large public buildings, ΔL represents the lighting adjustment of large public buildings, ΔH heating represents the heating regulation of large public buildings, ΔGr is the trading volume of large public buildings in the electricity market, ΔCa is the trading volume of large public buildings in the carbon market, and ΔEl is the trading volume of large public buildings in the green certificate market; Reward function r i It is defined based on the current state and actions performed by the i-th single agent to guide the learning process of the agent; r i ＝r i,energy +r i,comfort +r i,grid +r i,carbon +r i,grbbn In the formula, r i,energy is the reward based on energy consumption, r i,comfort is a reward based on indoor comfort, r i,grid is the reward for participating in the electricity market demand response, r i,carbon is the reward for participating in carbon market transactions, r i,green It is a reward for participating in green certificate market transactions; The essence of using deep reinforcement learning algorithm to train the i-th agent is to learn O i →A i The strategy π corresponding to maximizing the expected cumulative reward under i , which is specifically expressed by the following mathematical model: In the formula, J i (π i ) is the learning strategy π of the i-th agent i performance indicators; represents the expected value, which is calculated by the distribution probability of the initial state s0 and the action a0; ρ(·|π i ) is in strategy π i The state distribution under is the discount factor of the ith agent at time t, which is used to balance the importance of immediate rewards and future rewards, 4. According to claim 3, a large-scale public building demand response intelligent control method based on a multimodal reinforcement learning agent system is characterized in that: The step S3 specifically includes the following steps: Step S31, establish the global joint action a of the multimodal reinforcement learning agent system, expressed as: a=[a1,a2,…,a i ,…,a N ] Among them, a i represents the joint action of the ith agent; Construct the global action space A of the multimodal reinforcement learning agent system, where A is the Cartesian product of all agent action spaces, specifically expressed as A = A1×A2×…×A i ×…×A N Among them, A i represents the action space of the i-th agent; Step S32, define the joint Q function represents the reward of the ith agent under the global state S and the global joint action a: Among them, φ i represents the Q network parameters of the i-th agent; The policy network μ of the i-th agent i (o i θ i ) through policy gradient Optimize, policy gradient It is expressed as: in, represents the gradient, is the gradient of the joint Q function with respect to the action of the ith agent; θ i are the parameters of the policy network; is the policy network μ i (o i θ i ) about the parameter θ i The gradient of Joint Q function By the mean square error L(φ i )renew: The target value y i for: Among them, a′=[a′1,a′2,…,a i ′,…,a′ N ,] represents the global joint action of the multimodal reinforcement learning agent system in the next state; S′ is the global state of the multimodal reinforcement learning agent system in the next state; φ i ′ is the Q network parameter of the i-th agent in the next state; γ i represents the discount factor of the ith agent; The coordinated control of each agent in the multimodal reinforcement learning agent system is achieved through the global goal J global It is expressed as: Among them, E gloobal,π represents the global expected value, R global is the global reward function, defined as the weighted sum of all agents’ rewards: w i is the weight of the ith agent; In the training phase, the agent strategy and Q function are optimized through centralized training; the distributed update rule of the strategy for each agent i is: The parameter update rule of the Q function is: Where η is the learning rate; Step S33, in distributed execution, the agent observes the local i and strategy μ i (o i θ i ) decision-making actions, and jointly realize the coordinated operation of each agent in the multimodal reinforcement learning agent system.

5. According to claim 4, a large-scale public building demand response intelligent control method based on a multimodal reinforcement learning agent system is characterized in that: The step S4 includes the following steps: Step S41, the upper-layer intelligent system is integrated with the comprehensive energy intelligent management and control platform, and the data interface is connected with the building management system and the energy market platform through the protocol to obtain market prices, building energy consumption forecast data, and historical transaction record information; on the other hand, a transaction request is submitted to the energy market through the comprehensive energy intelligent management and control platform; and the operation of the lower-layer intelligent system is dynamically adjusted according to the transaction results; Step S42, the lower-level intelligent agent system is integrated with the device controller and embeddedly deployed: the trained lower-level intelligent agent is embeddedly deployed in the device controller.

6. According to claim 5, a large-scale public building demand response intelligent control method based on a multimodal reinforcement learning agent system is characterized in that: The step S5 includes the following steps: Step S51, establishing a double-layer regulation model for large public buildings to participate in the electricity-carbon-green certificate market demand response, including an upper-layer economic model and a lower-layer day-ahead scheduling model; The upper economic model is based on the maximum profit obtained by large public buildings participating in the electricity market, carbon market and green certificate market demand response. up The goals are: In the formula, is the electricity purchase price of large public building b at time t, is the marginal cost of large public building b at time t, is the electricity consumption of large public building b at time t, is the green certificate price of large public building b at time t, is the transaction volume of green certificates purchased by large public building b at time t, is the carbon price of large public building b at time t, is the carbon emission of large public building b at time t; The lower-level day-ahead dispatch model is based on the minimization of the cost of each energy device in the demand response of large public buildings participating in the electricity market, carbon market and green certificate market. low The goals are: in, represents the day-ahead scheduling cost of a large public building b at time t, is the day-ahead scheduling decision variable of each key controllable equipment in large public building b within time t; represents the total cost of the electricity market for a large public building b in time t, including the electricity purchase cost and demand response cost; represents the total carbon market cost of large public building b in time t, including the cost of purchasing carbon quotas; represents the total cost or benefit of the green certificate market of a large public building b in time t, depending on whether it buys or sells green certificates; It is expressed as: Among them, eq1(t) refers to the day-ahead scheduling decision of the cold and heat source power equipment in the large public building b; eq2(t) refers to the day-ahead scheduling decision of the air-conditioning equipment in the large public building b; eq3(t) refers to the day-ahead scheduling decision of the energy storage equipment in the large public building b; eq4(t) refers to the day-ahead scheduling decision of the lighting equipment in the large public building b; eq5(t) refers to the day-ahead scheduling decision of the renewable energy equipment in the large public building b; The constraints of the lower-level day-ahead dispatch model include power supply and demand balance constraints, carbon emission limit constraints, green certificate trading limit constraints, and energy equipment output constraints; The power supply and demand balance constraint is: Among them, q b,buy (t) is the electricity purchase amount of large public building b at time t, q b,sell (t) is the electricity sales of large public building b at time t, q grid (t) is the amount of electricity purchased or sold by large public building b, It means that it holds true for all times t; The carbon emission limit constraints are: Among them, e b (t) is the carbon emission of large public building b at time t, E b is the carbon quota of large public building b; Green certificate trading restrictions are as follows: Among them, z b (t) is the green certificate transaction volume of large public building b at time t, G b (t) is the renewable energy generation of large public building b at time t; Energy equipment output constraints: P upper With P lower They refer to the lower and upper limits of the output of the ith energy equipment in a large public building respectively; Step S52: Use the upper-layer agent system and the lower-layer agent system to solve the upper-layer economic model and the lower-layer day-ahead scheduling model respectively, and realize information interaction and collaborative solution between the upper and lower-layer agent systems through the communication coordination agent system to solve the upper-layer economic model and obtain Solve the lower-level day-ahead scheduling model to obtain By establishing the mean square error function L(φ up ), the objective function of the upper agent system J up , the policy network gradient of the upper agent system Then solve the upper economic model to get Specifically: The mean square error function L(φ up )for, L(φ up )=E up [(Q up (S up ,a up ;f up )-y up ) 2 ] y up =R up +γ up ·Q up ′ In the formula, L(φ up ) is the upper mean square error function value of the upper agent system; E up is the expected value of the upper-level intelligent system; Q up (S up ,a up ; φ up ) is the joint Q function of the upper agent system in the multimodal reinforcement learning agent system; S up is the global state of the upper-level agent system; a up is the joint action of the upper-level agent system, expressed as φ up For adjusting Q up (S up ,a up ; φ up ) learning parameters of the function; y up is the target value of the upper-level intelligent system; R up is the reward function established by the upper-level intelligent system based on the upper-level economic model, that is, R up =obj up ; γ up The discount factor of the upper-level agent system is used to balance the rewards of the upper-level agent system; Q up ′ is the joint Q function of the upper-level agent system in the multimodal reinforcement learning agent system in the next state; The objective function of the upper agent system J up for: The policy update process of the upper-level intelligent system is based on the policy network gradient of the upper-level intelligent system. Implementation, specifically: In the formula, is the objective function of the upper agent system J up About its policy network parameters θ up The policy network gradient is used to guide the policy update to maximize J up ; is the upper-level agent system’s policy network parameter θ up The expectation of up is the policy network parameter of the upper-layer intelligent system; μ up (S up θ up ) is the upper-level agent system strategy network; is the upper-level agent system strategy network μ up (S up θ up )’s gradient; is the gradient of the joint Q function of the upper agent system. By establishing the lower-level mean square error function L(φ low ), the objective function of the lower-level agent system J low , the policy network gradient of the lower-level agent system Solving the lower-level day-ahead scheduling model yields Specifically: The upper mean square error function L(φ up )for, L(φ low )=E low [(Q low (S low ,a low ;f low )-y low ) 2 ] y low =R low +γ low ·Q low′ In the formula, L(φ low ) is the lower-level mean square error function of the lower-level agent system; E low is the expected value of the lower-level intelligent system; Q low (S low ,a low ; φ low ) is the joint Q function of the lower-level agent system in the multimodal reinforcement learning agent system; S low is the global state of the lower-level agent system; a low is the joint action of the lower-level agent system, expressed as φ low For adjusting Q low (S low ,a low ; φ low ) learning parameters of the function; y low is the target value of the lower-level intelligent system; R low is the reward function established by the lower-level agent system according to the lower-level day-ahead scheduling model, that is, R low =obj low ; γ low The discount factor of the lower-level agent system is used to balance the rewards of the lower-level agent system; Q low′ is the joint Q function of the lower-level agent system in the multimodal reinforcement learning agent system in the next state; The objective function of the lower-level agent system J low for: The policy update process of the lower-level agent system is based on the policy network gradient of the lower-level agent system. Implementation, specifically: In the formula, is the objective function of the upper agent system J low About its policy network parameters θ low The policy network gradient is used to guide the policy update to maximize J low ; is the lower-level agent system’s policy network parameter θ low The expectation of low is the policy network parameter of the lower-level intelligent system; μ low (S low θ low ) is the lower-level agent system strategy network; is the lower-level agent system strategy network μ low (S low θ low )’s gradient; is the gradient of the joint Q function of the lower-level agent system; Based on the model solution results Regulating the key adjustable equipment; Step S53, maintaining the multimodal reinforcement learning agent system, specifically by monitoring the energy usage in the building in real time through smart sensors, and further optimizing and maintaining the behavior of the multimodal reinforcement learning agent system based on feedback information.

Images