Community digital grid autonomous management method based on multi-agent system

Abstract

The invention discloses a community digital grid autonomous management method based on a multi-agent system. The method comprises the following steps: S1, designing an agent layered architecture; s2, establishing a distributed communication network; s3, deploying an intelligent agent; s4, collecting data in real time by a local agent, and performing autonomous learning and strategy updating through a conjugate gradient optimization technology and an adaptive meta-strategy; s5, collecting local agent data by the regional coordination agent, constructing a regional environment state model, performing strategy collaboration and adjustment with the local agent through a multi-agent entropy regularization random exploration algorithm, and executing regional resource scheduling optimization; s6, acquiring regional state information by the global coordination agent, and executing global learning and optimization in the whole community range; and S7, regularly adjusting the strategy through data analysis, and executing region and global continuous optimization. Through a multi-agent layered architecture and a reinforcement learning algorithm, adaptive optimization management of the community digital grid is realized.

Description

Technical Field

[0001] The present invention relates to the field of intelligent control and automation technology, and in particular to a community digital grid autonomous management method based on a multi-agent system. Background Art

[0002] With the rapid development of Internet of Things and artificial intelligence technologies, the concept of smart communities has gradually entered the public eye. Community management has gradually transitioned from traditional manual management to digital and automated management. In smart communities, collaborative management of multiple aspects, such as environmental monitoring, security systems, and energy management, has formed the concept of "community digital grid". The community digital grid uses a large number of sensor devices, smart devices, and management systems to comprehensively monitor and manage various resources and equipment in the community, aiming to improve the overall management efficiency of the community, reduce energy consumption, and improve the quality of life of residents. However, although existing technologies have achieved a certain degree of automation, they still have significant limitations in terms of flexibility, adaptability, and global optimization.

[0003] Existing community management systems mainly rely on centralized control architectures, and all monitoring and decision-making functions are usually concentrated in a central node. This centralized management method can certainly achieve a certain degree of automation, but its management capabilities for large-scale communities are limited, especially when faced with complex and dynamically changing community environments. Centralized systems are prone to delays, overloads, and untimely responses. In particular, when the community is large, there are many sensors, monitoring devices, and types of resources that need to be managed, and the centralized architecture is prone to bottlenecks due to insufficient data processing capabilities. In addition, once the centralized architecture encounters a failure or network interruption, the management capabilities of the entire system will be severely limited, and real-time monitoring and management of the community environment will not be possible.

[0004] In the management of community digital grids, multiple data sources such as environmental data, energy consumption, and security information need to be processed in a timely manner, and these data are significantly dynamic and complex. The community management system in the prior art is usually managed through preset rules, but this rule-based management method lacks sufficient adaptability and cannot dynamically adjust to the real-time changing community environment. For example, the energy demand of the community may fluctuate significantly in different time periods, and the existing system cannot make effective adjustments to the energy scheduling strategy in a short period of time, which often leads to energy waste or imbalance between supply and demand. Similarly, in terms of environmental monitoring and security management, the management method with fixed rules is difficult to cope with sudden abnormal situations, such as extreme weather, sudden fires or safety accidents. The system can often only respond after the fact and lacks the ability to respond to emergencies in real time and dynamically.

[0005] In addition, the smart community management system in the prior art is usually based on a single-level control method. Even if intelligent management methods are used, they are mostly based on the learning and control of a single agent, lacking the ability of collaborative management and global optimization. In a complex community environment, there are a large number of interdependencies between various devices and resources. Single-agent management cannot effectively coordinate the collaboration between multiple devices to achieve global optimization. For example, environmental conditioning equipment and energy management systems usually need to cooperate with each other to minimize energy consumption while regulating the environment, but a single agent control strategy is difficult to take into account these conflicting goals at the same time, resulting in local optimal rather than global optimal results.

[0006] In response to the above problems, multi-agent systems have gradually attracted attention in recent years. Multi-agent systems can effectively solve problems in complex and dynamic environments through the collaborative work of multiple agents, and are particularly suitable for large-scale distributed management systems. Each agent can independently handle part of the task and optimize the overall goal through information sharing and strategy collaboration with other agents. However, in existing technologies, the application of multi-agent systems in community digital grid management is still relatively limited, especially the global coordination and adaptive optimization problems in large-scale community environments have not been fully solved. Specifically, the collaborative optimization, real-time communication and ability to cope with dynamic changes among agents in multi-agent systems are still insufficient. Traditional multi-agent systems usually rely on fixed collaboration rules and cannot dynamically adjust the collaboration mode between agents according to real-time changes in the environment. In addition, due to the complexity of the community environment itself and the unpredictability of emergencies, it is difficult for existing technologies to achieve simultaneous optimization at the regional and global levels, resulting in inflexible resource scheduling, and the response speed and efficiency to emergencies need to be improved.

[0007] Another obvious flaw in existing technologies is the data processing capability. The sensors and monitoring equipment involved in the community digital grid generate a large amount of real-time data every day, which needs to be processed and responded to in a short period of time. Although IoT technology provides a means of data collection, the processing of data by existing systems is usually limited to simple rule judgments and threshold triggers, and it is impossible to fully utilize the potential value in the data for in-depth analysis and learning. In addition, when dealing with complex events, existing technologies find it difficult to effectively use historical data for prediction and risk prevention and control, and lack data-driven intelligent decision-making capabilities.

[0008] In general, the existing community digital grid management systems have the following major defects: First, the limitations of the centralized management architecture lead to poor system scalability, heavy load pressure and lack of flexibility; second, the lack of adaptive learning capabilities makes it impossible to dynamically adjust management strategies to cope with real-time changes in the community environment; third, the control mode of a single intelligent agent makes it difficult to achieve coordinated optimization among multiple devices and systems; fourth, the data processing capabilities are insufficient, making it difficult to effectively use large-scale data for intelligent decision-making and risk prediction.

[0009] Therefore, how to provide a community digital grid autonomous management method based on a multi-agent system is an urgent problem that technicians in this field need to solve. Summary of the invention

[0010] One purpose of the present invention is to propose a community digital grid autonomous management method based on a multi-agent system. The present invention realizes the adaptive optimization management of the community digital grid by adopting a multi-agent hierarchical architecture design, combining conjugate gradient optimization technology, adaptive meta-strategy and multi-agent entropy regularization random exploration algorithm. Through real-time communication, data sharing and collaborative optimization between local agents, regional coordination agents and global coordination agents, the community environment, energy consumption and security status can be effectively managed. The system is highly flexible and scalable, and can respond in real time in a dynamic and complex community environment, ensuring the global optimization of resource scheduling, and improving the efficiency of overall management and emergency response capabilities.

[0011] A community digital grid autonomous management method based on a multi-agent system according to an embodiment of the present invention comprises the following steps:

[0012] S1. Design a hierarchical architecture of intelligent agents, including local agents, regional coordination agents, and global coordination agents;

[0013] S2. Establish a distributed communication network, use low-latency communication protocols between local agents, use Wi-Fi communication between regional coordination agents and local agents, and the global coordination agent communicates with the regional coordination agent through a high-speed network;

[0014] S3, deploy local intelligent agents at key points in the community, including environmental sensors, security equipment and energy controllers, deploy regional coordination intelligent agent servers in community blocks, deploy global coordination intelligent agents in community control centers, install intelligent agent autonomous learning and control software, and configure regional coordination and global coordination software;

[0015] S4, local agents operate independently, collect data in real time, and autonomously learn and update strategies through conjugate gradient optimization technology and adaptive meta-strategy, and perform operations according to autonomous learning strategies;

[0016] S5, the regional coordination agent collects local agent data, builds a regional environmental state model, coordinates and adjusts strategies with local agents through a multi-agent entropy regularized random exploration algorithm, and performs regional resource scheduling optimization;

[0017] S6, the global coordination agent obtains regional status information, performs global learning and optimization across the entire community, adjusts resource scheduling and collaboration, and sends optimization strategies to regional coordination agents;

[0018] S7. Local agents upload status information to regional coordination agents for local status monitoring; global coordination agents monitor the overall status of the community; strategies are regularly adjusted through data analysis to perform regional and global continuous optimization, including real-time monitoring, preventive operations, and system adaptive adjustments.

[0019] Optionally, the local intelligent agents include perceptual intelligent agents, executive intelligent agents and decision-making intelligent agents, and are configured with environmental monitoring, security monitoring and energy management functions; the regional coordination intelligent agent is configured in the community block and is responsible for the collaborative work of the local intelligent agents; the global coordination intelligent agent is configured at the community level and is responsible for global optimization.

[0020] Optionally, the S4 specifically includes:

[0021] S41, local intelligent agents use deployed sensors to collect environmental data, security data and energy usage data in real time;

[0022] S42, preprocessing the collected raw data, including data filtering, denoising and standardization, to obtain a standardized data set D t ={d 1,t ,d 2,t ,,d n,t}, where d i,t represents the i-th data sample collected by the local agent at time t;

[0023] S43. Construct a learning model for the local agent through conjugate gradient optimization technology, define the objective function J(θ), where θ represents the parameters of the local agent strategy, calculate the gradient of the objective function, and obtain the gradient The conjugate gradient method is used to iteratively optimize the policy parameters of the local agent. The iterative formula is:

[0024]

[0025] Among them, θ k+1 represents the policy parameter of the k+1th iteration, θ k represents the strategy parameter of the kth iteration, α k represents the step size parameter of the kth iteration, p krepresents the conjugate direction of the kth iteration, γ k represents the dynamic adjustment coefficient of the kth iteration, represents the gradient of the policy parameter at the kth iteration, p k-1 represents the conjugate direction of the k-1th iteration;

[0026] S44. In the conjugate gradient optimization process, the local agent strategy is dynamically adjusted in combination with the adaptive meta-strategy module. The adjustment formula of the adaptive meta-strategy is:

[0027]

[0028] Among them, π ′ (θ) represents the updated strategy of the adaptive meta-strategy, λ represents the adjustment coefficient, T represents the number of time steps, and δ t represents the error term at time step t, v t represents the square moving average of the gradient, ∈ represents the smoothing term, represents the gradient of the policy parameters at time step t;

[0029] S45. Define the reward function R(s,a):

[0030]

[0031] Among them, s represents the current state, a represents the current action, β1 and β2 represent the policy sensitivity parameters, and Q env (t) represents the environmental quality parameter at time t, Q env,ideal Indicates the ideal environmental quality value, Q env,max Indicates the maximum environmental quality, Q env,min Indicates the minimum environmental quality, E consumed (t) represents the energy consumption at time t, E threshold Indicates the preset energy consumption threshold, E max Indicates the maximum energy capacity, S sec (t) represents the safety factor at time t, η represents the safety factor decay rate, ω1, ω2 and ω3 represent dynamic weight coefficients, which are used to balance the impact of environmental quality, energy consumption and safety, and are defined as:

[0032]

[0033] Where i = 1, 2, 3, κ represents the adjustment rate parameter, and t0 represents the time point when the agent pays attention to each target;

[0034] S46. The local agent uses the exploration and utilization mechanism in the reinforcement learning framework to execute the updated strategy π ′(θ) to maximize the reward function R(s,a), obtain reward feedback and adjust the policy parameters according to the reward feedback;

[0035] S47, the local agent will autonomously learn the strategy π ′ (θ) is applied in actual operation to determine the control behavior of the environment, security and energy systems according to the strategy, and store the experience data in the learning process in the experience pool, and conduct strategy evaluation and retraining regularly.

[0036] Optionally, the S5 specifically includes:

[0037] S51, the regional coordination agent regularly collects environmental status data from local agents, builds a regional environmental status model, aggregates and standardizes the collected environmental status data, and forms a regional state vector S region ={s1,s2,…,s n}, where s i Represents the state information of the i-th local agent, including environmental quality, energy consumption and safety status parameters;

[0038] S52, according to the regional state vector S region Constructing regional environmental status model M region , regional environmental status model M region Expressed as the state transition probability P(S ′ region |S region ,A), where S ′ region represents the regional state at the next moment, A represents the joint action set of local agents in the region, | represents the conditional symbol, and the multi-agent reinforcement learning algorithm is used to learn the model and calculate the joint strategy of each local agent in the region Π={π1,π2,…,π n}, where π i represents the strategy of the i-th local agent;

[0039] S53. Introduce the multi-agent entropy regularized random exploration algorithm, add the entropy regularization term in the strategy optimization process, and define the entropy regularization term H(Π) of the joint strategy as:

[0040]

[0041] Where n represents the number of local agents in the region, a i represents the action of the i-th local agent, A i represents the action space of the i-th local agent, s i Represents the state of the i-th local agent;

[0042] S54, maximizing the entropy regularization term H(Π) of the joint strategy to achieve entropy regularization of the joint strategy;

[0043] S55. During the strategy optimization process, the regional coordination agent randomly explores and evaluates the strategies of the local agents and executes the random exploration strategy Π ′ , get the joint reward feedback R region :

[0044]

[0045] Among them, τ represents the entropy regularization coefficient, R(s i ,a i ) represents the reward function of the i-th local agent;

[0046] S56, the regional coordination agent gives feedback R based on the joint reward region The joint strategy Π is optimized to achieve the optimal coordination of resource scheduling within the region. The regional coordination agent uses the optimized joint strategy Π * , issue adjustment commands to local intelligent agents and perform regional resource scheduling optimization, including environmental control, energy management and security response.

[0047] Optionally, the S6 specifically includes:

[0048] S61, the global coordination agent regularly obtains regional state information from the regional coordination agent, builds a global state model for the entire community, and converts all regional state vectors Aggregate to form a global state vector Where m represents the number of regions in the community, represents the state of the i-th local agent in region k;

[0049] S62, based on the global state vector S global , the global coordination agent builds the global environment state model M global , where the model describes the dynamic changes of the state within the entire community and defines the global state transition probability P(S ′ global |S global ,A global ), where S ′ global Represents the global state at the next moment, A global represents the set of joint actions in the entire community, | represents the conditional symbol, and the global coordination agent uses a hierarchical reinforcement learning algorithm to optimize the global strategy and solve the optimal global strategy in represents the optimal strategy for region k;

[0050] S63, the global coordination agent introduces a multi-level entropy regularization method. In the process of global strategy optimization, the entropy regularization term H(Π global ), the entropy regularization term is defined as:

[0051]

[0052] Among them, m represents the number of regional coordination agents in the whole community, represents the jth action in region k, represents the action space of region k;

[0053] S64, in the process of global strategy optimization, the global coordination agent according to the current global state S global and strategy global Perform community-wide joint actions and calculate the global reward R based on the operation of the entire community global :

[0054]

[0055] Among them, n k represents the number of local agents in region k, represents the reward of the i-th local agent in region k, ξ represents the entropy regularization coefficient, represents the entropy regularization term of the region k strategy;

[0056] S65. Global coordination agent uses global reward R global The global strategy Π is trained by a hierarchical reinforcement learning algorithm global Optimize and use the optimized global strategy Adjustment instructions are issued to each regional coordination agent, and each regional coordination agent executes the optimized strategy to adjust resource scheduling and collaboration within the region.

[0057] Optionally, the S7 specifically includes:

[0058] S71. During independent operation, the local agent monitors environmental data, energy usage and security status in real time through deployed sensors, and generates local status information after standardizing the collected data;

[0059] S72, local agent based on autonomous learning strategy π i , upload the current status and actions taken to the regional coordination agent to form a local monitoring data set, and upload the data set to the regional coordination agent for real-time monitoring;

[0060] S73, the regional coordination agent analyzes the overall operation status in the region based on the data uploaded by the local agent and generates regional status information S region, and combined with the dynamic changes in the area, the strategies of local agents are evaluated and adjusted in real time to form a closed-loop feedback mechanism;

[0061] S74, the global coordination agent receives the regional status information S transmitted by the regional coordination agent region , and the global state S of the entire community global Perform comprehensive analysis, based on global monitoring data and predictive analysis, the global coordination agent issues strategy adjustment instructions to the regional coordination agent global ;

[0062] S75, the regional coordination agent updates the strategy π of the local agent in the region according to the strategy adjustment instruction issued by the global coordination agent i , the local agent re-executes the task according to the adjusted strategy, and again feeds back the state and action to the regional coordination agent, forming a closed-loop management system from local to regional to global.

[0063] The beneficial effects of the present invention are:

[0064] First, the present invention designs a hierarchical structure of intelligent agents and adopts a multi-level structure of local intelligent agents, regional coordination intelligent agents and global coordination intelligent agents, so that the community management system can handle complex tasks within the community in a distributed manner, avoiding the bottleneck problem of the traditional centralized management architecture. This hierarchical architecture makes the system more scalable and flexible. Each layer of intelligent agents can independently handle local tasks, while optimizing resource scheduling on a global scale, effectively reducing the system's load pressure and improving the system's processing efficiency.

[0065] Secondly, the present invention introduces multi-agent reinforcement learning algorithms, such as conjugate gradient optimization technology, adaptive meta-strategy, entropy regularized random exploration algorithm, etc., so that local agents and regional coordination agents can autonomously learn and adjust strategies in a dynamic community environment, thus having adaptability. This adaptive capability enables the system to dynamically adjust management strategies according to real-time changes in the environment, energy demand, and security conditions, avoiding the limitations of fixed rules in existing systems that cannot cope with complex dynamic environments. Through the application of reinforcement learning and adaptive strategies, local agents can optimize their behavior in different scenarios, thereby achieving reasonable scheduling of resources and efficient use of energy, reducing unnecessary energy consumption, and improving the overall efficiency of the community.

[0066] In addition, the present invention solves the problem that it is difficult to achieve global optimization under the single-agent management mode in the prior art through collaborative learning and global optimization mechanisms in the multi-agent system. Through real-time communication and data sharing between local agents, regional coordination agents and global coordination agents, multi-level collaborative work can be achieved to ensure that global benefits are taken into account while local goals are achieved. Through collaborative optimization among multiple agents, the present invention can achieve global optimization in dealing with problems such as resource conflicts and scheduling balance, and avoid the phenomenon that local agents only pursue local optimization and ignore overall benefits. This optimization mechanism that takes both global and local into account enables the system to operate efficiently in a large-scale community environment, ensuring the reasonable allocation and dynamic scheduling of various resources in the community.

[0067] In view of the complexity of the community environment and the unpredictability of emergencies, the closed-loop management mechanism of the present invention provides great advantages. The local agent continuously adjusts its strategy based on the real-time collected data, and transmits feedback information to the regional coordination agent and the global coordination agent, forming a closed-loop management system from local to regional and then to global. This closed-loop feedback mechanism not only improves the response speed of the system, but also improves the flexibility and robustness of the system in dealing with emergencies. Through the global monitoring and predictive analysis of the global coordination agent, the system can predict the potential risks in the community environment and make response strategies in advance, thereby greatly improving the accuracy of community security and resource management.

[0068] The present invention also enhances the intelligence level of community management through a data-driven intelligent decision-making mechanism. Local agents, regional coordination agents, and global coordination agents can effectively use historical data for prediction and optimization decisions through the collection, preprocessing, and analysis of real-time data. In particular, through the designed reward function and entropy regularization strategy, the present invention can achieve a balance between ensuring environmental quality, energy consumption, and community safety, and ensure that various types of agents operate collaboratively in a dynamic environment. This data-based real-time analysis and decision-making mechanism effectively makes up for the problem of insufficient data processing capabilities in the prior art, enabling the system to quickly make reasonable decisions in the face of complex environmental changes. BRIEF DESCRIPTION OF THE DRAWINGS

[0069] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0070] Figure 1 A flowchart of a community digital grid autonomous management method based on a multi-agent system proposed by the present invention;

[0071] Figure 2 A schematic diagram of the system architecture of a community digital grid autonomous management method based on a multi-agent system proposed by the present invention;

[0072] Figure 3 This is a working schematic diagram of a closed-loop feedback mechanism of a community digital grid autonomous management method based on a multi-agent system proposed in the present invention. DETAILED DESCRIPTION

[0073] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, which only illustrate the basic structure of the present invention in a schematic manner, and therefore only show the components related to the present invention.

[0074] refer to Figure 1-3 , a community digital grid autonomous management method based on a multi-agent system, comprising the following steps:

[0075] S1. Design a hierarchical architecture of intelligent agents, including local agents, regional coordination agents, and global coordination agents;

[0076] S2. Establish a distributed communication network, use low-latency communication protocols between local agents, use Wi-Fi communication between regional coordination agents and local agents, and the global coordination agent communicates with the regional coordination agent through a high-speed network;

[0077] S3, deploy local intelligent agents at key points in the community, including environmental sensors, security equipment and energy controllers, deploy regional coordination intelligent agent servers in community blocks, deploy global coordination intelligent agents in community control centers, install intelligent agent autonomous learning and control software, and configure regional coordination and global coordination software;

[0078] S4, local agents operate independently, collect data in real time, and autonomously learn and update strategies through conjugate gradient optimization technology and adaptive meta-strategy, and perform operations according to autonomous learning strategies;

[0079] S5, the regional coordination agent collects local agent data, builds a regional environmental state model, coordinates and adjusts strategies with local agents through a multi-agent entropy regularized random exploration algorithm, and performs regional resource scheduling optimization;

[0080] S6, the global coordination agent obtains regional status information, performs global learning and optimization across the entire community, adjusts resource scheduling and collaboration, and sends optimization strategies to regional coordination agents;

[0081] S7. Local agents upload status information to regional coordination agents for local status monitoring; global coordination agents monitor the overall status of the community; strategies are regularly adjusted through data analysis to perform regional and global continuous optimization, including real-time monitoring, preventive operations, and system adaptive adjustments.

[0082] In this embodiment, the local intelligent agents include perceptual intelligent agents, executive intelligent agents and decision-making intelligent agents, which are equipped with environmental monitoring, security monitoring and energy management functions; the regional coordination intelligent agent is configured in the community block and is responsible for the collaborative work of the local intelligent agents; the global coordination intelligent agent is configured at the community level and is responsible for global optimization.

[0083] In this implementation, S4 specifically includes:

[0084] S41, local intelligent agents use deployed sensors to collect environmental data, security data and energy usage data in real time;

[0085] S42, preprocessing the collected raw data, including data filtering, denoising and standardization, to obtain a standardized data set D t ={d 1,t ,d 2,t ,,d n,t}, where d i,t represents the i-th data sample collected by the local agent at time t;

[0086] S43. Construct a learning model for the local agent through conjugate gradient optimization technology, define the objective function J(θ), where θ represents the parameters of the local agent strategy, calculate the gradient of the objective function, and obtain the gradient The conjugate gradient method is used to iteratively optimize the policy parameters of the local agent. The iterative formula is:

[0087]

[0088] Among them, θ k+1 represents the policy parameter of the k+1th iteration, θ k represents the strategy parameter of the kth iteration, α k represents the step size parameter of the kth iteration, p k represents the conjugate direction of the kth iteration, γ k represents the dynamic adjustment coefficient of the kth iteration, represents the gradient of the policy parameter at the kth iteration, p k-1 represents the conjugate direction of the k-1th iteration;

[0089] S44. In the conjugate gradient optimization process, the local agent strategy is dynamically adjusted in combination with the adaptive meta-strategy module. The adjustment formula of the adaptive meta-strategy is:

[0090]

[0091] Among them, π ′ (θ) represents the updated strategy of the adaptive meta-strategy, λ represents the adjustment coefficient, T represents the number of time steps, and δ trepresents the error term at time step t, v t represents the square moving average of the gradient, ∈ represents the smoothing term, represents the gradient of the policy parameters at time step t;

[0092] S45. Define the reward function R(s,a):

[0093]

[0094] Among them, s represents the current state, a represents the current action, β1 and β2 represent the policy sensitivity parameters, and Q env (t) represents the environmental quality parameter at time t, Q env,ideal Indicates the ideal environmental quality value, Q env,max Indicates the maximum environmental quality, Q env,min Indicates the minimum environmental quality, E consumed (t) represents the energy consumption at time t, E threshold Indicates the preset energy consumption threshold, E max Indicates the maximum energy capacity, S sec (t) represents the safety factor at time t, η represents the safety factor decay rate, ω1, ω2 and ω3 represent dynamic weight coefficients, which are used to balance the impact of environmental quality, energy consumption and safety, and are defined as:

[0095]

[0096] Where i = 1, 2, 3, κ represents the adjustment rate parameter, and t0 represents the time point when the agent pays attention to each target;

[0097] S46. The local agent uses the exploration and utilization mechanism in the reinforcement learning framework to execute the updated strategy π ′ (θ) to maximize the reward function R(s,a), obtain reward feedback and adjust the policy parameters according to the reward feedback;

[0098] S47, the local agent will autonomously learn the strategy π ′ (θ) is applied in actual operation to determine the control behavior of the environment, security and energy systems according to the strategy, and store the experience data in the learning process in the experience pool, and conduct strategy evaluation and retraining regularly.

[0099] In this implementation manner, S5 specifically includes:

[0100] S51, the regional coordination agent regularly collects environmental status data from local agents, builds a regional environmental status model, aggregates and standardizes the collected environmental status data, and forms a regional state vector S region ={s1,s2,…,s n}, where s i Represents the state information of the i-th local agent, including environmental quality, energy consumption and safety status parameters;

[0101] S52, according to the regional state vector S region Constructing regional environmental status model M region , regional environmental status model M region Expressed as the state transition probability P(S ′ region |S region ,A), where S ′ region represents the regional state at the next moment, A represents the joint action set of local agents in the region, | represents the conditional symbol, and the multi-agent reinforcement learning algorithm is used to learn the model and calculate the joint strategy of each local agent in the region Π={π1,π2,…,π n}, where π i represents the strategy of the i-th local agent;

[0102] S53. Introduce the multi-agent entropy regularized random exploration algorithm, add the entropy regularization term in the strategy optimization process, and define the entropy regularization term H(Π) of the joint strategy as:

[0103]

[0104] Where n represents the number of local agents in the region, a i represents the action of the i-th local agent, A i represents the action space of the i-th local agent, s i Represents the state of the i-th local agent;

[0105] S54, maximizing the entropy regularization term H(Π) of the joint strategy to achieve entropy regularization of the joint strategy;

[0106] S55. During the strategy optimization process, the regional coordination agent randomly explores and evaluates the strategies of the local agents and executes the random exploration strategy Π ′ , get the joint reward feedback R region :

[0107]

[0108] Among them, τ represents the entropy regularization coefficient, R(s i ,a i ) represents the reward function of the i-th local agent;

[0109] S56, the regional coordination agent gives feedback R based on the joint reward regionThe joint strategy Π is optimized to achieve the optimal coordination of resource scheduling within the region. The regional coordination agent uses the optimized joint strategy Π * , issue adjustment commands to local intelligent agents and perform regional resource scheduling optimization, including environmental control, energy management and security response.

[0110] In this implementation manner, S6 specifically includes:

[0111] S61, the global coordination agent regularly obtains regional state information from the regional coordination agent, builds a global state model for the entire community, and converts all regional state vectors Aggregate to form a global state vector Where m represents the number of regions in the community, represents the state of the i-th local agent in region k;

[0112] S62, based on the global state vector S global , the global coordination agent builds the global environment state model M global , where the model describes the dynamic changes of the state within the entire community and defines the global state transition probability P(S ′ global |S global ,A global ), where S ′ global Represents the global state at the next moment, A global represents the set of joint actions in the entire community, | represents the conditional symbol, and the global coordination agent uses a hierarchical reinforcement learning algorithm to optimize the global strategy and solve the optimal global strategy in represents the optimal strategy for region k;

[0113] S63, the global coordination agent introduces a multi-level entropy regularization method. In the process of global strategy optimization, the entropy regularization term H(Π global ), the entropy regularization term is defined as:

[0114]

[0115] Among them, m represents the number of regional coordination agents in the whole community, represents the jth action in region k, represents the action space of region k;

[0116] S64, in the process of global strategy optimization, the global coordination agent according to the current global state S global and strategy global Perform community-wide joint actions and calculate the global reward R based on the operation of the entire community global :

[0117]

[0118] Among them, n k represents the number of local agents in region k, represents the reward of the i-th local agent in region k, ξ represents the entropy regularization coefficient, represents the entropy regularization term of the region k strategy;

[0119] S65. Global coordination agent uses global reward R global The global strategy Π is trained by a hierarchical reinforcement learning algorithm global Optimize and use the optimized global strategy Adjustment instructions are issued to each regional coordination agent, and each regional coordination agent executes the optimized strategy to adjust resource scheduling and collaboration within the region.

[0120] In this implementation manner, the S7 specifically includes:

[0121] S71. During independent operation, the local agent monitors environmental data, energy usage and security status in real time through deployed sensors, and generates local status information after standardizing the collected data;

[0122] S72, local agent based on autonomous learning strategy π i , upload the current status and actions taken to the regional coordination agent to form a local monitoring data set, and upload the data set to the regional coordination agent for real-time monitoring;

[0123] S73, the regional coordination agent analyzes the overall operation status in the region based on the data uploaded by the local agent and generates regional status information S region , and combined with the dynamic changes in the area, the strategies of local agents are evaluated and adjusted in real time to form a closed-loop feedback mechanism;

[0124] S74, the global coordination agent receives the regional status information S transmitted by the regional coordination agent region , and the global state S of the entire community global Perform comprehensive analysis, based on global monitoring data and predictive analysis, the global coordination agent issues strategy adjustment instructions to the regional coordination agent global ;

[0125] S75, the regional coordination agent updates the strategy π of the local agent in the region according to the strategy adjustment instruction issued by the global coordination agent i , the local agent re-executes the task according to the adjusted strategy, and again feeds back the state and action to the regional coordination agent, forming a closed-loop management system from local to regional to global.

[0126] Embodiment 1:

[0127] In order to verify the feasibility of the present invention in implementation, the present invention is applied to a large city, where a newly built smart community covers about 5,000 households and covers an area of ​​about 50 hectares. The community is equipped with various intelligent facilities, including environmental monitoring systems, security systems, smart grid systems, etc. These facilities need to achieve efficient resource scheduling, environmental quality monitoring and safety management in the community. However, the energy demand, environmental conditions and security conditions in the community change dynamically due to different time and place. For example, the electricity demand fluctuates greatly during the day and night, and the air quality during peak hours in the morning and evening will be affected by external traffic, while the residents' security needs increase significantly when certain emergencies (such as fires and intrusions) occur.

[0128] Under existing technologies, community resource scheduling and management usually adopts a centralized control solution. This model cannot quickly respond to dynamic changes in the community, which easily leads to energy waste, environmental quality degradation and security loopholes. To this end, the community introduced an autonomous management solution based on a multi-agent system, aiming to achieve optimal scheduling and dynamic adjustment of resources in the community through the collaborative work of local agents, regional coordination agents and global coordination agents.

[0129] In this smart community, local agents are deployed at key nodes of each building, such as environmental sensors on the roof, smart meters in residents' homes, surveillance cameras, etc. Each local agent is responsible for collecting environmental data, energy usage, and security status. The regional coordination agent is deployed in the management center of each community block to receive real-time data from all local agents in the area and is responsible for optimizing resource scheduling within the region. The global coordination agent is deployed in the community control center to coordinate the operating status of the entire community and adjust the strategies of the regional agents when necessary.

[0130] In a typical summer daily scenario, residents' electricity demand surges, especially between 6pm and 9pm, due to the extensive use of high-energy-consuming equipment such as air conditioners, the energy load of the community has increased significantly. The traditional centralized energy dispatching system cannot be adjusted according to demand in a timely manner, which often leads to energy waste or insufficient supply. In the multi-agent system of the present invention, the local agent monitors the electricity consumption of each household in real time and feeds back to the regional coordination agent. Through conjugate gradient optimization technology and adaptive meta-strategy, the local agent can adjust energy allocation according to real-time electricity demand and reduce ineffective energy consumption. The regional coordination agent collects data from all local agents and uses a multi-agent entropy regularized random exploration algorithm to optimize energy dispatch in the entire region, thereby effectively balancing the energy consumption of each block. At the same time, the global coordination agent monitors the energy usage of the entire community and performs global optimization to ensure that the electricity dispatch between different blocks is reasonable and balanced.

[0131] During the implementation process, each of the 5,000 households in the community installed a local intelligent agent to monitor energy usage, environmental quality and security status. The experimental data records of a certain day showed that during the peak power consumption period (18:00-21:00) in the community, the local intelligent agent detected that the utilization rate of high-energy consumption equipment such as air conditioners reached 90%. Under the traditional centralized management system, the energy supply cannot meet the demand, resulting in unstable electricity loads in some households and an energy waste rate of 15%. Under the autonomous management of the multi-agent system of the present invention, through the autonomous optimization of local intelligent agents and the coordinated scheduling of regional coordination intelligent agents, the energy consumption of the community is effectively controlled, the energy waste rate is reduced to 6%, and the power supply stability is guaranteed.

[0132] In addition, during the same period, the global coordination agent detected an imbalance in energy supply and demand in different blocks of the community by monitoring the operating data of all regional coordination agents. For example, the energy demand of some blocks was close to the upper limit, while other blocks still had surplus energy. The global coordination agent immediately issued an optimization strategy to mobilize energy resources from surplus blocks to support high-demand blocks, ultimately achieving energy balance for the entire community and avoiding local power shortages.

[0133] At the same time, the data of the environmental monitoring part shows that during the morning rush hour (7:00-9:00), the local agent detected that the air quality index (AQI) in the community reached 150 (moderate pollution), and the demand for environmental regulation at this time is relatively large. In traditional systems, the startup of air purification equipment is usually a preset time period, and it does not have the ability to respond to emergencies. After using the multi-agent system of the present invention, after the local agent detects that the air quality has deteriorated, it quickly feeds back the status information to the regional coordination agent. The regional coordination agent analyzes the environmental conditions in the entire area, combines the entropy regularization algorithm, starts the necessary air purification equipment, and optimizes it by time period. According to data comparison, after the air purification equipment was started, the AQI in the community dropped from 150 to 80 (good) within 30 minutes, and the environment recovered significantly.

[0134] In terms of security, in a simulated fire drill in the community, the local agent detected the fire signal through the smoke sensor as soon as the fire occurred, and immediately fed back the information to the regional coordination agent. The regional coordination agent activated the emergency response mechanism and automatically controlled the security equipment in the area, including opening the emergency evacuation channel and starting the automatic fire extinguishing system. After receiving the fire alarm from the regional coordination agent, the global coordination agent immediately issued an early warning to other areas and mobilized resources from other areas, such as sending additional security forces and coordinating the fire protection system support of other blocks. The data shows that in the simulated exercise, the response time of the regional coordination agent was 5 seconds, the coordination time of the global coordination agent was 10 seconds, and the entire community completed the global early warning and resource scheduling within 15 seconds. Compared with the traditional system, the response speed has increased by more than 50%. Table 1 shows the specific performance comparison between the traditional system and the multi-agent system in community management.

[0135] Table 1 Performance comparison of traditional system and multi-agent system in community management

[0136]

[0137] After the introduction of the community digital grid autonomous management method of the multi-agent system, the community management system has achieved significant improvements in various key aspects, which are specifically reflected in the following aspects:

[0138] First, the energy consumption waste rate has been significantly reduced. The energy waste rate under the traditional system is 15%, which reflects the serious waste caused by untimely or unreasonable energy scheduling in the centralized management mode. By introducing a multi-agent system, local agents can monitor and adjust the energy needs of each household in real time, and regional coordination agents can further reduce ineffective energy consumption by optimizing energy scheduling, reducing the waste rate to 6%, a reduction of 9 percentage points. This shows that the system's intelligent regulation and control capabilities in energy management have been significantly improved.

[0139] Secondly, the effect of shortening the air quality recovery time is particularly significant. In traditional systems, the recovery of air quality usually depends on preset rules and lacks the ability to flexibly respond to sudden air pollution events, resulting in a long air quality recovery time of about 60 minutes. After the introduction of the multi-agent system, the local agent can quickly respond to the deterioration of air quality and immediately start the air purification equipment. At the same time, the regional coordination agent optimizes the operation strategy of the purification equipment, shortening the air quality recovery time to 30 minutes, a reduction of 50%. This result shows that the adaptive ability of the multi-agent system in environmental regulation has been greatly improved.

[0140] In terms of security emergency response time, the multi-agent system also showed significant improvements. In the traditional system, the security emergency response time is 30 seconds, but in the multi-agent system, through the real-time monitoring of local agents and the rapid response mechanism of regional coordination agents, the security emergency response time is shortened to 15 seconds, which is half the time. This improvement is crucial to improving the community's security capabilities, especially in responding to emergencies, so that emergency measures can be taken more quickly to ensure the safety of residents.

[0141] In terms of overall system response time, the traditional system has a slow overall response speed, usually requiring 137 seconds to coordinate and complete various tasks. The multi-agent system has greatly improved the response efficiency through hierarchical and regional coordination optimization, shortening the overall response time to 72 seconds, a reduction of 53%. This improvement means that when faced with complex community management tasks, the multi-agent system can more efficiently schedule resources and execute tasks, greatly improving the operating efficiency of the system.

[0142] The regional energy balance efficiency has also been significantly improved in the new system. The energy balance efficiency of the traditional system is 82%, indicating that the traditional system cannot fully consider the real-time load changes and demand differences when allocating energy between different regions. After the introduction of the multi-agent system, the local agent monitors the energy demand of each region in real time, and the regional coordination agent dynamically allocates according to the load situation, which increases the energy balance efficiency to 93%, an increase of 11%. This result reflects the significant advantages of the multi-agent system in energy scheduling optimization, which can more reasonably allocate the community's energy resources and avoid excessive or insufficient energy allocation.

[0143] Finally, the efficiency of emergency handling has also been significantly improved. In traditional systems, the efficiency of emergency handling is 74%, which shows that the system is relatively slow to respond to emergencies and has poor speed and flexibility in allocating resources. The multi-agent system can quickly identify and respond to incidents in the early stages through the collaborative work of local and regional coordination agents, and the handling efficiency has been increased to 92%, an increase of 18%. This improvement is particularly critical to improving the emergency management capabilities of the community, especially in the face of sudden disasters such as fires and earthquakes. The system can more quickly coordinate relevant resources for emergency response and protect the lives and property of residents.

[0144] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical scheme and inventive concept of the present invention within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. A community digital grid autonomous management method based on a multi-agent system, characterized in that: The steps include: S1. Design a hierarchical architecture of intelligent agents, including local agents, regional coordination agents, and global coordination agents; S2. Establish a distributed communication network. Local agents use low-latency communication protocols, regional coordination agents use Wi-Fi communication with local agents, and global coordination agents communicate with regional coordination agents through high-speed networks. S3, deploy local intelligent agents at key points in the community, including environmental sensors, security equipment and energy controllers, deploy regional coordination intelligent agent servers in community blocks, deploy global coordination intelligent agents in community control centers, install intelligent agent autonomous learning and control software, and configure regional coordination and global coordination software; S4, local agents operate independently, collect data in real time, and autonomously learn and update strategies through conjugate gradient optimization technology and adaptive meta-strategy, and perform operations according to autonomous learning strategies; S5, the regional coordination agent collects local agent data, builds a regional environmental state model, coordinates and adjusts strategies with local agents through a multi-agent entropy regularized random exploration algorithm, and performs regional resource scheduling optimization; S6, the global coordination agent obtains regional status information, performs global learning and optimization across the entire community, adjusts resource scheduling and collaboration, and sends optimization strategies to regional coordination agents; S7. Local agents upload status information to regional coordination agents for local status monitoring; global coordination agents monitor the overall status of the community; strategies are regularly adjusted through data analysis to perform regional and global continuous optimization, including real-time monitoring, preventive operations, and system adaptive adjustments.

2. According to claim 1, a community digital grid autonomous management method based on a multi-agent system is characterized in that: The local intelligent agents include perceptual agents, executive agents and decision-making agents, which are equipped with environmental monitoring, security monitoring and energy management functions; the regional coordination agent is configured in the community block and is responsible for the collaborative work of the local intelligent agents; the global coordination agent is configured at the community level and is responsible for global optimization.

3. According to the method of autonomous management of community digital grid based on multi-agent system in claim 1, it is characterized in that: The S4 specifically includes: S41, local intelligent agents use deployed sensors to collect environmental data, security data and energy usage data in real time; S42, preprocessing the collected raw data, including data filtering, denoising and standardization, to obtain a standardized data set D t ={d 1,t ,d 2,t ,,d n,t }, where d i,t represents the i-th data sample collected by the local agent at time t; S43. Construct a learning model for the local agent through conjugate gradient optimization technology, define the objective function J(θ), where θ represents the parameters of the local agent strategy, calculate the gradient of the objective function, and obtain the gradient The conjugate gradient method is used to iteratively optimize the policy parameters of the local agent. The iterative formula is: Among them, θ k+1 represents the policy parameter of the k+1th iteration, θ k represents the strategy parameter of the kth iteration, α k represents the step size parameter of the kth iteration, p k represents the conjugate direction of the kth iteration, γ k represents the dynamic adjustment coefficient of the kth iteration, represents the gradient of the policy parameter at the kth iteration, p k-1 represents the conjugate direction of the k-1th iteration; S44. In the conjugate gradient optimization process, the adaptive meta-strategy module is combined to dynamically adjust the local agent strategy. The adjustment formula of the adaptive meta-strategy is: Among them, π ′ (θ) represents the updated strategy of the adaptive meta-strategy, λ represents the adjustment coefficient, T represents the number of time steps, and δ t represents the error term at time step t, v t represents the square moving average of the gradient, ∈ represents the smoothing term, represents the gradient of the policy parameters at time step t; S45. Define the reward function R(s,a): Among them, s represents the current state, a represents the current action, β1 and β2 represent the policy sensitivity parameters, and Q env (t) represents the environmental quality parameter at time t, Q env,ideal Indicates the ideal environmental quality value, Q env,max Indicates the maximum environmental quality, Q env,min Indicates the minimum environmental quality, E consumed (t) represents the energy consumption at time t, E threshold Indicates the preset energy consumption threshold, E max Indicates the maximum energy capacity, S sec (t) represents the safety factor at time t, η represents the safety factor decay rate, ω1, ω2 and ω3 represent dynamic weight coefficients, which are used to balance the impact of environmental quality, energy consumption and safety, and are defined as: Where i = 1, 2, 3, κ represents the adjustment rate parameter, and t0 represents the time point when the agent pays attention to each target; S46. The local agent uses the exploration and utilization mechanism in the reinforcement learning framework to execute the updated strategy π ′ (θ) to maximize the reward function R(s,a), obtain reward feedback and adjust the policy parameters according to the reward feedback; S47, the local agent will autonomously learn the strategy π ′ (θ) is applied in actual operation to determine the control behavior of the environment, security and energy systems according to the strategy, and store the experience data in the learning process in the experience pool, and conduct strategy evaluation and retraining regularly.

4. According to claim 1, a community digital grid autonomous management method based on a multi-agent system is characterized in that: The S5 specifically includes: S51, the regional coordination agent regularly collects environmental status data from local agents, builds a regional environmental status model, aggregates and standardizes the collected environmental status data, and forms a regional state vector S region ={s1,s2,…,s n }, where s i Represents the state information of the i-th local agent, including environmental quality, energy consumption and safety status parameters; S52, according to the regional state vector S region Constructing regional environmental status model M region , regional environmental status model M region Expressed as the state transition probability P(S ′ region |S region ,A), where S ′ region represents the regional state at the next moment, A represents the joint action set of local agents in the region, | represents the conditional symbol, and the multi-agent reinforcement learning algorithm is used to learn the model and calculate the joint strategy of each local agent in the region Π={π1,π2,…,π n }, where π i represents the strategy of the i-th local agent; S53. Introduce the multi-agent entropy regularized random exploration algorithm, add the entropy regularization term in the strategy optimization process, and define the entropy regularization term H(Π) of the joint strategy as: Where n represents the number of local agents in the region, a i represents the action of the i-th local agent, A i represents the action space of the ith local agent, s i Represents the state of the i-th local agent; S54, maximizing the entropy regularization term H(Π) of the joint strategy to achieve entropy regularization of the joint strategy; S55. During the strategy optimization process, the regional coordination agent randomly explores and evaluates the strategies of the local agents and executes the random exploration strategy Π ′ , get the joint reward feedback R region : Among them, τ represents the entropy regularization coefficient, R(s i ,a i ) represents the reward function of the i-th local agent; S56, the regional coordination agent gives feedback R based on the joint reward region The joint strategy Π is optimized to achieve the optimal coordination of resource scheduling within the region. The regional coordination agent uses the optimized joint strategy Π * , issue adjustment commands to local intelligent agents and perform regional resource scheduling optimization, including environmental control, energy management and security response.

5. The community digital grid autonomous management method based on a multi-agent system according to claim 1 is characterized in that: The S6 specifically includes: S61, the global coordination agent regularly obtains regional status information from the regional coordination agent, builds a global status model for the entire community, and converts all regional status vectors Aggregate to form a global state vector Where m represents the number of regions in the community, represents the state of the i-th local agent in region k; S62, based on the global state vector S global , the global coordination agent builds the global environment state model M global , where the model describes the dynamic changes of the state within the entire community and defines the global state transition probability P(S ′ global |S global ,A global ), where S ′ global Represents the global state at the next moment, A global represents the set of joint actions in the entire community, | represents the conditional symbol, and the global coordination agent uses a hierarchical reinforcement learning algorithm to optimize the global strategy and solve the optimal global strategy in represents the optimal strategy for region k; S63, the global coordination agent introduces a multi-level entropy regularization method. In the process of global strategy optimization, the entropy regularization term H(Π global ), the entropy regularization term is defined as: Among them, m represents the number of regional coordination agents in the whole community, represents the jth action in region k, represents the action space of region k; S64, in the process of global strategy optimization, the global coordination agent according to the current global state S global and strategy global Perform community-wide joint actions and calculate the global reward R based on the operation of the entire community global : Among them, n k represents the number of local agents in region k, represents the reward of the i-th local agent in region k, ξ represents the entropy regularization coefficient, represents the entropy regularization term of the region k strategy; S65. Global coordination agent uses global reward R global The global strategy Π is trained by a hierarchical reinforcement learning algorithm global Optimize and use the optimized global strategy Adjustment instructions are issued to each regional coordination agent, and each regional coordination agent executes the optimized strategy to adjust resource scheduling and collaboration within the region.

6. The community digital grid autonomous management method based on a multi-agent system according to claim 1 is characterized in that: The S7 specifically includes: S71. During independent operation, the local agent monitors environmental data, energy usage, and security status in real time through deployed sensors, and generates local status information after standardizing the collected data; S72, local agent based on autonomous learning strategy π i , upload the current status and actions taken to the regional coordination agent to form a local monitoring data set, and upload the data set to the regional coordination agent for real-time monitoring; S73, the regional coordination agent analyzes the overall operation status in the region based on the data uploaded by the local agent and generates regional status information S region , and combined with the dynamic changes in the area, the strategies of local agents are evaluated and adjusted in real time to form a closed-loop feedback mechanism; S74, the global coordination agent receives the regional status information S transmitted by the regional coordination agent region , and the global state S of the entire community global Perform comprehensive analysis, based on global monitoring data and predictive analysis, the global coordination agent issues strategy adjustment instructions to the regional coordination agent global ; S75, the regional coordination agent updates the strategy π of the local agent in the region according to the strategy adjustment instruction issued by the global coordination agent i , the local agent re-executes the task according to the adjusted strategy, and again feeds back the state and action to the regional coordination agent, forming a closed-loop management system from local to regional to global.

Images