Electric power multi-agent disk monitoring operation and maintenance method based on multi-modal data

Abstract

The invention provides an electric power multi-agent disk monitoring operation and maintenance method based on multi-modal data, relates to the field of intelligent operation and maintenance of an electric power system, and solves the technical problems of limited perception capability, difficulty in collaborative decision making and low response efficiency caused by insufficient utilization of multivariate data in the prior art. The method comprises the steps of collecting multi-source heterogeneous data in a power system, and performing fusion processing through a multi-modal large model base to generate unified semantic representation; on the basis of the unified semantic representation, collaborative analysis and decision making of an agent cluster are carried out through a predefined model collaborative protocol, and a collaborative decision making scheme containing a control instruction sequence is generated; the agent cluster comprises a new energy prediction agent, a power grid stability analysis agent, a load side response agent and an energy storage coordination agent; and performing real-time monitoring on power equipment by using the intelligent agent cluster, and generating a control instruction based on the collaborative decision scheme to reduce power fluctuation caused by new energy grid connection.

Description

Technical Field

[0001] This invention belongs to The specific field is a power multi-agent monitoring and maintenance method based on multimodal data. Background Technology

[0002] With the continuous increase in the proportion of new energy power generation, the power system is facing unprecedented operational challenges. The inherent intermittent and fluctuating characteristics of renewable energy sources such as wind power and photovoltaics have significantly increased the difficulty of power grid power balancing. Traditional monitoring methods based on single data sources and fixed rules are no longer sufficient to meet the safe and stable operation requirements of the new power system. Existing monitoring systems generally suffer from three technical bottlenecks: First, at the data level, multi-source heterogeneous information such as time-series monitoring data, spatial topology data, equipment image data, and text log data generated by power grid operation form data silos, lacking an effective fusion and processing mechanism, resulting in incomplete and inaccurate state perception; second, at the analysis level, various analysis function modules are independent of each other, and specialized analysis tools such as new energy forecasting, stability analysis, and load control lack coordination, making it difficult to form a globally optimized decision-making scheme; finally, at the decision-making level, there is an over-reliance on human experience, resulting in slow response speed and insufficient adaptability to complex operating conditions.

[0003] Current power system monitoring methods lack the ability to collaboratively utilize multimodal data and cannot establish cross-modal semantic relationships. Furthermore, their rapid response mechanisms for renewable energy volatility are inadequate, hindering coordinated regulation of power generation, grid, load, and storage. Particularly in scenarios with high-proportion renewable energy integration, traditional methods exhibit significant limitations in volatility prediction accuracy, stability assessment accuracy, and control strategy adaptability. . Summary of the Invention

[0004] This application provides a power multi-agent monitoring and maintenance method based on multimodal data, which solves the problems of existing methods. Technical issues.

[0005] To achieve the above objectives, this application adopts the following technical solution: Firstly, a power multi-agent monitoring and maintenance method based on multimodal data is provided, including: Multi-source heterogeneous data from the power system are collected and fused using a multimodal large model base to generate a unified semantic representation; wherein, the multi-source heterogeneous data includes time-series data, spatial data, image data and text data; Based on the unified semantic representation, collaborative analysis and decision-making of the agent cluster are carried out through a predefined model collaboration protocol to generate a collaborative decision-making scheme containing a sequence of control commands; the agent cluster includes a new energy prediction agent, a power grid stability analysis agent, a load-side response agent, and an energy storage coordination agent; The intelligent agent cluster is used to monitor power equipment in real time and generate control commands based on the collaborative decision-making scheme to reduce power fluctuations caused by the grid connection of new energy sources.

[0006] Based on the above technical solutions, the power multi-agent monitoring and operation method based on multimodal data provided in this application achieves deep fusion and unified semantic representation of multi-source heterogeneous data of the power system by constructing a multimodal large model foundation. This effectively solves the problems of data isolation and information fragmentation in traditional monitoring systems, providing a comprehensive and accurate data foundation for intelligent decision-making. Next, by introducing an intelligent agent cluster encompassing new energy forecasting, grid stability analysis, load response, and energy storage coordination, and performing distributed collaborative analysis and decision-making based on a predefined model collaboration protocol, expert knowledge, physical mechanisms, and data-driven models are organically combined. This overcomes the limitations of isolated systems in dealing with the uncertainty, speed, and complexity brought about by high-proportion new energy access, realizing a transformation from passive response to proactive early warning, and from local optimization to global collaboration. Finally, by generating executable control command sequences and applying them to real-time monitoring, this method forms an intelligent operation and maintenance system, which can improve the grid's ability to suppress new energy fluctuations and enhance the stability, resilience, and operational efficiency of the power system.

[0007] Furthermore, the multimodal large model base is built based on the Transformer architecture, and the construction process includes: Construct a neural network model containing a multi-layer Transformer encoder-decoder architecture to obtain the initial large model; An initial large model is pre-trained using a general multimodal corpus, and the model parameters are optimized using masked language modeling and contrastive learning objective functions to obtain a general pre-trained large model; the general multimodal corpus includes multi-source datasets of text-image pairs, time series, and cross-modal association data; Historical multi-source heterogeneous data of the power system are collected to obtain a dedicated dataset for the power sector. Domain-adaptive pre-training technology is employed, inputting a power industry-specific dataset into a general pre-trained large model for further training. The learning rate and parameter fine-tuning methods are dynamically adjusted to minimize the domain adaptation loss, and a retrieval-enhanced generation mechanism is introduced to strengthen domain knowledge fusion, resulting in a multimodal large model foundation. The multimodal large model base is used to perform multimodal fusion and semantic representation generation on time-series monitoring data, spatial topology data, equipment image data and text log data in the power system, supporting collaborative decision-making of intelligent agent clusters; In the application of the multimodal large model base, only the encoder part is retained for feature extraction. The encoder performs forward propagation on the input multi-source heterogeneous data and outputs a unified semantic representation vector.

[0008] Furthermore, the introduction of a retrieval enhancement generation mechanism to enhance domain knowledge fusion includes: Construct a dedicated knowledge base for the power industry, which stores structured and unstructured knowledge in the power industry, including but not limited to equipment parameter entities, operating procedure texts, historical fault cases, and multimodal correlation data; The encoder maps the power industry-specific dataset into query vectors, and the nearest neighbor search algorithm is used to retrieve the top K knowledge points from the knowledge base to obtain the retrieval results. The search results are fused with an attention-based dataset in the power sector, and an enhanced semantic representation is generated through weighted summation or concatenation. The enhanced semantic representation is used as the key vector and value vector of the decoder's cross-attention module, and the hidden state output by the decoder at the previous time step is used as the query vector. Through the attention weight calculation and weighted summation of the cross-attention module, a context vector that integrates professional knowledge in the power field and contextual information is obtained. The context vector is processed by the feedforward neural network layer of the decoder to generate a target sequence containing control instructions. The generated content is then subjected to physical rationality verification to achieve enhanced domain knowledge fusion. The physical rationality verification includes boundary checks and security constraint verification.

[0009] Furthermore, the collaborative analysis and decision-making of the agent cluster through a predefined model collaboration protocol includes: A unified semantic representation is pushed to each agent in parallel through a message middleware; Each agent independently analyzes the unified semantic representation; among them, The new energy prediction agent analyzes the fluctuation of new energy output within a preset time period through a spatiotemporal attention mechanism, and obtains an output characteristic report that includes power prediction values ​​and confidence intervals. The power grid stability analysis agent calculates the system stability margin based on the transient stability assessment algorithm and obtains a system stability state assessment report; The load-side response agent uses a deep learning algorithm to generate an adjustable and optimized scheduling scheme, resulting in an assessment of the adjustment potential and a set of optimization strategies. The energy storage coordination agent formulates the charging and discharging strategy of the energy storage system through a multi-scale optimization model. Based on the output of each agent, decision-making negotiation among the agents is carried out through a predefined model collaboration protocol.

[0010] Furthermore, the decision negotiation among agents through a predefined model coordination protocol includes: Define standardized agent communication primitives; the communication primitives include notification, request, commitment, and negotiation; Establish an event-driven communication mechanism and set three types of triggering conditions: data update events, abnormal alarm events, and control command events; Design the agent negotiation and decision-making process, including: The Contract Network protocol is used for task allocation. Based on the matching degree between task type and the capability attributes of each agent, the leading agent is determined through a bidding-tendering-winning mechanism. The capability attributes include domain expertise, current load status, and historical task success rate. The leading intelligent agent issues decision-making tasks; the decision-making tasks include task descriptions, constraints, and optimization objectives. Each agent generates candidate solutions based on its own output and evaluates the merits of the solutions through a utility function. Using an argument-based negotiation framework, the agents engage in multiple rounds of debate on the feasibility, risks, and benefits of the proposed solutions based on power grid system simulation data and historical data, thereby obtaining an optimized set of candidate solutions. Based on the optimized set of candidate solutions, a collaborative decision-making scheme containing a sequence of control instructions is generated through a voting mechanism or an authority hierarchy strategy.

[0011] Furthermore, the process of determining the leading agent based on the matching degree between task type and the capability attributes of each agent through a bidding-tendering-winning mechanism includes: Define a classification system for monitoring task types, dividing tasks into three basic types: emergency control, optimization scheduling, and early warning analysis. For each type of task, establish a task description template that includes parameters such as time limit, computational complexity, and security requirements. Establish a quantitative evaluation system for the capabilities of intelligent agents. Domain expertise is quantified based on the accuracy and response time of the agent on the test set in its respective professional field. Current load status is quantified by the number of tasks currently being processed by the agent and the remaining completion time. Historical task success rate is calculated by statistically analyzing the completion rate and performance score of the agent in executing the same type of task within a historical time period. When a collaborative decision-making requirement is triggered, the bidding party issues a bidding announcement; the announcement includes a task type identifier, a description of the task requirements, and a bid deadline; the bidding party is the initial coordinator or the leading intelligent agent in the previous round; Each intelligent agent calculates the matching degree between its own capability attributes and task requirements using a weighted cosine similarity algorithm based on the received tender notice. If the matching degree exceeds a preset threshold, it returns a tender message to the tendering party containing capability attributes, estimated execution time, and resource requirements. After receiving the bidding information, the bidding entity uses a weighted scoring method to comprehensively score different bidding information before the bidding deadline, and selects the agent with the highest comprehensive score as the leading agent for this round of decision-making and negotiation.

[0012] Furthermore, each intelligent agent generates candidate solutions based on its own output, including

[0013] The renewable energy prediction agent generates a candidate set of renewable energy dispatch schemes at different confidence levels based on the power prediction values ​​and confidence intervals in the power output characteristic report; specifically including: For high confidence intervals with a confidence level above the first preset score, candidate schemes are generated that prioritize ensuring grid security and appropriately limit the upper limit of new energy output. For a medium confidence interval between the second preset score and the first preset score, generate candidate solutions that strike a balance between safety and economy. For confidence intervals with confidence levels below the second preset score, candidate solutions are generated with the goal of improving the renewable energy consumption rate. Based on the system stability assessment report, the power grid stability analysis intelligent agent generates corresponding candidate control measures for different stability problems; specifically including: To address transient stability issues, candidate solutions are generated, including emergency control measures such as machine shutdown and load shedding. To address the voltage stability issue, candidate solutions for voltage control measures, including switching on and off reactive power compensation devices and adjusting transformer taps, are generated. To address the frequency stability issue, candidate solutions are generated, including frequency regulation measures such as reserve capacity deployment and load frequency control. The load-side response agent generates load regulation candidate schemes that consider different response characteristics and user preferences based on load regulation potential assessment and optimization strategy set; specifically including: For industrial interruptible loads, generate short-term interruption scheduling schemes based on price signals or incentive signals; For commercial air conditioning loads, generate a flexible control scheme for adjusting the temperature setpoint; For residential electric vehicle loads, a coordinated charging scheme with charging time shifting is generated; each load adjustment scheme clearly indicates its adjustment capacity, response speed, duration, and evaluation indicators of its impact on users.

[0014] The energy storage coordination agent, based on a multi-scale optimization model, generates candidate schemes for coordinated energy storage operation that adapt to different time scales; specifically including: The second-level fluctuation smoothing scheme focuses on the power response characteristics and state of charge constraints of the energy storage system. The minute-level frequency regulation control scheme focuses on the regulation rate and capacity configuration of the energy storage system. Hourly energy management solutions focus on the charging and discharging strategies and operational economics of energy storage systems; During the candidate solution generation stage, each agent inputs its output into a preset solution generation language model to generate structured and standardized candidate solution text descriptions and parameter sets. The solution generation language model designs dedicated prompt word templates according to the different situations mentioned above. Based on the analysis results of each agent and the corresponding prompt word templates, the model automatically generates candidate solution descriptions that include specific control objects, control parameters, execution timing, and expected effects.

[0015] Furthermore, the evaluation of the merits of the scheme through the utility function includes: Each intelligent agent converts its output into candidate control schemes based on its domain knowledge and work objectives; A multi-attribute utility function is constructed for each candidate solution for quantitative evaluation; the multi-attribute utility function has the following form: ;in, This represents the i-th candidate solution. Represents a multi-attribute utility function. It is a set of evaluation index functions, including safety, economy, and reliability indicators. These are the weighting coefficients for each indicator; Each agent independently calculates the utility value of each candidate solution, internally ranks the candidate solutions according to their utility values, and proposes solutions during multiple rounds of debate.

[0016] Furthermore, the construction process of the new energy prediction intelligent agent includes: A prediction model based on an encoder-decoder architecture is constructed. A graph attention network is introduced at the encoder end of the prediction model to model the power grid topology, and a temporal attention mechanism is introduced at the decoder end to perform weighted processing on the historical sequence. The model was trained using a dedicated dataset for the power sector to obtain a trained new energy prediction agent. The training data included historical new energy output data, weather forecast data, terrain elevation data, and semantic features related to new energy in a unified semantic representation. When applied, the new energy prediction intelligent agent can parse information about weather conditions and equipment status in the unified semantic representation and generate an output characteristic report that includes power prediction values, i.e., confidence intervals.

[0017] Furthermore, the construction process of the power grid stability analysis intelligent agent includes: A hybrid intelligent analysis model is obtained by integrating a physical mechanism model and a data-driven model; wherein the physical mechanism model is constructed based on the differential-algebraic equations of the power system, and the data-driven model is constructed based on graph neural networks and deep belief networks. The hybrid intelligent analysis model is trained using power system simulation data and historical operating data to obtain the trained power grid stability analysis agent; the training data includes power flow calculation results, transient stability simulation results, actual waveform recording data, and semantic features related to system stability in the unified semantic representation. When applied, the power grid stability analysis agent can parse information about network topology, operation mode, and fault scenarios in the unified semantic representation, calculate the system stability margin, and generate a system stability state assessment report; the stability state assessment includes transient stability assessment, voltage stability assessment, and frequency stability assessment.

[0018] Furthermore, the construction process of the load-side response agent includes: Construct a multi-agent reinforcement learning framework, modeling each adjustable load as an agent; The reinforcement learning framework is trained using historical data, electricity price data, user behavior data, and semantic features related to load characteristics from a unified semantic representation. By using clustering algorithms to classify load resources and quantifying the price response capabilities of different types of load resources based on elasticity coefficient models, a load response evaluation module of a reinforcement learning model is obtained. During the training process of the reinforcement learning framework, a multi-objective deep deterministic policy gradient algorithm is used for optimization to obtain the load-side response agent; among which, the optimization objectives include, but are not limited to, peak shaving and valley filling effects, user satisfaction, and power grid security constraints. When applied, the load-side response agent can parse information about system balance demand, market electricity price signals, and user comfort constraints in the unified semantic representation, generate an optimized scheduling scheme for adjustable loads, and output load response evaluation results and an optimization strategy set.

[0019] Furthermore, the construction process of the energy storage coordinating agent includes: Construct an optimization model framework that includes second-level, minute-level, and hour-level calculations; In the process of solving the optimization model, the model predictive control method is used for rolling optimization, and the energy storage lifetime loss cost is added to the objective function to construct a multi-timescale optimization model with the goal of minimizing the total life cycle cost. The multi-timescale optimization model is trained using historical operating data, equipment status data, market price data, and semantic features related to energy storage operation in a unified semantic representation. The parameters of the multi-timescale optimization model are then adjusted using an online learning algorithm to obtain an energy storage coordination agent. When applied, the energy storage coordination agent can parse information about system power fluctuations, stability requirements, and energy-related factors in semantic representations, formulate charging and discharging strategies for the energy storage system, and output multi-timescale coordinated control schemes.

[0020] Secondly, a power multi-agent monitoring and maintenance system based on multimodal data is provided, comprising: a data fusion processing module, an agent collaborative decision-making module, and a real-time monitoring module; wherein... The data fusion processing module is used to collect multi-source heterogeneous data from the power system and perform fusion processing through a multimodal large model base to generate a unified semantic representation; wherein, the multi-source heterogeneous data includes time-series data, spatial data, image data and text data; The intelligent agent collaborative decision-making module is used to perform collaborative analysis and decision-making of the intelligent agent cluster based on the unified semantic representation and through a predefined model collaborative protocol, and generate a collaborative decision-making scheme containing a sequence of control instructions; the intelligent agent cluster includes a new energy prediction intelligent agent, a power grid stability analysis intelligent agent, a load-side response intelligent agent, and an energy storage coordination intelligent agent; The real-time monitoring module is used to monitor power equipment in real time using the intelligent agent cluster, and generate control commands based on the collaborative decision-making scheme to reduce power fluctuations caused by the grid connection of new energy sources.

[0021] Compared with the prior art, the beneficial effects of this application are: This application provides a power system multi-agent monitoring and maintenance method based on multimodal data, which achieves an intelligent upgrade of power system monitoring and maintenance through multi-level technological innovation. First, by constructing a multimodal large-scale model foundation based on the Transformer architecture and introducing a retrieval-enhanced generation mechanism, this method effectively solves the challenges of semantic alignment and deep fusion of multi-source heterogeneous data, significantly improving the system's comprehensive perception and understanding of the power grid's operating status. This foundation not only generates high-quality unified semantic representations but also ensures the accurate injection of professional knowledge and adherence to physical constraints through domain-adaptive pre-training and knowledge enhancement mechanisms, providing a reliable data foundation for intelligent decision-making. Next, this method designs a specialized intelligent agent cluster collaborative decision-making architecture. Each agent performs independent analysis based on a unified semantic representation, and then achieves efficient collaboration through a standardized model collaboration protocol. The bidding-tendering-winning task allocation mechanism ensures the optimal choice for the leading agent, while the argument-based negotiation framework and utility evaluation system guarantee the scientific and rational nature of the decision-making process. In particular, by introducing a large language model for solution generation, the system can automatically generate structured candidate solutions and, combined with a multi-round debate optimization mechanism, ultimately form a collaborative decision solution through voting or an authority-level strategy, significantly improving the system's decision-making quality and efficiency in handling complex scenarios. Ultimately, this method, by constructing a complete closed-loop system of perception, analysis, decision-making, and execution, can achieve accurate prediction and rapid response to fluctuations in renewable energy power, effectively mitigate grid fluctuations through multi-element collaborative control of power generation, grid, load, and storage, improve the safe and stable operation of the power grid based on a stability assessment system that integrates physical mechanisms and data-driven approaches, and ensure that the system has good adaptability and scalability through dynamic cooperation mechanisms among intelligent agents, providing reliable technical support for building new power systems. Attached Figure Description

[0022] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0023] Figure 1 This application provides a system architecture diagram of a power multi-agent monitoring and maintenance system based on multimodal data, as shown in the embodiments of this application. Figure 2 A flowchart illustrating the power multi-agent monitoring and maintenance method based on multimodal data provided in this application embodiment; Figure 3 A flowchart illustrating another power multi-agent monitoring and maintenance method based on multimodal data provided in this application embodiment; Figure 4 A flowchart illustrating another power multi-agent monitoring and maintenance method based on multimodal data provided in this application embodiment; Figure 5 This is a flowchart illustrating another power multi-agent monitoring and maintenance method based on multimodal data, provided in an embodiment of this application. Detailed Implementation

[0024] In the description of this application, unless otherwise stated, " / " means "or," for example, A / B can mean A or B. The "and / or" in this document is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, and B alone. Furthermore, "at least one" means one or more, and "multiple" means two or more. The terms "first," "second," etc., do not limit the quantity or order of execution, and "first," "second," etc., do not necessarily imply differences.

[0025] It should be noted that, in this application, the terms "exemplary" or "for example" are used to indicate that something is being described as an example, illustration, or illustration. Any embodiment or design described as "exemplary" or "for example" in this application should not be construed as being more preferred or advantageous than other embodiments or design solutions. Specifically, the use of terms such as "exemplary" or "for example" is intended to present the relevant concepts in a concrete manner.

[0026] The power multi-agent monitoring and maintenance method based on multimodal data provided in this application can be applied to, for example... Figure 1 In the power multi-agent monitoring and maintenance system shown, based on multimodal data, such as Figure 1 As shown, the system includes: a data fusion processing module, an intelligent agent collaborative decision-making module, and a real-time monitoring module; among which, The data fusion processing module is used to collect multi-source heterogeneous data from the power system and perform fusion processing through a multimodal large model base to generate a unified semantic representation; the multi-source heterogeneous data includes time series data, spatial data, image data and text data; The intelligent agent collaborative decision-making module is used to perform collaborative analysis and decision-making of intelligent agent clusters based on unified semantic representation and through predefined model collaborative protocols, and generate collaborative decision-making schemes containing control command sequences; wherein, the intelligent agent cluster includes new energy prediction intelligent agents, power grid stability analysis intelligent agents, load-side response intelligent agents, and energy storage coordination intelligent agents; The real-time monitoring module is used to monitor power equipment in real time using a cluster of intelligent agents and generate control commands based on a collaborative decision-making scheme.

[0027] To address the technical problems of data isolation and information fragmentation in existing power systems, resulting in passive responses and limited decision-making to local optimization during fluctuations in renewable energy grid connection, this application provides a power multi-agent monitoring and maintenance method based on multimodal data. This method includes: Multi-source heterogeneous data from the power system are collected and fused using a multimodal large model base to generate a unified semantic representation; the multi-source heterogeneous data includes time-series data, spatial data, image data, and text data; Based on a unified semantic representation, collaborative analysis and decision-making of an agent cluster are performed through a predefined model collaboration protocol to generate a collaborative decision-making scheme containing a sequence of control commands. The agent cluster includes a new energy prediction agent, a power grid stability analysis agent, a load-side response agent, and an energy storage coordination agent. By utilizing intelligent agent clusters to monitor power equipment in real time and generating control commands based on collaborative decision-making schemes, power fluctuations caused by the grid connection of new energy sources can be reduced.

[0028] Based on this, this application breaks through the technical bottlenecks of traditional monitoring systems through data fusion, intelligent collaboration, and real-time control processes. It achieves an upgrade in power operation and maintenance from passive response to proactive prevention and control, and from local decision-making to global collaboration. This effectively enhances the grid's ability to accommodate high proportions of renewable energy, strengthens the safe and stable operation of the power system, and provides key technical support for building a clean, low-carbon, safe, and efficient energy system. Through intelligent monitoring and operation and maintenance, power fluctuations caused by renewable energy grid integration can be significantly reduced, improving the economy and reliability of grid operation.

[0029] like Figure 2 As shown in the embodiments of this application, the power multi-agent monitoring and maintenance method based on multimodal data includes: S1. Collect multi-source heterogeneous data from the power system and perform fusion processing through a multimodal large model base to generate a unified semantic representation.

[0030] Among them, multi-source heterogeneous data includes time-series data, spatial data, image data, and text data. For example, real-time monitoring data of generator output, bus voltage, and line current; spatial data such as substation geographical location, power grid topology connection relationship, and equipment installation layout; image data such as equipment appearance inspection photos, infrared thermal images, and instrument panel display screens; and text data such as equipment operation logs, fault repair records, operation and maintenance procedure documents, and dispatch instruction texts.

[0031] In some implementations, the multimodal large model foundation can be built based on a hybrid architecture of Convolutional Neural Networks (CNNs) and Recurrent Neural Networks (RNNs), or existing deep learning frameworks such as Deep Belief Networks (DBNs). The construction process requires first cleaning, standardizing, and normalizing various types of raw data using data preprocessing techniques to remove noise and outliers. Then, modality alignment techniques are used to establish semantic relationships between different types of data. Key information is extracted through feature extraction modules specific to each modality. Finally, algorithms such as concatenated fusion, weighted fusion, or attention fusion are used to integrate multi-dimensional features into a unified semantic representation vector. The construction of the multimodal large model foundation aims to address the problem of diverse data formats and loose connections in power systems. Traditional processing methods struggle to extract information value across modalities, while this foundation can transform scattered temporal, spatial, image, and textual data into unified and interpretable semantic information, providing comprehensive and coherent data support for subsequent collaborative decision-making by intelligent agents, ensuring the integrity and accuracy of decision-making.

[0032] It should be noted that the processing efficiency and representation quality of the multimodal large model base directly affect the response speed and decision-making accuracy of the entire operation and maintenance system. Therefore, its construction must take into account both the real-time performance of data processing and the effectiveness of feature extraction.

[0033] For example, when collecting wind speed time-series data, wind farm geographic location spatial data, wind turbine blade inspection image data, and wind turbine operation and maintenance record text data of a wind farm, the large model base first smooths and denoises the time-series data, performs target detection to extract blade wear features from the image data, extracts keywords from the text data to obtain the operation and maintenance history, and then integrates these features through a fusion algorithm to generate a unified semantic representation that can comprehensively reflect the operating status of the wind farm.

[0034] S2. Based on unified semantic representation, collaborative analysis and decision-making of intelligent agent clusters are carried out through predefined model collaboration protocols to generate collaborative decision-making schemes containing control command sequences.

[0035] The intelligent agent cluster includes a new energy prediction intelligent agent, a power grid stability analysis intelligent agent, a load-side response intelligent agent, and an energy storage coordination intelligent agent.

[0036] In some implementations, the agents can be constructed using the following methods: the new energy prediction agent can be constructed based on Support Vector Machine (SVM) or Long Short-Term Memory (LSTM) network, integrating meteorological forecasting theory and knowledge of new energy power generation characteristics; the power grid stability analysis agent can be constructed using algorithms such as logistic regression and decision trees, combined with power system transient stability theory; the load-side response agent can be constructed based on fuzzy control theory and gradient boosting tree model, incorporating load characteristic analysis methods; and the energy storage coordination agent can be constructed using operations research methods such as linear programming and nonlinear programming, combined with neural network models.

[0037] It should be noted that each intelligent agent is key to achieving specialized analysis of specific dimensions of the power system. Specifically, the new energy prediction agent is used to uncover the output patterns and fluctuation trends of new energy generation, providing generation-side predictions for decision-making; the grid stability analysis agent is used to assess the system's voltage, frequency, transient stability, and other stable states, identifying security risks; the load-side response agent is used to analyze the regulation potential and response characteristics of various loads, providing direction for load scheduling; and the energy storage coordination agent is used to formulate operating strategies for energy storage devices, leveraging their peak-shaving and valley-filling functions. During operation, each intelligent agent receives a unified semantic representation, invokes its built-in analysis models and professional knowledge bases, processes relevant information in a targeted manner, outputs specialized analysis results, and then achieves information exchange and decision-making consultation through predefined model collaboration protocols.

[0038] For example, when the unified semantic representation shows that photovoltaic output will drop significantly during a certain period, the new energy prediction agent predicts the magnitude and duration of the output drop, the grid stability analysis agent assesses the impact of the fluctuation on the system frequency stability, the load-side response agent proposes adjustment suggestions for industrial load and residential load, the energy storage coordination agent formulates a supplementary energy storage discharge plan, and the agents exchange results through a collaborative protocol to jointly promote decision-making.

[0039] S3. Utilize intelligent agent clusters to monitor power equipment in real time and generate control commands based on collaborative decision-making schemes to reduce power fluctuations caused by the grid connection of new energy sources.

[0040] Real-time monitoring refers to the continuous dynamic monitoring and real-time control of the operating status, various operating parameters, and operation execution of power equipment. Its importance lies in the real-time, continuous, and high-risk nature of power system operation. Fluctuations brought about by the grid connection of new energy sources may cause equipment failures or system imbalances. Real-time monitoring can capture abnormal situations at the first time, providing a prerequisite for timely intervention. It is a core link in ensuring the safe and stable operation of the power system.

[0041] In some implementations, the generation of control commands based on collaborative decision-making schemes can be achieved through a command parsing and conversion module: first, the decision objectives, constraints, and operational requirements in the collaborative decision-making scheme are converted into a standardized command framework; then, combined with the control interface specifications and communication protocols of specific power equipment, the standardized framework is converted into digital or analog control signals that the equipment can recognize and execute, and transmitted to the corresponding generators, transformers, energy storage devices, load control devices, etc. through the industrial control system.

[0042] It should be noted that the reason why this process can reduce the fluctuations of new energy grid connection lies in the comprehensiveness and pertinence of the collaborative decision-making scheme: the scheme integrates the predictive information of new energy forecasts, the security requirements of grid stability, the adjustment potential of load response and the buffering capacity of energy storage coordination. The generated control commands can realize the coordinated linkage of multiple links of source-grid-load-storage. By rapidly charging and discharging energy storage devices, short-term fluctuations are suppressed; by flexibly adjusting the load, the supply and demand gap is balanced; and by adjusting the parameters on the generation side, fluctuation changes are adapted. The uncertainty brought about by new energy grid connection is offset from multiple dimensions, thereby stabilizing the grid operation status.

[0043] For example, when real-time monitoring detects that the bus voltage fluctuation exceeds the allowable range due to the grid connection of new energy sources in a certain area, the control command generated based on the collaborative decision-making scheme will instruct the energy storage equipment in that area to start the reactive power compensation mode, and at the same time adjust the power consumption of the surrounding adjustable loads to quickly restore the bus voltage to the stable range and effectively suppress the spread of fluctuations.

[0044] Based on the above technical solutions, the power multi-agent monitoring and operation and maintenance method based on multimodal data provided in this application breaks down the isolation barriers of multi-source data in the power system through a multimodal large model base, realizing deep integration and unified representation of information; through specialized agent clusters and collaborative protocols, a decision-making system with clear division of labor and efficient collaboration is constructed, improving the scientific nature and comprehensiveness of decision-making; through the closed-loop operation of real-time monitoring and collaborative control, proactive prediction and rapid response to fluctuations in new energy grid connection are achieved, which can enhance the stability, flexibility and operating efficiency of the power system, and provide reliable operation and maintenance technical support for new power systems with a high proportion of new energy access.

[0045] In one possible implementation of the embodiments of this application, combined with Figure 2 ,like Figure 3 As shown, the above S1 can be implemented through the following S101, S102 and S103, which are explained in detail below: S101. Construct an initial large model and complete general pre-training to lay the foundation for multimodal data processing.

[0046] The goal of this step is to build an initial model framework with the potential for multimodal data processing, and to train it with a general corpus to master basic cross-modal semantic association capabilities, laying the groundwork for subsequent adaptation in the power sector.

[0047] In some implementations, the initial large model adopts a multi-layer Transformer encoder-decoder architecture, with each encoder and decoder containing 6-12 Transformer blocks. Each Transformer block integrates a multi-head self-attention mechanism, a feedforward neural network, and a layer normalization module. The number of heads in the multi-head self-attention mechanism is set to 8-16 to balance feature extraction capability and computational efficiency.

[0048] In the general pre-training phase, a multi-source general multimodal corpus covering text-image pairs (such as news text and corresponding scene images, technical documents and equipment diagrams), time series (such as meteorological monitoring time series data, industrial equipment operation time series curves), and cross-modal association data (such as corresponding data of video frames and speech-transcribed text) is selected. The model parameters are jointly optimized using two types of objective functions: masked language modeling and contrastive learning. Masked language modeling involves randomly masking 15%-20% of the tokens in text data, allowing the model to predict the original content at the masked locations. The InfoNCE objective function, based on contrastive learning, is used, with the following formula: This approach aims to narrow the gap between modal features from the same data source and widen the gap between modal features from different data sources, enabling the model to learn cross-modal semantic consistency. and These are feature vectors from different modalities of the same data source, representing positive sample pairs. These are modal feature vectors from other data sources, representing negative samples. Represents the cosine similarity function. The temperature parameter (range 0.01-0.1) is used to adjust the smoothness of the similarity distribution.

[0049] It should be noted that the key to general pre-training is to enable the model to have general multimodal feature extraction and semantic association capabilities, so as to avoid getting stuck in local optima when adapting to the domain later. The diversity of its training data directly determines the model's cross-modal transfer capability. Therefore, it is necessary to ensure that the corpus covers different scenarios and different data types, and the data scale reaches more than one million.

[0050] For example, the publicly available COCO text-image dataset, Kinetics video time-series dataset, and WeatherBench meteorological time-series dataset are selected to construct a general multimodal corpus. The initial large model is pre-trained for 1 million steps. The prediction accuracy of masked language modeling reaches more than 85%. The average similarity of positive sample pairs in contrastive learning is more than 0.6 higher than that of negative sample pairs, thus completing the construction of the general pre-trained large model.

[0051] S102. Introduce a domain-adaptive pre-training and retrieval-enhanced generation mechanism to achieve knowledge fusion in the power field.

[0052] This step aims to adapt a general pre-trained large model to the power industry scenario, improve the model's ability to understand multi-source heterogeneous power data by injecting power expertise, and solve the problem of inaccurate semantic representation of general models in professional fields.

[0053] In some implementation methods, the first step is to collect historical multi-source heterogeneous data of the power system over the past 5-10 years to construct a dedicated dataset for the power sector, including but not limited to time-series data of power output from new energy power plants, power grid topology data, infrared inspection image data of equipment, text data of substation operation logs, and fault repair records.

[0054] In the domain-adaptive pre-training stage, a power domain-specific dataset is input into the general pre-trained large model. A cosine annealing learning rate adjustment strategy is adopted, combined with mini-batch gradient descent for parameter fine-tuning. By minimizing the domain-adaptive loss function, the model can learn power domain tasks while reducing the feature distribution deviation between general data and power data. ,in It is the loss of the power data semantic matching task based on cross-entropy calculation. It is the domain difference loss, calculated by using a domain classifier to distinguish the feature distribution differences between general data and power data. It is the balance coefficient (with a value of 0.3-0.5).

[0055] In some implementations, the specific implementations of the retrieval enhancement generation mechanism include: The first step is to build a dedicated knowledge base for the power industry, using a distributed database to store structured and unstructured knowledge, and establishing an index of relationships between knowledge points through knowledge graph technology. The second step involves mapping a single data point in the power industry-specific dataset to a multi-dimensional query vector using the encoder trained by S101, and then using the K-Nearest Neighbor (KNN) algorithm to retrieve the top 5-10 knowledge points with the highest similarity from the knowledge base as the retrieval results. The third step involves using a weighted summation method for attention fusion. This allows the model to retain key features of the data itself while incorporating relevant professional knowledge. The fusion formula is as follows: ,in It is the feature vector of the power data itself. It is the feature vector of the m-th retrieved knowledge point. This is the similarity weight of the m-th retrieved knowledge point, obtained by normalizing the similarity score using the softmax function. It is the balance coefficient between data features and retrieval features (value range: 0.6-0.8). The fourth step involves using the enhanced semantic representation as the key vector K and value vector V of the decoder's cross-attention module, and the hidden state output by the decoder at the previous time step as the query vector Q. A context vector integrating professional knowledge and contextual information is calculated through attention weights. This context vector is then processed by the decoder's feedforward neural network to generate control command-related sequences. Finally, physical rationality is verified through boundary checks and security constraint validation, eliminating generated content that does not conform to the operating rules of the power system. The attention weights are used to calculate the context vector. The calculation formula is: ,in It is the dimension of the key vector; boundary checks include verifying whether the parameters are within the rated range of the equipment, such as the generator output not exceeding the rated power; safety constraint verification includes verifying whether the operation complies with the power grid safety regulations, such as the load shedding not exceeding the system's allowable range.

[0056] It should be noted that the core of domain-adaptive pre-training is transfer and fine-tuning. It is necessary to retain the cross-modal processing capability of the general model while adapting the model to the data distribution and professional knowledge of the power field. The key to the retrieval enhancement generation mechanism lies in the completeness of the knowledge base and the accuracy of the retrieval algorithm. The knowledge base needs to be updated regularly to include knowledge of new equipment and new procedures. The K value of the retrieval algorithm needs to be dynamically adjusted according to the correlation density of knowledge points to avoid retrieval redundancy or omission of key information.

[0057] S103. Retain the encoder portion for feature extraction, perform fusion processing on multi-source heterogeneous power data, and generate a unified semantic representation.

[0058] This step is the final execution stage of S1. Through the multimodal large model foundation built in the previous step, the actual collected multi-source heterogeneous power data is transformed into a unified dimension and interpretable semantic representation vector, providing data support for subsequent collaborative decision-making by intelligent agents.

[0059] In some implementations, only the encoder portion is retained when using a multimodal large model base. The decoder is used for early model training and knowledge fusion verification, and is not needed during the actual data processing stage to reduce computational latency. The specific processing flow is as follows: First, the multimodal data is preprocessed, and the preprocessing operations are the same as those used in the construction phase of the multimodal large model base. For example, real-time acquired time-series data is standardized, spatial data is normalized by coordinates, image data is unified in size and pixel value is normalized, and text data is segmented and embedded. Then, the preprocessed data of various types are input in parallel into the corresponding modal feature extraction branches of the encoder, including temporal data input into the temporal Transformer branch, image data input into the convolutional Transformer branch, text data input into the text Transformer branch, and spatial data input into the graph Transformer branch. After each branch extracts the key features of a single modality through a self-attention mechanism, it is input into the cross-modal attention fusion layer. By calculating the correlation weights between different modal features, the multimodal features are integrated into a semantic representation vector of a unified dimension and output to the agent collaborative decision-making module.

[0060] It should be noted that retaining only the encoder is to meet the low latency requirements of real-time power system monitoring. Removing the decoder can improve data processing speed by more than 30%. The dimension of the unified semantic representation needs to be adjusted according to the actual application scenario. The higher the dimension, the richer the feature expression, but the higher the computational cost. In the power monitoring scenario, 1024 dimensions is the optimal choice that balances expressive power and computational efficiency. During feature extraction, it is necessary to ensure the time synchronization of data from each modality to avoid deviations in semantic representation caused by differences in data acquisition time.

[0061] Based on the above technical solution, S1 constructs basic capabilities through general pre-training in S101, injects professional attributes through domain adaptation and knowledge enhancement in S102, and generates unified representations through efficient feature fusion in S103, forming a complete technical chain of "general capabilities + professional adaptation + real-time processing". This step not only solves the problems of insufficient fusion of multi-source heterogeneous data and inconsistent semantic representations in traditional methods, but also ensures the professionalism and accuracy of semantic representations through retrieval enhancement generation mechanisms and physical rationality verification. This provides a high-quality and highly reliable data foundation for subsequent collaborative analysis and decision-making of intelligent agent clusters, significantly improving the level of intelligence in power system monitoring and maintenance.

[0062] In one possible implementation of the embodiments of this application, combined with Figure 2 ,like Figure 4 As shown, the above S2 can be implemented through the following S201, S202 and S203, which are explained in detail below: S201. Construct a specialized intelligent agent cluster and complete its initialization to lay the functional foundation for collaborative decision-making.

[0063] The goal of this step is to create a set of intelligent agents with analytical capabilities in various specialized fields of the power system, clarify the technical architecture, core algorithms and operating parameters of each intelligent agent, and ensure that they can independently complete specialized analysis tasks, providing functional support for subsequent collaborative decision-making.

[0064] In some implementations, the construction of each intelligent agent needs to integrate power industry expertise, physical mechanisms, and advanced algorithms. The specific construction process is as follows: The new energy prediction agent employs an encoder-decoder architecture to build the prediction model. At the encoder, a Graph Attention Network (GAT) is introduced to model the power grid topology, capturing the impact of topological relationships on new energy output by calculating attention weights between nodes. At the decoder, a temporal attention mechanism is introduced to assign differentiated weights to features at different time steps in the historical output sequence, strengthening the influence of data from key periods. During the training phase, historical new energy output data, corresponding time-period weather forecast data, terrain elevation data, and new energy-related semantic features from a unified semantic representation are used as training samples. The model training is completed by minimizing the prediction error loss function using an Adaptive Moment Estimation (Adam) optimizer. A power grid stability analysis intelligent agent is constructed, integrating a physical mechanism model and a data-driven model. The physical mechanism model is based on differential-algebraic equations (DAEs) of the power system to describe the fundamental physical laws governing power grid operation. The data-driven model employs a concatenated architecture of a graph neural network (GNN) and a deep belief network (DBN). The GNN is used to extract the correlation features between power grid topology and operating state, while the DBN is used to mine nonlinear patterns in the data. Training data includes power flow calculation results, transient stability simulation data, actual fault waveform data, and semantic features related to system stability from a unified semantic representation. Joint training enables the model to achieve both physical interpretability and predictive accuracy.

[0065] Load-side response agents: A multi-agent reinforcement learning framework is built, modeling various adjustable loads such as industrial interruptible loads, commercial air conditioning loads, and residential electric vehicle loads as independent load agents. During training, historical load data, electricity price fluctuation data, user behavior preference data, and semantic features related to load characteristics from a unified semantic representation are input. K-means clustering is used to classify load resources, and the price response capability of different loads is quantified based on an elasticity coefficient model, forming a load response assessment module. The elasticity coefficient is calculated using the following formula: , For the load change rate, The rate of change is the electricity price. Then, the reinforcement learning framework is optimized using the Multi-Objective Deep Deterministic Policy Gradient (MO-DDPG) algorithm. The optimization objectives cover peak shaving and valley filling effects, user satisfaction, and power grid security constraints, ultimately resulting in an agent capable of generating load regulation schemes.

[0066] Energy Storage Coordination Intelligent Agent: A multi-timescale optimization model framework encompassing second-, minute-, and hour-level calculations is constructed. Energy storage lifetime loss cost is incorporated into the objective function, forming an optimization function with minimizing the total lifecycle cost as its core objective. The energy storage lifetime loss cost is calculated based on a linear fitting model of cycle count and discharge depth. Model solving employs Model Predictive Control (MPC) for rolling optimization. Training data includes historical operating data of energy storage devices, device status monitoring data, electricity market price data, and semantic features related to energy storage operation from a unified semantic representation. Online learning algorithms are used to adjust model parameters in real time, ensuring the model adapts to the dynamic changes of actual operating scenarios.

[0067] It should be noted that the initialization process of each agent requires parameter calibration and functional testing to ensure that its core indicators such as domain expertise and response speed meet the preset standards. At the same time, the agent cluster needs to reserve interfaces for compatibility with message middleware and model collaboration protocols to avoid communication compatibility issues.

[0068] S202. Distribute unified semantic representations through message middleware, and each agent independently completes specialized analysis.

[0069] The goal of this step is to enable each intelligent agent to synchronously acquire complete power grid operation status information, output specialized analysis results based on its own professional algorithms, and provide original basis for subsequent decision-making and consultation.

[0070] In some implementations, low-latency, highly reliable message middleware is chosen as the data distribution carrier. The message middleware is deployed in a master-slave architecture to ensure high availability, and data transmission uses a binary serialization format to reduce transmission latency. During unified semantic representation distribution, the message middleware encapsulates the representation vector into message packets according to a preset format, containing fields such as data identifier, timestamp, vector dimension, and feature data. These packets are simultaneously sent to each agent in the agent cluster via a parallel push mechanism, ensuring that each agent starts analysis synchronously. After receiving the message, each agent first decapsulates and verifies the message packet, and then calls its own core algorithm to perform targeted analysis of the unified semantic representation. The new energy prediction agent uses a spatiotemporal attention mechanism to associate information such as weather conditions, equipment status, and power grid topology in the unified semantic representation with historical data, analyzes the fluctuation of new energy output in the next 15 minutes to 24 hours, and outputs a power output characteristic report that includes power prediction values ​​for different time periods and multiple confidence intervals such as 90%, 95%, and 99%. The power grid stability analysis agent, based on a transient stability assessment algorithm, extracts key information such as network topology, operating mode, and fault scenarios from a unified semantic representation, calculates the system stability margin, and generates a system stability state assessment report covering transient stability, voltage stability, and frequency stability. The formula for calculating the stability margin is: , For maximum transmission power, This represents the current transmission power. The load-side response agent uses deep learning algorithms to analyze information such as system balance demand, market electricity price signals, and user comfort constraints in the unified semantic representation, evaluates the regulation potential of various loads, and generates a set of load regulation potential assessment and optimization strategies that includes regulation capacity, response speed, and duration. The energy storage coordination agent uses a multi-scale optimization model to extract information such as system power fluctuations, stable demand, and energy gap from a unified semantic representation, and formulates energy storage charging and discharging strategies at the second, minute, and hour levels, clarifying key parameters such as charging and discharging power and state of charge constraints for each time period.

[0071] For example, when the system detects weather changes in the regional power grid, the multimodal large model base generates a unified semantic representation vector, such as a 1024-dimensional vector V=[v1,v2,v3,...,v1024], where some dimensions encode key features. For instance, dimensions 1-10 represent wind speed-related features, with values ​​corresponding to wind speed change rate +3.5m / s; dimensions 11-20 represent light intensity features, with values ​​corresponding to growth rate +40%; and dimensions 21-30 represent voltage deviation features, with values ​​corresponding to 0.05pu. The message middleware encapsulates this vector into a message packet (containing vector data, timestamps, and other fields) and pushes it in parallel to each agent. After receiving the data, each agent directly parses and analyzes the vector content: the new energy prediction agent extracts the weather-related dimensions from the vector and outputs the power prediction vector and confidence interval, such as a power output characteristic report stating that "the photovoltaic output will increase from 100MW to 150MW in the next hour, with a 95% confidence interval of [145MW, 155MW]"; the grid stability analysis agent calculates the system stability margin K=0.15 (lower than the safety threshold of 0.2) and outputs an assessment report stating that "voltage stability is at risk"; the load-side response agent assesses the adjustable capacity of the industrial load at 30MW; and the energy storage coordination agent formulates a charging and discharging strategy of "second-level power smoothing + minute-level voltage support".

[0072] S203. Conduct decision negotiation based on a predefined model collaborative protocol to generate a collaborative decision scheme containing a sequence of control instructions.

[0073] The goal of this step is to integrate the specialized analysis results of various agents through a standardized collaborative mechanism, resolve decision-making conflicts among multiple agents, and form a globally optimal and physically feasible collaborative decision-making scheme.

[0074] In some implementations, the predefined model collaboration protocol consists of three parts: standardized communication primitives, event-driven communication mechanisms, and negotiation decision-making processes. The specific implementation is as follows: 1. Standardized Communication Primitives Definition: This definition clarifies the format and usage scenarios of four core communication primitives: notification, request, commitment, and negotiation. "Notification" is used by agents to push analysis results and status information; "Request" is used by agents to obtain supplementary data or collaborative support from other agents; "Commitment" is used by agents to confirm acceptance of decision-making tasks or execution of control commands; and "Negotiation" is used by agents to exchange opinions on candidate solutions. Communication primitives adopt a structured data format, including elements such as sender identifier, receiver identifier, primitive type, content fields, and timestamps, ensuring the accuracy of information exchange.

[0075] 2. Establishment of event-driven communication mechanism: Three types of triggering conditions are set: data update events, including unified semantic representation update, analysis report generation, etc.; abnormal alarm events, including stability margin below threshold, output fluctuation exceeding expectations, etc.; control command events, including decision scheme generation, command execution feedback, etc. When any triggering condition is met, the communication mechanism is automatically triggered to promote the decision-making process and avoid ineffective waiting.

[0076] 3. Design of the consultation and decision-making process: Step 1: Task allocation.

[0077] First, a classification system for monitoring task types is defined, dividing tasks into emergency control (such as insufficient stability margin and equipment failure), optimization scheduling (such as load balancing and economical operation of energy storage), and early warning analysis (such as prediction of potential fluctuations and risk assessment). For each type of task, a task description template containing parameters such as time limit, computational complexity, and safety requirements is established. Simultaneously, a quantitative evaluation system for the capabilities and attributes of intelligent agents will be established, and domain expertise will be assessed through the accuracy of a professional test set for intelligent agents. And response time (T) quantized as The current load status is determined by the current number of tasks (N) and the remaining completion time (N). ) Quantified as Historical task success rate is measured by the completion rate (R) of similar tasks and the effect score ( ) Quantified as .

[0078] Once the collaborative decision-making requirement is triggered, the initial coordinator or the previous leading agent issues a tender notice as the tendering party. Each agent calculates its own capability attributes using a weighted cosine similarity algorithm. The degree of matching between the task requirements and the requirements ,in, , , These are weighting coefficients, and the weighting coefficients are set according to the task type, such as emergency control tasks. =0.4, =0.4, =0.2; when the matching degree exceeds the preset threshold (e.g., 0.7), a bidding message is submitted; The bidding party uses a weighted scoring method, for example... The bidding information is scored, and the agent with the highest score is selected as the dominant agent.

[0079] Step 2: Task release and candidate solution generation.

[0080] The leading intelligent agent issues decision-making tasks based on bidding requirements, specifying task descriptions, constraints (such as safety thresholds and user comfort limits) and optimization objectives (such as minimizing fluctuations and minimizing costs). Each agent generates candidate solutions based on its own analysis report. The solution generation rules for each agent are as follows: the new energy prediction agent generates scheduling solutions according to confidence intervals; the power grid stability analysis agent generates control measures according to stability problem types; the load-side response agent generates adjustment solutions according to load types; and the energy storage coordination agent generates operation solutions according to time scales. Based on the solution generation rules, the output results are input into a pre-defined solution generation language model. The model automatically generates structured and standardized candidate solution text descriptions and parameter sets using dedicated prompt word templates. Step 3: Utility function evaluation.

[0081] For each candidate solution, a multi-attribute utility function is constructed for quantitative evaluation, with the function form as follows: ,in For the i-th candidate solution, The utility value (ranging from 0 to 1). It is a set of evaluation index functions, including safety, economy, and reliability indicators, such as safety indicators. Economic indicators , These are the weighting coefficients. Each agent independently calculates and ranks the utility values ​​of candidate solutions, and selects the top 3-5 solutions with the highest utility values ​​to enter the debate stage.

[0082] Step 4: Multiple rounds of debate and solution optimization.

[0083] Using an argument-based negotiation framework, intelligent agents engage in multiple rounds of debate on the feasibility (e.g., whether it meets equipment rating constraints) and risk-reward (e.g., the effect of fluctuation suppression versus cost) of candidate solutions based on power grid system simulation data (such as PSCAD / EMTDC simulation results) and historical operating data. After each round of debate, solutions with greater controversy are eliminated by voting, ultimately forming an optimized set of candidate solutions.

[0084] Step 5: Decision-making scheme generation.

[0085] Based on the candidate solution optimization set, emergency control and optimization scheduling tasks adopt a voting mechanism (each agent has 1 vote, and votes are weighted according to utility value, with weights...). For early warning analysis tasks, an authoritative hierarchical strategy is adopted (the dominant agent has a weight of 0.5, and other agents each have a weight of 0.125). The scheme with the highest score is selected and transformed into a sequence of control instructions that includes the controlled object, control parameters, execution timing, and security verification standards, forming a collaborative decision-making scheme.

[0086] Step 6: Physical rationality verification.

[0087] The generated control command sequence is subjected to boundary checks and safety constraint verification, unreasonable commands are eliminated and re-optimized; the boundary checks include verifying whether the control parameters are within the rated range of the equipment, such as the energy storage charging and discharging power not exceeding the rated power; the safety constraint verification includes verifying whether the operation complies with the power grid safety regulations, such as the load shedding amount not exceeding 5% of the total system load.

[0088] In some implementations, the dedicated prompt word template is: New energy prediction intelligent agent dedicated prompt word template: You are a power system renewable energy dispatching expert. Please generate candidate dispatching schemes based on the following analysis results: [Input data: Output characteristic report, including power prediction values, confidence intervals, etc.]; Please generate the solution according to the following rules: 1. High confidence interval (≥90%): Generate a conservative dispatch scheme with power grid security as the primary objective; 2. Medium confidence interval (50%-90%): Generate an optimized scheduling scheme that balances security and economy; 3. Low confidence interval (<50%): Generate an aggressive dispatch scheme aimed at improving the renewable energy absorption rate.

[0089] Output format requirements: {"Solution Type":"Conservative / Optimized / Aggressive","Scheduling Strategy":"Description of Specific Control Measures","Power Setting":{"Time Period":"Power Value"},"Confidence Level":"Value","Risk Assessment":"Text Description"}.

[0090] Dedicated prompt word template for power grid stability analysis intelligent agents: You are a power system stability analysis expert. Please generate a control scheme based on the following stability assessment report: [Input data: System stability assessment report, including stability margin, risk level, etc.]; Please generate corresponding measures for different types of stability problems: 1. Transient stability issues: emergency control measures such as machine shutdown and load shedding; 2. Voltage stability issues: reactive power compensation, transformer voltage regulation, etc.; 3. Frequency stability issues: reserve capacity deployment and load frequency control.

[0091] Output format requirements: {"Problem type":"Transient / voltage / frequency stability","Control measures":"Detailed operation description","Target":"Equipment or node identifier","Control parameters":"Specific values","Expected effect":"Degree of stability improvement"}.

[0092] Load-side response agent-specific prompt word template: You are a load-side management expert. Please generate an optimized solution based on the load regulation potential assessment: [Input data: Load regulation potential assessment and optimization strategy set].

[0093] Please generate differentiated solutions based on load type: 1. Industrial interruptible loads: short-term interruptions based on price / incentive signals; 2. Commercial air conditioning load: Temperature setpoint adjustment; 3. Residential electric vehicle load: charging time shift.

[0094] Output format requirements: {"Load type":"Industrial / Commercial / Residential","Regulation strategy":"Specific control method","Regulation capacity":"Value (MW)","Response time":"Value (minutes)","User impact":"Assessment level"}.

[0095] Energy Storage Coordination Agent Dedicated Prompt Text Template: You are an energy storage system optimization expert. Please generate a multi-timescale coordination scheme based on system requirements: [Input data: Information such as system power fluctuations and stability requirements].

[0096] Please design in layers according to time scale: 1. Second-level: Fluctuation smoothing, with a focus on power response characteristics. 2. Minute-level: Frequency modulation control, with a focus on adjustment rate. 3. Hourly level: Energy management, with a focus on economical operation. Output format requirements: {"Time scale":"seconds / minutes / hours","Control mode":"charge / discharge and power values","SOC constraint":"state of charge range","Optimization objective":"main optimization indicators","coordination strategy":"coordination method with other equipment"}.

[0097] For example, a sudden increase in photovoltaic power output in a certain region causes voltage stability risks (an emergency control task). After the bidding party issues a bidding announcement, the power grid stability analysis agent (matching degree M=0.85) and the energy storage coordination agent (M=0.78) submit bids. After weighted scoring, the power grid stability analysis agent is selected as the leading agent. After the leading agent issues the task of "restoring the bus voltage to 10-10.5kV within 1 minute", each agent generates candidate solutions: the power grid stability analysis agent proposes "putting on 1 set of reactive power compensation device" (U=0.92), the energy storage coordination agent proposes "energy storage output reactive power support" (U=0.88), and the load-side response agent proposes "reducing 5MW of industrial load" (U=0.8). After one round of debate, the "reducing industrial load" solution is eliminated (due to its significant impact on users). Finally, the combined solution of "putting on reactive power compensation device + energy storage reactive power support" is selected through weighted voting. After the control command sequence is physically verified (reactive power compensation capacity is within the rated range and energy storage state of charge meets the constraints), the final collaborative decision solution is formed.

[0098] Based on the above technical solution, S2 constructs a specialized intelligent agent cluster through S201 to ensure the depth and accuracy of analysis at each stage; through S202, it achieves efficient distribution and independent analysis of unified semantic representations, providing comprehensive original evidence for decision-making; through the standardized model collaboration protocol in S203, it realizes orderly negotiation and scientific decision-making among intelligent agents, and the final collaborative decision-making scheme has both global optimality and physical feasibility. This step deeply integrates expert knowledge, physical mechanisms, and data-driven models, breaking through the decision-making limitations of isolated systems, significantly improving the decision-making quality and efficiency of power systems in response to new energy fluctuations, and providing reliable support for subsequent real-time monitoring and control command generation.

[0099] In one possible implementation of the embodiments of this application, combined with Figure 2 ,like Figure 5 As shown, the above S3 can be implemented through the following S301, S302 and S303, which are explained in detail below: S301, intelligent agent cluster collaboratively executes real-time monitoring to comprehensively capture the operating status of power equipment and the fluctuation trend of the power grid.

[0100] The goal of this step is to leverage the specialized division of labor within the intelligent agent cluster to achieve uninterrupted monitoring of all aspects of the power system, from power source to grid to load to storage, and to summarize equipment operation data and status information in real time, providing a dynamic basis for the generation of control commands.

[0101] In some implementations, real-time monitoring adopts a distributed monitoring and centralized aggregation model: The new energy forecasting intelligent agent collects the output data of wind farms and photovoltaic power plants in real time, compares the deviation between the predicted value and the real-time value, and monitors the fluctuation trend of new energy output. The power grid stability analysis agent continuously extracts key operating parameters such as bus voltage, line current, and system frequency, calculates system stability margin in real time, and dynamically evaluates transient, voltage, and frequency stability. The load-side response intelligent agent monitors the real-time operating status of adjustable loads such as industrial loads, commercial air conditioning, and residential electric vehicles, and updates load adjustment potential data. The energy storage coordination intelligent agent monitors key indicators such as the state of charge, charging and discharging power, and equipment temperature of energy storage devices in real time to ensure that the energy storage system is in a responsive state.

[0102] It should be noted that the core value of real-time monitoring lies in dynamic perception. Unlike traditional fixed-period monitoring, collaborative monitoring by intelligent agent clusters can achieve synchronous collection and correlation analysis of multi-dimensional data. It can capture the local state of individual devices and link the overall power grid operation trend, avoiding the lag in fluctuation prediction caused by the omission of single-point monitoring.

[0103] S302. Analyze collaborative decision-making schemes, generate standardized control instructions, and complete physical rationality verification.

[0104] The goal of this step is to transform the abstract collaborative decision-making scheme into structured control instructions that the equipment can directly execute, and to ensure the security and feasibility of the instructions through rigorous verification.

[0105] In some implementations, the control command generation and verification process is as follows: The first step is instruction structure parsing. From the collaborative decision-making scheme, the controlled object (e.g., a wind farm, a reactive power compensation device, an energy storage power station), control parameters (e.g., output adjustment value, charging / discharging power, voltage setpoint), execution sequence (e.g., immediate execution, execution with a 30-second delay), and expected effect (e.g., restoring the voltage to 10.5kV, smoothing out 20MW output fluctuations) are extracted. Following the IEC 61850 standard power communication protocol, a standardized instruction format containing "instruction ID - executing entity - parameter type - parameter value - execution time limit - safety threshold" is constructed. The second step, physical rationality verification, includes boundary checks and safety constraint verification: boundary checks verify whether control parameters are within the allowable range of the equipment using the equipment's rated parameter library; safety constraint verification, based on power system safety regulations, verifies whether the executed commands meet grid operation constraints. To quantify the fluctuation control objective, a fluctuation deviation calculation formula is introduced to quantify the current fluctuation level, ensuring that the parameter settings of the control commands can account for the fluctuation deviation. Keep within the allowable range: ,in The fluctuation deviation between the actual output and the target output of new energy sources. Real-time power output data for new energy sources The target output value determined by the collaborative decision-making scheme.

[0106] S303 transmits and executes control commands, provides real-time feedback on fluctuation suppression effects, and dynamically adjusts them.

[0107] The goal of this step is to ensure that control commands are reliably transmitted to the target equipment and executed accurately. By monitoring the fluctuation suppression effect through a closed-loop feedback mechanism, command parameters are adjusted in a timely manner to achieve continuous smoothing of fluctuations in new energy grid connection.

[0108] In some implementations, control command transmission employs a dual-link redundancy mode of industrial Ethernet and 5G to avoid command transmission failure due to a single link interruption. The transmission protocol selected is MQTT (Message Queuing Telemetry Transport) to ensure low latency and high reliability. During the command execution phase, the target device automatically parses and executes the operation after receiving the command, while simultaneously providing real-time feedback on the execution status. Fluctuation suppression effect monitoring is achieved through secondary monitoring by an intelligent agent cluster: the power grid stability analysis intelligent agent calculates the system stability margin after command execution, and the new energy prediction intelligent agent compares the output fluctuation deviation to jointly quantify the suppression effect, introducing a suppression efficiency calculation formula: ,in For fluctuation suppression efficiency, This refers to the fluctuation deviation before the instruction is executed. This represents the fluctuation deviation after execution. If... If the efficiency is less than 80%, the collaborative decision-making module will be triggered to re-optimize the scheme and generate adjustment instructions, such as increasing the energy storage discharge power or calling in standby load for regulation; if We will continuously monitor fluctuation trends to ensure the stable operation of the power grid.

[0109] It should be noted that the closed-loop feedback mechanism is the key to achieving "fluctuation-suppression-re-optimization" - the fluctuation of new energy output is random and dynamic, and a single command is difficult to fully adapt to the continuously changing scenario. Through real-time feedback and dynamic adjustment, the control command can always fit the actual fluctuation situation, ensuring the stability and continuity of the suppression effect.

[0110] Based on the above technical solutions, S3 achieves dynamic and accurate perception of the power grid's operating status through multi-dimensional collaborative real-time monitoring via S301; ensures the security and feasibility of control commands through structured command generation and physical rationality verification via S302; and achieves continuous and effective suppression of new energy grid connection fluctuations through reliable transmission, closed-loop feedback, and dynamic adjustment via S303. This step deeply integrates the professional capabilities of the intelligent agent cluster with real-time control, breaking the limitations of the traditional monitoring-decision-execution disconnect, forming a complete fluctuation prevention and control closed loop, significantly improving the power system's rapid response capability and stable operation level in the face of new energy fluctuations, and providing reliable operation and maintenance execution support for scenarios with a high proportion of new energy access.

[0111] Although this application has been described herein in conjunction with various embodiments, those skilled in the art, by reviewing the accompanying drawings, disclosure, and appended claims, will understand and implement other variations of the disclosed embodiments in carrying out the claimed application. In the claims, the word "comprising" does not exclude other components or steps, and "a" or "an" does not exclude multiple instances. A single processor or other unit can implement several functions listed in the claims. While different dependent claims may recite certain measures, this does not mean that these measures cannot be combined to produce good results.

[0112] Although this application has been described in conjunction with specific features and embodiments, it is obvious that various modifications and combinations can be made thereto without departing from the spirit and scope of this application. Accordingly, this specification and drawings are merely exemplary illustrations of this application as defined by the appended claims, and are considered to cover any and all modifications, variations, combinations, or equivalents within the scope of this application. Clearly, those skilled in the art can make various alterations and modifications to this application without departing from the spirit and scope of this application. Thus, if such modifications and modifications of this application fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and modifications.

Claims

1. A power multi-agent monitoring and maintenance method based on multimodal data, characterized in that, include: Multi-source heterogeneous data from the power system are collected and fused using a multimodal large model base to generate a unified semantic representation; wherein, the multi-source heterogeneous data includes time-series data, spatial data, image data and text data; Based on the unified semantic representation, collaborative analysis and decision-making of the agent cluster are carried out through a predefined model collaboration protocol to generate a collaborative decision-making scheme containing a sequence of control commands; the agent cluster includes a new energy prediction agent, a power grid stability analysis agent, a load-side response agent, and an energy storage coordination agent; The intelligent agent cluster is used to monitor power equipment in real time and generate control commands based on the collaborative decision-making scheme.

2. The power multi-agent monitoring and operation method based on multi-modal data according to claim 1, characterized in that, The multimodal large model base is built based on the Transformer architecture, and the construction process includes: Construct a neural network model containing a multi-layer Transformer encoder-decoder architecture to obtain the initial large model; An initial large model is pre-trained using a general multimodal corpus, and the model parameters are optimized using masked language modeling and contrastive learning objective functions to obtain a general pre-trained large model; the general multimodal corpus includes multi-source datasets of text-image pairs, time series, and cross-modal association data; Historical multi-source heterogeneous data of the power system are collected to obtain a dedicated dataset for the power sector. Domain-adaptive pre-training technology is employed, inputting a power industry-specific dataset into a general pre-trained large model for further training. The learning rate and parameter fine-tuning methods are dynamically adjusted to minimize the domain adaptation loss, and a retrieval-enhanced generation mechanism is introduced to strengthen domain knowledge fusion, resulting in a multimodal large model foundation. The multimodal large model base is used to perform multimodal fusion and semantic representation generation on time-series monitoring data, spatial topology data, equipment image data and text log data in the power system, supporting collaborative decision-making of intelligent agent clusters; In the application of the multimodal large model base, only the encoder part is retained for feature extraction. The encoder performs forward propagation on the input multi-source heterogeneous data and outputs a unified semantic representation vector.

3. The power multi-agent monitoring and operation method based on multi-modal data according to claim 2, characterized in that, The introduced retrieval enhancement generation mechanism enhances domain knowledge fusion, including: Construct a dedicated knowledge base for the power industry, which stores structured and unstructured knowledge in the power industry, including equipment parameter entities, operating procedure texts, historical fault cases, and multimodal correlation data; The encoder maps the power industry-specific dataset into query vectors, and the nearest neighbor search algorithm is used to retrieve the top K knowledge points from the knowledge base to obtain the retrieval results. The search results are fused with an attention-based dataset in the power sector, and an enhanced semantic representation is generated through weighted summation or concatenation. The enhanced semantic representation is used as the key vector and value vector of the decoder's cross-attention module, and the hidden state output by the decoder at the previous time step is used as the query vector. Through the attention weight calculation and weighted summation of the cross-attention module, a context vector that integrates professional knowledge in the power field and contextual information is obtained. The context vector is processed by the feedforward neural network layer of the decoder to generate a target sequence containing control instructions, and the generated content is physically validated to achieve enhanced domain knowledge fusion; the physical validation includes boundary checks and security constraint verification.

4. The power multi-agent monitoring and operation method based on multi-modal data according to claim 1, characterized in that, The collaborative analysis and decision-making of the agent cluster through a predefined model collaboration protocol includes: A unified semantic representation is pushed to each agent in parallel through a message middleware; Each agent independently analyzes the unified semantic representation; among them, The new energy prediction agent analyzes the fluctuation of new energy output within a preset time period through a spatiotemporal attention mechanism, and obtains an output characteristic report that includes power prediction values ​​and confidence intervals. The power grid stability analysis agent calculates the system stability margin based on the transient stability assessment algorithm and obtains a system stability state assessment report; The load-side response agent uses a deep learning algorithm to generate an adjustable and optimized scheduling scheme, resulting in an assessment of the adjustment potential and a set of optimization strategies. The energy storage coordination agent formulates the charging and discharging strategy of the energy storage system through a multi-scale optimization model. Based on the output of each agent, decision-making negotiation among the agents is carried out through a predefined model collaboration protocol.

5. The power multi-agent monitoring and operation method based on multi-modal data according to claim 1, characterized in that, The decision negotiation among agents through a predefined model collaboration protocol includes: Define standardized agent communication primitives; the communication primitives include notification, request, commitment, and negotiation; Establish an event-driven communication mechanism and set three types of triggering conditions: data update events, abnormal alarm events, and control command events; Design the agent negotiation and decision-making process, including: The Contract Network protocol is used for task allocation. Based on the matching degree between task type and the capability attributes of each agent, the leading agent is determined through a bidding-tendering-winning mechanism. The capability attributes include domain expertise, current load status, and historical task success rate. The leading intelligent agent issues decision-making tasks; the decision-making tasks include task descriptions, constraints, and optimization objectives. Each agent generates candidate solutions based on its own output and evaluates the merits of the solutions through a utility function. Using an argument-based negotiation framework, the agents engage in multiple rounds of debate on the feasibility, risks, and benefits of the proposed solutions based on power grid system simulation data and historical data, thereby obtaining an optimized set of candidate solutions. Based on the optimized set of candidate solutions, a collaborative decision-making scheme containing a sequence of control instructions is generated through a voting mechanism or an authority hierarchy strategy.

6. The power multi-agent monitoring and maintenance method based on multimodal data according to claim 5, characterized in that, The process of determining the leading agent based on the matching degree between task type and the capability attributes of each agent, through a bidding-tendering-winning mechanism, includes: Define a classification system for monitoring task types, dividing tasks into three types: emergency control, optimization scheduling, and early warning analysis. For each type of task, establish a task description template that includes parameters such as time limit, computational complexity, and security requirements. Establish a quantitative evaluation system for the capabilities of intelligent agents. Domain expertise is quantified based on the accuracy and response time of the agent on the test set in its respective professional field. Current load status is quantified by the number of tasks currently being processed by the agent and the remaining completion time. Historical task success rate is calculated by statistically analyzing the completion rate and performance score of the agent in executing the same type of task within a historical time period. When a collaborative decision-making requirement is triggered, the bidding party issues a bidding announcement; the announcement includes a task type identifier, a description of the task requirements, and a bid deadline; the bidding party is the initial coordinator or the leading intelligent agent in the previous round; Each intelligent agent calculates the matching degree between its own capability attributes and task requirements using a weighted cosine similarity algorithm based on the received tender notice. If the matching degree exceeds a preset threshold, it returns a tender message to the tendering party containing capability attributes, estimated execution time, and resource requirements. After receiving the bidding information, the bidding entity uses a weighted scoring method to comprehensively score different bidding information before the bidding deadline, and selects the agent with the highest comprehensive score as the leading agent for this round of decision-making and negotiation.

7. The power multi-agent monitoring and operation method based on multi-modal data according to claim 1, characterized in that, The construction process of the new energy prediction intelligent agent includes: A prediction model based on an encoder-decoder architecture is constructed. A graph attention network is introduced at the encoder end of the prediction model to model the power grid topology, and a temporal attention mechanism is introduced at the decoder end to perform weighted processing on the historical sequence. The model was trained using a dedicated dataset for the power sector to obtain a trained new energy prediction agent. The training data included historical new energy output data, weather forecast data, terrain elevation data, and semantic features related to new energy in a unified semantic representation. When applied, the new energy prediction intelligent agent can parse information about weather conditions and equipment status in the unified semantic representation and generate an output characteristic report that includes power prediction values, i.e., confidence intervals.

8. The power multi-agent monitoring and operation method based on multi-modal data according to claim 1, characterized in that, The construction process of the power grid stability analysis agent includes: A hybrid intelligent analysis model is obtained by integrating a physical mechanism model and a data-driven model; wherein the physical mechanism model is constructed based on a system of differential-algebraic equations of a power system, and the data-driven model is constructed based on a graph neural network and a deep belief network. The hybrid intelligent analysis model is trained using power system simulation data and historical operating data to obtain the trained power grid stability analysis agent; the training data includes power flow calculation results, transient stability simulation results, actual waveform recording data, and semantic features related to system stability in the unified semantic representation. When applied, the power grid stability analysis agent can parse information about network topology, operation mode, and fault scenarios in the unified semantic representation, calculate the system stability margin, and generate a system stability state assessment report; the stability state assessment includes transient stability assessment, voltage stability assessment, and frequency stability assessment. 9.The power multi-agent monitoring and operation method based on multi-modal data according to claim 1, characterized in that, The construction process of the load-side response agent includes: Construct a multi-agent reinforcement learning framework, modeling each adjustable load as an agent; The reinforcement learning framework is trained using historical data, electricity price data, user behavior data, and semantic features related to load characteristics from a unified semantic representation. By using clustering algorithms to classify load resources and quantifying the price response capabilities of different types of load resources based on elasticity coefficient models, a load response evaluation module of a reinforcement learning model is obtained. During the training process of the reinforcement learning framework, a multi-objective deep deterministic policy gradient algorithm is used for optimization to obtain the load-side response agent; among which, the optimization objectives include peak shaving and valley filling effect, user satisfaction, and power grid security constraints. When applied, the load-side response agent can parse information about system balance demand, market electricity price signals, and user comfort constraints in the unified semantic representation, generate an optimized scheduling scheme for adjustable loads, and output load response evaluation results and an optimization strategy set.

10. The power multi-agent monitoring and maintenance method based on multimodal data according to claim 1, characterized in that, The construction process of the energy storage coordination agent includes: Construct an optimization model framework that includes second-level, minute-level, and hour-level calculations; In the process of solving the optimization model, the model predictive control method is used for rolling optimization, and the energy storage lifetime loss cost is added to the objective function to construct a multi-timescale optimization model with the goal of minimizing the total life cycle cost. The multi-timescale optimization model is trained using historical operating data, equipment status data, market price data, and semantic features related to energy storage operation in a unified semantic representation. The parameters of the multi-timescale optimization model are then adjusted using an online learning algorithm to obtain an energy storage coordination agent. When applied, the energy storage coordination agent can parse information about system power fluctuations, stability requirements, and energy-related factors in semantic representations, formulate charging and discharging strategies for the energy storage system, and output multi-timescale coordinated control schemes.

Images