A new energy power station digital twin autonomous operation and maintenance system and method based on NeRF and multi-agent

Abstract

The application relates to a new energy power station digital twin autonomous operation and maintenance system and method based on NeRF and multiple agents, and belongs to the technical field of new energy power station operation and maintenance. A technical scheme: multi-view images of a power station site are collected by using a drone or a fixed camera, a super-realistic and three-dimensional roamable geometric appearance model of a power plant is constructed through NeRF technology; on this basis, real-time operation data is accessed, and a full-factor digital twin of the power station covering a physical model, a data model and a rule model is constructed; and a multiple agent system is introduced, the system contains multiple agents with different functions, is responsible for cooperatively carrying out twin data fusion, state monitoring, fault diagnosis, risk prediction and autonomous operation and maintenance decision, and finally feeds back a control site execution mechanism or guides manual operation and maintenance through the digital twin. The application has more intelligent and more collaborative complex fault diagnosis and root cause analysis capability, and realizes closed-loop autonomous operation and maintenance from state perception to autonomous decision and execution.

Description

Technical Field

[0001] This invention relates to a digital twin autonomous operation and maintenance system and method for new energy power plants based on NeRF and multi-agent technology, belonging to the field of new energy power plant operation and maintenance technology. Background Technology

[0002] With the advancement of "dual carbon" targets, the installed capacity and complexity of new energy power plants are increasing dramatically, posing significant challenges to their operation and maintenance. Traditional operation and maintenance models heavily rely on periodic manual inspections and post-incident repairs, resulting in high costs, low efficiency, significant risks (such as high-altitude photovoltaic panels and wind turbines), and difficulty in real-time monitoring of the overall status. "Digital twin" technology provides a framework for achieving visualized, analyzable, and predictable intelligent operation and maintenance of industrial facilities. It simulates the behavior of physical entities in a real environment by replicating them 1:1 in a virtual space and combining real-time data. For new energy power plants, a high-fidelity 3D model is fundamental to building a high-quality digital twin. However, traditional 3D modeling methods (such as laser scanning and manual modeling) suffer from high costs, long development cycles, and difficulties in updating models.

[0003] In recent years, NeRF technology based on deep learning has achieved breakthroughs in the field of 3D reconstruction. Compared with traditional methods, NeRF only needs to collect multi-view photos of the target scene to reconstruct a 3D scene with realistic visual effects and fine geometric details. This provides a new approach for generating 3D models of power plants quickly, at low cost, and with high accuracy. Meanwhile, complex operation and maintenance tasks are difficult to complete with a single algorithm or module. The concept of multi-agent systems is well-suited for such scenarios, modularizing different analysis and decision-making functions into independent agents, and addressing complex, dynamic, and multi-objective power plant operation and maintenance problems through division of labor, collaboration, and communication.

[0004] The closest existing technology to this application is "an intelligent operation and maintenance system for new energy power plants based on digital twins"; this solution typically includes the following parts: 3D modeling module: Typically uses LiDAR scanning or oblique photogrammetry technology to create 3D models of the power plant site and equipment, generating interactive 3D models; Data acquisition and access module: Deploy various sensors (such as current, voltage, temperature sensors, environmental monitors), cameras, and SCADA systems to collect real-time operating data and environmental data of the power plant; Digital Twin Building Module: This module combines 3D models with real-time data and equipment information models to form a visualized digital twin platform. The platform can display equipment location, real-time data annotations, alarm information, etc. Intelligent operation and maintenance application module: Provides functions such as data analysis, fault diagnosis (e.g., identifying hot spots on photovoltaic panels based on image recognition, and identifying wind turbine anomalies based on data analysis), and visual monitoring. Some advanced systems may integrate simple predictive maintenance models.

[0005] While existing technologies have incorporated the concept of digital twins, they still suffer from significant technical shortcomings in construction and application, resulting in limited operational effectiveness: ① High cost and difficulty in updating 3D models: Existing technologies primarily rely on laser scanning or oblique photography. Laser scanning equipment is expensive, and data acquisition requires specialized expertise; while oblique photography generates 3D models quickly, its texture and geometric details (especially for equipment surfaces and internal structures) are limited in accuracy, insufficient to support refined diagnostics. Furthermore, the cost and time of remodeling when equipment is replaced or the layout is adjusted are exorbitant, causing the twin to gradually become "unrealistic" compared to the physical entity. ② Insufficient realism and immersion in 3D models: Traditional 3D models are mostly "geometric models + textures," resulting in significant differences from real-world scenes in terms of lighting effects, reflections, and detailed material representation. This affects the experience of virtual inspections and remote collaboration, and makes it difficult to support automated training for identifying subtle faults (such as fine cracks or specific stains) based on highly realistic visual features. ③ Insufficient "intelligence" and "autonomy" in operational decision-making: Existing systems are mostly a combination of "data visualization + single-point analysis tools." Fault diagnosis and predictive analysis modules often operate in isolation, resulting in weak capabilities to handle complex, interconnected faults. For example, a decrease in photovoltaic array output may be caused by a combination of factors, such as localized contamination, shading, component degradation, and inverter failure. Existing systems struggle to dynamically coordinate different analysis modules for correlation reasoning and root cause localization. Furthermore, the closed loop from "diagnosis" to "treatment" typically requires manual intervention, lacking the ability for autonomous decision-making and execution. ④ Weak system scalability and collaboration: System functions are usually fixed or modular; adding new analysis algorithms or integrating new equipment types often necessitates modifications to the overall architecture. The system exhibits poor adaptability when facing constantly emerging new fault modes and maintenance scenarios. Analysis models developed by different manufacturers and at different times also struggle to work effectively together. Summary of the Invention

[0006] This invention proposes a digital twin autonomous operation and maintenance system and method for new energy power plants based on NeRF and multi-agent systems. It constructs a more high-fidelity, lower-cost, and easily updatable three-dimensional base of the power plant, and has more intelligent and collaborative capabilities for complex fault diagnosis and root cause analysis. It realizes closed-loop autonomous operation and maintenance from "state perception" to "autonomous decision-making and execution", improves the overall modularity, scalability and adaptability of the system, and solves the problems of existing new energy (such as photovoltaic and wind power) power plant operation and maintenance relying on manual inspection, low inspection efficiency, delayed fault detection and handling, and weak collaborative analysis capabilities of multiple types of equipment.

[0007] The technical solution of this invention is: A new energy power plant digital twin autonomous operation and maintenance method based on NeRF and multi-agent technology is proposed. This method utilizes drones or fixed cameras to collect multi-view images of the power plant site, and constructs a hyper-realistic, 3D roamable geometric appearance model of the power plant using NeRF technology. Based on this, real-time operational data is integrated to construct a digital twin of all elements of the power plant, encompassing physical, data, and rule models. A multi-agent system is introduced, comprising multiple agents with different functions, responsible for collaborative twin data fusion, status monitoring, fault diagnosis, risk prediction, and autonomous operation and maintenance decision-making. Ultimately, the digital twin provides feedback to control on-site actuators or guides manual operation and maintenance.

[0008] The field actuators include cleaning robots and fault isolation devices.

[0009] The method includes the following steps: S101, Multi-view Image Acquisition and Processing: Deployed autonomous flight drones or fixed-point camera arrays at new energy power plants to periodically or irregularly acquire multi-view two-dimensional images of the covered equipment; perform distortion correction, alignment, and preprocessing on the images; S102, High-fidelity 3D reconstruction based on NeRF: The preprocessed multi-view images are input into the trained NeRF model; the model contains an MLP neural network that learns the mapping from spatial position and viewing direction to color and volume density; after training, this step outputs a high-precision, high-fidelity 3D neural representation of the power station; this representation not only supports photorealistic rendering of any viewpoint, but also implicitly contains accurate geometric information. S103, integrating real-time data to construct a dynamic digital twin: using the NeRF 3D model generated in S102 as the core geometric and visual basis, it connects to the real-time operation data, environmental monitoring data, and monitoring video stream of the power plant SCADA system through an IoT gateway and data interface; establishing a one-to-one mapping of equipment objects in the twin space to form a perceptible, interactive, and computable dynamic digital twin; the state of the twin is updated synchronously with the physical world. S104, Deployment and Operation of Multi-Agent Autonomous Operation and Maintenance System (MAS): Deploy a multi-agent system on the server or cloud platform where the digital twin resides; the agents interact, request, and collaborate with each other through standardized communication protocols; they share a knowledge base containing device models, rule bases, and historical cases; S105, Decision Feedback and Synchronization with the Digital Twin: The task scheduling and execution agent sends the generated decision instructions to the execution units or maintenance personnel in the physical world; the results of instruction execution are fed back to the system through sensors and SCADA data, and received by the perception and fusion agent; the system updates the status of the corresponding device in the digital twin accordingly, and records this maintenance case to form a closed loop; the entire maintenance process is fully simulated, recorded and visualized in the digital twin.

[0010] The equipment in step S101 includes a photovoltaic panel array, wind turbine blades, a transformer, and a combiner box.

[0011] The geometric information in step S102 includes surface normals and depth.

[0012] The real-time operating data in step S103 includes power, voltage, current, and temperature; the environmental monitoring data includes irradiance, wind speed, and temperature; and the equipment objects include each photovoltaic panel and each wind turbine.

[0013] The multi-agent system in step S104 includes, but is not limited to, the following types of agents, which are instantiated and work together. These agents interact, request, and collaborate through communication protocols based on WebSocket or message queues: ① Perception and Fusion Agent: Responsible for receiving external sensor and SCADA data, cleaning, aligning, and fusing it, and publishing the standardized data to a shared data blackboard or directly pushing it to other agents; ② 3D Vision Analysis Agent: Utilizes high-definition images or geometric information rendered from arbitrary perspectives using the NeRF model, runs computer vision-based detection algorithms, and identifies physical anomalies in the equipment; ③ Operational Status Diagnosis Agent: Based on a data-driven model or physical knowledge model... The system analyzes operational data to diagnose electrical and performance anomalies; ④ Risk prediction and decision-making intelligent agent: integrating visual analysis, data diagnostic results, historical data, and weather forecast information, using reinforcement learning or decision tree models to predict equipment performance degradation trends, failure probability, and power generation loss risks; when a fault or high risk is detected, the intelligent agent generates autonomous operation and maintenance decisions based on preset rules and cost-benefit analysis; ⑤ Task scheduling and execution intelligent agent: receiving instructions from the decision-making intelligent agent, decomposing them into executable tasks, scheduling physical execution units to perform operations, or pushing detailed work orders to the handheld terminals of operation and maintenance personnel, and providing AR remote collaboration guidance; the intelligent agent also monitors the execution status of tasks and provides feedback to the decision-making intelligent agent.

[0014] The three-dimensional visual analysis intelligent agent identifies physical anomalies in the equipment, including hot spots, cracks, stains, vegetation obstruction, and surface damage and icing on the wind turbine blades; the data-driven model for the operational status diagnosis intelligent agent is a deep learning LSTM network, and the performance anomalies include string faults, inverter efficiency degradation, and abnormal wind turbine vibration.

[0015] When the risk prediction and decision-making agent detects a fault or high risk, it generates autonomous operation and maintenance decisions based on preset rules and cost-benefit analysis, such as: "triggering the A12 area photovoltaic panel cleaning robot to start operation", "marking the B03 wind turbine for preventive maintenance and performing it in the next windless window", and "isolating the faulty combiner box C05 and switching to the backup line".

[0016] The task scheduling and execution intelligent agent scheduling physical execution unit includes cleaning robots, drones, on-site mobile inspection terminals, and remote control switches.

[0017] The decision instruction in step S105 includes control signals and work orders, and the result of the instruction execution includes whether the fault has been eliminated and whether the cleaning has been completed.

[0018] A NeRF-based and multi-agent-based digital twin autonomous operation and maintenance system for new energy power plants, used to run the aforementioned NeRF-based and multi-agent-based digital twin autonomous operation and maintenance method for new energy power plants, the system includes: ① Data Acquisition and Processing Module: Corresponding to step S101, this module includes a UAV control unit and an image processing unit, used to acquire and preprocess multi-view images; ② High-Fidelity 3D Modeling Module: Corresponding to step S102, this module has a built-in NeRF model training and inference engine, generating neural radiation fields from input images; ③ Twin Construction and Driving Module: Corresponding to step S103, this module is responsible for fusing the NeRF model, real-time data, and equipment information model to generate and update a dynamic digital twin; ④ Multi-Agent Autonomous Operation and Maintenance Module: Corresponding to steps S104 and S105, this module internally instantiates and runs various agents, including an agent management unit, communication bus, knowledge base, and decision engine; ⑤ Human-Computer Interaction and Visualization Module: Based on NeRF rendering technology and a 3D engine, this module provides an immersive digital twin visualization interface for operation and maintenance personnel, displaying real-time status, alarms, analysis results, and operation and maintenance instructions; ⑥ Instruction Issuance and Feedback Interface Module: This module is responsible for converting autonomous operation and maintenance decisions into specific control instructions and sending them to various actuators or personnel terminals at the power plant site, while also receiving execution feedback.

[0019] Key aspects of this invention: ① NeRF as a high-fidelity 3D visual foundation for digital twins: Utilizing NeRF technology to transform low-cost multi-view images into high-fidelity, 3D roamable scene models rich in geometric information suitable for digital twins. ② "Multi-agent collaboration" driving twin intelligence: Reconstructing traditional single or loosely coupled intelligent operation and maintenance functions into a system composed of multiple clearly defined, autonomously interactive, and collaborative intelligent agents, deeply embedded in the digital twin framework. ③ Intelligent agent-level fusion of visual perception and data perception: Designing specialized visual analysis and data diagnostic intelligent agents, and deeply associating and fusing them at the decision-making level to overcome the limitations of single-modal analysis. ④ Forming an autonomous operation and maintenance closed loop: By introducing decision-making and execution intelligent agents, the entire chain from digital space analysis and decision-making to physical space execution feedback is connected, truly achieving autonomous operation and maintenance, rather than merely auxiliary monitoring.

[0020] Compared with the prior art, the present invention has the following beneficial effects: Advantage 1: A higher-fidelity, lower-cost, and easily updated 3D power plant foundation is constructed. This invention uses NeRF technology based on multi-view images (S102) to replace laser scanning or traditional photogrammetry. The direct technical effect is that only ordinary cameras are needed to acquire images, greatly reducing hardware and data acquisition costs. NeRF can learn implicit scene representations, and the final result is that the generated 3D model has photorealistic feel and accurate geometry, far exceeding the visual and geometric accuracy of traditional modeling. At the same time, when small-scale changes occur on site, only new images of the affected area need to be taken, and incremental updates can be performed by combining them with the original model (S101), which directly brings convenience and low cost to model updates.

[0021] Advantage 2: Achieves more intelligent and collaborative capabilities for complex fault diagnosis and root cause analysis. This invention sets up "3D visual analysis" (identifying physical damage) and "operational status diagnosis" (analyzing electrical performance) as independent but collaborative agents (S104). This changes the existing system's isolated multi-module analysis. When power generation efficiency decreases, the visual analysis agent can detect obstructions or stains, and the data diagnosis agent can detect abnormal string currents. Both agents share their findings with the "risk prediction and decision-making agent" through a communication mechanism. The decision-making agent performs associative reasoning, enabling a more accurate determination that the fault is a coupled fault where "stains cause hot spots, leading to a decrease in current," rather than just an isolated phenomenon. This achieves collaborative diagnosis of complex coupled problems.

[0022] Advantage 3: It achieves a closed-loop autonomous operation and maintenance system from "state awareness" to "autonomous decision-making and execution." Most existing digital twin platforms stop at "visualization" and "analysis and alarms." This invention introduces a multi-agent system, specifically establishing a "risk prediction and decision-making agent" and a "task scheduling and execution agent" (S104). Based on multi-dimensional analysis results and a preset strategy library, the decision-making agent can automatically generate handling suggestions or even make direct decisions (e.g., triggering automatic cleaning or initiating preventative maintenance work orders). The scheduling and execution agent is responsible for translating decisions into specific, executable action instructions and issuing them (S105). This forms a complete closed loop of "perception -> analysis -> decision -> execution -> feedback," significantly improving the speed of operation and maintenance response and the level of automation.

[0023] Advantage 4: Enhanced system modularity, scalability, and adaptability. Employing a multi-agent architecture, each agent has cohesive functions and clear interfaces, communicating via standard protocols. The direct benefit is that when new diagnostic algorithms (such as new AI recognition models) or new operational scenarios (such as adding energy storage devices) need to be added, only a new corresponding agent needs to be developed and "registered" into the system, or the capabilities of existing agents need to be extended; there is no need to reconstruct the entire system architecture. This improves the system's adaptability to new requirements and technologies and reduces the complexity of later functional expansion. Detailed Implementation

[0024] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be described in detail below with reference to specific embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0025] A new energy power plant digital twin autonomous operation and maintenance method based on NeRF and multi-agent technology is proposed. This method utilizes drones or fixed cameras to collect multi-view images of the power plant site, and constructs a hyper-realistic, 3D roamable geometric appearance model of the power plant using NeRF technology. Based on this, real-time operational data is integrated to construct a digital twin of all elements of the power plant, encompassing physical, data, and rule models. A multi-agent system is introduced, comprising multiple agents with different functions, responsible for collaborative twin data fusion, status monitoring, fault diagnosis, risk prediction, and autonomous operation and maintenance decision-making. Ultimately, the digital twin provides feedback to control on-site actuators or guides manual operation and maintenance.

[0026] The field actuators include cleaning robots and fault isolation devices.

[0027] The method includes the following steps: S101, Multi-view Image Acquisition and Processing: Deployed autonomous flight drones or fixed-point camera arrays at new energy power plants to periodically or irregularly acquire multi-view two-dimensional images of the covered equipment; perform distortion correction, alignment, and preprocessing on the images; S102, High-fidelity 3D reconstruction based on NeRF: The preprocessed multi-view images are input into the trained NeRF model; the model contains an MLP neural network that learns the mapping from spatial position and viewing direction to color and volume density; after training, this step outputs a high-precision, high-fidelity 3D neural representation of the power station; this representation not only supports photorealistic rendering of any viewpoint, but also implicitly contains accurate geometric information. S103, integrating real-time data to construct a dynamic digital twin: using the NeRF 3D model generated in S102 as the core geometric and visual basis, it connects to the real-time operation data, environmental monitoring data, and monitoring video stream of the power plant SCADA system through an IoT gateway and data interface; establishing a one-to-one mapping of equipment objects in the twin space to form a perceptible, interactive, and computable dynamic digital twin; the state of the twin is updated synchronously with the physical world. S104, Deployment and Operation of Multi-Agent Autonomous Operation and Maintenance System (MAS): Deploy a multi-agent system on the server or cloud platform where the digital twin resides; the agents interact, request, and collaborate with each other through standardized communication protocols; they share a knowledge base containing device models, rule bases, and historical cases; S105, Decision Feedback and Synchronization with the Digital Twin: The task scheduling and execution agent sends the generated decision instructions to the execution units or maintenance personnel in the physical world; the results of instruction execution are fed back to the system through sensors and SCADA data, and received by the perception and fusion agent; the system updates the status of the corresponding device in the digital twin accordingly, and records this maintenance case to form a closed loop; the entire maintenance process is fully simulated, recorded and visualized in the digital twin.

[0028] The equipment in step S101 includes a photovoltaic panel array, wind turbine blades, a transformer, and a combiner box.

[0029] The geometric information in step S102 includes surface normals and depth.

[0030] The real-time operating data in step S103 includes power, voltage, current, and temperature; the environmental monitoring data includes irradiance, wind speed, and temperature; and the equipment objects include each photovoltaic panel and each wind turbine.

[0031] The multi-agent system in step S104 includes, but is not limited to, the following types of agents, which are instantiated and work together. These agents interact, request, and collaborate through communication protocols based on WebSocket or message queues: ① Perception and Fusion Agent: Responsible for receiving external sensor and SCADA data, cleaning, aligning, and fusing it, and publishing the standardized data to a shared data blackboard or directly pushing it to other agents; ② 3D Vision Analysis Agent: Utilizes high-definition images or geometric information rendered from arbitrary perspectives using the NeRF model, runs computer vision-based detection algorithms, and identifies physical anomalies in the equipment; ③ Operational Status Diagnosis Agent: Based on... The system analyzes operational data using data-driven models or physical knowledge models to diagnose electrical and performance anomalies; ④ Risk prediction and decision-making intelligent agent: integrating visual analysis, data diagnostic results, historical data, and weather forecast information, using reinforcement learning or decision tree models to predict equipment performance degradation trends, failure probabilities, and power generation loss risks; when a fault or high risk is detected, this intelligent agent generates autonomous operation and maintenance decisions based on preset rules and cost-benefit analysis; ⑤ Task scheduling and execution intelligent agent: receiving instructions from the decision-making intelligent agent, decomposing them into executable tasks, scheduling physical execution units to perform operations, or pushing detailed work orders to the handheld terminals of operation and maintenance personnel, and providing AR remote collaboration guidance. This intelligent agent also monitors the execution status of tasks and provides feedback to the decision-making intelligent agent.

[0032] The three-dimensional visual analysis intelligent agent identifies physical anomalies in the equipment, including hot spots, cracks, stains, and vegetation obstruction on photovoltaic panels, as well as surface damage and icing on wind turbine blades. The data-driven model for the operational status diagnosis intelligent agent is a deep learning LSTM network, and the performance anomalies include string faults, inverter efficiency degradation, and abnormal wind turbine vibration.

[0033] When the risk prediction and decision-making agent detects a fault or high risk, it generates autonomous operation and maintenance decisions based on preset rules and cost-benefit analysis, such as: "triggering the A12 area photovoltaic panel cleaning robot to start operation", "marking the B03 wind turbine for preventive maintenance and performing it in the next windless window", and "isolating the faulty combiner box C05 and switching to the backup line".

[0034] The task scheduling and execution intelligent agent scheduling physical execution unit includes cleaning robots, drones, on-site mobile inspection terminals, and remote control switches.

[0035] The decision instruction in step S105 includes control signals and work orders, and the result of the instruction execution includes whether the fault has been eliminated and whether the cleaning has been completed.

[0036] A NeRF-based and multi-agent-based digital twin autonomous operation and maintenance system for new energy power plants, used to run the aforementioned NeRF-based and multi-agent-based digital twin autonomous operation and maintenance method for new energy power plants, the system includes: ① Data Acquisition and Processing Module: Corresponding to step S101, this module includes a UAV control unit and an image processing unit, used to acquire and preprocess multi-view images; ② High-Fidelity 3D Modeling Module: Corresponding to step S102, this module has a built-in NeRF model training and inference engine, generating neural radiation fields from input images; ③ Twin Construction and Driving Module: Corresponding to step S103, this module is responsible for fusing the NeRF model, real-time data, and equipment information model to generate and update a dynamic digital twin; ④ Multi-Agent Autonomous Operation and Maintenance Module: Corresponding to steps S104 and S105, this module internally instantiates and runs various agents, including an agent management unit, communication bus, knowledge base, and decision engine; ⑤ Human-Computer Interaction and Visualization Module: Based on NeRF rendering technology and a 3D engine, this module provides an immersive digital twin visualization interface for operation and maintenance personnel, displaying real-time status, alarms, analysis results, and operation and maintenance instructions; ⑥ Instruction Issuance and Feedback Interface Module: This module is responsible for converting autonomous operation and maintenance decisions into specific control instructions and sending them to various actuators or personnel terminals at the power plant site, while also receiving execution feedback.

[0037] Explanation of terms related to this invention: Digital twin: A dynamic virtual mapping of the entire lifecycle of a physical object or system in virtual space, enabling simulation, prediction, and optimization through data-driven approaches.

[0038] NeRF (Neural Radiance Fields): A deep learning method. It implicitly represents a 3D scene through a neural network. It can synthesize highly realistic images from new perspectives by taking sparse 2D multi-view images as input, and can reconstruct detailed geometric structures and material information.

[0039] Multi-Agent System (MAS): A computing system composed of multiple interacting, autonomous agents. In this invention, each agent is responsible for a specific task (such as data acquisition, diagnosis, and scheduling), achieving complex global operational goals through cooperation and communication.

[0040] Autonomous operation and maintenance: refers to the system's ability to automatically complete the closed-loop process of equipment status monitoring, anomaly detection, diagnosis, decision-making, and action execution based on preset rules, models, or through learning, with little or no human intervention.

[0041] New energy power plants: specifically refers to facilities that generate electricity using renewable energy sources such as solar and wind power, such as photovoltaic power plants and wind farms.

[0042] Alternative technical solutions to the present invention: Alternative Solution 1 (Partial Alternative): Replacement of the 3D Modeling Base. While NeRF offers significant advantages in realism and convenience, if there are extreme requirements for absolute geometric accuracy of the model (e.g., for construction layout), step S102 can be combined with laser point cloud-assisted reconstruction technology. That is: images acquired by a drone are still used for NeRF to generate textures and visual details, while a lightweight LiDAR is used to acquire sparse but accurate ranging point clouds to assist in the geometric constraints during NeRF training, resulting in a 3D model that combines high visual quality and accurate geometry.

[0043] Alternative Solution Two (Complete Solution Replacement): Replacement of Agent Organization Form. The multi-agent system in S104 can adopt different architectures. For example, a federated architecture of "central coordinator + functional agents" can be used, where one central coordinating agent is responsible for task decomposition, result aggregation, and conflict resolution, while the other agents are functional. A fully decentralized peer-to-peer architecture can also be used, where agents collaborate through a negotiation mechanism, offering greater robustness but more complex design. Both architectures can achieve the goals of collaborative work and autonomous operation.

[0044] Alternative Solution 3 (Application Scenario Replacement): Replacement of specific types of new energy power plants and operation and maintenance tasks. The method of this invention is not only applicable to large-scale centralized photovoltaic power plants and wind farms, but also to distributed photovoltaic rooftops, integrated wind-solar-storage power plants, and other scenarios. Specific operation and maintenance tasks, in addition to cleaning and fault diagnosis, can be replaced with "prediction-based power optimization scheduling," "energy storage system health management and charging / discharging strategy optimization," and "equipment protection self-decision-making under extreme weather conditions," etc.

[0045] The above description is merely a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A digital twin autonomous operation and maintenance method for new energy power plants based on NeRF and multi-agent systems, characterized in that: By using drones or fixed cameras to collect multi-view images of the power plant site, and using NeRF technology to construct an ultra-realistic, three-dimensional, roamable geometric appearance model of the power plant; on this basis, by integrating real-time operation data, a digital twin of all elements of the power plant, including physical model, data model and rule model, is constructed. A multi-agent system is introduced, which includes multiple agents with different functions. These agents are responsible for collaboratively performing twin data fusion, status monitoring, fault diagnosis, risk prediction, and autonomous operation and maintenance decision-making. Ultimately, the digital twins provide feedback to control on-site actuators or guide manual operation and maintenance.

2. The autonomous operation and maintenance method for a new energy power plant based on NeRF and multi-agent digital twin as described in claim 1, characterized in that... The method includes the following steps: S101, Multi-view Image Acquisition and Processing: Deployed autonomous flight drones or fixed-point camera arrays at new energy power plants to periodically or irregularly acquire multi-view two-dimensional images of the covered equipment; perform distortion correction, alignment, and preprocessing on the images; S102, High-fidelity 3D reconstruction based on NeRF: The preprocessed multi-view images are input into the trained NeRF model; the model contains an MLP neural network that learns the mapping from spatial position and viewing direction to color and volume density; after training, this step outputs a high-precision, high-fidelity 3D neural representation of the power station; this representation not only supports photorealistic rendering of any viewpoint, but also implicitly contains accurate geometric information. S103, integrating real-time data to construct a dynamic digital twin: using the NeRF 3D model generated in S102 as the core geometric and visual basis, it connects to the real-time operation data, environmental monitoring data, and monitoring video stream of the power plant SCADA system through an IoT gateway and data interface; establishing a one-to-one mapping of equipment objects in the twin space to form a perceptible, interactive, and computable dynamic digital twin; the state of the twin is updated synchronously with the physical world. S104, Deployment and Operation of Multi-Agent Autonomous Operation and Maintenance System (MAS): Deploy a multi-agent system on the server or cloud platform where the digital twin resides; the agents interact, request, and collaborate with each other through standardized communication protocols; they share a knowledge base containing device models, rule bases, and historical cases; S105, Decision Feedback and Synchronization with the Digital Twin: The task scheduling and execution agent sends the generated decision instructions to the execution units or maintenance personnel in the physical world; the results of instruction execution are fed back to the system through sensors and SCADA data, and received by the perception and fusion agent; the system updates the status of the corresponding device in the digital twin accordingly, and records this maintenance case to form a closed loop; the entire maintenance process is fully simulated, recorded and visualized in the digital twin.

3. The autonomous operation and maintenance method for a new energy power plant based on NeRF and multi-agent digital twins as described in claim 2, characterized in that: The equipment in step S101 includes a photovoltaic panel array, wind turbine blades, a transformer, and a combiner box.

4. The autonomous operation and maintenance method for a new energy power plant based on NeRF and multi-agent digital twin as described in claim 2, characterized in that: The geometric information in step S102 includes surface normals and depth.

5. The autonomous operation and maintenance method for a new energy power plant based on NeRF and multi-agent digital twins according to claim 2, characterized in that: The real-time operating data in step S103 includes power, voltage, current, and temperature; the environmental monitoring data includes irradiance, wind speed, and temperature; and the equipment objects include each photovoltaic panel and each wind turbine.

6. The autonomous operation and maintenance method for a new energy power plant based on NeRF and multi-agent digital twin as described in claim 2, characterized in that: The multi-agent system in step S104 includes at least, but is not limited to, the following types of agents, which are instantiated and work together. The agents interact, request and cooperate with each other through communication protocols based on WebSocket or message queues: ① Perception and fusion agent: responsible for receiving external sensor and SCADA data, cleaning, aligning and fusing it, and publishing the standardized data to the shared data blackboard or directly pushing it to other agents; ② 3D visual analysis intelligent agent: Utilizes high-definition images or geometric information rendered from any viewpoint by the NeRF model, runs computer vision-based detection algorithms, and identifies physical anomalies in equipment; ③ Operational Status Diagnosis Agent: Based on a data-driven model or physical knowledge model, analyzes operational data to diagnose electrical and performance anomalies; ④ Risk Prediction and Decision-Making Agent: Integrates visual analysis, data diagnosis results, historical data, and weather forecast information, and uses reinforcement learning or decision tree models to predict equipment performance degradation trends, failure probabilities, and power generation loss risks; when a fault or high risk is detected, this agent generates autonomous operation and maintenance decisions based on preset rules and cost-benefit analysis; ⑤ Task Scheduling and Execution Agent: Receives instructions from the decision-making agent, decomposes them into executable tasks, schedules physical execution units to perform operations, or pushes detailed work orders to the handheld terminals of operation and maintenance personnel, and provides AR remote collaboration guidance; this agent also monitors the execution status of tasks and provides feedback to the decision-making agent.

7. The autonomous operation and maintenance method for a new energy power plant based on NeRF and multi-agent digital twin as described in claim 6, characterized in that: The decision instruction in step S105 includes control signals and work orders, and the result of the instruction execution includes whether the fault has been eliminated and whether the cleaning has been completed.

8. A new energy power plant digital twin autonomous operation and maintenance system based on NeRF and multi-agent technology, used to run the new energy power plant digital twin autonomous operation and maintenance method based on NeRF and multi-agent technology as described in any one of claims 1-7, characterized in that... The system includes: ① Data Acquisition and Processing Module: Corresponding to step S101, this module includes a UAV control unit and an image processing unit, used to acquire and preprocess multi-view images; ② High-Fidelity 3D Modeling Module: Corresponding to step S102, this module has a built-in NeRF model training and inference engine, generating neural radiation fields from input images; ③ Twin Construction and Driving Module: Corresponding to step S103, this module is responsible for fusing the NeRF model, real-time data, and equipment information model to generate and update a dynamic digital twin; ④ Multi-Agent Autonomous Operation and Maintenance Module: Corresponding to steps S104 and S105, this module internally instantiates and runs various agents, including an agent management unit, communication bus, knowledge base, and decision engine; ⑤ Human-Computer Interaction and Visualization Module: Based on NeRF rendering technology and a 3D engine, this module provides an immersive digital twin visualization interface for operation and maintenance personnel, displaying real-time status, alarms, analysis results, and operation and maintenance instructions; ⑥ Instruction Issuance and Feedback Interface Module: This module is responsible for converting autonomous operation and maintenance decisions into specific control instructions and sending them to various actuators or personnel terminals at the power plant site, while also receiving execution feedback.