Wind power plant disaster emergency system and method based on image recognition and unmanned aerial vehicle

The real-time status data of the fan is obtained through drone clusters and multimodal sensors, and combined with edge computing and cloud computing platforms, a real-time fault model is built, which solves the problem of low accuracy of disaster emergency information in the existing system and realizes an efficient wind farm Disaster identification and emergency response.

CN118941992APending Publication Date: 2024-11-12HUANENG HEZHANG WIND POWER CO LTD +1
View PDF 0 Cites 3 Cited by

Patent Information

Application Number
CN202411033998.5
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-30
Publication Date
2024-11-12

Smart Images

  • Figure CN118941992A_ABST
    Figure CN118941992A_ABST
Patent Text Reader

Abstract

The invention discloses a wind power plant disaster emergency system based on image recognition and unmanned aerial vehicles, which comprises an unmanned aerial vehicle cluster, the unmanned aerial vehicle cluster is in data connection with an edge computing node, the edge computing node is installed on an unmanned aerial vehicle of the unmanned aerial vehicle cluster, and the edge computing node is in data connection with a ground control station. The ground control station is in data connection with a cloud computing platform. The system senses the state of a fan through multi-source heterogeneous data, realizes intelligent analysis by means of cloud edge collaboration, and generates an interpretable decision by using a knowledge graph and digital twinning; the invention also discloses a wind power plant disaster emergency method based on image recognition, and the method comprises the steps: carrying out the preprocessing based on the obtained real-time state data of the fan, and obtaining the real-time state data; constructing a real-time fault model based on the fan damage detection data and the obtained historical fault data of the wind turbine generator; the real-time fault model is analyzed, a disaster emergency plan is made, emergency repair is carried out, and the method improves the safety of the wind power plant.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the technical field of intelligent operation and maintenance of wind power generation equipment, and specifically belongs to a wind farm disaster emergency response system and method based on image recognition and drones. Background Art

[0002] Wind power is a clean, renewable energy source, playing an increasingly important role in the global energy transition. To ensure the stable operation of wind turbines, regular inspections and condition monitoring of wind farms are crucial. Traditional manual inspections are inefficient and difficult to accurately diagnose internal defects in wind turbines. With the development of new technologies such as artificial intelligence and big data, there is an urgent need to build an efficient and intelligent wind farm monitoring and operation and maintenance system.

[0003] Currently, existing wind farms have begun to introduce new management systems. For example, existing management systems use airborne high-definition cameras to capture blade images, utilize computer vision algorithms for defect detection, and utilize vibration sensors for condition monitoring and fault diagnosis of wind turbine drive trains. While this system has improved the automation and intelligence of wind farm operations and maintenance, and enhanced the timeliness of wind farm disaster response, it only captures blade images through airborne high-definition cameras. The captured data is single-source, and processing of this image data can only determine the operating status of the blades. This simply determines whether the wind turbine is faulty, but cannot identify deeper faults. Furthermore, due to the numerous factors that influence blade operating status, determining wind farm disaster response information solely through blade images is inaccurate and carries a high risk of decision-making errors. Summary of the Invention

[0004] In order to solve the problem that the existing system only collects blade images to determine the wind farm disaster emergency information, which has low accuracy and high risk of decision-making errors, the present invention provides a wind farm disaster emergency system and method based on image recognition and drones.

[0005] To achieve the above object, the present invention provides the following technical solutions:

[0006] The present invention discloses a wind farm disaster emergency response system based on image recognition and drones, comprising a drone cluster, wherein the drone cluster is data-connected to an edge computing node, wherein the edge computing node is installed on a drone in the drone cluster, wherein the edge computing node is data-connected to a ground control station, and wherein the ground control station is data-connected to a cloud computing platform;

[0007] The drone cluster is used to obtain real-time status data of the wind turbine;

[0008] The edge computing node is used to pre-process the real-time status data and then transmit it to the ground control station;

[0009] The ground control station is used to control and monitor the drone cluster and collect data pre-processed by the edge computing nodes;

[0010] The cloud computing platform is used to obtain data from the ground control station, make disaster emergency decisions, and assist staff in disaster emergency response.

[0011] Preferably, the drones of the drone cluster are equipped with high-definition visible light cameras, long-wave infrared thermal imagers and hyperspectral imagers. The data of the high-definition visible light camera, the long-wave infrared thermal imager and the hyperspectral imager are respectively used to obtain real-time status images of the wind turbines, and then the real-time status images are transmitted to the edge computing nodes.

[0012] Preferably, the edge computing node includes a computing module, the computing module includes an image processor, a data processor, a user control terminal and a drone control module, and the image processor and the data processor are integrated on the drones of the drone cluster;

[0013] The image processor is used to process images acquired by the high-definition visible light camera, the long-wave infrared thermal imager, and the hyperspectral imager;

[0014] The data processor is used to extract features from the data processed by the image processor and then transmit the data to the ground control station;

[0015] The drone control module is used to control and monitor the drone cluster;

[0016] The user control terminal is used to monitor the working conditions of the image processor, the data processor and the drone control module, and to adjust the operating parameters of the image processor, the data processor and the drone control module.

[0017] Preferably, the ground control station comprises:

[0018] A drone control and dispatch module, configured to detect the position and status of drones in the drone cluster, specify real-time dispatch information for the drone cluster based on the position and status of the drones, and plan flight missions for the drone cluster;

[0019] Status detection module, used to collect real-time operating data of wind turbines;

[0020] an integration module, which integrates the real-time operation data and the real-time status data to obtain real-time detection data of the wind farm, and then transmits the data to the output module;

[0021] The output module is used to transmit the real-time monitoring data to the cloud computing platform, and transmit the flight mission and the real-time scheduling information to the UAV control module.

[0022] Preferably, the cloud computing platform includes:

[0023] The damage map module is used to obtain historical fault data of wind turbines and build a wind turbine fault map based on the historical fault data;

[0024] A damage detection module, which annotates the real-time monitoring data based on the wind turbine fault map to construct a real-time fault model;

[0025] An emergency processing module analyzes the real-time fault model to obtain a damage type and damage degree, and formulates an emergency processing strategy for the damage type based on the damage type and damage degree;

[0026] A decision detection module is used to perform virtual simulation training on the emergency response strategy to obtain disaster emergency decision-making;

[0027] The assistance processing module assists staff in handling faults based on the disaster emergency decision.

[0028] Preferably, the emergency processing module includes:

[0029] An image reasoning unit, configured to perform image reasoning on the real-time fault model, analyze features in the image, and identify the fault type and fault level in the real-time fault model;

[0030] a source tracing analysis unit, which constructs a knowledge graph based on the wind turbine fault graph, compares the knowledge graph with the fault type and fault level, and determines the fault cause causing the fault type and fault level, as well as the fault hazard caused by the fault type and fault level;

[0031] The decision-making plan unit formulates a disaster emergency plan based on the cause of the fault and the harm of the fault, and decomposes the disaster emergency plan to obtain the emergency inspection task of the drone cluster and the emergency handling strategy.

[0032] Preferably, the decision detection module includes:

[0033] A model building unit, configured to acquire wind farm data to build a wind farm model, and to build a simulation model based on the wind farm model, the fault type, and the fault level;

[0034] A virtual simulation unit simulates the execution of the emergency treatment strategy in the simulation model based on the emergency inspection task and the emergency treatment strategy, verifies the emergency treatment strategy, and obtains a strategy verification result;

[0035] A simulation display unit, used to display the process of executing the simulated emergency response strategy and virtual information annotations during the execution of the simulated emergency response strategy;

[0036] The strategy determination unit obtains the disaster emergency decision based on the strategy verification result.

[0037] Preferably, the system further includes a blockchain network module, and the blockchain network module includes:

[0038] An emergency recording unit, configured to record the disaster emergency decision, the fault type, the fault level, the fault cause, and the fault hazard;

[0039] The regional chain sharing unit is used to establish a block sharing space and share the emergency processing data of wind farms in the region;

[0040] The identity coding module is used to perform identity coding on the wind farms in the area, obtain the emergency processing data of the wind farms after the identity coding, and upload it to the block shared space.

[0041] The present invention also discloses a wind farm disaster emergency method based on image recognition, which includes the following steps:

[0042] Preprocessing is performed based on the acquired real-time status data of the fan to obtain real-time status data;

[0043] Building a real-time fault model based on the wind turbine damage detection data and the acquired historical fault data of the wind turbine;

[0044] The real-time fault model is analyzed, a disaster emergency plan is formulated, and emergency repairs are carried out.

[0045] Preferably, the real-time fault model is analyzed to formulate a disaster emergency plan including:

[0046] Analyzing the real-time fault model to obtain damage type and damage degree;

[0047] Formulate a disaster emergency plan based on the damage degree and the damage type, and decompose the disaster emergency plan to obtain the emergency inspection task of the drone cluster and the emergency handling strategy;

[0048] Based on the emergency inspection task and the emergency response strategy, the emergency response strategy execution is simulated to verify the emergency response strategy and obtain the disaster emergency decision.

[0049] Compared with the prior art, the present invention has the following beneficial technical effects:

[0050] The present invention discloses a wind farm disaster emergency response system based on image recognition and drones. This system utilizes a swarm of drones to comprehensively perceive the health status of wind turbines and rapidly acquire real-time status data. Combined with image recognition technology, this system can accurately identify abnormalities such as blade damage and tower tilt, significantly improving the accuracy and timeliness of disaster warnings. Edge computing nodes then immediately preprocess the acquired real-time status data, reducing data transmission delays and increasing the system's response speed. Furthermore, the use of edge computing nodes reduces the processing pressure on the cloud computing platform, making the entire system more efficient. The ground control station, as the core control node of the entire system, not only controls and monitors the drone swarm in real time but also aggregates preprocessed data from the edge computing nodes, providing comprehensive data support for the cloud computing platform, making the entire system easier to manage and maintain. Finally, the cloud computing platform, as the system's decision-making center, can make scientific disaster emergency response decisions based on data provided by the ground control station and combined with historical wind turbine fault data. It also assists personnel in disaster emergency response through various means.

[0051] Furthermore, the drones in the drone cluster in this system are equipped with high-definition visible light cameras, long-wave infrared thermal imagers and hyperspectral imagers. The drones are equipped with multi-modal sensors such as visible light, infrared, and hyperspectral sensors to conduct a full-scale scan of the wind farm and obtain real-time status data from multiple sources. Compared with a single data source, multi-source real-time status data can characterize the status of the wind turbine from different physical perspectives, greatly improving the ability to distinguish defect characteristics and making it impossible for risks and hidden dangers to hide.

[0052] Furthermore, this system utilizes a combined edge computing and cloud computing architecture. Preliminary data analysis is performed locally on the drone, alleviating communication and storage pressures. Massive data aggregation and training is performed in the cloud to generate high-precision fault diagnosis models. The dynamic collaboration of these two levels of computing resources not only meets real-time requirements but also leverages data scale effects, forming deep intelligent analysis capabilities.

[0053] Furthermore, this system achieves accurate identification and rapid response to wind turbine faults in wind farms through the damage map module, damage detection module, and emergency response module. Specifically, the damage map module can construct detailed wind turbine fault maps, providing a rich reference for damage detection; the damage detection module can use these maps to accurately annotate real-time monitoring data and construct real-time fault models; the emergency response module can quickly analyze these models, determine the type and extent of damage, and formulate emergency response strategies accordingly, improving the accuracy of fault identification, reducing the false alarm rate, and speeding up emergency response, allowing staff to take countermeasures more quickly and reduce the impact of faults on wind farm operations. In addition, the system also improves the safety and reliability of wind farms, providing a strong guarantee for the stable operation of wind farms.

[0054] Furthermore, the blockchain network module within this system leverages blockchain technology to store and share data across the entire wind turbine lifecycle. Smart contracts are also used to trigger disaster information and issue tasks in real time. This blockchain mechanism ensures data immutability and traceability, building a secure and trustworthy wind farm system from the very beginning. This ensures clear responsibilities for all participants and efficient collaboration.

[0055] The present invention also discloses a wind farm disaster emergency response method based on image recognition and drones. This method improves the accuracy and reliability of the data by pre-processing the real-time status data of the wind turbine, effectively avoids noise and interference in the original data, and makes the results of subsequent analysis more accurate. Secondly, the real-time fault model constructed based on wind turbine damage detection data and historical fault data can reflect the operating status and fault conditions of the wind farm in real time. This model not only takes into account the current status of the wind turbine, but also combines historical fault data to make the analysis results more comprehensive and in-depth. Through the real-time fault model, the staff can quickly understand the overall situation of the wind farm. Finally, through the analysis of the real-time fault model, a disaster emergency plan is formulated and emergency repairs are carried out to ensure that the wind farm can quickly resume operation when a fault occurs, improve the fault handling capability of the wind farm, and reduce the impact of the fault on the operation of the wind farm. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Figure 1 This is the overall architecture diagram of the wind farm disaster emergency response system based on image recognition and drones provided by the present invention;

[0057] Figure 2 This is a schematic diagram of the intelligent analysis of the wind farm disaster emergency system based on image recognition and drones provided by the present invention;

[0058] Figure 3 A data processing flow chart of an image processor in a wind farm disaster emergency system based on image recognition and drones provided by the present invention;

[0059] Figure 4 This is a data processing flow chart of the AC framework of the source tracing analysis unit in the wind farm disaster emergency system based on image recognition and drones provided by the present invention;

[0060] Figure 5 This is a processing flow chart of the simulation display unit in the wind farm disaster emergency system based on image recognition and drones provided by the present invention. DETAILED DESCRIPTION

[0061] Hereinafter, only certain exemplary embodiments are briefly described. As will be appreciated by those skilled in the art, the described embodiments may be modified in various ways without departing from the spirit or scope of the present invention. Therefore, the drawings and description are to be considered as illustrative in nature and not restrictive.

[0062] In the description of the present invention, it should be understood that the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", "clockwise", "counterclockwise", "axial", "radial", "circumferential" and the like to indicate orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as limiting the present invention.

[0063] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of the technical features being referred to. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of such features. In the description of the present invention, "plurality" means two or more, unless otherwise specifically defined.

[0064] In the present invention, unless otherwise expressly specified or limited, terms such as "mounted," "connected," "connect," and "fixed" should be understood broadly. For example, they may refer to fixed connection, detachable connection, or integration; mechanical connection, electrical connection, or communication; direct connection or indirect connection through an intermediate medium; and internal communication between two components or interaction between two components. Those skilled in the art will understand the specific meanings of the above terms in the present invention based on specific circumstances.

[0065] In the present invention, unless otherwise expressly specified or limited, a first feature being "above" or "below" a second feature may include the first and second features being in direct contact, or may include the first and second features being in contact not directly but through another feature between them. Furthermore, a first feature being "above," "above," and "above" a second feature may include the first feature being directly above or obliquely above the second feature, or may simply mean that the first feature is higher in level than the second feature. A first feature being "below," "below," and "below" a second feature may include the first feature being directly above or obliquely above the second feature, or may simply mean that the first feature is lower in level than the second feature.

[0066] The embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0067] See also Figure 1 The present invention discloses a wind farm disaster emergency response system based on image recognition and drones. The system includes a drone cluster, edge computing nodes, a ground control station, and a cloud computing platform. The drone cluster is data-connected to the edge computing nodes, which are installed on drones in the drone cluster. The edge computing nodes are data-connected to the ground control station, and the ground control station is data-connected to the cloud computing platform. The drone cluster is used to obtain real-time status data of wind turbines; the edge computing nodes are used to pre-process the real-time status data and then transmit it to the ground control station; the ground control station is used to control and monitor the drone cluster and aggregate the data pre-processed by the edge computing nodes; and the cloud computing platform is used to obtain data from the ground control station, make disaster emergency decisions, and assist personnel in disaster emergency response.

[0068] The drone swarm consists of multiple drones, numbering no fewer than 10. Each drone is interconnected via 5G communications. Each drone is equipped with a high-definition visible light camera, a long-wave infrared thermal imager, and a hyperspectral imager. These cameras capture real-time status data of the wind turbines, including images of the turbines, blades, and nacelles, enabling multimodal data collection for the wind farm. This data is then transmitted to edge computing nodes. The high-definition visible light camera is the Sony RX1R II with a resolution of 42.4 megapixels; the infrared thermal imager is the FLIR Vue Pro R with a resolution of 640×512; and the hyperspectral imager is the Resonon Pika L with a wavelength range of 400 to 1000 nm.

[0069] The edge computing node includes a computing module, which is integrated on the drone, and 1 to 2 are installed on each drone. The computing module is an NVIDIA Jetson AGX Xavier series module, which includes an image processor, a data processor, a user control terminal and a drone control module. The image processor and the data processor are integrated on the drones of the drone cluster. The image processor uses an improved YOLOv5 algorithm to process the original images obtained by the high-definition visible light camera, long-wave infrared thermal imager and hyperspectral imager to achieve real-time target detection of key parts of the wind turbine, with a detection accuracy of more than 95%. Figure 2As shown, the original image is subjected to target detection to obtain blade crack images and cabin fire images. The image processor is a 512-core Volta GPU. The data processor is used to extract features from the data processed by the image processor and then transmit it to the ground control station. The data processor is an 8-core ARM CPU.

[0070] The drone control module is used to control and monitor the drone cluster;

[0071] The user control terminal is used to monitor the working conditions of the image processor, data processor and drone control module, and to adjust the operating parameters of the image processor, data processor and drone control module. The edge computing node is also pre-installed with the CUDA toolkit and TensorRT inference engine, which can realize real-time image processing and target detection during flight. The image processing performance of this edge computing node can reach 32TOPS.

[0072] The ground control station is equipped with a high-performance workstation, which includes the UAV control module, status detection module, integration module, and output module. It uses the Dell Precision 7920Tower model, equipped with a dual-channel Intel Xeon Gold 6248 processor, equipped with 768GB DDR4 memory and four NVIDIA Quadro RTX 8000 graphics cards;

[0073] The drone control and dispatch module installs the ROS robot operating system, detects the location and status of drones in the drone cluster by programming ROS nodes, formulates real-time dispatch information of the drone cluster based on the location and status of the drones, and realizes unified dispatch and monitoring of the drone cluster;

[0074] The status detection module integrates wind turbine SCADA, CMS, etc. to collect real-time operation data of wind turbines, that is, real-time parameter data of wind turbine operation;

[0075] The integration module is used to integrate real-time operation data and real-time status data to obtain real-time detection data of the wind farm, and then transmit it to the output module;

[0076] The output module is used to transmit real-time monitoring data to the cloud computing platform and transmit real-time scheduling information to the drone control module.

[0077] The cloud computing platform is built on Amazon AWS EC2 P3 instances, employing multiple NVIDIA V100 GPUs to build a deep learning training cluster. It also utilizes the S3 object storage service to build a petabyte-scale wind power big data center, which is used for offline processing, mining, and model optimization of massive amounts of wind power data. The cloud computing platform includes a damage mapping module, a damage detection module, an emergency response module, a decision detection module, and an auxiliary processing module, all of which are integrated into the petabyte-scale wind power big data center.

[0078] The damage map module is used to obtain the historical fault data of the wind turbine and construct the wind turbine fault map based on the historical fault data, such as Figure 3 As shown in the figure, the atlas covers 25 common fault data of the main components of wind turbines, such as blade failure, i.e. blade cracking and blade falling off; transmission chain failure, i.e. gear wear and bearing peeling; generator failure, i.e. winding short circuit and bearing overheating, etc. Figure 3 As mentioned above, there are no fewer than 1,000 defect samples for each category. Manual annotation uses the Elf Annotation Assistant, and automatic annotation uses YOLO v5 pre-annotation and manual verification to ensure pixel-level annotation accuracy.

[0079] The damage detection module is used to annotate the image data in the real-time monitoring data through the fan fault map, filter out the image data with fan faults, extract features from the image data, and combine the real-time operation data corresponding to the image data to construct a real-time fault model of the fan;

[0080] The emergency response module uses the real-time fault model to infer high-definition images of abnormal parts of the wind turbine to obtain the damage type and damage extent, and formulates an emergency response strategy based on the damage type and damage extent;

[0081] Among them, the emergency processing module includes image reasoning unit, source tracing analysis unit and decision-making plan unit;

[0082] The image inference unit is used to perform image inference on the real-time fault model, analyze the features in the image, identify the fault type and fault level in the real-time fault model, and then integrate the real-time operation data obtained by the wind turbine SCADA, CMS, etc. to perform high-confidence diagnosis through the decision-level information fusion algorithm to obtain the wind turbine fault diagnosis result;

[0083] The decision-level information fusion algorithm is based on the Dempster-Shafer theory, which resolves conflicts and optimizes the combination of different diagnostic conclusions obtained by fault type and fault level. That is, there are n independent diagnostic conclusions E1, E2, ..., E n, their identification framework (i.e., the set of fault modes) is Θ. Through evidence theory, the system can obtain more comprehensive and reliable wind turbine fault diagnosis results, thereby making the optimal emergency decision. Among them, for any failure mode subset The i-th diagnostic conclusion E i The basic probability assignment (confidence) supporting A is m i (A) Combine multiple sources of evidence using a combination calculation formula;

[0084] The combination calculation formula is:

[0085]

[0086] Among them, m(A) is the confidence level supporting fault mode A after fusion, K is the conflict factor between evidences, and the calculation expression of the conflict factor is:

[0087]

[0088] Where n is the number of diagnostic conclusions,

[0089] When K=1, it indicates that there is a complete conflict between the diagnosis conclusions. At this time, the fault type and fault level are reselected.

[0090] The traceability analysis unit presets historical fault data, including preset historical fault types of wind turbines, historical fault levels corresponding to historical fault types, causes of historical fault types and historical hazard impacts, and constructs a knowledge graph of wind turbine faults through the preset historical fault data. By comparing the knowledge graph with the fault type and fault level, the fault cause causing the fault type and fault level, as well as the fault hazard caused by the fault type and the fault level are determined; the decision-making plan unit compares the fault cause and fault hazard with the cause and historical hazard impact caused by the historical fault type, and obtains the corresponding historical disaster emergency response strategy caused by the preset cause and historical hazard impact of the corresponding historical fault type. Based on the historical disaster emergency response strategy, a disaster emergency response plan is formulated, and the disaster emergency response plan is decomposed to obtain the emergency inspection task and emergency handling strategy of the drone cluster;

[0091] Among them, in the formulation of emergency inspection tasks of drone clusters, this module adopts the multi-agent reinforcement learning (MARL) algorithm to optimize the emergency inspection path of drone clusters in real time and minimize the coverage time of the entire field. The MARL algorithm uses the digital twin of the wind farm as the environment, that is, random events such as wind turbine failures and extreme weather, and drone coverage and flight distance as constraints. It uses the Actor-Critic (AC) framework to train the agent strategy and improve the robustness of the strategy through dynamic interaction with the environment. Among them, the AC framework includes an Actor network (strategy function) and a Critic network (value function), such as Figure 4 As shown in the figure, by obtaining wind farm environmental data, the wind farm environmental data is processed by the Critic network (value function), and then the value evaluation and wind farm environmental data are input into the Actor network for processing. Then, the action of the drone cluster is selected to obtain the emergency inspection task of the drone cluster. This process realizes the iterative optimization of emergency inspection tasks and values ​​through gradient feedback.

[0092] Furthermore, we improved the environment construction method in MARL. We introduced wind field models and energy consumption models to refine the UAV flight dynamics and improve the simulation accuracy of the digital twin. When generating UAV scheduling strategies, we not only consider mission coverage efficiency, but also track smoothness and energy optimization, making the execution plan more feasible.

[0093] The decision detection module is used to conduct virtual simulation training on emergency response strategies to obtain disaster emergency decisions; it includes a model building unit, a virtual simulation unit, a simulation display unit, and a strategy determination unit;

[0094] A model building unit, configured to acquire wind farm data to build a wind farm model, and to build a simulation model based on the wind farm model, the fault type, and the fault level;

[0095] The virtual simulation unit simulates the execution of the emergency treatment strategy in the simulation model based on the emergency inspection task and the emergency treatment strategy, verifies the emergency treatment strategy, and obtains a strategy verification result;

[0096] The simulation display unit is an AR-assisted display based on HoloLens 2, which is used to display the process of simulating the execution of emergency response strategies and the virtual information annotations in the simulation of emergency response strategy execution. That is, when staff wear AR glasses, they can see the virtual information annotations superimposed on the real scene of the wind turbine, such as Figure 5As shown, the system includes real-time operating parameters, historical fault records, and damage warnings. The system supports gesture and voice interaction and can simulate emergency plan execution in a virtual scene, allowing workers to experience the emergency repair process firsthand. This system uses AR / VR to assist in maintenance operations, projecting information such as wind turbine status, health assessments, and emergency plans into a three-dimensional virtual scene and supporting real-time human-machine interaction. Maintenance personnel can experience wind farm dynamics in an immersive way and conduct on-site operations using visualization tools, significantly reducing the risk of decision-making errors and improving operational safety.

[0097] Strategy determination unit, which makes disaster emergency decisions based on strategy verification results and emergency inspection tasks;

[0098] The assistance processing module controls the unmanned cluster based on disaster emergency decision-making to assist staff in handling faults.

[0099] The system also includes a blockchain network module, which uses Hyperledger Fabric 2.0 to build a private blockchain network. By designing smart contracts for wind turbine digital identity, data collection, disaster reporting, and maintenance records, it enables secure storage and trusted sharing of data throughout the wind turbine lifecycle. The blockchain network module is installed in the ground control station and cloud computing platform and includes an emergency recording unit, a regional chain sharing unit, and an identity encoding module.

[0100] Emergency recording unit, used to record disaster emergency decisions, fault type, fault level, fault cause and fault hazard;

[0101] The regional chain sharing unit is used to establish a block sharing space and share the emergency processing data of wind farms in the region;

[0102] The identity coding module is used to perform identity coding on the wind farms in the area, obtain the emergency processing data of the wind farms after identity coding, and upload it to the block shared space.

[0103] Furthermore, the blockchain network module also includes an extended blockchain module. This module stores all wind turbine operating data, disaster response decisions, fault types, fault levels, causes, and hazards on the blockchain, creating an electronic archive of the wind turbine's entire lifecycle. Staff can access complete operating records for each wind turbine at any time, tracing potential accidents and guiding maintenance decisions. The extended blockchain module ensures data immutability and authenticity.

[0104] This system integrates perception, analysis, decision-making, and interaction to build a multi-technology wind farm disaster emergency response system. This system integrates the precise perception of drone swarms, the intelligent analysis of AI algorithms, the trusted collaboration of blockchain, and the immersive experience of AR, forming a closed loop from data to decision-making. This effectively enhances the intelligent level of wind farm disaster prevention and mitigation. Through the integration of multiple technologies, data, and entities, a wind farm disaster emergency response system integrating perception, analysis, decision-making, management, control, and services has been constructed, forming a complete data intelligence closed loop. This system will significantly enhance wind farms' proactive defense capabilities against various disasters, minimize and reduce disaster losses, and provide strong support for the safe, intelligent, and resilient development of the wind power industry.

[0105] The present invention also discloses a wind farm disaster emergency response method based on image recognition and drones, comprising the following steps:

[0106] Preprocessing is performed based on the acquired real-time status data of the fan to obtain real-time status data;

[0107] Specifically, the real-time status data of the wind turbine is obtained, that is, the blade status image of the wind turbine, the real-time image of the transmission chain and the real-time image of the generator are obtained, and the blade status image, the real-time image of the transmission chain and the real-time image of the generator are sequentially subjected to image enhancement, image filtering and image contrast adjustment to eliminate interference from factors such as noise, light and background to obtain real-time image data, and the real-time image data is cleaned to remove invalid data (such as outliers generated by sensor failure) and noise, and the cleaned data is standardized or normalized to eliminate dimensional differences and order of magnitude differences between different parameters. After data cleaning and standardization, features related to the wind turbine status are extracted, and the extracted features are verified and checked to ensure that they can accurately reflect the real-time status of the wind turbine. The acquired historical data or expert experience is used for verification to obtain wind turbine damage detection data.

[0108] Building a real-time fault model based on the wind turbine damage detection data and the acquired historical fault data of the wind turbine;

[0109] Specifically, historical fault data of wind turbines is obtained, and a wind turbine fault map is constructed based on the historical fault data. Image data in real-time monitoring data is annotated using the wind turbine fault map to screen out image data with wind turbine faults. Feature extraction is performed on the image data, and the real-time operating data corresponding to the image data is combined to construct a real-time fault model for the wind turbine.

[0110] Analyze real-time fault models, develop disaster emergency plans, and carry out emergency repairs.

[0111] Based on the degree and type of damage, a disaster emergency plan is formulated and decomposed to obtain the emergency inspection tasks and emergency response strategies of the drone cluster.

[0112] Specifically, image reasoning is performed on the real-time fault model, features in the image are analyzed, and the fault type and fault level in the real-time fault model are identified;

[0113] Preset historical fault data, including preset historical fault types of wind turbines, historical fault levels corresponding to historical fault types, causes of historical fault types and historical hazard impacts. A knowledge graph of wind turbine faults is constructed through the preset historical fault data. The knowledge graph is compared with the fault type and fault level to determine the fault cause of the fault type and fault level, as well as the fault hazard caused by the fault type and the fault level. Based on the fault cause and fault hazard, the historical disaster emergency response strategy corresponding to the preset cause and historical hazard impact of the historical fault type is obtained. A disaster emergency response plan is formulated based on the historical disaster emergency response strategy, and the disaster emergency response plan is decomposed to obtain the emergency inspection task and emergency handling strategy of the drone cluster.

[0114] Based on emergency inspection tasks and emergency response strategies, simulate the execution of emergency response strategies, verify the emergency response strategies, obtain disaster emergency decisions, control the flight of drone clusters according to the disaster emergency decisions, and assist staff in carrying out emergency repairs.

[0115] The above shows and describes the basic principles and main features of the present invention and the advantages of the present invention. It is obvious to those skilled in the art that the present invention is not limited to the details of the above exemplary embodiments, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention. Therefore, from all points of view, the embodiments should be regarded as illustrative and non-restrictive. The scope of the present invention is defined by the appended claims rather than the above description, and it is intended that all changes that fall within the meaning and range of equivalents of the claims are included in the present invention. Any reference signs in the claims should not be construed as limiting the claim to which they relate.

[0116] In addition, it should be understood that although this specification describes the embodiments, not every embodiment contains only one independent technical solution. This description is for clarity only. Those skilled in the art should consider the specification as a whole. The technical solutions in each embodiment can also be appropriately combined to form other embodiments that can be understood by those skilled in the art. The above content is only for the purpose of illustrating the technical concept of the present invention and cannot be used to limit the scope of protection of the present invention. Any changes made based on the technical solution in accordance with the technical concept proposed by the present invention fall within the scope of protection of the claims of the present invention.

Claims

1. Wind farm disaster emergency response system based on image recognition and drones, characterized by: It includes a drone cluster, the drone cluster is data-connected with an edge computing node, the edge computing node is installed on a drone of the drone cluster, the edge computing node is data-connected with a ground control station, and the ground control station is data-connected with a cloud computing platform; The drone cluster is used to obtain real-time status data of the wind turbine; The edge computing node is used to pre-process the real-time status data and then transmit it to the ground control station; The ground control station is used to control and monitor the drone cluster and collect the data pre-processed by the edge computing node; The cloud computing platform is used to obtain data from the ground control station, make disaster emergency decisions, and assist staff in handling disaster emergencies.

2. The wind farm disaster emergency response system based on image recognition and drone according to claim 1 is characterized in that: The drones of the drone cluster are equipped with high-definition visible light cameras, long-wave infrared thermal imagers and hyperspectral imagers. The data of the high-definition visible light camera, the long-wave infrared thermal imager and the hyperspectral imager are respectively used to obtain real-time status images of the wind turbines, and then the real-time status images are transmitted to the edge computing nodes.

3. The wind farm disaster emergency response system based on image recognition and drone according to claim 2 is characterized in that: The edge computing node includes a computing module, the computing module includes an image processor, a data processor, a user control terminal and a drone control module, and the image processor and the data processor are integrated on the drones of the drone cluster; The image processor is used to process images acquired by a high-definition visible light camera, a long-wave infrared thermal imager, and a hyperspectral imager; The data processor is used to extract features from the data processed by the image processor and then transmit the data to the ground control station; The drone control module is used to control and monitor the drone cluster; The user control terminal is used to monitor the working conditions of the image processor, the data processor and the drone control module, and to adjust the operating parameters of the image processor, the data processor and the drone control module.

4. The wind farm disaster emergency response system based on image recognition and drone according to claim 3 is characterized in that: The ground control station comprises: A drone control and dispatch module, used to detect the position and status of drones in the drone cluster, and formulate real-time dispatch information of the drone cluster based on the position and status of the drones; Status detection module, used to collect real-time operation data of wind turbines; An integration module, integrating the real-time operation data and the real-time status data to obtain real-time detection data of the wind farm, and then transmitting the data to the output module; The output module is used to transmit the real-time monitoring data to the cloud computing platform, and transmit the flight mission and the real-time scheduling information to the UAV control module.

5. The wind farm disaster emergency response system based on image recognition and drones according to claim 4 is characterized in that: The cloud computing platform includes: The damage map module is used to obtain the historical fault data of the wind turbine and build a wind turbine fault map based on the historical fault data; A damage detection module, which annotates the real-time monitoring data based on the fan fault map to construct a real-time fault model; An emergency processing module analyzes the real-time fault model to obtain a damage type and a damage degree, and formulates an emergency processing strategy for the damage type based on the damage type and the damage degree; A decision detection module is used to perform virtual simulation training on the emergency response strategy to obtain disaster emergency decisions; The assisting processing module controls the unmanned cluster to assist the staff in handling faults based on the disaster emergency decision.

6. The wind farm disaster emergency response system based on image recognition and drones according to claim 5 is characterized in that: The emergency processing module comprises: An image reasoning unit, used to perform image reasoning on the real-time fault model, analyze features in the image, and identify the fault type and fault level in the real-time fault model; A tracing analysis unit constructs a knowledge graph based on the wind turbine fault graph, and compares the knowledge graph with the fault type and the fault level to determine the fault cause causing the fault type and the fault level, and the fault hazard caused by the fault type and the fault level; The decision-making plan unit formulates a disaster emergency plan based on the cause of the fault and the hazard of the fault, and decomposes the disaster emergency plan to obtain the emergency inspection task of the drone cluster and the emergency handling strategy.

7. The wind farm disaster emergency response system based on image recognition and drones according to claim 6 is characterized in that: The decision detection module comprises: A model building unit, used to acquire wind farm data to build a wind farm model, and to build a simulation model based on the wind farm model, the fault type and the fault level; A virtual simulation unit simulates the execution of the emergency processing strategy in the simulation model based on the emergency inspection task and the emergency processing strategy, verifies the emergency processing strategy, and obtains a strategy verification result; A simulation display unit, used to display the process of executing the simulated emergency handling strategy and virtual information annotations in the execution of the simulated emergency handling strategy; A strategy determination unit obtains the disaster emergency decision based on the strategy verification result and the emergency inspection task.

8. The wind farm disaster emergency response system based on image recognition and drones according to claim 1 is characterized in that: The system also includes a blockchain network module, which includes: An emergency recording unit, used to record the disaster emergency decision, the fault type, the fault level, the fault cause and the fault hazard; The regional chain sharing unit is used to establish a block sharing space and share the emergency processing data of wind farms in the region; The identity coding module is used to perform identity coding on the wind farms in the area, and obtain the emergency processing data in the wind farms after the identity coding, and upload it to the block shared space.

9. A wind farm disaster emergency response method based on image recognition and drones, characterized in that: The following steps are involved: Preprocessing is performed based on the acquired real-time status data of the fan to obtain fan damage detection data; Building a real-time fault model based on the wind turbine damage detection data and the acquired historical fault data of the wind turbine set; The real-time fault model is analyzed, a disaster emergency plan is formulated, and emergency repairs are carried out.

10. The wind farm disaster emergency response method based on image recognition and drones according to claim 9 is characterized in that: The real-time fault model is analyzed to formulate a disaster emergency plan including: Analyze the real-time fault model to obtain damage type and damage degree; Based on the damage degree and the damage type, a disaster emergency plan is formulated, and the disaster emergency plan is decomposed to obtain the emergency inspection task of the drone cluster and the emergency handling strategy; Based on the emergency inspection task and the emergency handling strategy, the emergency handling strategy is simulated and executed to verify the emergency handling strategy and obtain the disaster emergency decision.

Citation Information

Cited By

  • Emergency medical material transportation system based on unmanned aerial vehicle

    CN119494597A

  • An emergency medical supplies transportation system based on drones

    CN119494597B

  • Emergency rescue method and system based on unmanned aerial vehicle and urban three-dimensional platform

    CN119648011A