Fan running state analysis system and method based on image recognition

Abstract

The invention belongs to the technical field of wind power generation, and discloses a fan running state analysis system and method based on image recognition, and the system comprises a data collection system, an edge calculation system and a control decision system; the data acquisition system comprises an image acquisition unit and an acoustic sensing unit; the edge calculation system comprises an image preprocessing module, a deep learning analysis module and a state diagnosis module. The control decision-making system comprises a state evaluation module, a fault early warning module and a maintenance decision-making module. According to the system, potential faults of the fan can be found as soon as possible, decision basis is provided for preventive maintenance, and the intelligent operation and maintenance level of a wind power plant is remarkably improved. The system adopts a distributed edge computing architecture, has real-time monitoring capability, adapts to various complex working environments, and provides powerful guarantee for safe and stable operation of the wind power plant.

Description

Technical Field

[0001] The present invention belongs to the technical field of wind power generation, and in particular relates to a system and method for analyzing the operating status of a wind turbine based on image recognition. Background Art

[0002] The safe and stable operation of wind turbines is directly related to the reliability of the power system. Analysis of wind turbine operating status is an important means to ensure the safe operation of wind turbines. Through real-time monitoring and analysis of wind turbine operating parameters, potential faults can be discovered in time to avoid equipment damage and reduced power generation efficiency. With the continuous increase in wind turbine installed capacity and the continuous improvement of single-machine power, higher requirements are placed on wind turbine operating status analysis.

[0003] At present, the analysis of wind turbine operation status mainly relies on traditional methods such as vibration monitoring, temperature monitoring and manual inspection. The vibration monitoring system can only monitor the vibration characteristics of local components; the temperature monitoring system is greatly affected by environmental factors and has limited monitoring accuracy; the manual inspection method has problems such as low efficiency, high cost, and inability to monitor in real time. These traditional monitoring methods are difficult to meet the needs of intelligent operation and maintenance of large wind farms. In the process of wind turbine operation status analysis, existing technologies are difficult to achieve comprehensive perception and intelligent diagnosis of wind turbine status. Especially for key components such as wind turbine blades, because they are in a high-speed motion state and the working environment is complex and changeable, existing monitoring methods are difficult to accurately identify and locate their early signs of failure, which restricts the level of preventive maintenance of wind farms.

[0004] The Chinese patent publication number is CN118587477A, ​​and the name is a patent application for an internal detection method and system for an offshore wind turbine cabin based on image recognition. The system includes multiple smart cameras with edge computing capabilities deployed inside the offshore wind turbine cabin to collect high-definition video images and environmental parameter data. The smart camera uses an image super-resolution reconstruction and denoising algorithm based on deep learning to improve image quality. The smart camera has a built-in edge computing device, which uses an improved deep learning target detection algorithm to quickly identify the internal equipment of the cabin and monitor its working status in real time. This patent application can realize the monitoring of the internal equipment of the offshore wind turbine cabin, but it cannot make status judgments on the images of the external blades of the wind turbine. Summary of the invention

[0005] In order to overcome the problems existing in the above-mentioned prior art, the purpose of the present invention is to provide a fan operating status analysis system and method based on image recognition, which realizes comprehensive perception and intelligent diagnosis of the fan operating status by adopting a three-layer architecture design of data acquisition system, edge computing system and control decision system.

[0006] To achieve the above object, the technical solution adopted by the present invention is: In the first aspect, the present invention provides a fan operation status analysis system based on image recognition, comprising: a data acquisition system, an edge computing system and a control decision system; the data acquisition system comprises: an image acquisition unit and an acoustic sensing unit; the edge computing system comprises: an image preprocessing module, a deep learning analysis module and a status diagnosis module; the image preprocessing module is communicatively connected to the image acquisition unit; the deep learning analysis module is communicatively connected to the image preprocessing module, and the status diagnosis module is communicatively connected to the acoustic sensing unit and the deep learning analysis module; the control decision system comprises: a status assessment module, a fault warning module and a maintenance decision module; the status assessment module is communicatively connected to the status diagnosis module, and the fault warning module is communicatively connected to the status assessment module; the maintenance decision module is communicatively connected to the fault warning module.

[0007] Optionally, the image acquisition unit includes: an infrared thermal imaging device, a high-speed camera and a full-day monitoring device; the acoustic sensing unit includes: an acoustic camera array and an ultrasonic sensor.

[0008] Optionally, the infrared thermal imaging device adopts a non-cooled focal plane detector; the all-day monitoring device has an IP67 protection level.

[0009] Optionally, the image preprocessing module includes a visible light image enhancement unit and an infrared thermal imaging enhancement unit.

[0010] Optionally, the image preprocessing module is embedded with an adaptive enhancement algorithm and a contrast enhancement algorithm.

[0011] Optionally, the deep learning analysis module has a YOLOv5 target detection network and a fine segmentation network with a U-Net++ structure.

[0012] Optionally, the edge computing system adopts a distributed edge computing architecture, and an edge computing system is provided on each wind turbine tower.

[0013] Optionally, the status diagnosis module has a defect classification and evaluation network.

[0014] In a second aspect, the present invention provides a method for analyzing the operating status of a fan based on image recognition, using the above-mentioned system for analyzing the operating status of a fan based on image recognition, comprising the following steps: Synchronously collect multi-source data through an image acquisition unit and an acoustic sensing unit of a data acquisition system; and send the collected multi-source data to an edge computing system; Complete image preprocessing, feature extraction and preliminary analysis through the edge computing system; send the analysis results to the control decision system; Perform in-depth analysis, status assessment and generate decision outputs through control decision systems.

[0015] Optionally, the preliminary analysis uses a loss function for calculation, and performs defect classification and severity assessment based on the calculated loss results.

[0016] Compared with the prior art, the present invention has the following beneficial effects: The present invention provides a fan operating status analysis system and method based on image recognition, which realizes comprehensive perception and intelligent diagnosis of the fan operating status by adopting a three-layer architecture design of a data acquisition system, an edge computing system, and a control decision system. Among them, the data acquisition system adopts a multimodal data acquisition method, combined with an image acquisition unit and an acoustic sensing unit, to overcome the limitations of a single sensing method; the edge computing system adopts a three-level recognition architecture, and through the coordinated work of a target detection network, a fine segmentation network, and a defect classification and evaluation network, the accuracy and reliability of the system's fan status recognition are improved; the multimodal data cross-validation mechanism set by the control decision system effectively reduces the system's false alarm rate.

[0017] Furthermore, the system of the present invention can detect potential faults of wind turbines early, provide decision-making basis for preventive maintenance, and significantly improve the intelligent operation and maintenance level of wind farms. The system adopts a distributed edge computing architecture, has real-time monitoring capabilities, adapts to various complex working environments, and provides a strong guarantee for the safe and stable operation of wind farms. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The drawings described herein are for explanation purposes only and are not intended to limit the scope of the present invention in any way. In addition, the shapes and proportional dimensions of the components in the drawings are only for illustration purposes to help understand the present invention, and are not intended to specifically limit the shapes and proportional dimensions of the components of the present invention. In the drawings: Figure 1 This is an overall architecture diagram of a wind turbine operating status analysis system based on image recognition according to an embodiment of the present invention; Figure 2 A multi-scale feature extraction network structure diagram of an embodiment of the present invention; Figure 3 This is a diagram of the target detection network architecture of an embodiment of the present invention; Figure 4 A fine segmentation network structure diagram of an embodiment of the present invention; Figure 5 A diagram of a multimodal cross-validation mechanism according to an embodiment of the present invention; Figure 6 This is a workflow diagram of a wind turbine operating status analysis system based on image recognition according to an embodiment of the present invention. DETAILED DESCRIPTION

[0019] In order to enable those skilled in the art to better understand the technical solutions in the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0020] Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the invention claimed for protection, but merely represents selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0021] In the description of the embodiments of the present invention, it should be noted that if the terms "upper", "lower", "horizontal", "inner", etc. appear, the orientation or position relationship indicated is based on the orientation or position relationship shown in the accompanying drawings, or is the orientation or position relationship in which the product of the invention is usually placed when used. It is only for the convenience of describing the present invention and simplifying the description, and does not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present invention.

[0022] When an element is referred to as being "disposed on" another element, it may be directly on the other element or there may also be a central element. When an element is considered to be "connected to" another element, it may be directly connected to the other element or there may also be a central element. The terms "vertical", "horizontal", "left", "right" and similar expressions used herein are for illustrative purposes only and are not intended to be the only embodiments. If the term "horizontal" appears, it does not mean that the component is required to be absolutely horizontal, but it can be slightly tilted. For example, "horizontal" only means that its direction is more horizontal than "vertical", which does not mean that the structure must be completely horizontal, but it can be slightly tilted.

[0023] It should be noted that similar reference numerals and letters represent similar items in the following figures, so once an item is defined in one figure, it does not need to be further defined and explained in the subsequent figures. In the description of the present invention, it is understood that the terms "include" and "comprise" indicate the presence of the described features, wholes, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or their collections.

[0024] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those generally understood by those skilled in the art of the technical field of the present invention. The terms used herein in the specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. As used in the specification of the present invention and the appended claims, the singular forms of "a", "an" and "the" are intended to include plural forms unless the context clearly indicates otherwise.

[0025] The present invention is described in detail below with reference to the accompanying drawings.

[0026] like Figure 1 As shown, a fan operation status analysis system based on image recognition of the present invention includes: a data acquisition system, an edge computing system and a control decision system.

[0027] The data acquisition system includes: an image acquisition unit and an acoustic sensing unit.

[0028] The edge computing system includes: an image preprocessing module, a deep learning analysis module and a status diagnosis module.

[0029] The control decision system includes: a state evaluation module, a fault warning module and a maintenance decision module.

[0030] The image preprocessing module is communicatively connected to the image acquisition unit; the deep learning analysis module is communicatively connected to the image preprocessing module; and the state diagnosis module is communicatively connected to the acoustic sensing unit and the deep learning analysis module.

[0031] The state evaluation module is communicatively connected to the state diagnosis module, the fault warning module is communicatively connected to the state evaluation module; and the maintenance decision module is communicatively connected to the fault warning module.

[0032] The present invention realizes comprehensive perception and intelligent diagnosis of the operating status of the wind turbine by adopting a three-layer architecture design of a data acquisition system, an edge computing system, and a control decision-making system. Among them, the data acquisition system adopts a multimodal data acquisition method, combined with an image acquisition unit and an acoustic sensing unit, to overcome the limitations of a single sensing method. The system of the present invention can detect potential faults of wind turbines early, provide a decision-making basis for preventive maintenance, and significantly improve the level of intelligent operation and maintenance of wind farms. The system adopts a distributed edge computing architecture, has real-time monitoring capabilities, and can adapt to various complex working environments, providing a strong guarantee for the safe and stable operation of wind farms.

[0033] Example 1 The wind turbine operating status analysis system based on image recognition provided by the present invention adopts a three-layer architecture design, including a data acquisition system, an edge computing system and a control decision system. The system of the present invention realizes real-time monitoring and intelligent diagnosis of the wind turbine operating status through multi-source image analysis and deep learning algorithms.

[0034] The data acquisition system includes an image acquisition unit and an acoustic sensing unit. The image acquisition unit includes an infrared thermal imaging device, a high-speed camera, and a full-day monitoring device. The acoustic sensing unit includes an acoustic camera array and an ultrasonic sensor. The edge computing system has three core modules: image preprocessing, deep learning analysis, and status diagnosis to complete data processing and analysis. The control decision system realizes intelligent decision-making of the system through three links: status evaluation, fault warning, and maintenance decision-making.

[0035] Information is transmitted between systems at all levels through data flows to ensure the real-time and reliability of system operation.

[0036] Specifically, the all-day monitoring device is an all-weather camera.

[0037] In the image acquisition unit, the infrared thermal imaging device uses an uncooled focal plane detector with a resolution of 640×480, a temperature measurement accuracy of ±2°C, and a field of view of 32°×24°.

[0038] Specifically, the infrared thermal imaging device uses an infrared thermal imaging camera, which uses dual-spectrum imaging technology to achieve synchronous acquisition and registration of visible light images and thermal imaging images. The high-speed camera uses a CMOS sensor with a maximum acquisition frame rate of 200fps and a resolution of 4K. The high-speed camera is equipped with a five-axis optical image stabilization system and uses a real-time motion compensation algorithm to eliminate image jitter. The all-day monitoring device has an IP67 protection level, uses an optical zoom lens with a zoom range of 5-50mm, and is equipped with an automatic rain and defogging device.

[0039] For different types of image data, an adaptive enhancement algorithm is used based on the image preprocessing module. For visible light images, an adaptive enhancement algorithm based on local contrast is used:

[0040] In the formula, represents the enhanced image; represents the original image; represents the local variance; Represents the adaptive gain coefficient. The algorithm can dynamically adjust the enhancement strength according to the local features of the image and effectively improve the visibility of details.

[0041] For infrared thermal imaging, a temperature feature-preserving contrast enhancement algorithm is used:

[0042] In the formula, represents the enhanced temperature image; represents the original temperature image; represents the temperature Laplace operator; represents the enhancement coefficient. This algorithm improves the image contrast while maintaining the accuracy of temperature measurement.

[0043] like Figure 2 As shown in the figure, based on the image preprocessing module, a multi-scale feature extraction network (MS-Net) is used to extract multi-level features of wind turbine blades. The network adopts a multi-branch parallel structure, including detail branch, texture branch and structure branch. The detail branch uses 1×1 convolution to extract local features, the texture branch uses 3×3 convolution to extract medium-scale features, and the structure branch uses 5×5 convolution to extract large-scale features. The feature fusion module dynamically fuses features of different scales through an adaptive weight mechanism and outputs a comprehensive feature map. This network structure effectively improves the ability to extract features of different scales of wind turbine blades.

[0044] In view of the high-speed rotation characteristics of wind turbine blades, an analysis method based on time-series image sequences is adopted. The phase-related sub-pixel registration algorithm is:

[0045] In the formula, , Respectively represent the Fourier transform of adjacent frames; represents a conjugate complex number; Represents the phase correlation function. The algorithm achieves high-precision inter-frame motion estimation and provides a reliable basis for subsequent state analysis.

[0046] Specifically, the edge computing system adopts a distributed edge computing architecture. The edge computing system of each wind turbine tower is equipped with an NVIDIA Jetson Xavier NX processor, 16GB RAM and 512GB SSD storage for local image processing and feature extraction. The control decision system adopts a cluster server architecture to achieve parallel monitoring of multiple wind turbines through load balancing.

[0047] In terms of image recognition framework, the edge computing system proposes a three-level recognition architecture.

[0048] like Figure 3 As shown, the first layer is based on a deep learning analysis module and uses an improved YOLOv5 target detection network for rapid screening.

[0049] The target detection network adopts a multi-module collaborative architecture, including a backbone network optimization module, a feature fusion module, and a detection optimization module. The backbone optimization network module adopts the CSPNet structure and includes a spatial attention module, a channel attention module, and a jump connection, which enhances the feature extraction capability. The feature fusion module achieves effective fusion of multi-scale features through three sub-modules: upsampling path, lateral connection, and feature aggregation. The detection optimization module adopts anchor optimization, multi-scale detection, and non-maximum suppression (NMS) optimization strategies to improve the accuracy and robustness of target detection.

[0050] like Figure 4 As shown, the second layer uses the U-Net++ network to perform fine defect segmentation on the results of the first layer based on the deep learning analysis module.

[0051] The fine segmentation network adopts an encoder-decoder structure and introduces two innovative modules: feature extraction enhancement and multi-scale decoding. The feature extraction enhancement part contains three key components: spatial attention, channel attention, and edge enhancement, which improves the feature expression ability of the network. The multi-scale decoding part achieves high-precision defect segmentation through skip connections, deep supervision, and feature recombination mechanisms. This network structure is particularly suitable for processing subtle defects on the surface of wind turbine blades and has strong edge perception capabilities.

[0052] The third layer uses a defect classification and evaluation network based on the state diagnosis module, and adopts a multi-task learning strategy to complete defect classification and severity evaluation at the same time. The loss function is designed as follows:

[0053] In the formula, represents the overall loss, represents the target detection loss function; represents the segmentation loss function; represents the classification loss function; represents the regression loss function; to Represents the weight coefficient of each loss function. Through multi-task joint optimization, the generalization ability of the model is improved.

[0054] like Figure 5 As shown, the control decision system adopts a cross-validation method based on multimodal data.

[0055] The cross-validation method based on multimodal data comprehensively utilizes image recognition results, temperature distribution and vibration characteristics to conduct a comprehensive evaluation through three verification strategies: consistency test, correlation analysis and historical comparison. After the evaluation results are judged for reliability, reliable results are directly output, and unreliable results enter the secondary analysis link, forming a closed-loop verification process, which effectively reduces the false alarm rate of the system.

[0056] Example 2 A method for analyzing the operation status of a fan based on image recognition, such as Figure 6 As shown, the system workflow diagram shows the complete link of data processing.

[0057] The fan operation status analysis system based on image recognition includes the following steps: Synchronously collect multi-source data through an image acquisition unit and an acoustic sensing unit of a data acquisition system; and send the collected multi-source data to an edge computing system; Complete image preprocessing, feature extraction and preliminary analysis through the edge computing system; send the analysis results to the control decision system; Perform in-depth analysis, status assessment and generate decision outputs through control decision systems.

[0058] The data transmission between the systems adopts a high-speed network channel to ensure the real-time data processing. The wind turbine operation status analysis system based on image recognition adopts a parallel processing mechanism, and multiple links run simultaneously, which improves the overall processing efficiency. At the same time, through the closed-loop feedback mechanism, the system can dynamically adjust the acquisition parameters and processing strategies according to the analysis results to achieve adaptive optimization. The design of this workflow effectively ensures the reliability and real-time performance of the system operation.

[0059] The present invention realizes intelligent analysis of the operating status of the fan through image recognition. The minimum defect size that can be detected reaches 1 mm, and the detection accuracy rate exceeds 95%; the end-to-end response time of the system of the present invention is less than 100 milliseconds, and the false alarm rate is less than 1%; the fan operating status analysis system based on image recognition of the present invention can warn of potential failures 72 hours in advance, and the prediction accuracy rate exceeds 90%; the fan operating status analysis system based on image recognition of the present invention can maintain stable operation under all-weather conditions, including complex environments such as strong light, rain and fog, and night.

[0060] Unless otherwise specified, the device elements involved in the above embodiments are all conventional device elements, and the structural settings, working modes or control modes involved are all conventional settings, working modes or control modes in the art unless otherwise specified.

[0061] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Other modifications or equivalent substitutions made to the technical solution of the present invention by ordinary technicians in the field should be included in the scope of the claims of the present invention as long as they do not depart from the spirit and scope of the technical solution of the present invention.

Claims

1. A fan operation status analysis system based on image recognition, characterized in that: include: Data acquisition system, edge computing system and control decision system; The data acquisition system includes: an image acquisition unit and an acoustic sensing unit; the edge computing system includes: an image preprocessing module, a deep learning analysis module and a status diagnosis module; the image preprocessing module is communicatively connected to the image acquisition unit; the deep learning analysis module is communicatively connected to the image preprocessing module, and the status diagnosis module is communicatively connected to the acoustic sensing unit and the deep learning analysis module; the control decision system includes: a status assessment module, a fault warning module and a maintenance decision module; the status assessment module is communicatively connected to the status diagnosis module, and the fault warning module is communicatively connected to the status assessment module; the maintenance decision module is communicatively connected to the fault warning module.

2. A fan operation status analysis system based on image recognition according to claim 1, characterized in that: The image acquisition unit includes: an infrared thermal imaging device, a high-speed camera and a full-day monitoring device; the acoustic sensing unit includes: an acoustic camera array and an ultrasonic sensor.

3. A fan operation status analysis system based on image recognition according to claim 2, characterized in that: The infrared thermal imaging device adopts a non-cooled focal plane detector; the all-day monitoring device has an IP67 protection level.

4. The wind turbine operating status analysis system based on image recognition according to claim 2, characterized in that: The image preprocessing module includes a visible light image enhancement unit and an infrared thermal imaging enhancement unit.

5. The wind turbine operating status analysis system based on image recognition according to claim 1, characterized in that: The image preprocessing module is embedded with an adaptive enhancement algorithm and a contrast enhancement algorithm.

6. The wind turbine operating status analysis system based on image recognition according to claim 1, characterized in that: The deep learning analysis module has a YOLOv5 target detection network and a fine segmentation network with a U-Net++ structure.

7. The wind turbine operating status analysis system based on image recognition according to claim 1, characterized in that: The edge computing system adopts a distributed edge computing architecture, and an edge computing system is provided on each wind turbine tower.

8. The wind turbine operating status analysis system based on image recognition according to claim 1, characterized in that: The status diagnosis module has a defect classification and evaluation network.

9. A method for analyzing the operation status of a fan based on image recognition, characterized in that: The fan operation status analysis system based on image recognition according to any one of claims 1 to 8 comprises the following steps: Synchronously collect multi-source data through an image acquisition unit and an acoustic sensing unit of a data acquisition system; and send the collected multi-source data to an edge computing system; Complete image preprocessing, feature extraction and preliminary analysis through the edge computing system; send the analysis results to the control decision system; Perform in-depth analysis, status assessment and generate decision outputs through control decision systems.

10. A method for analyzing wind turbine operating status based on image recognition according to claim 9, characterized in that: The preliminary analysis uses a loss function for calculation, and classifies defects and evaluates their severity based on the calculated loss results.

Images