Fan abnormality monitoring method, device and equipment based on image recognition and medium

Abstract

The application discloses an image recognition-based fan abnormality monitoring method and device, equipment and a medium, relates to the technical field of image recognition, and is applied to a wind turbine nacelle, and comprises the following steps: collecting a raw image sequence of a first candidate image area inside the wind turbine nacelle in real time, eliminating environmental interference factors in the raw image sequence of the first candidate image area, and obtaining a target image sequence; extracting visual features in the target image sequence of the first candidate image area, identifying visual abnormalities in the target image sequence of the first candidate image area based on the visual features of the first candidate image area; and in the case where the visual abnormalities are identified in the target image sequence of the first candidate image area, outputting alarm information containing the visual abnormalities of the first candidate image area. The application solves the technical problem of poor monitoring effect in the current fan monitoring scheme.

Description

Technical Field

[0001] This application relates to the field of image recognition technology, and in particular to a method, apparatus, equipment and medium for monitoring wind turbine anomalies based on image recognition. Background Technology

[0002] With the rapid development of the wind power industry and the continuous growth of wind turbine unit capacity, the operational safety and stability requirements of wind turbine units, as core energy conversion equipment, are increasingly stringent. Current wind turbine monitoring technologies primarily rely on traditional physical sensors such as temperature and vibration sensors deployed at key locations. However, these methods are limited by their working principles, typically only acquiring point-specific parameter information. They struggle to provide synchronous, visual monitoring of large areas within the wind turbine nacelle, lacking the ability to detect early, localized visual anomalies. Furthermore, alarm triggering by physical sensors often depends on parameter thresholds, resulting in response delays. Therefore, current wind turbine monitoring solutions suffer from poor monitoring performance.

[0003] The above content is only used to help understand the technical solution of this application and does not represent an admission that the above content is prior art. Summary of the Invention

[0004] The main purpose of this application is to provide a method, device, equipment and medium for wind turbine anomaly monitoring based on image recognition, which aims to solve the technical problem of poor monitoring effect in the current wind turbine monitoring scheme.

[0005] To achieve the above objectives, this application proposes an image recognition-based wind turbine anomaly monitoring method, applied to the nacelle of a wind turbine unit. The image recognition-based wind turbine anomaly monitoring method includes:

[0006] The original image sequence inside the wind turbine nacelle is acquired in real time, and environmental interference factors in the original image sequence are eliminated to obtain the target image sequence;

[0007] Visual features are extracted from the target image sequence, and visual anomalies in the target image sequence are identified based on the visual features;

[0008] If a visual anomaly is detected in the target image sequence, an alarm message containing the visual anomaly is output.

[0009] In one embodiment, the step of eliminating environmental interference factors in the original image sequence to obtain the target image sequence includes:

[0010] Identify environmental interference factors in the original image sequence;

[0011] When the environmental interference factor is static stain interference, the occlusion area of ​​the static stain interference is located, and the occlusion area is restored based on a preset uncontaminated image to obtain a first intermediate image sequence.

[0012] When the environmental interference factor is dynamic particle interference, the particle noise corresponding to the dynamic particle interference in the original image sequence is extracted and the particle noise is removed to obtain the second intermediate image sequence.

[0013] Perform contrast enhancement and color space transformation operations on both the first intermediate image sequence and / or the second intermediate image sequence to obtain the target image sequence.

[0014] In one embodiment, the step of restoring the occluded area based on a preset uncontaminated image to obtain a first intermediate image sequence includes:

[0015] Differential calculations are performed on each image frame in the original image sequence and a preset uncontaminated image to determine the target position of the occluded region in the preset uncontaminated image;

[0016] Pixel information of the target location is extracted from the preset uncontaminated image, and the pixel information is used to replace the occluded area to obtain the filled area;

[0017] A smooth transition process is performed on the boundary between the filled region and the unoccluded region of the original image sequence to obtain a first intermediate image sequence.

[0018] In one embodiment, the step of extracting particle noise corresponding to dynamic particle interference in the original image sequence includes:

[0019] For each spatial location in the original image sequence, a time-series statistical analysis is performed on the set of pixel values ​​at the spatial location to identify abnormal pixels that deviate from a preset normal value range in the set of pixel values, and the regions corresponding to all identified abnormal pixels are taken as candidate noise regions.

[0020] Extract the morphological change information and spatial distribution information of the candidate noise region in the original image sequence, and calculate the dynamic behavior feature measure of the candidate noise region based on the morphological change information and the spatial distribution information;

[0021] Based on the dynamic behavior feature metric, target noise that meets the preset dynamic particle interference determination metric is determined in the candidate noise region, and the target noise is used as the particle noise corresponding to the dynamic particle interference in the original image sequence.

[0022] In one embodiment, the visual features include at least color features, texture features, and shape features, and the visual anomalies include lubricating oil leakage anomalies. The step of identifying visual anomalies in the target image sequence based on the visual features includes:

[0023] Based on the color features, a first candidate image region that conforms to the preset oil stain color range is extracted from the target image sequence;

[0024] Calculate the first matching degree between the texture features of the first candidate image region and the preset oil stain texture features, and calculate the second matching degree between the shape features of the first candidate image region and the preset oil stain shape features;

[0025] If both the first matching degree and the second matching degree reach a preset matching degree threshold, it is determined that a visual abnormality is identified in the target image sequence, and the visual abnormality is a lubricating oil leakage abnormality.

[0026] In one embodiment, the visual features further include motion pattern features, and the visual anomalies further include smoke anomalies. The step of identifying visual anomalies in the target image sequence based on the visual features further includes:

[0027] Based on the texture features, a second candidate image region conforming to a preset smoke texture is identified in the target image sequence;

[0028] Identify the diffusion behavior of the second candidate image region based on the motion pattern features;

[0029] If the diffusion direction of the diffusion behavior remains unchanged and the diffusion process of the diffusion behavior is not interrupted, it is determined that a visual anomaly is identified in the target image, and the visual anomaly is a smoke anomaly.

[0030] In one embodiment, the visual features further include morphological change features, and the visual anomalies further include open flame anomalies. The step of identifying visual anomalies in the target image sequence based on the visual features further includes:

[0031] Based on the color features, a third candidate image region that conforms to a preset open flame spectral distribution is extracted from the target image sequence;

[0032] The flicker frequency and contour shape change rate in the third candidate image region are determined based on the morphological change characteristics.

[0033] If the flickering frequency reaches a preset frequency threshold, the mean of the contour shape change rate reaches a preset mean, and the variance of the contour shape change rate reaches a preset variance, then a visual abnormality is identified in the target image, and the visual abnormality is an open flame abnormality.

[0034] Furthermore, to achieve the above objectives, this application also proposes an image recognition-based wind turbine anomaly monitoring device, applied to the nacelle of a wind turbine unit. The image recognition-based wind turbine anomaly monitoring device includes:

[0035] The image acquisition module is used to acquire raw images of the interior of the wind turbine nacelle in real time, eliminate environmental interference factors in the raw images, and obtain the target image;

[0036] Anomaly detection module is used to extract visual features from the target image and identify visual anomalies in the target image based on the visual features;

[0037] An alarm output module is used to output alarm information containing the visual abnormality when a visual abnormality is detected in the target image.

[0038] In addition, to achieve the above objectives, this application also proposes an electronic device, the device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the image recognition-based wind turbine anomaly monitoring method described above.

[0039] In addition, to achieve the above objectives, this application also proposes a storage medium, which is a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, it implements the steps of the image recognition-based wind turbine anomaly monitoring method described above.

[0040] In addition, to achieve the above objectives, this application also provides a computer program product, which includes a computer program that, when executed by a processor, implements the steps of the image recognition-based wind turbine anomaly monitoring method described above.

[0041] This application provides a wind turbine anomaly monitoring method based on image recognition, applied to the nacelle of a wind turbine. The method includes: real-time acquisition of original image sequences of the interior of the wind turbine nacelle in a first candidate image region; elimination of environmental interference factors in the original image sequences of the first candidate image region to obtain a target image sequence; extraction of visual features from the target image sequence of the first candidate image region; identification of visual anomalies in the target image sequence of the first candidate image region based on the visual features; and output of alarm information containing visual anomalies in the first candidate image region when visual anomalies are identified in the target image sequence of the first candidate image region.

[0042] This application overcomes the problem of image quality degradation under harsh operating conditions by acquiring raw image sequences of the nacelle in real time and eliminating environmental interference factors. This provides a clear and stable visual data foundation for subsequent analysis. Visual features are extracted from the processed target image sequence, and visual anomalies are identified based on these features. This enables synchronous, non-contact monitoring of a large area inside the nacelle, which can keenly capture signs of visual anomalies, significantly improving the comprehensiveness and early warning capabilities of monitoring. When an anomaly is detected, alarm information is automatically output, realizing an automated closed loop from image perception to decision feedback, effectively reducing the risk of fault escalation due to response delays. Compared with related solutions that struggle to synchronously and visually monitor a large area inside the wind turbine nacelle and suffer from response lag, this application, through visual perception, intelligent analysis, and real-time early warning, solves the inherent defects of existing wind turbine monitoring solutions in terms of coverage, response speed, and recognition capabilities, significantly improving the comprehensiveness and reliability of wind turbine status monitoring. Attached Figure Description

[0043] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0044] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0045] Figure 1 This is a flowchart illustrating an embodiment of the image recognition-based wind turbine anomaly monitoring method of this application.

[0046] Figure 2 This is a flowchart illustrating Embodiment 2 of the image recognition-based wind turbine anomaly monitoring method of this application;

[0047] Figure 3 This is a flowchart illustrating Embodiment 3 of the image recognition-based wind turbine anomaly monitoring method of this application;

[0048] Figure 4 This is a schematic diagram of the module structure of the wind turbine anomaly monitoring device based on image recognition, as described in an embodiment of this application.

[0049] Figure 5 This is a schematic diagram of the equipment structure of the hardware operating environment involved in the image recognition-based wind turbine anomaly monitoring method in this application embodiment.

[0050] The purpose, features, and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0051] It should be understood that the first embodiment described herein is merely used to explain the technical solution of this application and is not intended to limit this application.

[0052] To better understand the technical solution of this application, a detailed description will be provided below in conjunction with the accompanying drawings and specific implementation methods.

[0053] The first embodiment of this application is applied to the nacelle of a wind turbine. The main solution is: a wind turbine anomaly monitoring method based on image recognition includes: real-time acquisition of the original image sequence inside the nacelle of the wind turbine in the first candidate image region; elimination of environmental interference factors in the original image sequence of the first candidate image region to obtain the target image sequence; extraction of visual features in the target image sequence of the first candidate image region; identification of visual anomalies in the target image sequence of the first candidate image region based on the visual features of the first candidate image region; and output of alarm information containing visual anomalies in the first candidate image region when visual anomalies are identified in the target image sequence of the first candidate image region.

[0054] In the first embodiment, for ease of description, the following description will focus on an image recognition-based wind turbine anomaly monitoring system.

[0055] Existing technologies primarily rely on traditional physical sensors deployed at key locations for monitoring, such as those for temperature and vibration. However, due to the limitations of their operating principles, these methods typically only acquire point-specific parameter information, making it difficult to perform synchronous visual monitoring of large areas inside the wind turbine nacelle. They also lack the ability to detect early, localized visual anomalies. Furthermore, the alarm triggering of physical sensors often depends on parameter thresholds, resulting in response delays.

[0056] This application provides a solution that effectively overcomes the problem of image quality degradation under harsh operating conditions by acquiring raw image sequences of the wind turbine nacelle in real time and eliminating environmental interference factors. This provides a clear and stable visual data foundation for subsequent analysis. Visual features are extracted from the processed target image sequence, and visual anomalies are identified based on these features. This enables synchronous, non-contact monitoring of a large area inside the nacelle, keenly capturing signs of visual anomalies and significantly improving the comprehensiveness and early warning capabilities of monitoring. When an anomaly is detected, alarm information is automatically output, realizing an automated closed loop from image perception to decision feedback, effectively reducing the risk of escalation of faults due to response delays. Compared to related solutions that struggle to synchronously and visually monitor large areas inside the wind turbine nacelle and suffer from response lag, this application, through visual perception, intelligent analysis, and real-time early warning, solves the inherent deficiencies of existing wind turbine monitoring solutions in terms of coverage, response speed, and recognition capabilities, significantly improving the comprehensiveness and reliability of wind turbine status monitoring.

[0057] It should be noted that the executing entity of the first embodiment can be a computing service device with data processing, network communication, and program execution functions, such as a tablet computer, personal computer, mobile phone, or other electronic device, or a system, application, or program capable of implementing the above functions. The first embodiment and the following embodiments will be described using an image recognition-based wind turbine anomaly monitoring system as an example.

[0058] All actions involving the acquisition of signals, information, or data in this application are carried out in accordance with the relevant data protection laws and policies of the country where the application is located, and with the authorization of the owner of the relevant device.

[0059] Based on this, embodiments of this application provide a wind turbine anomaly monitoring method based on image recognition, referring to... Figure 1 , Figure 1 This is a flowchart illustrating the first embodiment of the wind turbine anomaly monitoring method based on image recognition in this application.

[0060] In this embodiment, the image recognition-based wind turbine anomaly monitoring method is applied to the nacelle of a wind turbine unit, including steps S01~S03:

[0061] Step S01: Real-time acquisition of the original image sequence inside the wind turbine nacelle, elimination of environmental interference factors in the original image sequence, and obtaining the target image sequence;

[0062] It should be noted that the wind turbine nacelle refers to the sealed enclosure on top of the wind turbine that houses the core equipment such as the gearbox, generator, and control system. The raw image sequence is a continuous, unprocessed sequence of frames directly output from the camera, which may contain imperfections such as blurriness and noise. Environmental interference factors refer to internal nacelle conditions that affect image quality, such as vibration blurring caused by high-speed equipment rotation, lens obstruction by airborne dust particles, and uneven lighting due to changes in sunlight. The target image sequence refers to a high-quality image sequence obtained after preprocessing the raw image sequence, including denoising and enhancement, making it more suitable for computer analysis.

[0063] Additionally, it should be noted that cameras are installed inside the wind turbine nacelle. These cameras can be deployed on the top of the nacelle, near the gearbox and generator, and near the electrical cabinet. The top of the nacelle can provide global situational awareness, monitor the spread of smoke over a large area, the location of open flames, and the relative positions of various subsystems. The gearbox, generator, and other core equipment, as well as the electrical cabinet, are key areas prone to anomalies, and therefore also need to be monitored.

[0064] Additionally, it should be noted that multiple cameras deployed inside the wind turbine nacelle continuously capture images synchronously at a set frame rate, generating concurrent video streams covering different angles and focal lengths, which together constitute a complete sequence of original images.

[0065] Understandably, step S01 obtains a clear and stable target image sequence by acquiring and eliminating environmental interference in the original image sequence in real time. This overcomes the defects of false alarms and missed alarms caused by environmental factors in existing technologies, improves the environmental adaptability of monitoring and the reliability of data input, and lays a robust visual data foundation for subsequent accurate analysis.

[0066] Step S02: Extract visual features from the target image sequence and identify visual anomalies in the target image sequence based on the visual features;

[0067] It should be noted that visual features are information extracted from the target image sequence to describe and distinguish different visual patterns. These include quantifiable features such as the shape, color, and texture of the core equipment surface inside the wind turbine nacelle, such as color and texture features. Visual anomalies are signs in the target image sequence that deviate from the normal visual state inside the wind turbine nacelle and indicate potential malfunctions, such as discoloration and flow marks caused by lubricating oil leaks, trace amounts of smoke generated in the early stages of electrical equipment overheating, or sparks generated by friction-induced fire.

[0068] Understandably, step S02 extracts and analyzes visual features in the target image sequence to identify anomalies, expanding the monitoring dimension from discrete point parameters to a continuous spatial visual field. This solves the technical problem that traditional point-based sensors have limited monitoring coverage and cannot fully perceive early visual morphological anomalies in a large area inside the cabin, and overcomes the monitoring blind spots and perception lag of existing technologies.

[0069] Step S03: If a visual abnormality is identified in the target image sequence, output an alarm message containing the visual abnormality.

[0070] It should be noted that alarm information refers to structured data containing information such as the type, location, and severity of visual anomalies, used to notify users of system failures.

[0071] For example, alarm information may include the type of visual anomaly (such as "lubricating oil leak"), visual location (such as "below the southeast side of the gearbox"), time of occurrence, and relevant image snapshots as evidence. It can be prompted locally through audio-visual devices and pushed to the remote monitoring center and the mobile terminals of maintenance personnel in real time via the network.

[0072] Understandably, step S03 automatically outputs alarm information when visual anomalies are detected, which solves the response delay problem caused by traditional sensors relying on parameter thresholds for triggering. It can trigger early warnings in the early stages of visual anomalies, providing maintenance personnel with sufficient emergency response time and significantly improving the timeliness and proactive early warning capabilities of the monitoring system.

[0073] Based on the first embodiment of this application, in the second embodiment of this application, the content that is the same as or similar to the first embodiment described above can be referred to the above description, and will not be repeated hereafter. Based on this, please refer to... Figure 2 In step S01, the step of eliminating environmental interference factors in the original image sequence to obtain the target image sequence includes steps S11 to S14:

[0074] Step S11: Identify environmental interference factors in the original image sequence;

[0075] It should be noted that environmental interference factors include static contamination and dynamic particulate interference. Static contamination refers to fixed contaminants that adhere to the camera lens over a long period, such as water stains and dust clumps. Dynamic particulate interference refers to transient particulate matter that floats and moves in the cabin air, such as drifting dust, lint, and small insects.

[0076] For example, the original image sequence is analyzed to calculate the stability of pixel values ​​at each spatial location in the image sequence in the time dimension. Regions with fixed positions and change rates below a threshold in consecutive frames are identified from the original image sequence, and these regions are determined to have static dirt interference. At the same time, the pixel-level differences between consecutive frames in the original image sequence are calculated and the motion characteristics of the difference regions are analyzed to identify regions with continuously changing positions and random motion in the original image sequence, and these regions are determined to have dynamic particle interference.

[0077] Additionally, it should be noted that the original image sequence can also be identified using a pre-trained machine learning model for recognizing environmental interference factors. This machine learning model is trained on a dataset containing numerous examples of static dirt interference and dynamic particle interference, enabling it to identify environmental interference factors in the original image sequence and distinguish the type of these interference factors.

[0078] Step S12: When the environmental interference factor is static stain interference, locate the occlusion area of ​​static stain interference, and restore the occlusion area based on the preset uncontaminated image to obtain the first intermediate image sequence.

[0079] It should be noted that the obscured area refers to the image range where scene information is lost due to being covered by fixed contaminants. The preset uncontaminated image refers to a reference image that is pre-collected and stored in a clean state without any stains obscuring the image. The first intermediate image sequence is an image sequence after static stain restoration, and its device surface morphology is consistent with the uncontaminated state.

[0080] Additionally, it should be noted that when identifying environmental interference factors in the original image sequence, the occlusion area is the region with a fixed position and an intensity change rate lower than the change rate threshold that is identified in multiple consecutive frames of images.

[0081] Additionally, it should be noted that after identifying the area affected by static stains, the percentage of that area in the camera lens is calculated. When this percentage reaches a preset ratio (e.g., 10%), a cleaning notification is sent to maintenance personnel to prevent large static stains from obscuring important information and affecting anomaly identification.

[0082] Step S13: When the environmental interference factor is dynamic particle interference, extract the particle noise corresponding to the dynamic particle interference in the original image sequence, remove the particle noise, and obtain the second intermediate image sequence.

[0083] It should be noted that dynamic particle interference refers to particles that move randomly in the short term, and their position and shape change over time. Particle noise is an abnormal set of pixels formed by dynamic particles in the original image sequence, consisting of small, bright or dark spots that are distributed and move rapidly. The second intermediate image sequence is the image sequence after removing dynamic particles, and its image clarity and the sharpness of static object edges are significantly improved.

[0084] Additionally, it should be noted that particle noise removal can be achieved through various filtering methods, such as spatial domain filtering, frequency domain filtering, and time-series filtering. For details, please refer to existing technologies; further elaboration will not be provided here.

[0085] Step S14: Perform contrast enhancement and color space transformation operations on both the first intermediate image sequence and / or the second intermediate image sequence to obtain the target image sequence.

[0086] It should be noted that uneven lighting and dimness in some areas inside the wind turbine nacelle can easily lead to insufficient overall or local contrast in the image. Therefore, contrast enhancement is necessary for both the first and / or second intermediate image sequences. Contrast enhancement is a technique that adjusts the pixel intensity distribution of an image using algorithms to amplify the differences in brightness between different regions. Essentially, it remaps the image's grayscale levels, stretching or transforming pixel values ​​that were originally concentrated within a narrow range to a wider dynamic range. By stretching the effective grayscale range, key details in the image (such as the edges of dark oil stains and the initial texture of light smoke) become more distinct and easily distinguishable, significantly improving the sensitivity and reliability of subsequent feature extraction steps.

[0087] Additionally, it should be noted that the lighting conditions inside the wind turbine nacelle are complex and variable (e.g., light entering through windows, equipment shadows), causing the color of the same object to change drastically with brightness in the RGB space, affecting recognition stability. Therefore, it is necessary to perform color space transformation operations on both the first and / or second intermediate image sequences. Color space transformation (e.g., conversion to HSV) decouples color attributes from brightness, enabling the system to make judgments based on stable hue characteristics, thereby significantly reducing lighting interference and improving the accuracy and environmental adaptability of color anomaly recognition. For example, converting images from the RGB color space to spaces such as HSV / HSL, and primarily utilizing the hue component, can relatively stably describe color types, reducing the impact of changes in light intensity. HSV refers to decomposing color into Hue, Saturation, and Value, which is more consistent with human color perception. HSL refers to decomposing color into Hue, Saturation, and Lightness.

[0088] In this embodiment, intelligent identification of interference types enables adaptive selection of processing strategies. By eliminating static interference such as lens stains, visual information of obscured areas is restored, solving the problem of blind spots in key areas caused by lens dirt. By filtering out dynamic particle noise such as airborne dust, false alarms caused by this are significantly reduced, enhancing the stability and reliability of monitoring. Through unified contrast enhancement and color space transformation, the recognizability and consistency of image features are improved, laying a solid foundation for subsequent high-precision anomaly identification.

[0089] In one feasible implementation, step S12, which involves restoring the occluded area based on a preset uncontaminated image to obtain a first intermediate image sequence, includes steps A01 to A03:

[0090] Step A01: Perform differential calculation on each image frame in the original image sequence and the preset uncontaminated image to determine the target position of the occluded area in the preset uncontaminated image;

[0091] It should be noted that an image frame is a single image in the original image sequence. Differential calculation refers to the operation of comparing brightness or color differences pixel by pixel. The target location refers to the coordinates of the occluded area in the preset uncontaminated image.

[0092] Step A02: Extract pixel information of the target location from the preset uncontaminated image, and replace the pixel information with the occluded area to obtain the filled area;

[0093] It should be noted that pixel information refers to the color (RGB) and brightness (grayscale) values ​​of a single pixel in an image. The filled area refers to the area after the occluded area has been filled with pixel information.

[0094] Step A03: Perform a smooth transition process on the boundary between the filled region and the unoccluded region of the original image sequence to obtain the first intermediate image sequence.

[0095] It should be noted that the unobstructed area refers to the normal display area in the original image that is not covered by static stains. Smooth transition processing is an algorithmic process used to eliminate abrupt changes in image stitching edges, ensuring a natural and seamless restoration.

[0096] For example, firstly, differential calculation is performed. By comparing the contaminated current image frame with a pre-acquired and stored reference image in a clean state (i.e., a preset uncontaminated image) at the pixel level, the pixel difference region caused by the static stain is calculated, thereby determining the original scene content and its location obscured by the static stain in the reference image. Subsequently, pixel information migration is performed. Uncontaminated original pixel data is extracted from the corresponding position in the reference image and filled into the occluded area of ​​the static stain in the current image frame, replacing the contaminated pixel values, thus directly restoring the obscured scene information. Finally, boundary fusion processing is performed. For possible brightness or color jumps between the filled area and the surrounding original uncontaminated area, an image fusion algorithm is used to perform weighted smooth transition on the boundary pixels, ensuring that the repaired area visually seamlessly connects with the entire image, generating a first intermediate image sequence.

[0097] In this embodiment, the corresponding position of the occluded area in the uncontaminated image is determined by differential calculation, which solves the problem of inaccurate image target positioning caused by static stain interference. Pixel information of the corresponding position is extracted from the preset uncontaminated image and filled into the occluded area, effectively restoring the original scene visual data that was continuously covered by stains. The boundary between the filled area and the original uncontaminated area is smoothly transitioned, eliminating visual artifacts and seams that may be introduced by the repair, and ensuring the visual naturalness and spatiotemporal coherence of the repaired image sequence.

[0098] In one feasible implementation, step S13, the step of extracting the particle noise corresponding to the dynamic particle interference in the original image sequence, includes steps A11 to A13:

[0099] Step A11: For each spatial location in the original image sequence, perform temporal statistical analysis on the set of pixel values ​​at the spatial location, identify abnormal pixels in the set of pixel values ​​that deviate from the preset normal value range, and take the regions corresponding to all identified abnormal pixels as candidate noise regions.

[0100] It should be noted that spatial location refers to the fixed coordinates of each pixel in the original image sequence. The pixel value set refers to the sequence of grayscale / RGB values ​​of a spatial location across all image frames; for example, the brightness or color values ​​of a spatial location over 30 consecutive frames. Temporal statistical analysis refers to performing statistical calculations (such as mean, standard deviation, and extreme value distribution) on the pixel value sequence over time. The preset normal value range is the pixel value fluctuation range based on historical clean image statistics (such as the mean grayscale value ± 3σ), representing a reasonable fluctuation range of pixel values ​​under normal, interference-free conditions. Abnormal pixels are pixels whose values ​​deviate from the normal value range. Candidate noise regions are continuous regions formed by the aggregation of abnormal pixels.

[0101] For example, a probabilistic model (e.g., a Gaussian mixture model) based on historical frames is built from the original image sequence to describe the range of brightness and color fluctuations (i.e., the "normal range") under normal image conditions. When the pixel value of a point in a new frame deviates significantly from this model, it is marked as an anomalous pixel. After spatially clustering all anomalous pixels, candidate noise regions are formed. For example, for a pixel on a background wall in an airplane cabin, its model describes the color range under normal lighting; when a speck of dust flies quickly past that point, it causes the color of that point to momentarily brighten or darken, thus being identified as an anomalous point by the model.

[0102] Step A12: Extract the morphological change information and spatial distribution information of the candidate noise region in the original image sequence, and calculate the dynamic behavior feature measure of the candidate noise region based on the morphological change information and spatial distribution information.

[0103] It should be noted that morphological change information refers to the shape and area change characteristics of the candidate noise region over time (such as the area expansion rate). Spatial distribution information refers to the positional change characteristics of the candidate noise region in image space (such as the trajectory of the center coordinate movement). Dynamic behavior feature metrics are parameters used to quantify the motion characteristics of the candidate noise region (such as directional consistency and velocity fluctuation rate).

[0104] For example, the evolution of each candidate region across multiple frames of images is tracked to extract morphological change information (such as the drastic increase or decrease in area and the variance of contour stability) and spatial distribution information (such as the regularity of the motion trajectory of the region's centroid and the dispersion of distribution among multiple regions). Based on this information, a comprehensive dynamic behavior feature metric is calculated. For instance, a region formed by dust may have a fluctuating area and a random motion trajectory; while the edge of a potential oil stain changes relatively slowly and is directional.

[0105] Step A13: Based on the dynamic behavior feature measurement, determine the target noise in the candidate noise region that meets the preset dynamic particle interference judgment metric, and use the target noise as the particle noise corresponding to the dynamic particle interference in the original image sequence.

[0106] It should be noted that the preset dynamic particle interference judgment metric is a combination of thresholds that distinguish dynamic particles from static stains (e.g., directional consistency > 0.7 and velocity fluctuation rate > 0.3). The target noise is a candidate noise region that meets the corresponding conditions of the preset dynamic particle interference judgment metric.

[0107] For example, based on a preset dynamic particle interference determination metric, the calculated feature metric is used to perform final classification of candidate regions. The dynamic particle interference determination metric is essentially a classifier decision boundary, dividing the high-dimensional feature space into two categories: dynamic particle interference and other foreground. For instance, the dynamic particle interference determination metric can be set as follows: when the randomness index of a region's motion trajectory is higher than a preset randomness threshold X, and its area change variance is higher than a preset change threshold Y, it is determined to be target noise. This allows for the identification and localization of transient particles such as drifting dust and water mist from the image, providing a precise target for subsequent noise removal.

[0108] In this embodiment, transient anomalies deviating from the normal range of pixel values ​​are identified and candidate noise regions are located through temporal statistical analysis. This solves the technical problem that traditional methods struggle to distinguish between transient noise and real abnormal signals under dynamic particle interference, achieving preliminary quantitative perception of transient interference. By extracting and analyzing the temporal morphological changes and spatial distribution information of candidate noise regions and calculating their dynamic behavior feature metrics, the problem that relying solely on a single frame or single feature cannot accurately describe the dynamic behavior pattern of particles is solved. This provides multi-dimensional criteria for distinguishing different types of dynamic interference. Based on the calculated dynamic behavior feature metrics, target noise that conforms to typical dynamic particle interference patterns is selected according to preset judgment rules. This solves the technical challenge of accurately separating and confirming particle noise in complex backgrounds and effectively suppresses false alarms caused by non-abnormal dynamic factors such as dust.

[0109] Based on the first and second embodiments of this application, in the third embodiment of this application, the content that is the same as or similar to the first and second embodiments described above can be referred to the above description, and will not be repeated hereafter. Based on this, please refer to... Figure 3 In step S02, the visual features include at least color features, texture features, and shape features, and the visual anomalies include lubricating oil leakage anomalies. The steps for identifying visual anomalies in the target image sequence based on visual features include steps S21 to S23:

[0110] Step S21: Extract the first candidate image region that matches the preset oil stain color range from the target image sequence based on color features;

[0111] It should be noted that color features refer to the data attributes related to pixel color that are quantified and extracted from the target image sequence. For example, after converting an image from the RGB color space to HSV / HSL, the hue components in the image are considered. The preset oil stain color range is a pre-defined range of values ​​within a selected color space (such as the hue channel of HSV) based on the typical colors of lubricating oils (such as mineral oil, which range from yellow to brown). The first candidate image region is the connected region formed by all pixels in the entire image that conform to the preset oil stain color range, identified through color thresholding and connected component analysis.

[0112] For example, after the color space transformation operation in step S14, a preset abnormal color threshold range (e.g., the yellow to brown hue range corresponding to mineral lubricating oil) is applied to the hue and saturation channels of the color space (such as HSV), and the image is scanned pixel by pixel and connected component analysis is performed to extract all image regions that conform to the preset oil stain color range as the first candidate image regions.

[0113] Step S22: Calculate the first matching degree between the texture features of the first candidate image region and the preset oil stain texture features, and calculate the second matching degree between the shape features of the first candidate image region and the preset oil stain shape features;

[0114] It should be noted that texture features are mathematical descriptors used to quantify and describe the spatial patterns of grayscale or color variations within a target image sequence. These include local contrast (reflecting the changes in brightness caused by oil stain reflection), uniformity (the oil film area may be relatively uniform), or roughness. Preset oil stain texture features are predefined texture descriptor templates or thresholds that characterize typical surface patterns of lubricating oil stains. For example, liquid oil stains often exhibit a specific gloss and wetness, and their texture patterns differ from dry rust or dust. Shape features are mathematical descriptors used to quantify and describe the external contours and overall shape of an image region, including the region's aspect ratio, roundness, and contour compactness. Preset oil stain shape features are predefined shape descriptor templates or thresholds that characterize typical diffusion patterns of lubricating oil leaks. For example, leaked oil stains often appear as irregular but flowing strips, accumulated drips at low points, or patches with wetting edges. The first matching degree is the degree of similarity between the texture features of the first candidate image region and the preset oil stain texture features, calculated using a preset contrast calculation algorithm (such as calculating the distance between feature vectors). The second matching degree is the degree of similarity between the shape features of the first candidate image region and the preset oil stain shape features. These two values ​​are usually normalized to a specific range (such as 0 to 1), with higher values ​​indicating greater similarity.

[0115] For example, texture features (such as local contrast and uniformity indices reflecting surface reflectivity and wetness) of the first candidate image region are extracted, and the similarity between the texture features and preset oil stain texture features (i.e., typical texture patterns of oil stains) is calculated to obtain a first matching degree. Simultaneously, shape features (such as aspect ratio, compactness, and similarity to fluid morphology) of the first candidate image region are extracted, and the similarity between the shape features and preset oil stain shape features is calculated to obtain a second matching degree. For example, for rust spots with matching colors but dry texture and sharp edges, the first matching degree will be significantly lower than the threshold; while for regions with matching colors and a wet sheen and a flowing, extended outline, both matching degrees will remain at a high level.

[0116] Step S23: If both the first matching degree and the second matching degree reach the preset matching degree threshold, it is determined that a visual abnormality is identified in the target image sequence, and the visual abnormality is a lubricating oil leakage abnormality.

[0117] It should be noted that abnormal lubricating oil leakage refers to the unexpected escape of lubricating oil from sealed components or pipelines, forming visible oil stains, droplets, or flows on the equipment surface. The core functions of lubricating oil are to reduce friction, cool, and clean. Leakage can lead to insufficient lubrication of critical moving parts such as gearboxes and bearings, causing dry friction and irreversible mechanical damage such as gear pitting and bearing sintering in a short period of time. Furthermore, leaked lubricating oil can vaporize and potentially ignite upon contact with high-temperature surfaces such as gearboxes, causing a fire in the nacelle. In the enclosed and high-altitude environment of a wind turbine nacelle, a fire could result in a catastrophic accident that destroys the entire machine.

[0118] For example, the first matching degree and the second matching degree are compared with preset texture matching degree threshold and shape matching degree threshold respectively. If and only if both matching degrees reach or exceed their respective thresholds, the current candidate region is determined to be an abnormal lubricating oil leakage.

[0119] Additionally, it should be noted that when lubricating oil leaks are detected, they are usually cleaned by maintenance personnel. However, during the cleaning process, due to lubricating oil penetration, some areas in the engine compartment may not be completely cleaned, leaving a historical oil stain mark. When identifying static stain interference, since historical oil stain marks are always static, they may be misidentified as static stain interference during the identification process. Therefore, after each lubricating oil leak anomaly occurs, if the lubricating oil leak anomaly is detected and cleaned (e.g., receiving a cleanup completion instruction from maintenance personnel), the first image frame after cleanup is acquired. The first image frame is compared with the corresponding preset uncontaminated image to determine whether there is a historical oil stain mark in the area corresponding to the static stain interference. If a historical oil stain mark is identified, the area corresponding to the historical oil stain mark is marked as historical environmental stain. Historical environmental stains are used to distinguish static stain interference and prevent this area from being identified as static stain interference.

[0120] In this embodiment, the suspected lubricating oil leakage area is quickly located by preset oil stain color range, solving the problem of false detection caused by color interference (such as equipment corrosion and oil accumulation) in traditional methods, and improving the initial screening efficiency. By calculating the matching degree of texture and shape, interference sources with similar colors but inconsistent textures (such as granular rust) or shapes (regular geometric shapes) are eliminated, solving the technical defect of insufficient specificity of single feature recognition. The dual matching degree threshold collaborative judgment is adopted to ensure the reliability of anomaly recognition, significantly improving the perception accuracy of the monitoring system for lubricating oil leakage anomalies, thereby effectively improving the problem of poor monitoring effect of traditional solutions.

[0121] In one feasible implementation, in step S02, the visual features further include motion pattern features, and the visual anomalies further include smoke anomalies. The step of identifying visual anomalies in the target image sequence based on visual features further includes steps B01 to B03:

[0122] Step B01: Identify a second candidate image region in the target image sequence that matches a preset smoke texture based on texture features;

[0123] It should be noted that the preset smoke texture is a mathematical model or feature template used to describe the typical texture characteristics of smoke, not a specific image, but a set of quantified features (such as high local entropy, low edge gradient magnitude, and specific spatial frequency distribution). These features collectively describe the visual characteristics of smoke: blurred edges, uneven internal texture, and a flocculent appearance. The second candidate image region is a set of one or more connected pixels segmented from the target image sequence based on the degree of matching between the texture features and the preset smoke texture.

[0124] For example, the local texture of each region in the target image sequence is calculated. By analyzing the consistency of the image gradient direction or the uniformity of local contrast, regions with blurred edges and semi-transparent and non-uniform internal textures are identified. Specifically, the target image sequence is divided into multiple sub-blocks, and the local binary mode variance and entropy value of each sub-block are calculated. Sub-blocks with variances below a set threshold and entropy values ​​above a set threshold are identified as having blurred and semi-transparent characteristics. Adjacent sub-blocks of this type are merged to mark the second candidate image region.

[0125] Step B02: Identify the diffusion behavior of the second candidate image region based on motion pattern features;

[0126] It should be noted that motion pattern features are data vectors calculated from the target image sequence to quantify the regularity of the target's motion. They can be composed of optical flow field statistics (such as average motion direction and direction consistency variance), trajectory continuity indices, etc. Diffusion behavior is a quantitative description of the evolution of the second candidate image region in the time dimension. It can be determined by tracking the area change rate, centroid movement trajectory, and contour deformation degree of the second candidate image region in consecutive frames. It aims to capture the characteristic of smoke continuously spreading rather than moving back and forth or appearing and disappearing instantaneously.

[0127] For example, a dense optical flow field is calculated for consecutive image frames containing the second candidate image region, and the consistency of the overall direction and amplitude of pixel motion vectors within the second candidate image region is analyzed. Specifically, the centroid position and area changes of the second candidate image region in five consecutive image frames are tracked, the direction angle sequence of centroid movement is calculated, and the continuity of the area growth rate between adjacent frames is calculated. If the standard deviation of the direction angle sequence is less than a preset angle tolerance, and the area growth rate is consistently positive, then the second candidate image region is determined to exhibit diffusion behavior where the diffusion direction remains unchanged and the diffusion process is uninterrupted. The diffusion direction is a vector direction that characterizes the overall movement trend, obtained by statistically calculating the centroid displacement or main optical flow direction of the second candidate image region. Uninterrupted diffusion means that the second candidate image region is continuously and stably tracked in the consecutive frame sequence, and its area or energy does not suddenly decrease or return to zero.

[0128] Step B03: If the diffusion direction of the diffusion behavior remains unchanged and the diffusion process of the diffusion behavior is not interrupted, it is determined that a visual abnormality is identified in the target image, and the visual abnormality is a smoke abnormality.

[0129] It should be noted that smoke anomalies refer to abnormal visual patterns in the target image sequence that conform to the physical diffusion characteristics of smoke. Smoke is essentially a mixture of visible solid, liquid particles, and gases produced by the thermal decomposition or combustion of substances. If smoke is detected, it means that some equipment components have suffered substantial thermal damage or are in the early stages of combustion, indicating that the equipment is on the verge of failure or a safety accident.

[0130] In this embodiment, the suspected smoke area is accurately located from the target image sequence by using preset smoke texture features, which solves the problem of misjudgment caused by changes in lighting or equipment vibration in traditional solutions, improves the accuracy of the initial screening, and makes the final judgment by verifying the consistency of the diffusion direction and the continuity of the diffusion process. This solves the problem of false alarm caused by instantaneous local airflow disturbance or isolated interference objects, and achieves highly reliable confirmation of real smoke anomalies.

[0131] In one feasible implementation, in step S02, the visual features further include morphological change features, and the visual anomalies further include open flame anomalies. The step of identifying visual anomalies in the target image sequence based on visual features includes steps B11 to B13:

[0132] Step B11: Extract a third candidate image region from the target image sequence that conforms to the preset open flame spectral distribution based on color features;

[0133] It should be noted that the preset open flame spectral distribution is a predefined range of color parameters used to identify open flames, and is used to initially filter pixels in the target image sequence that may belong to flames. The third candidate image region is the set of connected regions in the target image sequence that conform to the preset open flame spectral distribution after color feature filtering.

[0134] Step B12: Determine the flicker frequency and contour shape change rate in the third candidate image region based on morphological change features.

[0135] It should be noted that morphological change characteristics refer to the dynamic attributes of the overall shape and appearance of the third candidate image region in the target image sequence as it evolves over time. This includes the periodic fluctuations in pixel brightness within the third candidate image region (i.e., flicker frequency) and the degree of fluctuation in the geometry of the outer contour of the third candidate image region (i.e., contour morphological change rate). For example, a region with high morphological change characteristics will exhibit rapid and irregular fluctuations in brightness and contour. Flicker frequency refers to the rate at which the average brightness value of the third candidate image region undergoes significant periodic changes in the target image sequence. The flicker frequency of a real flame is typically within a certain range (e.g., 1-10 Hz), while stable light sources or irregular noise do not possess this characteristic or their frequency characteristics do not match. The contour morphological change rate is an indicator used to quantify the degree of change in the outer contour shape of the third candidate image region in the target image sequence. The statistical characteristics (mean and variance) of this change rate reflect the average intensity and instability of the contour fluctuations. The contour morphological change rate of a real flame typically has a high mean (continuous and drastic changes) and a high variance (unstable amplitude of change).

[0136] For example, based on morphological change characteristics, a temporal sequence analysis is performed on the third candidate image region to quantify its dynamic characteristics. Specifically, the flicker frequency is extracted by calculating the periodic fluctuation of the average brightness of the third candidate image region over time; simultaneously, the contour morphological change rate is calculated by analyzing the matching degree (such as Hausdorff distance) or shape moment difference of the contour point set in the third candidate image region in consecutive frames of the target image sequence.

[0137] Step B13: When the flickering frequency reaches a preset frequency threshold, the mean of the contour shape change rate reaches a preset mean, and the variance of the contour shape change rate reaches a preset variance, it is determined that a visual abnormality has been identified in the target image, and the visual abnormality is an open flame abnormality.

[0138] It should be noted that open flame anomalies refer to open combustion flames appearing inside the wind turbine nacelle. Identification is based not only on specific color spectra, but more importantly on their unique and unstable combustion dynamics. The preset mean is a standard value for judging the average intensity of contour changes. When the mean of the contour shape change rate is higher than this value, it indicates that the contour of the candidate region is continuously in a state of severe deformation, consistent with the jumping characteristics of a flame. The preset variance is a standard value for judging the instability of contour changes. When the variance of the contour shape change rate is higher than this value, it indicates that the amplitude of contour changes fluctuates greatly, exhibiting non-periodic and irregular characteristics, which is significantly different from the stable contour changes produced by mechanical motion or regular reflection.

[0139] In this embodiment, candidate regions that conform to the spectral distribution of open flames are extracted based on color features, which solves the technical problem that traditional sensors cannot perceive the spectral characteristics of flames through visual information. This enables rapid initial screening of potential open flame regions. By determining the flicker frequency and contour morphology change rate through morphological change features, the problem of difficulty in quantifying and analyzing the dynamic behavior of flames is solved, and the unique flickering and jumping patterns of flames are accurately characterized. By comprehensively judging whether the statistical features of flicker frequency and contour morphology change rate reach a preset threshold, the problem of false alarms caused by interference from high-temperature light sources, reflections, etc., in single feature judgment is solved, achieving a high-reliability technical effect for identifying open flame anomalies.

[0140] It should be noted that the above examples are only for understanding this application and do not constitute a limitation on the wind turbine anomaly monitoring method based on image recognition in this application. Any simple modifications based on this technical concept are within the protection scope of this application.

[0141] This application also provides an image recognition-based wind turbine anomaly monitoring device, applied to the nacelle of a wind turbine unit. Please refer to [reference needed]. Figure 4 The image recognition-based wind turbine anomaly monitoring device includes:

[0142] Image acquisition module 10 is used to acquire raw images of the interior of the wind turbine nacelle in real time, eliminate environmental interference factors in the raw images, and obtain the target image;

[0143] The anomaly detection module 20 is used to extract visual features from the target image and identify visual anomalies in the target image based on the visual features.

[0144] The alarm output module 30 is used to output alarm information containing visual abnormalities when visual abnormalities are detected in the target image.

[0145] The wind turbine anomaly monitoring device based on image recognition provided in this application, employing the wind turbine anomaly monitoring method based on image recognition in the above embodiments, can solve the technical problem of poor monitoring effect in current wind turbine monitoring schemes. Compared with the prior art, the beneficial effects of the wind turbine anomaly monitoring device based on image recognition provided in this application are the same as the beneficial effects of the wind turbine anomaly monitoring method based on image recognition provided in the above embodiments, and other technical features in the wind turbine anomaly monitoring device based on image recognition are the same as the features disclosed in the methods of the above embodiments, and will not be repeated here.

[0146] This application provides an electronic device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the image recognition-based wind turbine anomaly monitoring method described in Embodiment 1 above.

[0147] The following is for reference. Figure 5 The diagram illustrates a structural schematic of an electronic device suitable for implementing embodiments of this application. The electronic devices in these embodiments may include, but are not limited to, mobile terminals such as mobile phones, laptops, and PADs (Portable Application Description: Tablet computers), as well as fixed terminals such as digital TVs and desktop computers. Figure 5 The electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0148] like Figure 5As shown, the electronic device may include a processing unit 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory 1002 or a program loaded from a storage device 1003 into a random access memory 1004. The random access memory 1004 also stores various programs and data required for the operation of the electronic device. The processing unit 1001, the read-only memory 1002, and the random access memory 1004 are interconnected via a bus 1005. An input / output interface 1006 is also connected to the bus. Typically, the following systems can be connected to the input / output interface 1006: input devices 1007 including, for example, touchscreens, touchpads, keyboards, mice, image sensors, microphones, accelerometers, gyroscopes, etc.; output devices 1008 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 1003 including, for example, magnetic tapes, hard disks, etc.; and communication devices 1009. The communication device 1009 allows the electronic device to communicate wirelessly or wiredly with other devices to exchange data. Although the diagrams show electronic devices with various systems, it should be understood that it is not required to implement or have all of the systems shown. More or fewer systems may be implemented alternatively.

[0149] Specifically, according to the embodiments disclosed in this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from storage device 1003, or installed from read-only memory 1002. When the computer program is executed by processing device 1001, it performs the functions defined in the methods of the embodiments disclosed in this application.

[0150] The electronic device provided in this application employs the image recognition-based wind turbine anomaly monitoring method described in the above embodiments, which can solve the technical problem of poor monitoring effect in current wind turbine monitoring schemes. Compared with the prior art, the beneficial effects of the electronic device provided in this application are the same as those of the image recognition-based wind turbine anomaly monitoring method provided in the above embodiments, and other technical features of this electronic device are the same as those disclosed in the previous embodiment method, and will not be repeated here.

[0151] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any suitable manner in one or more embodiments or examples.

[0152] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0153] This application provides a computer-readable storage medium having computer-readable program instructions (i.e., a computer program) stored thereon, which are used to execute the image recognition-based wind turbine anomaly monitoring method in the above embodiments.

[0154] The computer-readable storage medium provided in this application may be, for example, a USB flash drive, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems or devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination thereof.

[0155] The aforementioned computer-readable storage medium may be included in an electronic device or may exist independently without being assembled into an electronic device.

[0156] The aforementioned computer-readable storage medium carries one or more programs. When the aforementioned one or more programs are executed by an electronic device, the image recognition-based wind turbine anomaly monitoring device is applied to the wind turbine nacelle, enabling it to acquire in real time the original image sequence inside the wind turbine nacelle in the first candidate image region, eliminate environmental interference factors in the original image sequence of the first candidate image region, and obtain the target image sequence; extract visual features from the target image sequence of the first candidate image region, identify visual anomalies in the target image sequence of the first candidate image region based on the visual features of the first candidate image region; and output alarm information containing visual anomalies in the target image sequence of the first candidate image region when visual anomalies are identified in the target image sequence of the first candidate image region.

[0157] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a Local Area Network (LAN) or a Wide Area Network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0158] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0159] The modules described in the embodiments of this application can be implemented in software or hardware. The names of the modules do not necessarily limit the functionality of the unit itself.

[0160] The readable storage medium provided in this application is a computer-readable storage medium that stores computer-readable program instructions (i.e., a computer program) for executing the above-described image recognition-based wind turbine anomaly monitoring method, thereby solving the technical problem of poor monitoring effect in current wind turbine monitoring schemes. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in this application are the same as those of the image recognition-based wind turbine anomaly monitoring method provided in the above embodiments, and will not be repeated here.

[0161] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the image recognition-based wind turbine anomaly monitoring method described above.

[0162] The computer program product provided in this application can solve the technical problem of poor monitoring effect in current wind turbine monitoring schemes. Compared with the prior art, the beneficial effects of the computer program product provided in this application are the same as those of the wind turbine anomaly monitoring method based on image recognition provided in the above embodiments, and will not be repeated here.

[0163] The above description is only a part of the embodiments of this application and does not limit the patent scope of this application. All equivalent structural transformations made under the technical concept of this application and using the contents of the specification and drawings of this application, or direct / indirect applications in other related technical fields, are included in the patent protection scope of this application.

Claims

1. A method for monitoring wind turbine anomalies based on image recognition, characterized in that, Applied to wind turbine nacelles, the image recognition-based wind turbine anomaly monitoring method includes: The original image sequence inside the wind turbine nacelle is acquired in real time, and environmental interference factors in the original image sequence are eliminated to obtain the target image sequence; Visual features are extracted from the target image sequence, and visual anomalies in the target image sequence are identified based on the visual features; If a visual anomaly is detected in the target image sequence, an alarm message containing the visual anomaly is output. The step of eliminating environmental interference factors in the original image sequence to obtain the target image sequence includes: Identify environmental interference factors in the original image sequence; When the environmental interference factor is static stain interference, the occlusion area of ​​the static stain interference is located, and the occlusion area is restored based on a preset uncontaminated image to obtain a first intermediate image sequence. When the environmental interference factor is dynamic particle interference, the particle noise corresponding to the dynamic particle interference in the original image sequence is extracted and the particle noise is removed to obtain the second intermediate image sequence. Perform contrast enhancement and color space transformation operations on the first intermediate image sequence and / or the second intermediate image sequence to obtain the target image sequence; The step of extracting particle noise corresponding to dynamic particle interference in the original image sequence includes: For each spatial location in the original image sequence, a time-series statistical analysis is performed on the set of pixel values ​​at the spatial location to identify abnormal pixels that deviate from a preset normal value range in the set of pixel values, and the regions corresponding to all identified abnormal pixels are taken as candidate noise regions. Extract the morphological change information and spatial distribution information of the candidate noise region in the original image sequence, and calculate the dynamic behavior feature measure of the candidate noise region based on the morphological change information and the spatial distribution information; Based on the dynamic behavior feature metric, target noise that meets the preset dynamic particle interference determination metric is determined in the candidate noise region, and the target noise is used as the particle noise corresponding to the dynamic particle interference in the original image sequence.

2. The wind turbine anomaly monitoring method based on image recognition as described in claim 1, characterized in that, The step of restoring the occluded area based on a preset uncontaminated image to obtain a first intermediate image sequence includes: Differential calculations are performed on each image frame in the original image sequence and a preset uncontaminated image to determine the target position of the occluded region in the preset uncontaminated image; Pixel information of the target location is extracted from the preset uncontaminated image, and the pixel information is used to replace the occluded area to obtain the filled area; A smooth transition process is performed on the boundary between the filled region and the unoccluded region of the original image sequence to obtain a first intermediate image sequence.

3. The wind turbine anomaly monitoring method based on image recognition as described in claim 1, characterized in that, The visual features include at least color features, texture features, and shape features; the visual anomalies include lubricating oil leakage anomalies; and the step of identifying visual anomalies in the target image sequence based on the visual features includes: Based on the color features, a first candidate image region that conforms to the preset oil stain color range is extracted from the target image sequence; Calculate the first matching degree between the texture features of the first candidate image region and the preset oil stain texture features, and calculate the second matching degree between the shape features of the first candidate image region and the preset oil stain shape features; If both the first matching degree and the second matching degree reach a preset matching degree threshold, it is determined that a visual abnormality is identified in the target image sequence, and the visual abnormality is a lubricating oil leakage abnormality.

4. The wind turbine anomaly monitoring method based on image recognition as described in claim 3, characterized in that, The visual features also include motion pattern features, and the visual anomalies also include smoke anomalies. The step of identifying visual anomalies in the target image sequence based on the visual features further includes: Based on the texture features, a second candidate image region conforming to a preset smoke texture is identified in the target image sequence; Identify the diffusion behavior of the second candidate image region based on the motion pattern features; If the diffusion direction of the diffusion behavior remains unchanged and the diffusion process of the diffusion behavior is not interrupted, it is determined that a visual anomaly is identified in the target image, and the visual anomaly is a smoke anomaly.

5. The wind turbine anomaly monitoring method based on image recognition as described in claim 4, characterized in that, The visual features also include morphological change features, and the visual anomalies also include open flame anomalies. The step of identifying visual anomalies in the target image sequence based on the visual features further includes: Based on the color features, a third candidate image region that conforms to a preset open flame spectral distribution is extracted from the target image sequence; The flicker frequency and contour shape change rate in the third candidate image region are determined based on the morphological change characteristics. If the flickering frequency reaches a preset frequency threshold, the mean of the contour shape change rate reaches a preset mean, and the variance of the contour shape change rate reaches a preset variance, then a visual abnormality is identified in the target image, and the visual abnormality is an open flame abnormality.

6. A wind turbine anomaly monitoring device based on image recognition, characterized in that, The image recognition-based wind turbine anomaly monitoring device, applied to wind turbine nacelles, includes: The image acquisition module is used to acquire raw images of the interior of the wind turbine nacelle in real time, eliminate environmental interference factors in the raw images, and obtain the target image; Anomaly detection module is used to extract visual features from the target image and identify visual anomalies in the target image based on the visual features; An alarm output module is used to output alarm information containing the visual abnormality when a visual abnormality is detected in the target image. The step of eliminating environmental interference factors in the original image sequence to obtain the target image sequence includes: Identify environmental interference factors in the original image sequence; When the environmental interference factor is static stain interference, the occlusion area of ​​the static stain interference is located, and the occlusion area is restored based on a preset uncontaminated image to obtain a first intermediate image sequence. When the environmental interference factor is dynamic particle interference, the particle noise corresponding to the dynamic particle interference in the original image sequence is extracted and the particle noise is removed to obtain the second intermediate image sequence. Perform contrast enhancement and color space transformation operations on the first intermediate image sequence and / or the second intermediate image sequence to obtain the target image sequence; The step of extracting particle noise corresponding to dynamic particle interference in the original image sequence includes: For each spatial location in the original image sequence, a time-series statistical analysis is performed on the set of pixel values ​​at the spatial location to identify abnormal pixels that deviate from a preset normal value range in the set of pixel values, and the regions corresponding to all identified abnormal pixels are taken as candidate noise regions. Extract the morphological change information and spatial distribution information of the candidate noise region in the original image sequence, and calculate the dynamic behavior feature measure of the candidate noise region based on the morphological change information and the spatial distribution information; Based on the dynamic behavior feature metric, target noise that meets the preset dynamic particle interference determination metric is determined in the candidate noise region, and the target noise is used as the particle noise corresponding to the dynamic particle interference in the original image sequence.

7. An electronic device, characterized in that, The device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the image recognition-based wind turbine anomaly monitoring method as described in any one of claims 1 to 5.

8. A storage medium, characterized in that, The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, it implements the steps of the wind turbine anomaly monitoring method based on image recognition as described in any one of claims 1 to 5.

Images