Intelligent inspection monitoring system and method for wind power plant

Abstract

The embodiment of the invention provides an intelligent inspection monitoring system and method for a wind power plant. The method comprises the following steps: firstly, acquiring a blade inspection image of a to-be-detected fan acquired by a camera of an unmanned aerial vehicle, then, carrying out image preprocessing on the blade inspection image to obtain a clipped blade inspection image, then, carrying out multi-scale feature extraction on the clipped blade inspection image to obtain a multi-scale blade state feature map, and finally, carrying out multi-scale feature extraction on the clipped blade inspection image to obtain a multi-scale blade state feature map. And determining whether to generate a fan blade fault prompt based on the multi-scale blade state feature graph. Therefore, the fault prompt can be generated in time, and the inspection cost and risk are reduced.

Description

Technical Field

[0001] Embodiments of the present invention relate to the field of intelligent inspection, and more specifically, to an intelligent inspection and monitoring system and method for a wind farm. Background Art

[0002] my country's current offshore wind power development is primarily located along the coast. Operations and maintenance are significantly impacted by tides, including typhoons and other adverse conditions, as well as frequent strong winds, fog, and thunderstorms from the Jianghuai cyclone. These conditions also include large shallows and the significant influence of tides in the intertidal zone. This makes access difficult, and transportation equipment selection challenging. Consequently, offshore maintenance operations are limited in time, pose significant safety risks, and lack significant repair equipment.

[0003] Wind turbine blades, in particular, are exposed to harsh natural environments for long periods of time, subject to erosion by factors such as wind, rain, and ice, making them prone to failure. Timely and accurate detection and diagnosis of wind turbine blade faults and defects is crucial to ensuring the normal operation of wind turbines and extending their service life.

[0004] Traditional inspection methods require technicians to climb onto wind turbines for visual inspections, which not only poses certain safety risks but is also time-consuming, labor-intensive, and inefficient. Therefore, an optimized inspection and monitoring solution for wind farms is desired. Summary of the Invention

[0005] The present invention aims to solve at least one of the technical problems existing in the prior art, and provides an intelligent inspection and monitoring system for a wind farm and a method thereof.

[0006] In a first aspect, an embodiment of the present invention provides an intelligent inspection and monitoring method for a wind farm, comprising:

[0007] Obtaining blade inspection images of the wind turbine to be inspected captured by the camera of the drone;

[0008] performing image preprocessing on the blade inspection image to obtain a cropped blade inspection image;

[0009] performing multi-scale feature extraction on the cropped blade inspection image to obtain a multi-scale blade state feature map; and

[0010] Based on the multi-scale blade state characteristic diagram, it is determined whether to generate a wind turbine blade fault prompt.

[0011] In some possible embodiments, performing image preprocessing on the blade inspection image to obtain a cropped blade inspection image includes:

[0012] The blade inspection image is randomly cropped to obtain the cropped blade inspection image.

[0013] In some possible embodiments, performing multi-scale feature extraction on the cropped blade inspection image to obtain a multi-scale blade state feature map includes:

[0014] A deep learning network model is used to perform multi-scale feature extraction on the cropped blade inspection image to obtain the multi-scale blade state feature map.

[0015] In some possible embodiments, the deep learning network model is a leaf state feature extractor including a multi-scale convolutional structure.

[0016] In some possible embodiments, the leaf state feature extractor including a multi-scale convolution structure includes a first convolution layer, a second convolution layer, and a multi-scale feature fusion layer connected to the first convolution layer and the second convolution layer, wherein the first convolution layer and the second convolution layer respectively use two-dimensional convolution kernels with different scales.

[0017] In some possible embodiments, the performing multi-scale feature extraction on the cropped blade inspection image using a deep learning network model to obtain the multi-scale blade state feature map includes:

[0018] The cropped blade inspection image is passed through the blade state feature extractor including the multi-scale convolution structure to obtain the multi-scale blade state feature map.

[0019] In some possible embodiments, the step of passing the cropped blade inspection image through the blade state feature extractor including the multi-scale convolution structure to obtain the multi-scale blade state feature map includes:

[0020] Using the first convolution layer of the blade state feature extractor including the multi-scale convolution structure to perform two-dimensional convolution encoding on the cropped blade inspection image with a two-dimensional convolution kernel having a first scale to obtain a first-scale blade state feature map;

[0021] Using the second convolution layer of the blade state feature extractor including the multi-scale convolution structure to perform two-dimensional convolution encoding on the cropped blade inspection image with a two-dimensional convolution kernel having a second scale to obtain a second-scale blade state feature map; and

[0022] The first-scale leaf state feature map and the second-scale leaf state feature map are cascaded through the multi-scale feature fusion layer of the leaf state feature extractor including the multi-scale convolution structure to obtain the multi-scale leaf state feature map.

[0023] In some possible embodiments, determining whether to generate a wind turbine blade fault prompt based on the multi-scale blade state characteristic diagram includes:

[0024] performing feature distribution correction on the multi-scale blade state characteristic graph to obtain a corrected multi-scale blade state characteristic graph; and

[0025] The corrected multi-scale blade state feature map is passed through a multi-task classification head module to obtain a classification result, and the classification result is used to indicate whether a wind turbine blade fault prompt is generated.

[0026] In some possible embodiments, the step of passing the corrected multi-scale blade state feature map through a multi-task classification head module to obtain a classification result, wherein the classification result is used to indicate whether a wind turbine blade fault prompt is generated, includes:

[0027] Passing the multi-scale blade state feature map through a fine-grained classifier to obtain multiple blade defect category probability values;

[0028] Passing the multi-scale leaf state characteristic graph through a coarse-grained classifier to obtain a first probability value and a second probability value;

[0029] fusing the plurality of blade defect category probability values, the first probability value, and the second probability value in a probability-weighted fusion manner to obtain a comprehensive expression probability value; and

[0030] The classification result is obtained based on the comprehensive expression probability value.

[0031] In a second aspect, an embodiment of the present invention provides an intelligent inspection and monitoring system for a wind farm, comprising:

[0032] An image acquisition module is used to obtain inspection images of the blades of the wind turbine to be inspected, which are captured by the camera of the UAV;

[0033] An image preprocessing module, configured to perform image preprocessing on the blade inspection image to obtain a cropped blade inspection image;

[0034] a multi-scale feature extraction module, configured to perform multi-scale feature extraction on the cropped blade inspection image to obtain a multi-scale blade state feature map; and

[0035] A fault prompt judgment module is used to determine whether to generate a wind turbine blade fault prompt based on the multi-scale blade state characteristic diagram.

[0036] Compared to existing technologies, the intelligent inspection and monitoring system and method for wind farms provided by embodiments of the present invention first acquires blade inspection images of the wind turbine to be inspected, captured by a drone's camera. Next, these images are preprocessed to produce cropped blade inspection images. Multi-scale feature extraction is then performed on these cropped images to produce a multi-scale blade status feature map. Finally, based on this multi-scale blade status feature map, a determination is made as to whether to generate a wind turbine blade fault alert. This allows for timely generation of fault alerts, reducing inspection costs and risks. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the specific embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0038] Figure 1 4 is a flow chart of an intelligent inspection and monitoring method for a wind farm according to an embodiment of the present invention.

[0039] Figure 2 FIG. 4 is a schematic diagram of the architecture of an intelligent inspection and monitoring method for a wind farm according to an embodiment of the present invention.

[0040] Figure 3 4 is a flowchart of sub-step S140 of the intelligent inspection and monitoring method for a wind farm according to an embodiment of the present invention.

[0041] Figure 4 4 is a flowchart of sub-step S142 of the intelligent inspection and monitoring method for a wind farm according to an embodiment of the present invention.

[0042] Figure 5 FIG. 4 is a block diagram of an intelligent inspection and monitoring system for a wind farm according to an embodiment of the present invention.

[0043] Figure 6 2 is a diagram showing an application scenario of an intelligent inspection and monitoring method for a wind farm according to an embodiment of the present invention. DETAILED DESCRIPTION

[0044] To enable those skilled in the art to better understand the technical solutions of the present invention, the present invention is further described below in conjunction with the accompanying drawings and specific embodiments. It is apparent that the described embodiments are only a portion of the embodiments of the present invention, rather than all of them. Based on the described embodiments of the present invention, all other embodiments obtained by those skilled in the art without requiring creative effort are within the scope of protection of the present invention.

[0045] Unless otherwise specified, the technical terms or scientific terms used in the embodiments of the present invention should be understood by people with ordinary skills in the field to which the present invention belongs. The terms "including" or "comprising" used in the embodiments of the present invention neither limit the shapes, numbers, steps, actions, operations, components, originals and / or their groups mentioned, nor exclude the appearance or addition of one or more other different shapes, numbers, steps, actions, operations, components, originals and / or their groups, or the addition of these. In addition, the terms "first" and "second" are only used for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly indicating the number and order of the indicated technical features. Thus, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the embodiments of the present invention, the meaning of "multiple" is two or more, unless otherwise clearly and specifically defined.

[0046] Unless otherwise specifically stated, the relative arrangements of the components and steps, numerical expressions, and numerical values set forth in these embodiments do not limit the scope of the present invention. At the same time, it should be understood that, for ease of description, the dimensions of the various parts shown in the drawings are not drawn in accordance with actual proportional relationships, and that the techniques, methods, and devices known to those of ordinary skill in the relevant art may not be discussed in detail, but where appropriate, the techniques, methods, and devices shown should be considered part of the authorized specification. In all examples shown and discussed herein, any specific other examples may have different values. It should be noted that similar symbols and letters represent similar items in the following figures, and therefore, once an item is defined in one figure, it does not need to be further discussed in subsequent figures.

[0047] In the description of the embodiments of the present invention, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In the embodiments of the present invention, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine different embodiments or examples described in the embodiments of the present invention and the features of different embodiments or examples, unless they are mutually inconsistent.

[0048] Flowcharts are used in this disclosure to illustrate the operations performed by systems according to embodiments of the present invention. It should be understood that the preceding or following operations do not necessarily need to be performed in exact order. Instead, various steps may be processed in reverse order or simultaneously, as needed. Furthermore, other operations may be added to these processes, or one or more operations may be removed from these processes.

[0049] Below, the exemplary embodiments according to the present invention will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments of the present invention, and it should be understood that the present invention is not limited to the exemplary embodiments described herein.

[0050] In response to the above technical problems, the technical concept of the present invention is to collect image data of wind blades through drones, and combine machine vision technology and artificial intelligence technology based on deep learning to analyze and process the image data of wind blades, so as to realize intelligent judgment of whether there is a fault in the wind blades and generate fault prompts in time, reducing inspection costs and risks.

[0051] Based on this, Figure 1 4 is a flow chart of an intelligent inspection and monitoring method for a wind farm according to an embodiment of the present invention. Figure 2 FIG. 1 is a schematic diagram of the architecture of an intelligent inspection and monitoring method for a wind farm according to an embodiment of the present invention. Figure 1 and Figure 2 As shown, the intelligent inspection and monitoring method for a wind farm according to an embodiment of the present invention includes the following steps: S110, obtaining a blade inspection image of the wind turbine to be inspected captured by a camera of a drone; S120, performing image preprocessing on the blade inspection image to obtain a cropped blade inspection image; S130, performing multi-scale feature extraction on the cropped blade inspection image to obtain a multi-scale blade state feature map; and, S140, determining whether to generate a wind turbine blade fault prompt based on the multi-scale blade state feature map.

[0052] It should be understood that the purpose of step S110 is to use a drone's camera to capture images of the wind turbine blades to be inspected. Drones can be equipped with cameras or other sensors, enabling them to capture images from high altitudes over wind farms. Compared to traditional ground-based inspection methods, drones offer a more comprehensive and high-resolution perspective, allowing for clearer visibility of blade details. Drones can fly quickly and cover large wind farm areas. Traditional inspection methods, which may require manual labor or the use of equipment such as hanging baskets, are time-consuming and inefficient. However, drones can complete an entire wind farm inspection in a shorter timeframe, significantly improving inspection efficiency. Because drones can replace human personnel in inspecting wind turbine blades, they reduce the risks and safety hazards associated with high-altitude environments. Drone operations can be performed from the ground, minimizing risk exposure. Drones can be equipped with sensors and cameras, enabling them to capture high-quality images and other relevant data. This data can be used for subsequent image processing, feature extraction, and fault diagnosis. Drones can also transmit data to ground-based sites in real time via wireless communication, enabling timely data processing and analysis. In step S120, the blade inspection image undergoes image preprocessing, such as denoising, contrast enhancement, and brightness adjustment, to improve the accuracy and effectiveness of subsequent processing. After preprocessing, the image can be cropped to retain only the blade portion, reducing the computational effort required for subsequent processing. In step S130, multi-scale feature extraction is performed on the cropped blade inspection image. Multi-scale feature extraction can include using filters of different scales to extract features such as texture, shape, and color to capture the various details and characteristics of the blade. By extracting multi-scale features, a more comprehensive description of the blade's condition can be achieved. In step S140, fault detection and diagnosis are performed based on the multi-scale blade state feature map. By analyzing and comparing the feature map, it is possible to determine whether a blade fault or abnormality exists. If a fault or abnormality is detected, the system can generate a corresponding wind turbine blade fault alert, allowing timely repair measures to ensure the proper operation of the wind turbine. These steps, combined, constitute an intelligent inspection and monitoring method for wind farms. Through drone image acquisition, image preprocessing, feature extraction, and fault detection, wind turbine blade condition monitoring and fault diagnosis are achieved.

[0053] Specifically, in the technical solution of the present invention, a blade inspection image of the wind turbine to be inspected, captured by a drone camera, is first obtained; then, the blade inspection image is randomly cropped to obtain a cropped blade inspection image. Here, in an embodiment of the present invention, the random cropping of the blade inspection image is performed based on the blade region. During the cropping process, if the blade region occupies more than 80% of the cropped area, the region is retained. In this way, the influence of the background can be removed to a certain extent, allowing the model to focus on learning information from the blade region.

[0054] Accordingly, in step S120, the blade inspection image is preprocessed to obtain a cropped blade inspection image, including randomly cropping the blade inspection image to obtain the cropped blade inspection image. It should be understood that using random cropping in the blade inspection image preprocessing step can achieve the following benefits: 1. Reducing computational complexity: Blade inspection images may have high resolution and size, which may result in excessive computational complexity for subsequent processing. Random cropping can reduce the image size to an appropriate size, thereby reducing the computational complexity of subsequent processing and improving processing efficiency. 2. Removing redundant information: Blade inspection images may contain redundant areas unrelated to the blade status, such as the surrounding sky or support. Random cropping can select areas of interest and remove this redundant information, allowing subsequent processing to focus more on key areas of the blade and improving processing accuracy. 3. Enhancing robustness: Random cropping can introduce a certain degree of randomness, making the processing robust to blade inspection images of varying positions and sizes. This allows for better adaptation to the size and position variations of different wind turbine blades and improves the algorithm's generalization capability. 4. Data Augmentation: Random cropping can generate multiple differently cropped leaf inspection images, thereby expanding the dataset. This is particularly useful for training deep learning models, increasing their generalization and robustness, and improving their ability to recognize different leaf states. Overall, random cropping reduces computational complexity, removes redundant information, enhances robustness, and enhances data augmentation in the image preprocessing of leaf inspection images, helping to improve the efficiency and accuracy of subsequent processing.

[0055] The cropped blade inspection image is then passed through a blade state feature extractor comprising a multi-scale convolutional structure to obtain a multi-scale blade state feature map. The purpose of performing multi-scale convolutional encoding on the cropped blade inspection image through the blade state feature extractor comprising a multi-scale convolutional structure is to extract features at different levels and scales in the cropped blade inspection image, thereby enhancing the model's expressive power and generalization capabilities. Multi-scale convolutional encoding can capture local details and global structures in the blade inspection image, as well as faults and defects of varying sizes and shapes, enabling the model to effectively classify blade states of varying types and degrees.

[0056] Specifically, there are multi-scale features in the leaf inspection images. For example, the texture features of the leaf surface, such as cracks, scratches, stains, etc., are usually small and unevenly distributed, and usually require a convolution kernel with a small receptive field to extract; the geometric features of the leaf shape, such as bending, deformation, fracture, etc., are usually large and have a certain regularity, and usually require a convolution kernel with a large receptive field to extract.

[0057] Accordingly, in step S130, multi-scale feature extraction is performed on the cropped blade inspection image to obtain a multi-scale blade state feature map, including: utilizing a deep learning network model to perform multi-scale feature extraction on the cropped blade inspection image to obtain the multi-scale blade state feature map. It should be understood that multi-scale feature extraction has the following uses in blade inspection image processing: 1. Extracting rich feature information: Through multi-scale feature extraction, rich feature information can be obtained from images of different scales. Features at different scales can capture different levels of detail and structure, thereby more comprehensively describing the blade state. This helps to improve the accuracy of blade state identification and fault detection. 2. Improving scale invariance: The size and shape of wind turbine blades may vary due to changes in distance and angle. Through multi-scale feature extraction, the algorithm can achieve a certain degree of scale invariance for blades of different scales, that is, it can effectively identify and analyze blade states at different scales. 3. Enhancing robustness: Multi-scale feature extraction can increase the robustness of the algorithm to noise, occlusion, and other interference factors. By extracting features at different scales, the impact of these interference factors can be reduced to a certain extent, improving the stability and accuracy of the algorithm. 4. Information fusion: Multi-scale feature extraction can fuse features at different scales to obtain a more global and comprehensive blade status feature map. This helps to improve the understanding and analysis of the overall state of the blade and provide more accurate information for subsequent fault diagnosis and prediction. In summary, multi-scale feature extraction can extract rich feature information, improve scale invariance, enhance robustness, and achieve information fusion in blade inspection image processing, which helps to improve the accuracy and reliability of blade status recognition and fault detection.

[0058] The deep learning network model is a leaf state feature extractor with a multi-scale convolutional structure. The leaf state feature extractor with a multi-scale convolutional structure includes a first convolutional layer, a second convolutional layer, and a multi-scale feature fusion layer connected to the first and second convolutional layers, wherein the first and second convolutional layers respectively use two-dimensional convolution kernels of different scales.

[0059] Specifically, a deep learning network model is used to perform multi-scale feature extraction on the cropped blade inspection image to obtain the multi-scale blade state feature map, including: passing the cropped blade inspection image through a blade state feature extractor comprising a multi-scale convolutional structure to obtain the multi-scale blade state feature map. It should be understood that the application of the blade state feature extractor comprising a multi-scale convolutional structure in the deep learning network model has the following uses: 1. Extracting multi-scale features: By using two-dimensional convolutional kernels of different scales, the first and second convolutional layers can capture features at different scales. This allows feature representations with different receptive fields to be obtained at different layers in the network, thereby achieving multi-scale feature extraction. This helps better understand the details and structure of the blade and improves the accuracy of blade state identification and analysis. 2. Multi-scale feature fusion: The multi-scale feature fusion layer is used to fuse features from different scales. By fusing features from different scales, a more global and comprehensive blade state feature map can be obtained. This helps improve the understanding and analysis of the overall blade state, providing more accurate information for subsequent fault diagnosis and prediction. 3. Enhance the model's expressiveness: The multi-scale convolutional structure can enhance the model's expressiveness, enabling it to better capture important features in blade images. By using multi-scale convolutional kernels, the network can learn richer and more diverse feature representations, improving the model's ability to discern blade status. 4. Improve the model's robustness: Multi-scale feature extraction can increase the model's robustness to blade images of varying scales and positions. Blades may exhibit varying scales and angles during actual inspections. Through multi-scale feature extraction, the model can better adapt to these variations and improve its ability to discern blade status. In summary, using a blade status feature extractor containing a multi-scale convolutional structure to perform multi-scale feature extraction on cropped blade inspection images can improve the accuracy of blade status recognition and analysis, and enhance the model's expressiveness and robustness. This has important application value for blade inspection and monitoring in wind farms.

[0060] More specifically, in one example, the cropped blade inspection image is passed through the blade state feature extractor including a multi-scale convolution structure to obtain the multi-scale blade state feature map, including: using the first convolution layer of the blade state feature extractor including a multi-scale convolution structure to perform two-dimensional convolution encoding on the cropped blade inspection image with a two-dimensional convolution kernel having a first scale to obtain a first-scale blade state feature map; using the second convolution layer of the blade state feature extractor including a multi-scale convolution structure to perform two-dimensional convolution encoding on the cropped blade inspection image with a two-dimensional convolution kernel having a second scale to obtain a second-scale blade state feature map; and, cascading the first-scale blade state feature map and the second-scale blade state feature map through the multi-scale feature fusion layer of the blade state feature extractor including a multi-scale convolution structure to obtain the multi-scale blade state feature map.

[0061] Next, the multi-scale blade state feature map is passed through a multi-task classification head module to obtain a classification result, and the classification result is used to indicate whether a wind turbine blade fault prompt is generated. Here, the multi-task classification head module can perform multiple classification tasks simultaneously. Specifically, in the technical solution of the present invention, the multi-task classification head module can use a fine-grained classifier to determine the type of defects existing in the blade, and at the same time use a coarse-grained classifier to determine whether the blade has defects, thereby improving the description ability and detailed expression of the blade state. The blade fault prompt information generated based on this judgment can provide timely and effective reference information for operation and maintenance personnel.

[0062] Accordingly, if Figure 3As shown, in step S140, based on the multi-scale blade state feature map, determining whether to generate a wind turbine blade fault prompt includes: S141, performing feature distribution correction on the multi-scale blade state feature map to obtain a corrected multi-scale blade state feature map; and S142, passing the corrected multi-scale blade state feature map through a multi-task classification head module to obtain a classification result, which is used to indicate whether a wind turbine blade fault prompt is generated. It should be understood that the purpose of step S141 is to further optimize the feature representation by correcting the feature distribution of the multi-scale blade state feature map. Feature distribution correction can be achieved through statistical methods or normalization techniques, such as mean-variance normalization or maximum-minimum normalization. The corrected feature map can reduce redundancy between features, improve feature discrimination and identification, and thus better represent blade state information. Step S142, using the multi-task classification head module, classifies the corrected multi-scale blade state feature map to determine whether to generate a wind turbine blade fault prompt. The multi-task classification head module is typically a neural network module with multiple outputs, each corresponding to a classification task. In this case, the classification task is to determine whether there is a fault on the blade. By training the model, the association between blade status features and fault prompts can be learned, thereby generating corresponding classification results based on the multi-scale blade status feature map. In summary, step S141 optimizes the representation of the multi-scale blade status feature map through feature distribution correction, and step S142 classifies the corrected feature map through the multi-task classification head module, thereby realizing the generation of wind turbine blade fault prompts. These two steps work together to improve the model's understanding of blade status and fault identification capabilities, providing effective support for blade inspection and monitoring in wind farms.

[0063] It's worth noting that the multi-task classification head module is a structure in deep learning models that simultaneously handles multiple related but distinct classification tasks. This module is typically one or more fully connected or convolutional layers added on top of a backbone network (such as a convolutional neural network). The multi-task classification head module transforms the feature maps extracted by the backbone network into distinct classification results. Each classification task has a separate output representing the classification result for that task. Within this module, each task has its own classifier, which can be a fully connected layer, a convolutional layer, or other task-specific structure. The output layer of these classifiers typically uses an appropriate activation function (such as softmax) to produce the classification result. The design of the multi-task classification head module can be flexibly adjusted to meet the needs of different tasks. For example, for the wind turbine blade fault detection task, the multi-task classification head module can include a binary classifier to determine whether the blade is faulty. If there are other related tasks, such as blade icing detection or blade damage type classification, additional classifiers can be added to the multi-task classification head module. By using the multi-task classification head module, deep learning models can simultaneously handle multiple related tasks, share the computational overhead of feature extraction, and leverage the correlations between tasks for joint learning. This can improve the efficiency and generalization ability of the model while reducing the number of parameters and training time. The multi-task classification head module has been widely used in many computer vision and natural language processing tasks, and can effectively handle the feature representation and classification problems of multiple related tasks.

[0064] Here, the cropped blade inspection image is passed through a blade state feature extractor including a multi-scale convolution structure to obtain a multi-scale blade state feature map, and each feature matrix of the multi-scale blade state feature map is used to express the image local neighborhood semantic association features of the cropped blade inspection image at different scales, and the feature matrices follow the channel distribution of the blade state feature extractor including the multi-scale convolution structure. However, considering that the scale difference of the two-dimensional convolution kernels with different scales will bring significant distribution differences to the feature representation of each feature matrix, the multi-scale blade state feature map will still have a sparse local feature distribution sparsification, that is, a sparse sub-manifold outside the distribution relative to the overall high-dimensional feature manifold, so that when the multi-scale blade state feature map is subjected to class probability regression mapping through a multi-task classification head module, the convergence of the multi-scale blade state feature map to the predetermined class probability category representation in the probability space is poor, affecting the accuracy of the classification result.

[0065] Therefore, preferably, when the multi-scale leaf state feature map is classified by a multi-task classification head module, the multi-scale leaf state feature vector obtained by expanding the multi-scale leaf state feature map is optimized for its position-by-position feature value.

[0066] Accordingly, in one example, performing feature distribution correction on the multi-scale blade state characteristic graph to obtain a corrected multi-scale blade state characteristic graph includes: performing feature distribution correction on the multi-scale blade state characteristic graph using the following optimization formula to obtain the corrected multi-scale blade state characteristic graph; wherein the optimization formula is:

[0067]

[0068] Where V is the multi-scale leaf state feature vector obtained by expanding the multi-scale leaf state feature map, v i is the eigenvalue of the i-th position of the multi-scale blade state eigenvector V, exp(·) represents the exponential operation of the value, and the exponential operation of the value represents the calculation of the natural exponential function value with the value as the power, v i ′ is the modified multi-scale leaf state feature vector obtained by expanding the modified multi-scale leaf state feature map.

[0069] That is, the sparse distribution in the high-dimensional feature space is processed by regularization based on heavy probability to activate the natural distribution transfer of the geometric manifold of the multi-scale blade state feature vector V in the high-dimensional feature space to the probability space, thereby improving the category convergence of the complex high-dimensional feature manifold with high spatial sparsity under the predetermined class probability by performing heavy-probability-based smoothing regularization on the distribution sparse sub-manifold of the high-dimensional feature manifold of the multi-scale blade state feature vector V, thereby improving the accuracy of the classification result obtained by the multi-task classification head module of the multi-scale blade state feature vector V.

[0070] Furthermore, in a specific example of the present invention, Figure 4 As shown, in step S142, the corrected multi-scale blade state feature map is passed through a multi-task classification head module to obtain a classification result, and the classification result is used to indicate whether an encoding process for generating a wind turbine blade fault prompt is generated, including: S1421, passing the multi-scale blade state feature map through a fine-grained classifier to obtain multiple blade defect category probability values; S1422, passing the multi-scale blade state feature map through a coarse-grained classifier to obtain a first probability value and a second probability value; S1423, fusing the multiple blade defect category probability values, the first probability value and the second probability value in a probability-weighted fusion manner to obtain a comprehensive expression probability value; and, S1424, obtaining the classification result based on the comprehensive expression probability value.

[0071] It should be understood that in step S1421, the fine-grained classifier is a classifier for specific blade defect categories. It takes the modified multi-scale blade state feature map as input and outputs a probability value for each blade defect category. Through this step, classification results for different blade defect categories can be obtained, which are used to indicate the likelihood of each defect existing in the blade. In step S1422, the coarse-grained classifier is a classifier for processing the overall blade state. It takes the modified multi-scale blade state feature map as input and outputs a first probability value and a second probability value representing the overall blade state. These probability values can be used to determine whether the blade is faulty or abnormal. In step S1423, multiple probability values are weighted and fused to obtain a comprehensive expression probability value. Weighted fusion can use different weight allocation strategies, such as dynamic adjustment based on the performance of different classifiers or fixed weights. The comprehensive expression probability value comprehensively considers the blade defect category, the overall blade state, and the relationship between them, providing a comprehensive assessment of the blade state. In step S1424, based on the comprehensive expression probability value, a threshold determination or other decision rules can be performed to obtain the final classification result. This classification result can indicate whether a wind turbine blade fault prompt is generated, or specifically indicate the type and severity of the blade fault. In summary, step S1421 obtains the probability value of the blade defect category through a fine-grained classifier, step S1422 obtains the probability value of the overall blade state through a coarse-grained classifier, step S1423 obtains the comprehensive expression probability value through probability weighted fusion, and step S1424 obtains the final classification result based on the comprehensive expression probability value. These steps together constitute the encoding process of converting the corrected multi-scale blade state feature map into a classification result, which is used to represent the generation of a wind turbine blade fault prompt.

[0072] It is worth mentioning that the blade defect categories of the fine-grained classifier are blade gel coat shedding, damage, transverse cracks and oil stains, and the probability values of multiple blade defect categories respectively represent the probability values of the occurrence of the above-mentioned blade defects; the first probability value of the coarse-grained classifier represents the probability of the occurrence of defects, and the second probability value represents the probability of no defects; obtaining the classification result based on the comprehensive expression probability value means determining the classification result based on the comprehensive expression probability value and the threshold obtained by training.

[0073] In summary, an intelligent inspection and monitoring method for wind farms based on an embodiment of the present invention is explained, which can combine machine vision technology and deep learning-based artificial intelligence technology to analyze and process the image data of wind turbine blades, thereby realizing intelligent judgment on whether there is a fault in the wind turbine blades, and generating fault prompts in a timely manner, thereby reducing inspection costs and risks.

[0074] Figure 5 FIG is a block diagram of an intelligent inspection and monitoring system 100 for a wind farm according to an embodiment of the present invention. Figure 5 As shown, the intelligent inspection and monitoring system 100 for a wind farm according to an embodiment of the present invention includes: an image acquisition module 110, used to obtain a blade inspection image of the wind turbine to be inspected acquired by a camera of a drone; an image preprocessing module 120, used to perform image preprocessing on the blade inspection image to obtain a cropped blade inspection image; a multi-scale feature extraction module 130, used to perform multi-scale feature extraction on the cropped blade inspection image to obtain a multi-scale blade state feature map; and a fault prompt judgment module 140, used to determine whether to generate a wind turbine blade fault prompt based on the multi-scale blade state feature map.

[0075] In one example, in the above-mentioned intelligent inspection and monitoring system 100 for a wind farm, the image preprocessing module 120 is configured to randomly crop the blade inspection image to obtain the cropped blade inspection image.

[0076] Here, those skilled in the art will appreciate that the specific functions and operations of the various modules in the intelligent inspection and monitoring system 100 for wind farms have been described in detail above. Figures 1 to 4 The invention has been described in detail in the description of the intelligent inspection and monitoring method for a wind farm, and therefore, its repeated description will be omitted.

[0077] As described above, the intelligent inspection and monitoring system 100 for wind farms according to an embodiment of the present invention can be implemented in various wireless terminals, such as a server equipped with an intelligent inspection and monitoring algorithm for wind farms. In one example, the intelligent inspection and monitoring system 100 for wind farms according to an embodiment of the present invention can be integrated into a wireless terminal as a software module and / or a hardware module. For example, the intelligent inspection and monitoring system 100 for wind farms can be a software module within the operating system of the wireless terminal, or an application developed specifically for the wireless terminal. Of course, the intelligent inspection and monitoring system 100 for wind farms can also be one of the many hardware modules of the wireless terminal.

[0078] Alternatively, in another example, the intelligent patrol monitoring system 100 for wind farms and the wireless terminal may also be separate devices, and the intelligent patrol monitoring system 100 for wind farms may be connected to the wireless terminal via a wired and / or wireless network and transmit interactive information in accordance with an agreed data format.

[0079] Figure 6 FIG is an application scenario diagram of the intelligent inspection and monitoring method for a wind farm according to an embodiment of the present invention. Figure 6 As shown, in this application scenario, first, obtain the data generated by the drone (for example, Figure 6 The inspection image of the blades of the wind turbine to be inspected (e.g., Figure 6 Then, the blade inspection image is input to a server (eg, Figure 6 In S) shown in , the server is capable of using the intelligent inspection and monitoring algorithm for wind farms to process the blade inspection image to obtain a classification result indicating whether a wind turbine blade fault prompt is generated.

[0080] It's worth mentioning that wind turbine blades generally consist of the following main components: 1. Blade Frame: The blade frame is the main structure of the blade, typically made of composite materials (such as glass fiber reinforced polyester resin) or metal (such as aluminum alloy). It provides strength and rigidity to the blade and supports other blade components. 2. Blade Surface: The blade surface is the outer covering of the blade, typically made of composite materials or glass fiber reinforced polyester resin. Its smooth appearance helps reduce air resistance and improve wind energy conversion efficiency. 3. Blade Root: The blade root connects the blade to the wind turbine main shaft. It typically uses a special design and connection method to ensure blade stability and safety and transmit the power generated by the blade to the wind turbine main shaft. 4. Blade Airfoil: The blade airfoil is the cross-sectional shape of the blade, which determines the blade's aerodynamic performance in the wind. Common blade airfoils include the NACA series airfoils and symmetrical airfoils, which have different lift and drag characteristics to suit different wind energy conversion requirements. 5. Blade Tip: The blade tip is the end of the blade. Because centrifugal forces are generated during blade operation, blade tips are typically designed to be elongated to reduce the impact of centrifugal forces on the blades and reduce noise and vibration. Overall, the structure of a wind turbine blade is designed to provide sufficient strength and rigidity while minimizing air resistance for efficient wind energy conversion. Factors such as blade shape, material, and connection method vary depending on the specific wind turbine design and application requirements.

[0081] According to another aspect of the present invention, a non-volatile computer-readable storage medium is provided, on which computer-readable instructions are stored. When the instructions are executed by a computer, the above-mentioned method can be executed.

[0082] The program portion of the technology can be considered a "product" or "article of manufacture" in the form of executable code and / or related data, implemented or implemented through computer-readable media. Tangible, permanent storage media can include any memory or storage used by a computer, processor, or similar device or related module. For example, various semiconductor memories, tape drives, disk drives, or any similar device that can provide storage for software.

[0083] All or part of the software may sometimes be communicated over a network, such as the Internet or other communication network. Such communication can load the software from one computer device or processor to another. For example: loading from a server or host computer of a video target detection device to a hardware platform of a computer environment, or other computer environment that implements the system, or a system with similar functions related to providing information required for target detection. Therefore, another medium capable of transmitting software elements can also be used as a physical connection between local devices, such as light waves, radio waves, electromagnetic waves, etc., which are transmitted through cables, optical cables or air. Physical media used to carry carriers, such as cables, wireless connections or optical cables and the like, can also be considered as media that carry software. As used herein, unless limited to tangible "storage" media, other terms referring to computer or machine "readable media" refer to media that participate in the process of executing any instructions by the processor.

[0084] In addition, it will be understood by those skilled in the art that various aspects of the present invention may be illustrated and described by a number of patentable categories or situations, including any new and useful process, machine, product or combination of substances, or any new and useful improvements thereto. Accordingly, various aspects of the present invention may be performed entirely by hardware, entirely by software (including firmware, resident software, microcode, etc.), or by a combination of hardware and software. The above hardware or software may be referred to as "data blocks", "modules", "engines", "units", "components" or "systems". In addition, various aspects of the present invention may be represented as a computer product located in one or more computer-readable media, which includes computer-readable program code.

[0085] Unless otherwise defined, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. It should also be understood that terms such as those defined in common dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant technology and should not be interpreted in an idealized or highly formal sense, unless expressly defined as such herein.

[0086] It will be understood that the above embodiments are merely exemplary embodiments for illustrating the principles of the present invention, and the present invention is not limited thereto. Those skilled in the art will appreciate that various modifications and improvements can be made without departing from the spirit and substance of the present invention, and such modifications and improvements are also considered to be within the scope of protection of the present invention.

Claims

1. An intelligent inspection and monitoring method for a wind farm, characterized in that: The method comprises: Obtaining blade inspection images of the wind turbine to be inspected captured by the camera of the drone; performing image preprocessing on the blade inspection image to obtain a cropped blade inspection image; Performing multi-scale feature extraction on the cropped blade inspection image to obtain a multi-scale blade state feature map; and Based on the multi-scale blade state characteristic diagram, it is determined whether to generate a wind turbine blade fault prompt.

2. The intelligent inspection and monitoring method for a wind farm according to claim 1, characterized in that: The performing image preprocessing on the blade inspection image to obtain a cropped blade inspection image includes: The blade inspection image is randomly cropped to obtain the cropped blade inspection image.

3. The intelligent inspection and monitoring method for a wind farm according to claim 2, characterized in that: The performing multi-scale feature extraction on the cropped blade inspection image to obtain a multi-scale blade state feature map includes: A deep learning network model is used to perform multi-scale feature extraction on the cropped blade inspection image to obtain the multi-scale blade state feature map.

4. The intelligent inspection and monitoring method for a wind farm according to claim 3, characterized in that: The deep learning network model is a leaf state feature extractor comprising a multi-scale convolutional structure.

5. The intelligent inspection and monitoring method for a wind farm according to claim 4, characterized in that: The leaf state feature extractor including a multi-scale convolution structure includes a first convolution layer, a second convolution layer and a multi-scale feature fusion layer connected to the first convolution layer and the second convolution layer, wherein the first convolution layer and the second convolution layer respectively use two-dimensional convolution kernels with different scales.

6. The intelligent inspection and monitoring method for a wind farm according to claim 5, characterized in that: The method of performing multi-scale feature extraction on the cropped blade inspection image using a deep learning network model to obtain the multi-scale blade state feature map includes: The cropped blade inspection image is passed through the blade state feature extractor including the multi-scale convolution structure to obtain the multi-scale blade state feature map.

7. The intelligent inspection and monitoring method for a wind farm according to claim 6, characterized in that: The step of passing the cropped blade inspection image through the blade state feature extractor including the multi-scale convolution structure to obtain the multi-scale blade state feature map comprises: Using the first convolution layer of the blade state feature extractor including the multi-scale convolution structure to perform two-dimensional convolution encoding on the cropped blade inspection image with a two-dimensional convolution kernel having a first scale to obtain a first-scale blade state feature map; Using the second convolution layer of the blade state feature extractor including the multi-scale convolution structure to perform two-dimensional convolution encoding on the cropped blade inspection image with a two-dimensional convolution kernel having a second scale to obtain a second-scale blade state feature map; and The first-scale leaf state feature map and the second-scale leaf state feature map are cascaded through the multi-scale feature fusion layer of the leaf state feature extractor including the multi-scale convolution structure to obtain the multi-scale leaf state feature map.

8. The intelligent inspection and monitoring method for a wind farm according to claim 7, characterized in that: The determining whether to generate a wind turbine blade fault prompt based on the multi-scale blade state characteristic diagram includes: performing feature distribution correction on the multi-scale blade state characteristic graph to obtain a corrected multi-scale blade state characteristic graph; and The corrected multi-scale blade state feature map is passed through a multi-task classification head module to obtain a classification result, and the classification result is used to indicate whether a wind turbine blade fault prompt is generated.

9. The intelligent inspection and monitoring method for a wind farm according to claim 8, characterized in that: The step of passing the corrected multi-scale blade state feature map through a multi-task classification head module to obtain a classification result, wherein the classification result is used to indicate whether a wind turbine blade fault prompt is generated, includes: Passing the multi-scale blade state feature map through a fine-grained classifier to obtain multiple blade defect category probability values; Passing the multi-scale leaf state characteristic graph through a coarse-grained classifier to obtain a first probability value and a second probability value; fusing the plurality of blade defect category probability values, the first probability value, and the second probability value in a probability-weighted fusion manner to obtain a comprehensive expression probability value; and The classification result is obtained based on the comprehensive expression probability value.

10. An intelligent inspection and monitoring system for a wind farm, characterized in that: include: An image acquisition module is used to obtain inspection images of the blades of the wind turbine to be inspected, which are captured by the camera of the UAV; An image preprocessing module, configured to perform image preprocessing on the blade inspection image to obtain a cropped blade inspection image; A multi-scale feature extraction module is used to perform multi-scale feature extraction on the cropped blade inspection image to obtain a multi-scale blade state feature map; as well as A fault prompt judgment module is used to determine whether to generate a wind turbine blade fault prompt based on the multi-scale blade state characteristic diagram.

Images