Image-based fan blade aging detection method and system

Abstract

The invention discloses an image-based fan blade aging detection method and system, and the method comprises the steps: collecting a multispectral image of a fan blade, and dynamically adjusting the collection frequency according to the rotation speed and position of the blade in the collection process; preprocessing the collected multispectral images, and aligning and fusing the images of the same leaf under different wavelengths through image registration to generate a multispectral feature image; and establishing an aging detection model to process the multispectral feature image, automatically identifying and marking an aging area in the image, generating a blade aging report, and completing fan blade aging detection. According to the method, the problems of low image quality, inaccurate feature extraction and low detection efficiency in fan blade aging detection in the prior art are solved, the accuracy and the automation degree of fan blade aging detection are comprehensively improved, and powerful technical support is provided for efficient operation and maintenance of a wind power plant.

Description

Technical Field

[0001] The present invention relates to the technical field of image detection, and in particular to an image-based method and system for detecting aging of fan blades. Background Art

[0002] With the rapid development of wind power technology, wind turbines have occupied an increasingly important position in the global energy structure. As the core component of wind power equipment, the health status of wind turbine blades directly affects the power generation efficiency and the service life of the equipment. Traditional wind turbine blade detection methods mainly rely on manual inspections and ultrasonic detection technologies. These methods can identify obvious defects of blades to a certain extent, but due to limited detection methods, it is often difficult to capture tiny cracks or aging problems inside the blade material. In addition, wind turbine blades are exposed to harsh environments all year round and are eroded by multiple factors such as wind, rain, ice and snow. The aging process is complex and irreversible. Existing detection technologies have great limitations in dealing with aging feature extraction and precise positioning under multi-spectral conditions, which puts higher requirements on the operation and maintenance of wind farms.

[0003] Currently, image-based detection methods have gradually become a research hotspot for wind turbine blade aging detection. The introduction of multispectral imaging technology makes it possible to identify the aging characteristics of blades by analyzing image information under different spectra. However, existing multispectral image processing methods still have significant deficiencies in image fusion, feature extraction and aging recognition. For example, multispectral images are easily interfered by noise during the processing process, resulting in a decrease in the accuracy of image registration, which in turn affects the subsequent feature extraction and analysis results. In addition, since the performance of wind turbine blades under different spectra varies greatly, how to effectively fuse this information and use it for aging detection remains a difficulty. In the existing technology, there is a lack of a mechanism for dynamically adjusting the acquisition frequency of multispectral images, and it is impossible to adapt to the real-time motion changes of blades, which makes it challenging to obtain high-quality, multi-dimensional aging feature images. Summary of the invention

[0004] In view of the above-mentioned problems, the present invention is proposed.

[0005] Therefore, the technical problem solved by the present invention is that the prior art lacks a mechanism for dynamically adjusting the acquisition frequency of multispectral images and cannot adapt to the real-time movement changes of blades, which makes it challenging to obtain high-quality, multi-dimensional aging characteristic images.

[0006] To solve the above technical problems, the present invention provides the following technical solutions: an image-based method for detecting aging of fan blades, comprising: collecting multispectral images of fan blades, and dynamically adjusting the collection frequency according to the rotation speed and position of the blades during the collection process; preprocessing the collected multispectral images, aligning and fusing the images of the same blade at different wavelengths through image registration, and generating a multispectral feature image; establishing an aging detection model to process the multispectral feature image, automatically identifying and marking the aging area in the image, generating a blade aging report, and completing the aging detection of fan blades.

[0007] As a preferred solution of the image-based wind blade aging detection method described in the present invention, the multispectral image is obtained by collecting images of wind blades using a multispectral camera that switches spectrums. The multispectral camera uses five spectra: visible light, near-infrared light, short-wave infrared light, medium-wave infrared light and far-infrared light. After each spectrum switching, the acquisition time window is recalculated according to the rotation speed and position of the blades, and the camera's acquisition frequency is adjusted.

[0008] As a preferred embodiment of the image-based wind turbine blade aging detection method of the present invention, the dynamic adjustment of the acquisition frequency includes using a sensor to monitor the position and rotation speed of the blade in real time, and calculating the rotation period T of the blade according to the rotation speed of the blade. blade , according to the position and speed of the blade, set the image acquisition time point in each cycle; preset the spectral sequence to be collected, and automatically rotate the filter wheel according to the set spectral sequence. Each spectrum is collected within three blade rotation cycles T blade After switching, the basic acquisition frequency f is set according to the rotation period of the blade and the time required for spectrum switching. b :

[0009]

[0010] Among them, n i Indicates the number of images that need to be collected in each spectral band; when switching spectra, the acquisition frequency f is dynamically adjusted according to the blade rotation speed d :

[0011]

[0012] Where V represents the rotation speed of the blade; t switchIndicates the time required for the filter wheel to switch from one spectral band to the next band; for the three cycles of images collected for each spectrum, the image of one cycle with the best comprehensive quality is selected based on the clarity, contrast and noise indicators as the final image for analysis; the collected image is bound to the blade angle and acquisition timestamp at the time of acquisition, and the collected multispectral images are stored according to the preset spectral order and acquisition time, and sorted according to the blade angle.

[0013] As a preferred solution of the image-based wind turbine blade aging detection method described in the present invention, the image registration includes extracting initial feature points through the SIFT algorithm, performing multiple Gaussian blur processing on the input multispectral image, constructing a Gaussian pyramid, finding local extreme points as potential feature points by calculating the difference between adjacent scale images, locating the potential feature points and calculating the Harris response value, calculating the gradient direction of the feature point neighborhood, and selecting the direction with the largest gradient amplitude as the main direction of the feature point, generating a feature descriptor according to the direction information, and completing the feature point extraction;

[0014] The Euclidean distance matching algorithm is used to introduce the similarity of spectral signals as an additional constraint condition to match the feature points of images under different spectra; based on the matched feature points, the multispectral images are preliminarily spatially registered. For the nonlinear deformation in the multispectral images, the matched feature point pairs are used to fit the thin plate spline transformation model through the least squares method to obtain the preliminary transformation matrix. The calculated transformation matrix is ​​used to preliminarily register the images. The thin plate spline transformation model is expressed as:

[0015]

[0016] Among them, T TPS (x, y) represents the output coordinates after applying the thin plate spline transformation to the input coordinates; N represents the number of feature points; w i represents the weight coefficient of the i-th control point; U represents the radial basis function; p represents the input coordinates (x, y); p i Represents the coordinates of the ith control point; a represents the parameter of the linear transformation; on the basis of preliminary spatial registration, an iterative optimization algorithm based on a multi-resolution pyramid is adopted to continuously refine the image resolution, optimize the registration results step by step, fuse the registered multispectral images, and generate a multispectral feature image.

[0017] As a preferred embodiment of the image-based wind turbine blade aging detection method of the present invention, the feature point matching includes: using the Euclidean distance matching algorithm to perform feature point matching, for each feature point, calculating the descriptor f of the feature point i With all the feature point descriptors f under other spectra j The Euclidean distance between:

[0018] d ij =∥f i -f j ∥

[0019] Among them, d ij Indicates p i and p j The Euclidean distance between two feature points; the similarity of spectral signals is introduced as an additional constraint. In the matching process, not only the similarity of geometric features is considered, but also the spectral feature similarity S of the region where the feature points are located is calculated. ij :

[0020]

[0021] Among them, σ represents the constant that controls the similarity attenuation; g i Represents the feature point p i The spectral feature vector of g under different spectra; j Represents the feature point p j The spectral feature vectors under different spectra; the final matching score M ij It is determined by the Euclidean distance and the similarity of spectral features and is expressed as:

[0022]

[0023] Among them, α represents the weight coefficient; the random sampling consistency algorithm is used to remove false matches from the preliminary matched point pairs.

[0024] As a preferred solution of the image-based wind turbine blade aging detection method described in the present invention, the aging detection model is a neural network model established by combining a deep convolutional neural network with an attention mechanism, and the multi-spectral feature image is processed through multi-layer convolution and feature extraction operations. The structure of the aging detection model includes an input layer, a convolution layer, a spectral feature enhancement module, a multi-scale feature extraction module, a pooling layer, a fully connected layer and an output layer. The input layer receives the feature image after multi-spectral fusion, and the convolution layer uses convolution kernels of different sizes for multi-scale feature extraction; the spectral feature enhancement module is improved on the basis of the convolution layer 2, and an adaptive weight allocation mechanism of spectral features is introduced in the conventional convolution operation; the multi-scale feature extraction module weights and integrates feature maps of different scales, fuses information from different scales, and obtains the final feature representation:

[0025]

[0026] Among them, F final represents the final multi-scale feature representation; α lrepresents the weight coefficient of the lth layer; L represents the total number of layers of the spectral feature enhancement module; F g Represents the feature map after the l-th layer spectral feature enhancement module; A(l) represents the feature weighting factor of the l-th layer; ω represents the integral variable; the adaptive pooling layer is used to reduce the size of the feature map, reduce the dimension of the feature map, flatten the feature map into a one-dimensional vector, and pass the flattened one-dimensional vector to the fully connected layer to generate a preliminary representation of the prediction result. After the fully connected layer, normalization is performed, and the activation function and adjustment parameters are combined to further enhance the feature expression, generate the final result and output it through the output layer:

[0027]

[0028] Among them, O final represents the final output, Z represents the normalization constant; M represents the scale number of the feature map; β m Represents the weighting coefficient at different scales; σ represents the activation function; γ represents the adjustment parameter that controls the degree of feature amplification.

[0029] As a preferred solution of the image-based wind turbine blade aging detection method described in the present invention, the spectral feature enhancement module is a composite module, including a pooling layer, a fully connected layer and a feature enhancement layer, which performs pooling on the feature map of each spectral channel of the input spectral feature enhancement module to generate a set of values ​​representing the global features of the current image:

[0030]

[0031] Among them, G c represents the global eigenvalue of the cth spectral channel; W represents the width of the feature map; H represents the height of the feature map; Represents the feature map of the input spectral feature enhancement module; according to the global pooling result G c , the weight α of each spectral channel is calculated through the fully connected layer c :

[0032] α c =σ(W c ·G c +b c )

[0033] Among them, W c represents weight; b c Indicates bias;

[0034] The output F of the spectral feature enhancement module i,j,k It is expressed as:

[0035]

[0036] Where C represents the total number of spectral channels.

[0037] In a second aspect, the present invention also provides an image-based detection system for aging of wind blades, including: an image acquisition module, which acquires multispectral images of wind blades through a multispectral camera, and dynamically adjusts the acquisition frequency according to the speed and position of the wind blades when the spectrum is switched; an image processing module, which performs image registration on the acquired multispectral images, aligns and fuses images of the same blade at different wavelengths, and generates a multispectral feature image; an aging detection module, which establishes an aging detection model based on a deep convolutional neural network, uses a spectral feature enhancement module to enhance the spectral features of the image, and identifies the aging area of ​​the blade.

[0038] In a third aspect, the present invention further provides a computing device, comprising: a memory and a processor;

[0039] The memory is used to store computer executable instructions, and the processor is used to execute the computer executable instructions. When the computer executable instructions are executed by the processor, the steps of the image-based wind turbine blade aging detection method are implemented.

[0040] In a fourth aspect, the present invention further provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the steps of the image-based wind blade aging detection method.

[0041] Beneficial effects of the invention: The method of the invention solves the problems of low image quality, inaccurate feature extraction and low detection efficiency in the prior art of wind turbine blade aging detection through multispectral image acquisition, precise image registration and fusion, aging detection based on deep learning and the final generation of an aging report. It comprehensively improves the accuracy and automation of wind turbine blade aging detection and provides strong technical support for the efficient operation and maintenance of wind farms. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0043] Figure 1 An overall flow chart of an image-based wind turbine blade aging detection method provided by one embodiment of the present invention. DETAILED DESCRIPTION

[0044] In order to make the above-mentioned purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the drawings of the specification. Obviously, the described embodiments are part of the embodiments of the present invention, but not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary persons in the art without creative work should fall within the scope of protection of the present invention.

[0045] Example 1, reference Figure 1 , as an embodiment of the present invention, provides an image-based wind turbine blade aging detection method, comprising:

[0046] S1: Collect multispectral images of wind turbine blades and dynamically adjust the acquisition frequency according to the rotation speed and position of the blades during the acquisition process.

[0047] Furthermore, multispectral images are obtained by capturing images of wind turbine blades using a multispectral camera that switches spectrums. The multispectral camera uses five spectrums: visible light, near-infrared light, short-wave infrared light, medium-wave infrared light, and far-infrared light. After each spectrum switching, the acquisition time window is recalculated according to the rotation speed and position of the blades, and the camera's acquisition frequency is adjusted.

[0048] Each type of light corresponds to different aging conditions. Visible light (400-700nm) detects surface cracks, obvious corrosion spots and color changes; near-infrared light (700-1100nm) detects microscopic cracks, internal stress distribution and moisture content changes inside the material; short-wave infrared light (1100-2500nm) detects composite material delamination, coating degradation and deep cracks; medium-wave infrared light (3-5μm) detects thermal effects, fatigue damage and deep structural abnormalities in materials; far-infrared light (8-14μm) detects thermal distribution of materials, surface temperature abnormalities and changes in thermal conductivity characteristics caused by long-term aging.

[0049] Considering that data processing and analysis in the ultraviolet band are very complicated, and the characteristics of ultraviolet light are more significant under specific circumstances, the cost and benefit of additional ultraviolet light analysis in conventional leaf aging detection are not proportional, so ultraviolet spectral image collection is not performed.

[0050] It should be noted that multispectral cameras are divided into full-spectrum cameras that collect multiple spectral images at the same time and cameras that need to switch spectral cameras. Although multi-channel cameras can collect multiple spectral images at the same time, their cost is relatively high, while cameras that switch spectrums are cheaper and are particularly suitable for projects that require long-term aging monitoring on multiple wind turbines. Through reasonable image processing, cameras that switch spectrums can also provide accurate multispectral data.

[0051] From the functional point of view, the purpose of the present invention is to detect the aging of the fan rather than the real-time failure. The aging of the fan blades is a slow process. The required image acquisition does not require the synchronous acquisition of different spectra in a very short time. The aging detection focuses on the surface changes of the blades over a period of time, rather than the instantaneous spectral response.

[0052] Therefore, the present invention chooses to use a multi-spectral camera that switches spectrums for image acquisition, which can effectively reduce costs, and the disadvantage of lax time synchronization has no effect on the aging detection of wind turbine blades. Images under different spectra can be aligned and analyzed through post-processing.

[0053] Furthermore, sensors are used to monitor the angle and rotation speed of the blades in real time, and the rotation period of the blades is calculated based on the rotation speed of the blades:

[0054]

[0055] Among them, T blade represents the rotation period of the blade; ω represents the rotation angular velocity of the blade.

[0056] According to the position and speed of the blade, the image acquisition time point in each cycle is set to ensure that the image covers the entire blade surface. First, the acquisition time interval Δt of each image is determined to ensure that the image can evenly cover the entire blade surface during the blade rotation:

[0057]

[0058] Among them, n images Indicates the number of images required to be collected in one cycle. This value is set according to the coverage requirements of the blade surface and is usually a fixed value.

[0059] According to the current position and angular velocity of the blade, the specific acquisition time point of each image is dynamically calculated, and the acquisition time point is associated with the rotation angle of the blade to ensure that the image is evenly distributed on the entire blade surface, expressed as:

[0060] t i =t0+i×Δt

[0061] θ i =ω×t i

[0062] Among them, t i represents the acquisition time of the i-th image; t0 represents the time when the leaf first passes the spectral camera position; i represents the sequence number of the image; θ i represents the rotation angle of the blade when the i-th image is acquired.

[0063] It should be noted that t0 is the time when the blade reaches the initial position detected by the sensor, and each subsequent acquisition point t i All of them are increased according to the time interval Δt on this basis. Calibration of the acquisition time point according to the blade position angle can ensure that the acquisition time point t i Corresponding to the uniformly distributed angle θ on the blade surface i , thus ensuring that the entire blade surface is evenly covered.

[0064] If the rotation speed of the blade changes (i.e. ω changes), the acquisition time of subsequent images needs to be dynamically adjusted:

[0065]

[0066] Among them, t i+1 is the acquisition time point of the i+1th image; Δθ represents the desired rotation angle difference between two adjacent images; ω current Indicates the current rotational angular velocity of the blade.

[0067] Under normal circumstances, ω will not change significantly in each acquisition cycle to affect image acquisition. Therefore, the acquisition time point is adjusted once after each spectrum switch. If ω changes by more than 10%, the acquisition time point is dynamically adjusted. Otherwise, the last setting is retained without change.

[0068] The order of spectra to be collected is preset, and the filter wheel is automatically rotated according to the set spectral order. Each spectrum is collected within three blade rotation cycles T. blade After switching, the basic acquisition frequency f is set according to the rotation period of the blade and the time required for spectrum switching. b :

[0069]

[0070] Among them, n i Indicates the number of images that need to be collected in each spectral band; when switching spectra, the acquisition frequency f is dynamically adjusted according to the blade rotation speed d :

[0071]

[0072] Where V represents the rotation speed of the blade; t switch Represents the time it takes for the filter wheel to switch from one spectral band to the next.

[0073] For each spectrum acquisition of three cycles of images, the image of one cycle with the best comprehensive quality was selected based on the clarity, contrast and noise indicators as the final image for analysis.

[0074] The acquired images are bound to the blade angle and acquisition timestamp at the time of acquisition, and the acquired multispectral images are stored according to the preset spectral sequence and acquisition time, and are sorted according to the blade angle.

[0075] S2: Preprocess the collected multispectral images, align and fuse the images of the same leaf at different wavelengths through image registration, and generate a multispectral feature image.

[0076] Image registration refers to aligning wind turbine blade images at different wavelengths so that they overlap in space for subsequent image fusion and feature extraction. Since multispectral images differ in spectral bands, they may be distorted and offset during the imaging process, so accurate registration is required.

[0077] Furthermore, in the multispectral images of wind turbine blades, images under different spectra may have problems such as geometric deformation and spectral differences. In order to accurately register the images, it is first necessary to extract stable and representative feature points in each image. Due to the difference in wavelength, multispectral images may have scale, rotation, illumination and other deformations. In order to cope with these deformations, the multiscale feature extraction algorithm can ensure that stable feature points are extracted under different scales and rotations.

[0078] When performing image registration, the initial feature points are first extracted using the SIFT algorithm. The input multispectral image is subjected to multiple Gaussian blurring processes to construct a Gaussian pyramid. The local extreme points are found as potential feature points by calculating the differences between adjacent scale images. The potential feature points are located and the Harris response values ​​are calculated. The gradient direction of the feature point neighborhood is calculated, and the direction with the largest gradient amplitude is selected as the main direction of the feature point. A feature descriptor is generated based on the direction information to complete the feature point extraction.

[0079] The Euclidean distance matching algorithm is used to introduce the similarity of spectral signals as an additional constraint to match the feature points of images under different spectra. For each feature point, the descriptor f of the feature point is calculated. i With all the feature point descriptors f under other spectra j The Euclidean distance between:

[0080] d ij =∥f i -f j ∥

[0081] Among them, d ij Indicates p i and p j The Euclidean distance between two feature points.

[0082] The similarity of spectral signals is introduced as an additional constraint. In the matching process, not only the similarity of geometric features is considered, but also the spectral feature similarity S of the region where the feature point is located is calculated. ij :

[0083]

[0084] Among them, σ represents the constant that controls the similarity attenuation; g i Represents the feature point p i The spectral feature vector of g under different spectra; j Represents the feature point p j The spectral feature vectors under different spectra.

[0085] The final matching score M ij It is determined by the Euclidean distance and the similarity of spectral features and is expressed as:

[0086]

[0087] Among them, α represents the weight coefficient.

[0088] Through the random sampling consistency algorithm, false matches are removed from the preliminary matching point pairs, and a minimum set of point pairs is randomly selected from the preliminary matching feature point pairs. An initial geometric transformation model is estimated based on these point pairs. For the remaining matching point pairs, the geometric transformation model is applied and their transformed positions are calculated. If the error between the transformed position and the actual position is within a preset threshold, the point pair is regarded as an "inlier". Random sampling and model estimation are repeated many times until a model containing the most inliers is found.

[0089] The traditional RANSAC algorithm only considers geometric features, while the present invention introduces spectral feature similarity as a further optimization method. In the process of interior point judgment, not only the geometric error is required to be within the threshold, but also the spectral feature similarity S ij When a certain level is reached, the retained feature point pairs have high similarity in both geometry and spectrum, ensuring that in a multispectral environment, the matching point pairs are not only geometrically aligned but also consistent in spectral information.

[0090] Based on the matched feature points, the multispectral images are preliminarily spatially registered. For the nonlinear deformation in the multispectral images, the matched feature point pairs are used to fit the thin plate spline transformation model by the least squares method to obtain the preliminary transformation matrix. The calculated transformation matrix is ​​used to preliminarily register the images. The thin plate spline transformation model is expressed as:

[0091]

[0092] Among them, T TPS(x, y) represents the output coordinates after applying the thin plate spline transformation to the input coordinates; N represents the number of feature points; w i represents the weight coefficient of the i-th control point; U represents the radial basis function; p represents the input coordinates (x, y); p i represents the coordinates of the i-th control point; a represents the parameters of the linear transformation.

[0093] The calculated transformation matrix is ​​used to perform preliminary image registration. The source image is resampled according to the transformation matrix through bilinear interpolation to generate the registered image. For the transformed image boundary, an expansion or cropping strategy is adopted to avoid edge information loss.

[0094] On the basis of preliminary spatial registration, an iterative optimization algorithm based on a multi-resolution pyramid is adopted to continuously refine the image resolution, optimize the registration results step by step, and fuse the registered multispectral images to generate a multispectral feature image.

[0095] S3: Establish an aging detection model to process multispectral feature images, automatically identify and mark the aging areas in the image, generate a blade aging report, and complete the wind turbine blade aging detection.

[0096] Furthermore, the aging detection model is a neural network model established by combining a deep convolutional neural network with an attention mechanism. The multispectral feature image is processed through multi-layer convolution and feature extraction operations. The structure of the aging detection model includes an input layer, a convolution layer, a spectral feature enhancement module, a multi-scale feature extraction module, a pooling layer, a fully connected layer, and an output layer. The input layer receives the feature image after multi-spectral fusion, and the convolution layer uses convolution kernels of different sizes for multi-scale feature extraction.

[0097] In conventional convolutional layers, the feature map of each spectral channel is processed by the same convolution kernel, lacking specialized optimization for different spectral features. This means that when processing multispectral images, features at different wavelengths may be treated the same, failing to highlight the specific information of certain spectral channels. Therefore, a spectral feature enhancement module is established to analyze spectral features.

[0098] The spectral feature enhancement module is improved on the basis of convolution layer 2. It is a composite module including a pooling layer, a fully connected layer and a feature enhancement layer. After the traditional convolution operation, an adaptive mechanism is introduced to calculate the weight for each spectral channel. The weight is dynamically adjusted according to the current image content to highlight the features under a specific spectral channel.

[0099] After the feature map is input, the pooling layer pools the feature map of each spectral channel of the input spectral feature enhancement module to generate a set of values ​​representing the global features of the current image:

[0100]

[0101] Among them, G c represents the global eigenvalue of the cth spectral channel; W represents the width of the feature map; H represents the height of the feature map; Feature map representing the input spectral feature enhancement module.

[0102] According to the global pooling result G c , the weight α of each spectral channel is calculated through the fully connected layer c :

[0103] α c =σ(W c ·G c +b c )

[0104] Among them, W c represents weight; b c Indicates bias.

[0105] The output F of the spectral feature enhancement module i,j,k It is expressed as:

[0106]

[0107] Where C represents the total number of spectral channels.

[0108] Through the adaptive spectral weighting mechanism, the model can distinguish the manifestations of aging characteristics between different spectral channels, especially in the spectral manifestations of characteristics such as cracks and corrosion, significantly enhancing the differentiated expression between different spectral channels.

[0109] The multi-scale feature extraction module weights and integrates feature maps of different scales, fuses information from different scales, and obtains the final feature representation:

[0110]

[0111] Among them, F final represents the final multi-scale feature representation; α l represents the weight coefficient of the lth layer; L represents the total number of layers of the spectral feature enhancement module; F g It represents the feature map after the l-th layer spectral feature enhancement module; A(l) represents the feature weighting factor of the l-th layer; ω represents the integral variable.

[0112] Use the adaptive pooling layer to reduce the size of the feature map, reduce the dimension of the feature map, flatten the feature map into a one-dimensional vector, pass the flattened one-dimensional vector to the fully connected layer, generate a preliminary representation of the prediction result, and after the fully connected layer, perform normalization, and combine the activation function and adjustment parameters to further enhance the feature expression, generate the final result and output it through the output layer:

[0113]

[0114] Among them, O final represents the final output, Z represents the normalization constant; M represents the scale number of the feature map; β m Represents the weighting coefficient at different scales; σ represents the activation function; γ represents the adjustment parameter that controls the degree of feature amplification.

[0115] Furthermore, the fully connected neural network (FCNN) is used as a classifier to transform the output of the aging detection model into final As the input of the classifier, the classifier includes several hidden layers, each layer applies the activation function ReLU, and the last layer outputs the classification result of the aging degree. The loss function uses the cross entropy function. According to the output result of the classifier, the aging degree of the leaves is determined and the corresponding aging report is generated.

[0116] The present embodiment also provides an image-based wind blade aging detection system, including: an image acquisition module, which acquires multispectral images of wind blades through a multispectral camera, and dynamically adjusts the acquisition frequency according to the speed and position of the wind blades when the spectrum is switched; an image processing module, which performs image registration on the acquired multispectral images, aligns and fuses images of the same blade at different wavelengths, and generates a multispectral feature image; an aging detection module, which establishes an aging detection model based on a deep convolutional neural network, uses a spectral feature enhancement module to enhance the spectral features of the image, and identifies the aging area of ​​the blade.

[0117] This embodiment also provides a computing device, including a memory and a processor; the memory is used to store computer-executable instructions, and the processor is used to execute computer-executable instructions to implement the image-based wind blade aging detection method proposed in the above embodiment.

[0118] This embodiment further provides a storage medium on which a computer program is stored. When the program is executed by a processor, the image-based wind turbine blade aging detection method proposed in the above embodiment is implemented.

[0119] The storage medium proposed in this embodiment and the image-based wind blade aging detection method proposed in the above embodiment belong to the same inventive concept. The technical details not fully described in this embodiment can be referred to the above embodiment, and this embodiment has the same beneficial effects as the above embodiment.

[0120] Through the above description of the implementation method, the technicians in the relevant field can clearly understand that the present invention can be implemented by means of software and necessary general hardware, and of course it can also be implemented by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as a computer floppy disk, read-only memory (ReadOnly, Memory, ROM), random access memory (RandomAccessMemory, RAM), flash memory (FLASH), hard disk or optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, server, or network device, etc.) to perform the methods of various embodiments of the present invention.

[0121] Example 2, below is an embodiment of the present invention, which provides an image-based method for detecting aging of wind turbine blades. In order to verify the beneficial effects of the present invention, a scientific demonstration is carried out through simulation experiments.

[0122] A simulation experiment was conducted on 10 wind turbine blade samples collected from actual wind farms, including blades with different degrees of aging. Multispectral images of the wind turbine blades were collected through a multispectral camera system, supporting five spectral bands (visible light, near infrared, short-wave infrared, medium-wave infrared, and far infrared).

[0123] First, the camera acquisition frequency is set according to the rotation speed and position of each blade. The rotation period of the wind turbine blades is monitored in real time and fed back to the control system through sensors to dynamically adjust the acquisition frequency of the multispectral camera. The number of images collected in each spectral band is set to 30, and the spectral switching time is 0.2 seconds.

[0124] The multispectral image acquisition process adopts a spectral switching strategy, that is, the spectrum is switched once every three fan blade rotation cycles to ensure that clear images can be obtained in each spectral band. The entire acquisition process is completed within 20 minutes, and blade images under five spectra are collected. The collected spectral images are analyzed by the method of the present invention, and the experimental data are shown in Table 1.

[0125] Table 1 Experimental data table

[0126]

[0127] Among them, 1-10 represents the number of the wind turbine blades, and bands 1-4 represent the clarity of the collected spectral images, expressed in percentage. Band 1 corresponds to the visible light band, capturing the appearance characteristics of the blades under visible light, such as surface cracks and rust; Band 2 corresponds to the near-infrared light band, used to detect temperature changes or moisture content of the blades; Band 3 corresponds to the short-wave infrared light band, used to observe the internal structure of the blades or the subtle features of the surface; Band 4 corresponds to the mid-wave and far-infrared light bands, used to detect thermal radiation or identify the specific chemical composition of the material.

[0128] The accuracy rate and error rate are the accuracy rate and error rate of aging detection, which are used to verify the beneficial effects of the method of the present invention. It can be seen from the data analysis in the table that the image-based wind turbine blade aging detection method of the present invention has shown significant advantages in image clarity and aging detection accuracy. The clarity data of spectral bands 1 to spectral bands 4 in the table are all maintained between 82.5% and 91.3%, indicating that the method of the present invention can obtain high-quality images in each spectral band, ensuring the reliability of subsequent image registration and fusion.

[0129] As for the aging detection accuracy, the detection accuracy of all leaves was above 90.5%, with the highest reaching 93.0%. This shows that the deep convolutional neural network combined with the model structure of the spectral feature enhancement module adopted by the present invention can effectively extract aging features in multi-spectral images, thereby achieving high-precision leaf aging detection. In addition, compared with the prior art, the present invention further improves the accuracy and robustness of detection and reduces the error rate by introducing technical means such as dynamic acquisition frequency adjustment and multi-resolution pyramid iterative optimization, so that the average error rate in the experiment is as low as 3.0%.

[0130] In summary, the performance of the method of the present invention in the experiment fully proves its innovation and superiority. By comparing the shortcomings of the prior art, it can be concluded that the method of the present invention not only shows higher accuracy in the acquisition and processing of multispectral images, but also achieves higher detection accuracy and lower error rate in the aging detection model through technical means such as spectral feature enhancement module, which has significant technical progress and practical value.

[0131] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. A method for detecting aging of wind turbine blades based on an image, characterized in that: include: Collect multispectral images of wind turbine blades and dynamically adjust the acquisition frequency according to the rotation speed and position of the blades during the acquisition process; Preprocess the collected multispectral images, align and fuse the images of the same leaf at different wavelengths through image registration, and generate a multispectral feature image; An aging detection model is established to process multispectral feature images, automatically identify and mark the aging areas in the images, generate blade aging reports, and complete wind turbine blade aging detection.

2. The image-based wind turbine blade aging detection method according to claim 1, characterized in that: The multispectral image is obtained by capturing images of wind turbine blades using a multispectral camera that switches spectrums. The multispectral camera uses five spectrums: visible light, near infrared light, short-wave infrared light, medium-wave infrared light, and far infrared light. After each spectrum switching, the acquisition time window is recalculated according to the rotation speed and position of the blades, and the camera's acquisition frequency is adjusted.

3. The image-based wind turbine blade aging detection method according to claim 2, characterized in that: The dynamic adjustment of the acquisition frequency includes using a sensor to monitor the position and rotation speed of the blade in real time, and calculating the rotation period T of the blade according to the rotation speed of the blade. blade , according to the position and speed of the blade, the image acquisition time point in each cycle is set; The order of spectra to be collected is preset, and the filter wheel is automatically rotated according to the set spectral order. Each spectrum is collected within three blade rotation cycles T. blade After switching, the basic acquisition frequency f is set according to the rotation period of the blade and the time required for spectrum switching. b : Among them, n i Indicates the number of images that need to be collected in each spectral band; when switching spectra, the acquisition frequency f is dynamically adjusted according to the blade rotation speed d : Where V represents the rotation speed of the blade; t switch Represents the time it takes for the filter wheel to switch from one spectral band to the next; For each spectrum acquisition of three cycles of images, the image of one cycle with the best comprehensive quality was selected based on the clarity, contrast and noise indicators as the final image for analysis; The acquired images are bound to the blade angle and acquisition timestamp at the time of acquisition, and the acquired multispectral images are stored according to the preset spectral sequence and acquisition time, and are sorted according to the blade angle.

4. The image-based wind turbine blade aging detection method according to claim 3, characterized in that: The image registration includes extracting initial feature points through SIFT algorithm, performing multiple Gaussian blur processing on the input multispectral image, constructing a Gaussian pyramid, finding local extreme points as potential feature points by calculating the difference between adjacent scale images, locating the potential feature points and calculating Harris response values, calculating the gradient direction of the neighborhood of the feature points, and selecting the direction with the largest gradient amplitude as the main direction of the feature points, generating feature descriptors according to the direction information, and completing feature point extraction; The Euclidean distance matching algorithm is used to introduce the similarity of spectral signals as an additional constraint to match the feature points of images under different spectra. Based on the matched feature points, the multispectral images are preliminarily spatially registered. For the nonlinear deformation in the multispectral images, the matched feature point pairs are used to fit the thin plate spline transformation model by the least squares method to obtain the preliminary transformation matrix. The calculated transformation matrix is ​​used to preliminarily register the images. The thin plate spline transformation model is expressed as: Among them, T TPS (x, y) represents the output coordinates after applying the thin plate spline transformation to the input coordinates; N represents the number of feature points; w i represents the weight coefficient of the i-th control point; U represents the radial basis function; p represents the input coordinates (x, y); p i represents the coordinates of the i-th control point; a represents the parameter of the linear transformation; On the basis of preliminary spatial registration, an iterative optimization algorithm based on a multi-resolution pyramid is adopted to continuously refine the image resolution, optimize the registration results step by step, and fuse the registered multispectral images to generate a multispectral feature image.

5. The image-based wind turbine blade aging detection method according to claim 4, characterized in that: The feature point matching includes using the Euclidean distance matching algorithm to perform feature point matching, and for each feature point, calculating the descriptor f of the feature point i With all the feature point descriptors f under other spectra j The Euclidean distance between: d ij =∥f i -f j ∥ Among them, d ij Indicates p i and p j Euclidean distance between two feature points; The similarity of spectral signals is introduced as an additional constraint. In the matching process, not only the similarity of geometric features is considered, but also the spectral feature similarity S of the region where the feature point is located is calculated. ij : Among them, σ represents the constant that controls the similarity attenuation; g i Represents the feature point p i The spectral feature vector of g under different spectra; j Represents the feature point p j The spectral feature vectors of under different spectra; The final matching score M ij It is determined by the Euclidean distance and the similarity of spectral features and is expressed as: Among them, α represents the weight coefficient; the random sampling consistency algorithm is used to remove false matches from the preliminary matched point pairs.

6. The method for detecting aging of wind turbine blades based on images according to claim 5, characterized in that: The aging detection model is a neural network model established by combining a deep convolutional neural network with an attention mechanism. The multispectral feature image is processed through multi-layer convolution and feature extraction operations. The structure of the aging detection model includes an input layer, a convolution layer, a spectral feature enhancement module, a multi-scale feature extraction module, a pooling layer, a fully connected layer and an output layer. The input layer receives the feature image after multi-spectral fusion, and the convolution layer uses convolution kernels of different sizes to extract multi-scale features. The spectral feature enhancement module is improved on the basis of convolution layer 2, and an adaptive weight allocation mechanism of spectral features is introduced in the conventional convolution operation; The multi-scale feature extraction module weights and integrates feature maps of different scales, fuses information from different scales, and obtains the final feature representation: Among them, F final represents the final multi-scale feature representation; α l represents the weight coefficient of the lth layer; L represents the total number of layers of the spectral feature enhancement module; F g represents the feature map after the l-th layer spectral feature enhancement module; A(l) represents the feature weighting factor of the l-th layer; ω represents the integral variable; Use the adaptive pooling layer to reduce the size of the feature map, reduce the dimension of the feature map, flatten the feature map into a one-dimensional vector, pass the flattened one-dimensional vector to the fully connected layer, generate a preliminary representation of the prediction result, and after the fully connected layer, perform normalization, and combine the activation function and adjustment parameters to further enhance the feature expression, generate the final result and output it through the output layer: Among them, O final represents the final output, Z represents the normalization constant; M represents the scale number of the feature map; β m Represents the weighting coefficient at different scales; σ represents the activation function; γ represents the adjustment parameter that controls the degree of feature amplification.

7. The method for detecting aging of wind turbine blades based on images according to claim 6, characterized in that: The spectral feature enhancement module is a composite module, including a pooling layer, a fully connected layer and a feature enhancement layer, which pools the feature map of each spectral channel of the input spectral feature enhancement module to generate a set of values ​​representing the global features of the current image: Among them, G c represents the global eigenvalue of the cth spectral channel; W represents the width of the feature map; H represents the height of the feature map; A feature map representing the input spectral feature enhancement module; According to the global pooling result G c , the weight α of each spectral channel is calculated through the fully connected layer c : a c =σ(W c ·G c +b c ) Among them, W c represents weight; b c Indicates bias; The output F of the spectral feature enhancement module i,j,k It is expressed as: Where C represents the total number of spectral channels.

8. An image-based wind turbine blade aging detection system using the method according to any one of claims 1 to 7, characterized in that: include, The image acquisition module collects multispectral images of the fan blades through a multispectral camera and dynamically adjusts the acquisition frequency according to the speed and position of the fan blades when the spectrum is switched; The image processing module performs image registration on the collected multispectral images, aligns and fuses the images of the same leaf at different wavelengths, and generates a multispectral feature image; The aging detection module establishes an aging detection model based on a deep convolutional neural network and uses the spectral feature enhancement module to enhance the spectral features of the image and identify the aging areas of the leaves.

9. A computing device comprising: Memory and processor; The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions. When the computer-executable instructions are executed by the processor, the steps of the method described in any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium storing computer-executable instructions, wherein the computer-executable instructions, when executed by a processor, implement the steps of the method according to any one of claims 1 to 7.

Images