An image-based detection method and system for fan blade aging

Abstract

The application discloses a kind of based on image's fan blade aging detection method and system, comprising: the multispectral image of fan blade is collected, in the acquisition process according to the rotation speed and position of blade dynamic adjustment acquisition frequency;The multispectral image collected is preprocessed, and the image under different wavelengths of the same blade is aligned and fused by image registration, and multispectral feature image is generated;Establish aging detection model and process multispectral feature image, automatically identify and label the aging area in image, generate blade aging report, complete fan blade aging detection.The method of the present application solves the problems of low image quality, inaccurate feature extraction and low detection efficiency in the existing technology of fan blade aging detection, and improves the accuracy and automation of fan blade aging detection, providing strong technical support for efficient operation and maintenance of wind farm.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image detection, in particular to a fan blade aging detection method and system based on images. BACKGROUND

[0002] With the rapid development of wind power technology, wind turbines have played an increasingly important role in the global energy structure. As the core component of wind power equipment, the health status of wind turbine blades directly affects the power generation efficiency and service life of the equipment. Traditional wind turbine blade detection methods mainly rely on manual inspection and ultrasonic detection technologies. These methods can identify obvious defects in the blades to some extent, but due to the limitations of detection methods, it is often difficult to capture small cracks or aging problems inside the blade material. In addition, wind turbine blades are exposed to harsh environments for years, and are eroded by wind, rain, ice and snow and other factors, and their aging process is complex and irreversible. Existing detection technologies have great limitations in dealing with aging feature extraction and accurate positioning under multiple spectra, which poses higher requirements for wind farm operation and maintenance.

[0003] Currently, image-based detection methods have gradually become a research hotspot for wind turbine blade aging detection. The introduction of multispectral imaging technology makes it possible to analyze image information under different spectra to identify aging characteristics of the blades. However, existing multispectral image processing methods still have significant shortcomings in image fusion, feature extraction and aging identification. For example, multispectral images are prone to noise interference during processing, which reduces the image registration accuracy and affects the subsequent feature extraction and analysis results. In addition, due to the large differences in the performance of wind turbine blades under different spectra, how to effectively fuse this information for aging detection remains a challenge. In existing technologies, there is a lack of a mechanism for dynamically adjusting the acquisition frequency of multispectral images, which cannot adapt to the real-time motion changes of the blades, making it a challenge to obtain high-quality, multi-dimensional aging feature images. SUMMARY

[0004] In view of the above problems, the present application is proposed.

[0005] Therefore, the technical problem solved by the present application is that existing technologies lack a mechanism for dynamically adjusting the acquisition frequency of multispectral images, which cannot adapt to the real-time motion changes of the blades, making it a challenge to obtain high-quality, multi-dimensional aging feature images.

[0006] To solve the above technical problems, the application provides the following technical scheme: a fan blade aging detection method based on images, comprising: collecting a multispectral image of a fan blade, dynamically adjusting a collection frequency according to a rotation speed and a position of the blade during the collection process; pre-processing the collected multispectral image, aligning and fusing images of the same blade under different wavelengths through image registration to generate a multispectral feature image; establishing an aging detection model to process the multispectral feature image, automatically identifying and labeling an aging area in the image, generating a blade aging report, and completing fan blade aging detection.

[0007] As a preferred scheme of the fan blade aging detection method based on images, the multispectral image is obtained by using a multispectral camera with a switched spectrum to collect images of the fan blade, the spectrum used by the multispectral camera is visible light, near-infrared light, short-wave infrared light, medium-wave infrared light and far-infrared light, after each spectrum switching, the collection time window is recalculated according to the rotation speed and the position of the blade, and the collection frequency of the camera is adjusted.

[0008] As a preferred scheme of the fan blade aging detection method based on images, the dynamic adjustment of the collection frequency comprises: using a sensor to monitor the position and the rotation speed of the blade in real time, calculating a rotation period T blade of the blade according to the rotation speed of the blade, setting an image collection time point in each period according to the position and the speed of the blade, presetting a spectrum order to be collected, automatically rotating a filter wheel according to the set spectrum order, switching each spectrum after three rotation periods T blade of the blade, setting a basic collection frequency f b according to the rotation period of the blade and the time required for spectrum switching.

[0009]

[0010] wherein n i represents the number of images to be collected under each spectrum band; the collection frequency f d is dynamically adjusted according to the rotation speed of the blade when the spectrum is switched.

[0011]

[0012] wherein V represents the rotation speed of the blade; t switchThe time required for the filter wheel to switch from one spectral band to the next band is represented; for three periodic images of each spectral acquisition, the one with the best comprehensive quality is selected based on the indicators of sharpness, contrast and noise as the final image for analysis; the acquired image is bound with the blade angle at the time of acquisition and the acquisition time stamp, and the acquired multi-spectral image is stored according to the preset spectral order and acquisition time, and is sorted according to the blade angle.

[0013] As a preferred scheme of the image-based fan blade aging detection method, the image registration comprises: extracting initial feature points by a SIFT algorithm, performing multiple Gaussian blur processing on the input multi-spectral image, constructing a Gaussian pyramid, finding local extreme points as potential feature points by calculating the difference between adjacent scale images, positioning the potential feature points and calculating Harris response values, calculating the gradient direction of the neighborhood of the feature points, and selecting the direction with the largest gradient amplitude as the main direction of the feature points, and generating feature descriptors according to the direction information to complete the feature point extraction.

[0014] An Euclidean distance matching algorithm is adopted to introduce the similarity of the spectral signals as an additional constraint condition to match the feature points of the images under different spectrums; based on the matched feature points, the multi-spectral images are preliminarily registered, and for the nonlinear deformation existing in the multi-spectral images, a matched feature point pair is used to fit a thin-plate spline transformation model by a least square method to obtain a preliminary transformation matrix, and the calculated transformation matrix is used to preliminarily register the images, and the thin-plate spline transformation model is expressed as:

[0015]

[0016] wherein, T TPS (x,y) represents the output coordinates after applying the thin-plate spline transformation to the input coordinates; N represents the number of feature points; w i represents the weight coefficient of the i th control point; U represents a radial basis function; p represents the input coordinates (x,y); p i represents the coordinates of the i th control point; a represents the parameters of the linear transformation; on the basis of the preliminary spatial registration, an iterative optimization algorithm based on a multi-resolution pyramid is adopted to gradually optimize the registration result by continuously refining the image resolution, and the registered multi-spectral images are fused to generate a multi-spectral feature image.

[0017] As a preferred scheme of the image-based fan blade aging detection method, the feature point matching comprises: adopting an Euclidean distance matching algorithm to match the feature points, and for each feature point, calculating the Euclidean distance between the descriptor f i of the feature point and all the feature point descriptors f j under other spectrums:

[0018] d ij =∥f i -f j ∥

[0019] Where, d ij p i and p j The Euclidean distance between two feature points; introducing the similarity of spectral signals as an additional constraint, in the matching process, not only is the similarity of geometric features considered, but also the spectral feature similarity S of the region where the feature point is located is calculated. ij :

[0020]

[0021] Where σ represents a constant controlling the similarity decay; g i Representing feature point p i Spectral eigenvectors of g under different spectra; j Representing feature point p j The spectral feature vectors under different spectra; the final matching score M ij Determined by both Euclidean distance and spectral feature similarity, it can be expressed as:

[0022]

[0023] Here, α represents the weighting coefficient; the random sampling consensus algorithm removes mismatches from the initially matched point pairs.

[0024] As a preferred embodiment of the image-based wind turbine blade aging detection method of the present invention, the aging detection model is a neural network model established by combining a deep convolutional neural network with an attention mechanism. Through multi-layer convolution and feature extraction operations, it processes multispectral feature images. The structure of the aging detection model includes an input layer, a convolutional layer, a spectral feature enhancement module, a multi-scale feature extraction module, a pooling layer, a fully connected layer, and an output layer. The input layer receives the feature image after multispectral fusion. The convolutional layer uses convolutional kernels of different sizes for multi-scale feature extraction. The spectral feature enhancement module improves upon convolutional layer 2 by introducing an adaptive weight allocation mechanism for spectral features in the conventional convolution operation. The multi-scale feature extraction module weights and integrates feature maps at different scales, fusing information from different scales to obtain the final feature representation.

[0025]

[0026] Among them, F final This represents the final multi-scale feature representation; α lrepresents the weight coefficient of the lth layer; L represents the total number of layers of the spectral feature enhancement module; F g represents the feature map after the lth layer of the spectral feature enhancement module; A(l) represents the feature weighting factor of the lth layer; ω represents the integral variable; the adaptive pooling layer is used to reduce the size of the feature map, reduce the dimension of the feature map, flatten the feature map into a one-dimensional vector, and the flattened one-dimensional vector is transmitted into the full connection layer to generate a preliminary representation of the prediction result; after the full connection layer, normalization processing is performed, and the feature expression is further enhanced by combining the activation function and the adjustment parameter to generate the final result and output by the output layer:

[0027]

[0028] wherein O final represents the final output, Z represents the normalization constant; M represents the scale number of the feature map; β m represents the weighting coefficient under different scales; σ represents the activation function; γ represents the adjustment parameter for controlling the amplification degree of the feature.

[0029] As a preferred scheme of the image-based fan blade aging detection method described in the application, wherein: the spectral feature enhancement module is a composite module, including a pooling layer, a full connection layer and a feature enhancement layer, the feature map of each spectral channel input into the spectral feature enhancement module is pooled to generate a group of values representing the global features of the current image:

[0030]

[0031] wherein G c represents the global feature value of the cth spectral channel; W represents the width of the feature map; H represents the height of the feature map; represents the feature map input into the spectral feature enhancement module; according to the global pooling result G c , the weight α c of each spectral channel is calculated by the full connection layer:

[0032] α c =σ(W c ·G c +b c )

[0033] wherein W c represents the weight; b c represents the bias;

[0034] The output F i,j,k of the spectral feature enhancement module is:

[0035]

[0036] wherein C represents the total number of spectral channels.

[0037] In a second aspect, the present application also provides an image-based fan blade aging detection system, comprising: an image acquisition module, which acquires a multi-spectral image of a fan blade by a multi-spectral camera and dynamically adjusts an acquisition frequency according to a speed and a position of the fan blade when a spectrum is switched; an image processing module, which performs image registration on the acquired multi-spectral image, aligns and fuses images of the same blade under different wavelengths, and generates a multi-spectral feature image; and an aging detection module, which establishes an aging detection model based on a deep convolutional neural network, enhances spectral features of the image by a spectral feature enhancement module, and identifies an aging area of the blade.

[0038] In a third aspect, the present application also provides a computing device, comprising: a memory and a processor;

[0039] The memory is configured to store computer executable instructions, and the processor is configured to execute the computer executable instructions, so as to implement steps of the image-based fan blade aging detection method.

[0040] In a fourth aspect, the present application also provides a computer readable storage medium, which stores computer executable instructions, and the computer executable instructions are executed by a processor, so as to implement steps of the image-based fan blade aging detection method.

[0041] The present application has the following beneficial effects: the method of the present application solves the problems of low image quality, inaccurate feature extraction and low detection efficiency in the fan blade aging detection in the prior art by the process of multi-spectral image acquisition, accurate image registration and fusion, aging detection based on deep learning and finally generating an aging report, and comprehensively improves the accuracy and automation degree of the fan blade aging detection, thereby providing strong technical support for efficient operation and maintenance of a wind farm. BRIEF DESCRIPTION OF DRAWINGS

[0042] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiment description. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can also be obtained by those skilled in the art without any creative effort on the basis of these drawings.

[0043] Figure 1 A flowchart of an image-based fan blade aging detection method provided by an embodiment of the present application. DETAILED DESCRIPTION

[0044] In order to make the above objectives, characteristics and advantages of the present application more obvious and easy to understand, the specific embodiments of the present application will be described in detail below with reference to the drawings of the specification. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor should fall within the scope of protection of the present application.

[0045] Embodiment 1, reference Figure 1 For an embodiment of the present application, an image-based detection method for fan blade aging is provided, comprising:

[0046] S1: Collecting a multi-spectral image of the fan blade, and dynamically adjusting the collection frequency according to the rotation speed and position of the blade during the collection process.

[0047] Further, the multi-spectral image is obtained by using a multi-spectral camera that switches spectra to collect images of the fan blade. The spectra used by the multi-spectral camera are visible light, near-infrared light, short-wave infrared light, mid-wave infrared light, and far-infrared light. After each spectrum switching, the collection time window is recalculated according to the rotation speed and position of the blade, and the collection frequency of the camera is adjusted.

[0048] Each light corresponds to a different aging condition. Visible light (400-700nm) detects surface cracks, obvious corrosion spots, and color changes; near-infrared light (700-1100nm) detects microscopic cracks, internal stress distribution, and moisture content changes in the material; short-wave infrared light (1100-2500nm) detects delamination of composite materials, degradation of coatings, and deep-seated cracks; mid-wave infrared light (3-5μm) detects thermal effects, fatigue damage, and deep structural abnormalities in the material; far-infrared light (8-14μm) detects thermal distribution, surface temperature abnormalities, and changes in thermal conductivity characteristics caused by long-term aging.

[0049] Considering that the data processing and analysis of the ultraviolet band are very complex, and the characteristics of ultraviolet light are more prominent only in certain cases, the cost and benefit of additional analysis of ultraviolet light in routine blade aging detection are not proportional, therefore, no ultraviolet spectral image collection is performed.

[0050] It should be noted that multi-spectral cameras are divided into full-spectrum cameras that simultaneously collect multiple spectral images and cameras that need to switch spectra. Although multi-channel cameras can simultaneously collect multiple spectral images, their cost is relatively high, while cameras that switch spectra have lower cost and are particularly suitable for projects that need to monitor long-term aging on multiple fans. Through reasonable image processing, cameras that switch spectra can also provide accurate multi-spectral data.

[0051] In terms of effect, the purpose of the application is to detect the aging of the fan instead of real-time failure, the aging of the fan blade is a slow process, and the required image acquisition does not need to complete the synchronous acquisition of different spectra in a very short time, and the aging detection focuses on the surface change of the blade in a period of time, instead of the instantaneous spectral response.

[0052] Therefore, the application selects a multi-spectral camera with switching spectrum for image acquisition, which can effectively reduce the cost, and the time synchronization is not strict, which has no effect on the fan blade aging detection, and the images under different spectra can be aligned and analyzed through post-processing.

[0053] Further, the angle and rotation speed of the blade are monitored in real time using a sensor, and the rotation period of the blade is calculated according to the rotation speed of the blade:

[0054]

[0055] Wherein, T blade represents the rotation period of the blade; and ω represents the rotation angular velocity of the blade.

[0056] According to the position and speed of the blade, the image acquisition time point in each cycle is set to ensure that the image covers the entire blade surface, first, the acquisition time interval Δt of each image is determined to ensure that the image can uniformly cover the entire blade surface during the rotation of the blade:

[0057]

[0058] Wherein, n images represents the number of images required to be collected in a cycle, which is set according to the coverage requirement of the blade surface, and is usually a fixed value.

[0059] According to the current position and angular velocity of the blade, the specific acquisition time point of each image is dynamically calculated, and the acquisition time point is associated with the rotation angle of the blade to ensure that the image is uniformly distributed on the entire blade surface, which is represented as:

[0060] t i =t0+i×Δt

[0061] θ i =ω×t i

[0062] Wherein, t i represents the acquisition time point of the i-th image; t0 represents the time point when the blade passes the spectral camera position for the first time; i represents the serial number of the image; and θ i represents the rotation angle of the blade when the i-th image is collected.

[0063] It should be noted that t0 is the time when the blade reaches the initial position detected by the sensor, and each subsequent collection point t i is increased by a time interval Δt on this basis. According to the blade position angle calibration collection time point, it can be ensured that the collection time point t i corresponds to the uniformly distributed angle θ on the blade surface i , so as to ensure that the entire blade surface is uniformly covered.

[0064] If the rotation speed of the blade changes (i.e. ω changes), the collection time points of subsequent images need to be dynamically adjusted:

[0065]

[0066] Where t i+1 is the collection time point of the i+1th image; Δθ represents the desired rotation angle difference between adjacent two images; ω current represents the current rotation angular velocity of the blade.

[0067] In general, ω will not change greatly in each collection period, so the adjustment of the collection time point is performed once after each spectrum switching. If ω changes by more than 10%, the collection time point is dynamically adjusted, otherwise the last setting is retained without change.

[0068] The preset spectrum order to be collected is automatically rotated according to the set spectrum order, and each spectrum is switched after three rotation periods T blade of the blade. According to the rotation period of the blade and the time required for spectrum switching, the base collection frequency f b is set:

[0069]

[0070] Where n i represents the number of images to be collected under each spectrum band; the collection frequency f d is dynamically adjusted according to the blade rotation speed when switching the spectrum:

[0071]

[0072] Where V represents the rotation speed of the blade; t switch represents the time required for the filter wheel to switch from one spectrum band to the next band.

[0073] For three periods of images collected for each spectrum, the one with the best overall quality based on clarity, contrast and noise indicators is selected as the final image for analysis.

[0074] The collected image is bound with the leaf angle at the time of collection and the collection timestamp, the collected multi-spectral image is stored according to the preset spectral order and collection time, and is sorted according to the leaf angle.

[0075] S2: Preprocessing the collected multi-spectral image, aligning and fusing the images of the same blade under different wavelengths through image registration to generate a multi-spectral feature image.

[0076] Image registration refers to aligning fan blade images under different wavelengths so that they coincide in space for subsequent image fusion and feature extraction. Due to differences in spectral bands of multi-spectral images, they may have some distortion and offset during imaging, so accurate registration is needed.

[0077] Further, in the multi-spectral image of the fan blade, images under different spectra may have geometric deformation, spectral differences, etc. In order to accurately register the images, first, stable and representative feature points need to be extracted from each image. Due to the difference in wavelengths between multi-spectral images, there may be scale, rotation, and illumination deformation. To deal with these deformations, a multi-scale feature extraction algorithm is used to ensure that stable feature points are extracted under different scales and rotations.

[0078] When performing image registration, first, the SIFT algorithm is used to extract initial feature points, the input multi-spectral image is subjected to multiple Gaussian blur processing, a Gaussian pyramid is constructed, the difference between adjacent scale images is calculated to find local extreme points as potential feature points, the potential feature points are located and the Harris response value is calculated, the gradient direction of the feature point neighborhood is calculated, and the direction with the largest gradient amplitude is selected as the main direction of the feature point, the feature descriptor is generated according to the direction information, and the feature point extraction is completed.

[0079] The Euclidean distance matching algorithm is used to introduce the similarity of spectral signals as an additional constraint condition to match the feature points of images under different spectra. For each feature point, the descriptor f i of the feature point is calculated, and the Euclidean distance between the descriptor f j of the feature point and the descriptor f j of all feature points under other spectra is calculated.

[0080] d ij =∥f i -f j ∥

[0081] where d ij represents the Euclidean distance between two feature points p i and p j .

[0082] The similarity of spectral signals is introduced as an additional constraint condition, in the matching process, not only the similarity of geometric features is considered, but also the spectral feature similarity S of the region where the feature points are located is calculated ij :

[0083]

[0084] Wherein, sigma represents a constant for controlling the similarity attenuation; g i Indicates the spectral feature vector of feature point p i Under different spectrums; g j Indicates the spectral feature vector of feature point p j Under different spectrums.

[0085] The final matching score M ij Is determined by the Euclidean distance and the spectral feature similarity, and is expressed as:

[0086]

[0087] Wherein, alpha represents a weight coefficient.

[0088] By using the random sample consensus algorithm, the false matching is removed from the preliminary matched point pairs, a minimum set of point pairs is randomly selected from the preliminary matched feature point pairs, an initial geometric transformation model is estimated based on these point pairs, for the remaining matched point pairs, the geometric transformation model is applied and their transformed positions are calculated, if the error between the transformed position and the actual position is within a preset threshold, the point pair is regarded as an 'inlier', the random sampling and model estimation are repeated for multiple times until a model containing the most inliers is found.

[0089] The traditional RANSAC algorithm only considers the geometric features, and the spectral feature similarity is introduced as a further optimization means in the present application. In the inlier judgment process, not only the geometric error is required to be within the threshold, but also the spectral feature similarity S ij Reaches a certain level, the reserved feature point pairs have high similarity in geometry and spectrum, and it is ensured that the matched point pairs are not only aligned in geometry, but also consistent in spectral information in the multi-spectral environment.

[0090] Based on the matched feature points, the multi-spectral images are preliminarily spatially registered, for the nonlinear deformation existing in the multi-spectral images, the matched feature point pairs are used, a thin plate spline transformation model is fitted by using the least square method, a preliminary transformation matrix is obtained, the calculated transformation matrix is used to preliminarily register the images, and the thin plate spline transformation model is expressed as:

[0091]

[0092] Wherein, T TPS(x, y) represents the output coordinates after applying the thin plate spline transformation to the input coordinates; N represents the number of feature points; w i represents the weight coefficient of the i-th control point; U represents the radial basis function; p represents the input coordinates (x, y); p i represents the coordinates of the i-th control point; a represents the parameters of the linear transformation.

[0093] The calculated transformation matrix is used to preliminarily register the image, and the source image is resampled according to the transformation matrix through bilinear interpolation to generate a registered image. For the boundary of the transformed image, an extension or cropping strategy is adopted to avoid loss of edge information.

[0094] On the basis of preliminary spatial registration, an iterative optimization algorithm based on a multi-resolution pyramid is adopted to gradually optimize the registration result by continuously refining the image resolution, and the registered multispectral image is fused to generate a multispectral feature image.

[0095] S3: Establish an aging detection model to process the multispectral feature image, automatically identify and label the aging area in the image, generate a leaf aging report, and complete the fan blade aging detection.

[0096] Further, the aging detection model is a neural network model established by combining a deep convolutional neural network with an attention mechanism. The multispectral feature image is processed through multi-layer convolution and feature extraction operations. The structure of the aging detection model includes an input layer, a convolutional layer, a spectral feature enhancement module, a multi-scale feature extraction module, a pooling layer, a fully connected layer, and an output layer. The input layer receives the multispectral fused feature image, and the convolutional layer uses different sizes of convolutional kernels for multi-scale feature extraction.

[0097] In a conventional convolutional layer, the feature maps of each spectral channel are processed by the same convolutional kernel, which lacks specialized optimization for different spectral features. This means that when processing multispectral images, features at different wavelengths may be treated the same, and certain spectral channel-specific information may not be highlighted. Therefore, a spectral feature enhancement module is established to analyze spectral features.

[0098] The spectral feature enhancement module is improved based on the convolutional layer 2 and is a composite module that includes a pooling layer, a fully connected layer, and a feature enhancement layer. After traditional convolutional operations, an adaptive mechanism is introduced to calculate weights for each spectral channel. The weights are dynamically adjusted according to the current image content to highlight features in specific spectral channels.

[0099] When the feature map is input, the pooling layer pools the feature map of each spectral channel input to the spectral feature enhancement module to generate a set of values representing the global features of the current image:

[0100]

[0101] where G c represents the global feature value of the c-th spectral channel; W represents the width of the feature map; H represents the height of the feature map; represents the feature map input into the spectral feature enhancement module.

[0102] According to the global pooling result G c , the weight α c of each spectral channel is calculated by a fully connected layer:

[0103] α c =σ(W c ·G c +b c )

[0104] where W c represents the weight; b c represents the bias.

[0105] The output F i,j,k of the spectral feature enhancement module is represented as:

[0106]

[0107] where C represents the total number of spectral channels.

[0108] Through the adaptive spectral weighting mechanism, the model can distinguish the performance of aging features between different spectral channels, especially in the spectral performance of features such as cracks and corrosion, significantly enhancing the differentiated expression between different spectral channels.

[0109] The multi-scale feature extraction module integrates the information from different scales by weighting and integrating the feature maps of different scales to obtain the final feature representation:

[0110]

[0111] where F final represents the final multi-scale feature representation; α l represents the weight coefficient of the l-th layer; L represents the total number of layers of the spectral feature enhancement module; F g represents the feature map after the l-th layer of the spectral feature enhancement module; A(l) represents the feature weighting factor of the l-th layer; ω represents the integration variable.

[0112] The adaptive pooling layer is used to reduce the size of the feature map, reduce the dimension of the feature map, and flatten the feature map into a one-dimensional vector. The flattened one-dimensional vector is input into the fully connected layer to generate a preliminary representation of the prediction result. After the fully connected layer, normalization processing is performed, and the feature expression is further enhanced by combining the activation function and the adjustment parameter to generate the final result and output through the output layer:

[0113]

[0114] wherein, O final represents the final output, Z represents a normalization constant; M represents the number of scales of the feature map; β m represents the weighting coefficient at different scales; σ represents an activation function; and γ represents a regulation parameter for controlling the amplification degree of the feature.

[0115] Further, a fully connected neural network (FCNN) is used as a classifier, and the output O final of the aging detection model is taken as the input of the classifier. The classifier includes several hidden layers, each layer applying an activation function ReLU, and the last layer outputs the classification result of the aging degree. A cross-entropy function is used as the loss function. According to the output result of the classifier, the aging degree of the blade is determined, and a corresponding aging report is generated.

[0116] The embodiment also provides an image-based detection system for the aging of a fan blade, which comprises an image acquisition module, a multi-spectral camera, an image processing module, and an aging detection module. The multi-spectral camera is used to acquire multi-spectral images of the fan blade, and the acquisition frequency is dynamically adjusted according to the speed and position of the fan blade when the spectrum is switched. The image processing module is used to perform image registration on the acquired multi-spectral images, align and fuse the images of the same blade at different wavelengths, and generate multi-spectral feature images. The aging detection module is used to establish an aging detection model based on a deep convolutional neural network, enhance the spectral features of the images by using a spectral feature enhancement module, and identify the aging area of the blade.

[0117] The embodiment also provides a computing device, which comprises a memory and a processor. The memory is used to store computer executable instructions, and the processor is used to execute the computer executable instructions to implement the image-based detection method for the aging of a fan blade as described in the above embodiment.

[0118] The embodiment also provides a storage medium having a computer program stored thereon. When the program is executed by a processor, the image-based detection method for the aging of a fan blade as described in the above embodiment is implemented.

[0119] The storage medium proposed in the embodiment belongs to the same inventive concept as the image-based detection method for the aging of a fan blade proposed in the above embodiment. The technical details not described in detail in the embodiment can be referred to the above embodiment, and the embodiment has the same beneficial effects as the above embodiment.

[0120] Through the above description of the embodiments, those skilled in the art can clearly understand that the present application can be realized by means of software and necessary general hardware, and of course can also be realized by hardware, but in many cases the former is a better embodiment. Based on such understanding, the technical solutions of the present application can be embodied in the form of a software product, which can be stored in a computer-readable storage medium, such as a floppy disk, a read-only memory (ROM), a random access memory (RAM), a FLASH, a hard disk or an optical disk, etc., and includes a number of instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute the methods of various embodiments of the present application.

[0121] The following is an embodiment of the present application, which provides an image-based detection method for fan blade aging. In order to verify the beneficial effects of the present application, a simulation experiment is conducted for scientific demonstration.

[0122] Ten fan blade samples collected from actual wind farms were used for simulation experiments, including blades with different aging degrees. A multispectral camera system was used to collect multispectral images of the fan blades, supporting five spectral bands (visible light, near-infrared, short-wave infrared, mid-wave infrared, and far-infrared).

[0123] First, the camera acquisition frequency was set according to the rotation speed and position of each blade. The rotation period of the fan blades was monitored in real time and fed back to the control system through sensors to dynamically adjust the acquisition frequency of the multispectral camera. The number of images collected under each spectral band was set to 30, and the spectral switching time was 0.2 seconds.

[0124] The multispectral image acquisition process adopted a spectral switching strategy, i.e., switching the spectrum every three fan blade rotation periods to ensure clear images under each spectral band. The entire acquisition process was completed within 20 minutes, and blade images under five spectral bands were collected. The collected spectral images were analyzed by the method of the present application, and the experimental data are shown in Table 1.

[0125] Table 1 Experimental data table

[0126]

[0127] Wherein 1-10 represents the number of fan blades, and wave band 1-4 represents the definition of the collected spectral image in the form of percentage. Wave band 1 corresponds to the visible light wave band, which captures the appearance characteristics of the blade under visible light, such as surface cracks and rust; wave band 2 corresponds to the near-infrared light wave band, which is used to detect the temperature change or moisture content of the blade; wave band 3 corresponds to the short-wave infrared light wave band, which is used to observe the internal structure or fine features of the blade surface; and wave band 4 corresponds to the mid-wave and far-infrared light wave band, which is used to detect thermal radiation or identify the specific chemical composition of the material.

[0128] The accuracy and error rate are the accuracy and error rate of the aging detection, which are used to verify the beneficial effects of the method of the application. As can be seen from the data analysis in the table, the image-based fan blade aging detection method of the application has shown significant advantages in image definition and aging detection accuracy. The definition data of spectral wave band 1 to spectral wave band 4 in the table are kept between 82.5% and 91.3%, indicating that the method of the application can obtain high-quality images under each spectral wave band, ensuring the reliability of subsequent image registration and fusion.

[0129] For the aging detection accuracy, the detection accuracy of all blades is above 90.5%, and the highest reaches 93.0%. This indicates that the model structure of the deep convolutional neural network combined with the spectral feature enhancement module used in the application can effectively extract the aging features in the multi-spectral image, thereby realizing high-precision blade aging detection. In addition, compared with the prior art, the application further improves the accuracy and robustness of the detection by introducing dynamic acquisition frequency adjustment and multi-resolution pyramid iterative optimization and other technical means, reduces the error rate, and makes the average error rate in the experiment as low as 3.0%.

[0130] In summary, the performance of the method of the application in the experiment fully proves its innovation and superiority. By comparing the shortcomings of the prior art, it can be concluded that the method of the application not only performs higher precision in multi-spectral image acquisition and processing, but also realizes higher detection accuracy and lower error rate in the aging detection model through technical means such as the spectral feature enhancement module, which has significant technical progress and practical value.

[0131] It should be noted that the above embodiments are only used to illustrate the technical solutions of the application and not to limit it. Although the application has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the application can be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the application, which should be covered in the scope of the claims of the application.

Claims

1. A method of image-based detection of fan blade aging, characterized in that, The method comprises the following steps: Collecting a multispectral image of a fan blade, dynamically adjusting the collection frequency according to the rotation speed and position of the blade during the collection process; Pretreating the collected multispectral image, aligning and fusing the images of the same blade under different wavelengths through image registration to generate a multispectral feature image; Establishing an aging detection model to process the multispectral feature image, automatically identifying and labeling the aging area in the image, generating a blade aging report, and completing the fan blade aging detection; The dynamic adjustment of the acquisition frequency comprises: monitoring the position and rotation speed of the blade in real time using a sensor, calculating the rotation period T of the blade according to the rotation speed of the blade blade , and setting the image acquisition time point in each period according to the position and speed of the blade. Pre-set the order of the spectrum to be collected, automatically rotate the filter wheel according to the set spectrum order, three blade rotation periods T for each spectrum blade After switching, set the basic collection frequency f according to the rotation period of the blade and the time required for spectrum switching b : where n i represents the number of images to be acquired at each spectral band; the acquisition frequency f d is dynamically adjusted according to the leaf rotation speed when switching the spectrum where V represents the rotational speed of the blades; t switch represents the time required for the filter wheel to switch from one spectral band to the next. For three cycles of images collected for each spectrum, select the one with the best overall quality based on the clarity, contrast and noise indicators as the final image for analysis; The collected image is bound with the blade angle and collection timestamp at the time of collection, and the collected multispectral image is stored according to the preset spectral order and collection time, and sorted according to the blade angle.

2. The image-based detection method of fan blade aging as claimed in claim 1, wherein: The multispectral image is obtained by using a multispectral camera with switched spectrum to collect images of the fan blade. The spectrum used by the multispectral camera is visible light, near-infrared light, short-wave infrared light, medium-wave infrared light and far-infrared light. After each spectrum switch, the collection time window is recalculated according to the rotation speed and position of the blade, and the collection frequency of the camera is adjusted.

3. The image-based detection method of fan blade aging as claimed in claim 2, wherein: The image registration comprises the following steps: extracting initial feature points through the SIFT algorithm, performing multiple Gaussian blur processing on the input multispectral image, constructing a Gaussian pyramid, finding local extreme points as potential feature points by calculating the difference between adjacent scale images, positioning the potential feature points and calculating the Harris response value, calculating the gradient direction of the neighborhood of the feature points, and selecting the direction with the largest gradient amplitude as the main direction of the feature points, generating a feature descriptor according to the direction information, and completing the feature point extraction; The Euclidean distance matching algorithm is adopted, and the similarity of the spectral signal is introduced as an additional constraint condition to match the feature points of the images under different spectrums; Based on the matched feature points, the multispectral images are preliminarily registered, and for the nonlinear deformation existing in the multispectral images, the matched feature point pairs are used to fit a thin plate spline transformation model through the least squares method to obtain a preliminary transformation matrix, and the transformation matrix is used to preliminarily register the images, and the thin plate spline transformation model is expressed as: where T TPS (x, y) represents the output coordinates after applying the thin plate spline transformation to the input coordinates; N represents the number of feature points; w i represents the weight coefficient of the i-th control point; U represents the radial basis function; p represents the input coordinates (x, y); p i represents the coordinates of the i-th control point; a represents the parameters of the linear transformation; On the basis of the preliminary spatial registration, an iterative optimization algorithm based on a multi-resolution pyramid is adopted, the image resolution is continuously refined, the registration result is optimized step by step, the registered multispectral images are fused, and a multispectral feature image is generated.

4. The image-based detection method of fan blade aging as claimed in claim 3, wherein: The feature point matching comprises: performing feature point matching by using an Euclidean distance matching algorithm; for each feature point, calculating an Euclidean distance between a descriptor f i of the feature point and all feature point descriptors f j under other spectrums. d ij =||f i -f j || where d ij represents p i and p j the Euclidean distance between two feature points; The similarity of the spectral signal is introduced as an additional constraint condition. In the matching process, not only the similarity of the geometric features is considered, but also the spectral feature similarity S of the region where the feature points are located is calculated ij : wherein σ represents a constant for controlling similarity degree attenuation;g i represents a spectral feature vector of the feature point p i under different spectrums;g j represents a spectral feature vector of the feature point p j under different spectrums; Final match score M ij is determined by both the Euclidean distance and the spectral feature similarity, and is expressed as: Wherein, α represents a weight coefficient; and the false matches are removed from the preliminary matched point pairs through the random sample consensus algorithm.

5. The image-based detection method of fan blade aging as claimed in claim 4, wherein: The aging detection model is a neural network model established by using a deep convolutional neural network combined with an attention mechanism, which processes the multispectral feature image through multiple convolution and feature extraction operations. The structure of the aging detection model comprises an input layer, a convolution layer, a spectral feature enhancement module, a multi-scale feature extraction module, a pooling layer, a full connection layer and an output layer. The input layer receives the multispectral fused feature image, and the convolution layer uses convolution kernels of different sizes for multi-scale feature extraction. The spectral feature enhancement module improves the convolutional layer and introduces an adaptive weight distribution mechanism of spectral features in the conventional convolution operation. The multi-scale feature extraction module integrates information from different scales by weighting and integrating feature maps of different scales to obtain the final feature representation. where F final represents the final multi-scale feature representation; a l represents the weight coefficient of the lth layer; L represents the total number of layers of the spectral feature enhancement module; F g represents the feature map after the lth layer of the spectral feature enhancement module; A(l) represents the feature weighting factor of the lth layer; ω represents the integral variable; The adaptive pooling layer is used to reduce the size of the feature map, reduce the dimension of the feature map, flatten the feature map into a one-dimensional vector, and then input the one-dimensional vector into the fully connected layer to generate a preliminary representation of the prediction result. wherein O final represents the final output, Z represents a normalization constant; M represents the number of scales of the feature map; β m represents the weighting coefficient at different scales; σ represents an activation function; and γ represents a regulation parameter for controlling the amplification degree of the features.

6. The image-based detection method of fan blade aging as claimed in claim 5, wherein: The spectral feature enhancement module is a composite module including a pooling layer, a fully connected layer, and a feature enhancement layer. wherein G c represents the global eigenvalue of the cth spectral channel; W represents the width of the feature map; H represents the height of the feature map; represents the feature map input into the spectral feature enhancement module; According to the global pooling result G c , the weight α c of each spectral channel is calculated by a fully connected layer: a c = σ(W c · G c + b c ) where W c represents a weight; b c represents a bias; The output F of the spectral feature enhancement module i,j,k is represented as: C represents the total number of spectral channels.

7. An image-based detection system for the ageing of fan blades, using the method according to any one of claims 1 to 6, characterised in that The image acquisition module acquires multispectral images of the fan blade through a multispectral camera and dynamically adjusts the acquisition frequency according to the speed and position of the fan blade during spectral switching. The image processing module performs image registration on the acquired multispectral images, aligns and fuses the images of the same blade under different wavelengths, and generates a multispectral feature image. The aging detection module establishes an aging detection model based on a deep convolutional neural network, enhances the spectral features of the image using the spectral feature enhancement module, and identifies the aging area of the blade. A memory and a processor; 8. A computing device comprising: The memory is configured to store computer executable instructions, and the processor is configured to execute the computer executable instructions to implement the steps of the method in any one of claims 1 to 6.

9. A computer readable storage medium storing computer executable instructions, which, when executed by a processor, implement the steps of the method in any one of claims 1 to 6. ​

Images