Multi-classification intelligent identification method and system for icing state of blade of wind driven generator

Abstract

The invention relates to the technical field of intelligent identification, in particular to a multi-classification intelligent identification method and system for the icing state of a blade of a wind driven generator. Comprising the steps that blade operation data are obtained through a multi-mode sensor array deployed on a wind driven generator, the multi-mode sensor array at least comprises an infrared thermal imager, a laser radar and a vibration acceleration sensor, and the infrared thermal imager is used for collecting blade surface temperature field distribution data; the laser radar is used for collecting blade surface contour point cloud data, and the vibration acceleration sensor is used for collecting blade vibration time domain signals; preprocessing the acquired multi-modal data, wherein the preprocessing comprises noise suppression, space-time alignment and normalization processing; according to the invention, multi-source information can be integrated, different icing degrees can be effectively distinguished, and high-precision multi-classification identification is realized.

Description

Technical Field

[0001] The present invention relates to the field of intelligent recognition technology, and in particular to a multi-classification intelligent recognition method and system for icing status of wind turbine blades. Background Art

[0002] In the field of wind power generation, accurately identifying the icing status of blades is key to ensuring the safe and efficient operation of the unit. Currently, wind turbine blade icing detection technology mainly relies on a single sensor to collect data, such as using a temperature sensor to monitor the blade surface temperature, or using a vibration sensor to analyze the blade vibration characteristics. However, the data information collected by a single sensor is limited in dimension and cannot fully reflect the complex state of blade icing. For example, judging the icing situation based solely on temperature changes is easily affected by ambient temperature fluctuations and may lead to misjudgment; relying solely on vibration signal analysis cannot accurately identify early mild icing phenomena. At the same time, the characteristic differences between different degrees of icing states in a single data dimension are not significant, making it difficult for existing technologies to accurately classify multiple states such as no icing, light icing, moderate icing, and heavy icing.

[0003] Based on the above problems, there is an urgent need for a technical solution that can integrate multi-source information, effectively distinguish different degrees of icing, and achieve high-precision multi-classification recognition to meet the urgent needs of the wind power industry for reliable detection of blade icing status. Summary of the Invention

[0004] The purpose of the present invention is to address the shortcomings of the prior art and to propose a multi-classification intelligent identification method for the icing status of wind turbine blades, comprising:

[0005] S1: Obtain blade operation data through a multimodal sensor array deployed on the wind turbine, wherein the multimodal sensor array includes at least an infrared thermal imager, a laser radar, and a vibration acceleration sensor, wherein the infrared thermal imager is used to collect blade surface temperature field distribution data, the laser radar is used to collect blade surface contour point cloud data, and the vibration acceleration sensor is used to collect blade vibration time domain signals;

[0006] S2: Preprocessing of the acquired multimodal data, including noise suppression, spatiotemporal alignment, and normalization;

[0007] S3: Extracting multidimensional features from the preprocessed multimodal data, the multidimensional features including temperature field gradient features, point cloud curvature features, and vibration signal time-frequency domain features, and performing dimensionality reduction and fusion on the multidimensional features using an adaptive feature selection algorithm to obtain key feature vectors for icing state identification;

[0008] S4: Construct a multi-classification recognition model based on a deep neural network. The deep neural network includes a feature encoding layer, an attention mechanism layer, and a classification decision layer. The deep neural network is trained using labeled icing state sample data. During the training process, a multi-task learning strategy is used to simultaneously optimize the classification loss and inter-class discrimination.

[0009] S5: Input the multimodal data acquired and preprocessed in real time into the trained deep neural network, and output the classification result of the current icing state of the blade, where the icing state includes at least four types: no icing, light icing, moderate icing, and heavy icing.

[0010] Preferably, the noise suppression processing specifically includes: median filtering processing on infrared thermal imaging temperature data, outlier detection and repair on lidar point cloud data, and wavelet denoising based on adaptive threshold on vibration acceleration signals; the time-space alignment processing is to unify the timestamps of data collected by different sensors into the wind turbine main shaft rotation period coordinate system, specifically through a synchronization trigger device and an interpolation algorithm to achieve time synchronization of multimodal data, and through blade three-dimensional modeling coordinate transformation to achieve spatial alignment; the normalization processing is to map the eigenvalues of each modal data to the [-1,1] interval, specifically using the Min-Max normalization method.

[0011] Further preferably, the method for extracting the temperature field gradient feature is: in the infrared thermal imaging temperature field image, calculating the gradient value of each pixel point in the horizontal and vertical directions, and constructing the temperature gradient matrix through the Sobel operator; the method for extracting the point cloud curvature feature is: performing local surface fitting on the lidar point cloud data, and calculating the Gaussian curvature and average curvature of each point; the method for extracting the time-frequency domain feature of the vibration signal is: converting the vibration time domain signal into a two-dimensional time-frequency image through short-time Fourier transform, and extracting the energy entropy, peak frequency and frequency band energy distribution characteristics of the time-frequency image.

[0012] Further preferably, the specific steps of the adaptive feature selection algorithm are: first, the mutual information value of each dimensional feature and the icing state label is calculated to obtain the initial feature importance ranking; then, through the feature subset search algorithm, a feature combination with a higher mutual information value and a lower mutual redundancy is selected within a preset feature dimension threshold, and the redundancy is calculated by the Pearson correlation coefficient. When the absolute value of the correlation coefficient of two features is greater than the preset threshold, the feature with the higher mutual information value is retained and the other feature is eliminated, and finally the key feature vector is obtained.

[0013] Further preferably, the calculation process of the attention mechanism layer of the deep neural network is as follows: Assume that the feature vector input to the attention mechanism layer is:

[0014]

[0015] Where n is the number of eigenvectors and d is the feature dimension; by querying the matrix Bond Matrix Sum Matrix Calculate attention weights:

[0016]

[0017] Among them, q i =H i Q is the query vector, k j =H j K is the key vector, α ij Represents the attention weight of the i-th eigenvector to the j-th eigenvector; the final output attention feature for: v j =H j In the formula, v j is a value vector, and the adaptive weighted fusion of different modal features is achieved through this attention mechanism.

[0018] Further preferably, the loss function of the multi-task learning strategy is constructed as follows: the classification loss function is defined as the cross entropy loss:

[0019]

[0020] Where N is the number of samples, C is the number of icing state categories, and y ic is the true label (0 or 1) of sample i belonging to category c, is the category probability predicted by the model; the inter-class discrimination loss function is defined as the triplet loss:

[0021]

[0022] Among them, f i is the feature encoding vector of sample i, is the feature encoding vector of the positive sample of the same type as sample i, is the feature encoding vector of negative samples of different classes from sample i, d(·,·) is the Euclidean distance, and m is the preset inter-class interval threshold; the total loss function is the weighted sum of the two:

[0023] L=L class +λL triplet ;

[0024] In the formula, λ is the balance coefficient, and the balance optimization of classification accuracy and inter-class discrimination is achieved by adjusting λ.

[0025] Further preferably, during the training process of the deep neural network, a transfer learning strategy is adopted, and the network is first pre-trained on a large-scale general mechanical equipment failure dataset, and then fine-tuned on a wind turbine blade icing state sample dataset; during the pre-training process, the classification decision layer is frozen, and only the parameters of the feature encoding layer and the attention mechanism layer are updated; during the fine-tuning process, the parameters of the entire network are optimized.

[0026] A multi-classification intelligent identification method for the icing state of a wind turbine blade, applicable to any of the above-described multi-classification intelligent identification methods for the icing state of a wind turbine blade, comprises a multimodal data acquisition module, a data preprocessing module, a feature extraction and fusion module, a multi-classification identification model, and a state output module. The multimodal data acquisition module comprises an infrared thermal imager, a laser radar, a vibration acceleration sensor, and a synchronous triggering device, and is used to collect blade surface temperature field data, contour point cloud data, and vibration signals in real time, and to achieve synchronous acquisition of multi-sensor data. The data preprocessing module comprises a noise suppression unit, a time-space alignment unit, and a normalization unit, respectively configured to perform noise removal, time-space alignment, and scale normalization on the multimodal data. The feature extraction and fusion module comprises a temperature field feature extraction unit, a point cloud feature extraction unit, a vibration signal feature extraction unit, and an adaptive feature selection unit, respectively configured to extract features from each modal data and perform fusion and dimensionality reduction. The multi-classification identification model comprises a model based on a deep neural network, and is used to classify the icing state of the fused key feature vectors. The state output module is used to visually display the classification results and transmit them to a wind turbine monitoring system via a communication interface.

[0027] Further preferably, the infrared thermal imager of the multimodal data acquisition module has a resolution of not less than 320×240 pixels, a temperature measurement accuracy of ±2°C, and a point cloud density of not less than 100 points / m 2 The measuring range of the vibration acceleration sensor is ±50g, and the sampling frequency is not less than 10kHz.

[0028] Further preferably, it also includes a data storage module and a model update module; the data storage module is used to store historically collected multimodal data, labeled icing state samples and trained deep neural network model parameters; the model update module regularly obtains new labeled data from the data storage module, performs incremental training on the multi-classification recognition model, and triggers a full retraining process when the model classification accuracy is lower than a preset threshold.

[0029] Technical effect:

[0030] The present invention uses a multimodal sensor array: infrared thermal imager, laser radar, vibration acceleration sensor to collect multi-dimensional data such as blade temperature field, contour point cloud, vibration signal, etc., to solve the problem of one-sided information of existing single sensor and comprehensively capture the characteristics of icing state. Noise suppression, spatiotemporal alignment and normalization processing in the pre-processing link ensure the reliability and consistency of multi-source data; multi-dimensional feature extraction is combined with an adaptive feature selection algorithm to effectively integrate key features such as temperature field gradient, point cloud curvature, vibration time and frequency domain, and avoid redundant information interference. The multi-classification model based on deep neural network realizes adaptive weighted fusion of multimodal features through the attention mechanism, and combines the multi-task learning strategy to optimize the classification loss and inter-class discrimination, significantly improving the recognition accuracy of different icing states, overcoming the technical bottleneck of the existing technology that is difficult to accurately distinguish multiple icing degrees, and providing a reliable state recognition solution for the safe operation of wind turbines. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 This is a flow chart of the multi-classification intelligent identification method for the icing status of wind turbine blades in this application;

[0032] Figure 2 This is the block diagram of the multi-classification intelligent recognition system for the icing status of wind turbine blades in this application. DETAILED DESCRIPTION

[0033] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0034] See also Figure 1 Conventional solutions have the following technical issues: icing on wind turbine blades significantly affects power generation efficiency and equipment safety. Existing identification methods suffer from insufficient single-modal data information, low recognition accuracy, and difficulty distinguishing between multiple icing states. Based on this, this embodiment provides a multi-classification intelligent identification method for wind turbine blade icing states, including:

[0035] S1: Obtain blade operation data through a multimodal sensor array deployed on the wind turbine, wherein the multimodal sensor array includes at least an infrared thermal imager, a laser radar, and a vibration acceleration sensor, wherein the infrared thermal imager is used to collect blade surface temperature field distribution data, the laser radar is used to collect blade surface contour point cloud data, and the vibration acceleration sensor is used to collect blade vibration time domain signals;

[0036] S2: Preprocessing of the acquired multimodal data, including noise suppression, spatiotemporal alignment, and normalization;

[0037] S3: Extracting multidimensional features from the preprocessed multimodal data, the multidimensional features including temperature field gradient features, point cloud curvature features, and vibration signal time-frequency domain features, and performing dimensionality reduction and fusion on the multidimensional features using an adaptive feature selection algorithm to obtain key feature vectors for icing state identification;

[0038] S4: Construct a multi-classification recognition model based on a deep neural network. The deep neural network includes a feature encoding layer, an attention mechanism layer, and a classification decision layer. The deep neural network is trained using labeled icing state sample data. During the training process, a multi-task learning strategy is used to simultaneously optimize the classification loss and inter-class discrimination.

[0039] S5: Input the multimodal data acquired and preprocessed in real time into the trained deep neural network, and output the classification result of the current icing state of the blade, where the icing state includes at least four types: no icing, light icing, moderate icing, and heavy icing.

[0040] It is worth mentioning that this embodiment proposes a complete multi-classification intelligent recognition method for the icing state of wind turbine blades. First, a multimodal sensor array is constructed, integrating infrared thermal imagers, lidars and vibration acceleration sensors to obtain blade operation data from multiple dimensions such as temperature field, surface profile, vibration, etc., providing a rich source of information for subsequent analysis. The data is then preprocessed, covering noise suppression, spatiotemporal alignment and normalization processing, to effectively improve data quality and availability. Multidimensional features are then extracted and fused for dimensionality reduction to obtain key feature vectors, reducing the data dimension while retaining core information. A deep neural network model consisting of a feature encoding layer, an attention mechanism layer and a classification decision layer is then constructed. Using labeled sample data, a multi-task learning strategy is adopted to train the model to enhance the model's ability to recognize different icing states. Finally, real-time data is input into the trained model to output accurate icing state classification results.

[0041] The technical effects achieved by the above-mentioned embodiments include: collecting data through a multimodal sensor array, overcoming the defects of one-sided information from a single sensor, and obtaining more comprehensive blade status information. The preprocessing of the data effectively removes noise interference, unifies the spatiotemporal scale of the data, and lays the foundation for accurate analysis. Multidimensional feature extraction and fusion, combined with an adaptive feature selection algorithm, can accurately capture key features related to the icing state and improve feature expression capabilities. The combination of deep neural networks and multi-task learning strategies significantly improves the classification accuracy and generalization ability of the model, and can accurately distinguish between various states such as no icing, light icing, moderate icing, and heavy icing, thereby helping operation and maintenance personnel to take timely measures to reduce downtime and equipment damage caused by icing, and improve the efficiency and safety of wind power generation.

[0042] For example, traditional technical solutions have the following technical problems: data collected by multimodal sensors often have noise interference, time asynchrony and spatial inconsistency, and large differences in data scales. These problems will seriously affect the accuracy of subsequent feature extraction and model recognition. Based on this, the noise suppression processing specifically includes: median filtering of infrared thermal imaging temperature data, outlier detection and repair of lidar point cloud data, and adaptive threshold-based wavelet denoising of vibration acceleration signals; the spatiotemporal alignment processing is to unify the timestamps of data collected by different sensors into the wind turbine main shaft rotation period coordinate system, specifically through a synchronous trigger device and an interpolation algorithm to achieve time synchronization of multimodal data, and through the three-dimensional modeling coordinate transformation of the blade to achieve spatial alignment; the normalization processing is to map the eigenvalues of each modal data to the [-1,1] interval, specifically using the Min-Max normalization method.

[0043] It is worth mentioning that this embodiment further refines the data preprocessing steps in the above embodiments. In terms of noise suppression, specific processing methods are used according to the characteristics of different sensor data, such as median filtering of infrared thermal imaging temperature data to effectively remove salt and pepper noise; outlier detection and repair of lidar point cloud data to ensure the integrity of point cloud data; and wavelet denoising based on adaptive thresholds for vibration acceleration signals to accurately remove noise while retaining signal characteristics. Spatiotemporal alignment processing uses a synchronous trigger device and interpolation algorithm to unify the multimodal data timestamps to the wind turbine main shaft rotation period coordinate system, and uses the blade three-dimensional modeling coordinate transformation to achieve spatial alignment to ensure data consistency in the spatiotemporal dimension. Normalization processing uses the Min-Max normalization method to map the eigenvalues of each modal data to the [-1,1] interval to eliminate data scale differences.

[0044] The technical effects achieved by the above embodiments include: precise noise suppression processing, effective removal of various types of noise, improved data purity, and making the data more realistically reflect the actual state of the blades. Spatiotemporal alignment processing ensures the consistency of multimodal data in time and space, facilitates multimodal data fusion and joint analysis, and gives full play to the advantages of multimodal data. Normalization processing eliminates data scale differences, avoids training deviations caused by data scale problems during model training, and improves model convergence speed and recognition accuracy. After these preprocessing steps, high-quality data is provided for subsequent feature extraction and model training, which ensures the accuracy and reliability of wind turbine blade icing status identification from the source.

[0045] For example, traditional technical solutions have the following technical problems: Accurately extracting features related to the icing state of the blade is the key to achieving high-precision recognition, but existing feature extraction methods are often unable to fully tap the effective information contained in multimodal data. Based on this, the method for extracting the temperature field gradient feature is as follows: in the infrared thermal imaging temperature field image, the gradient value of each pixel in the horizontal and vertical directions is calculated, and the temperature gradient matrix is constructed through the Sobel operator; the method for extracting the point cloud curvature feature is as follows: local surface fitting is performed on the lidar point cloud data, and the Gaussian curvature and average curvature of each point are calculated; the method for extracting the time-frequency domain features of the vibration signal is as follows: the vibration time domain signal is converted into a two-dimensional time-frequency image through short-time Fourier transform, and the energy entropy, peak frequency and frequency band energy distribution characteristics of the time-frequency image are extracted.

[0046] It is worth mentioning that: this embodiment further elaborates on the specific method of multi-dimensional feature extraction in the above embodiment. In the temperature field gradient feature extraction, the Sobel operator is used to calculate the horizontal and vertical gradient values of each pixel in the infrared thermal imaging temperature field image, and the temperature gradient matrix is constructed to obtain the change trend information of the temperature field. When extracting the point cloud curvature feature, the lidar point cloud data is locally fitted with a surface, and the Gaussian curvature and average curvature of each point are calculated to describe the geometric shape characteristics of the blade surface. The time-frequency domain feature extraction of the vibration signal converts the time domain signal into a two-dimensional time-frequency image through short-time Fourier transform, and then extracts features such as energy entropy, peak frequency and frequency band energy distribution, revealing the change law of the vibration signal in the time-frequency domain.

[0047] The technical effects achieved by the above embodiments include: temperature field gradient feature extraction can capture the changes in blade surface temperature. Icing will cause abnormal temperature distribution on the blade surface. This feature can effectively reflect this change and provide a basis for judging the icing state. The point cloud curvature feature describes the blade surface from the perspective of geometric shape. Icing will change the surface contour of the blade. By calculating the curvature feature, this change can be sensitively detected. The time-frequency domain feature extraction of the vibration signal analyzes the operating state of the blade from a dynamic perspective. Icing will affect the vibration characteristics of the blade. These time-frequency domain features can accurately capture vibration changes. Through these feature extraction methods, rich and effective feature information is obtained from multiple dimensions, which enhances the characterization ability of the blade icing state, improves the feature discrimination, provides high-quality feature input for subsequent icing state classification, and significantly improves the performance of the entire recognition system.

[0048] For example, the traditional technical solution has the following technical problems: although multi-dimensional feature extraction obtains rich information, too many features may be redundant, increase computational complexity, and may also introduce noise, affecting model performance. At the same time, not all features are equally important for icing state identification. This solution aims to solve how to automatically screen out the most representative and most critical features from the numerous extracted features, while reducing the data dimension and ensuring the effectiveness of the features in icing state identification, thereby improving the training efficiency and recognition accuracy of the model. Based on this, the specific steps of the adaptive feature selection algorithm are as follows: first, calculate the mutual information value between the features of each dimension and the icing state label to obtain the initial feature importance ranking; then, through the feature subset search algorithm, select feature combinations with higher mutual information values and lower mutual redundancy within the preset feature dimension threshold. The redundancy is calculated by the Pearson correlation coefficient. When the absolute value of the correlation coefficient of two features is greater than the preset threshold, the feature with the higher mutual information value is retained and the other feature is eliminated, and finally the key feature vector is obtained.

[0049] It is worth mentioning that this embodiment proposes an adaptive feature selection algorithm for dimensionality reduction and fusion of the multi-dimensional features extracted in the above embodiment. First, the mutual information value between each dimensional feature and the icing state label is calculated, and the feature importance is sorted according to the size of the mutual information, and the features with a high correlation with the icing state are screened out. Then, through the feature subset search algorithm, within the preset feature dimension threshold, the redundancy between features is calculated in combination with the Pearson correlation coefficient. When the absolute value of the correlation coefficient of two features is greater than the preset threshold, the features with high mutual information values are retained, and the redundant features are eliminated, and finally a key feature vector containing key information and independent of each other is obtained.

[0050] The technical effects achieved by the above-mentioned embodiments include: feature importance ranking based on mutual information, which can accurately measure the degree of correlation between each feature and the icing state, and give priority to retaining features that have important contributions to identification. Redundant features are removed in combination with the Pearson correlation coefficient, which avoids information duplication between features and makes the final key feature vector more streamlined and representative. Through this adaptive feature selection algorithm, the feature dimension is effectively reduced, the computational complexity and time cost of model training are reduced, and the training efficiency is improved. At the same time, after removing redundant and irrelevant features, the model can focus on key information, avoid overfitting, and enhance the generalization ability of the model, thereby further improving the accuracy and stability of multi-classification recognition of the icing state of wind turbine blades.

[0051] For example, traditional technical solutions have the following technical problems: During the multimodal data fusion process, different modal features contribute to different degrees to the recognition of ice conditions. Traditional fusion methods often treat all features equally, failing to fully utilize the role of key features, thus affecting recognition accuracy. Based on this, the calculation process of the attention mechanism layer of the deep neural network is as follows: Assume that the feature vector input to the attention mechanism layer is:

[0052]

[0053] Where n is the number of eigenvectors and d is the feature dimension; by querying the matrix Bond Matrix Sum Matrix Calculate attention weights:

[0054]

[0055] Among them, q i =H i Q is the query vector, k j =H j K is the key vector, α ij Represents the attention weight of the i-th eigenvector to the j-th eigenvector; the final output attention feature for: v j =H j In the formula, v j is a value vector, and the adaptive weighted fusion of different modal features is achieved through this attention mechanism.

[0056] The feature matrix of the input attention mechanism layer consists of n feature vectors h1,h2,…,h with dimension d n It is composed of feature information after multi-modal fusion of infrared thermal imaging, lidar, vibration signals, etc.

[0057] Trainable parameter matrices are used to map input features into query vectors, key vectors, and value vectors, respectively, where k and v are the dimensions after mapping, usually set according to the model structure.

[0058] q i =H i Q: The query vector corresponding to the i-th feature vector, used to "query" key information related to the current feature in the feature space.

[0059] k j =H j K: The key vector corresponding to the j-th feature vector, used as the "key" to match the relevance of the query vector.

[0060] α ij : The attention weight of the i-th eigenvector to the j-th eigenvector, indicating the degree of association between the two in the feature space.

[0061] This formula adopts the Scaled Dot-Product Attention mechanism, the core of which is to query the vector q i With key vector k j The dot product of is used to calculate the similarity, and Scaling is performed to avoid gradient vanishing. When k is large, the dot product result may be too large, causing the gradient after softmax to approach 0. The denominator normalizes the similarity of all feature vectors through the softmax function so that the sum of the weights is 1, ensuring that the weights of different features are comparable. In the identification of the icing state of wind turbine blades, multimodal features such as temperature field gradient, point cloud curvature, and vibration time-frequency domain features contribute differently to the icing state. For example, light icing may mainly affect the temperature field distribution, while heavy icing will significantly change the blade geometry and vibration characteristics. The attention mechanism adaptively calculates weights so that the model pays more attention to the features most relevant to the current icing state during fusion. For example, it gives higher weight to the point cloud curvature feature in heavy icing, suppresses irrelevant modal interference, and solves the defect that traditional fixed weight fusion methods cannot dynamically adapt to different icing states, thereby improving the pertinence of feature representation and recognition accuracy.

[0062] Attention feature output v j =H j V:

[0063] v j =H j V: The value vector obtained by mapping the j-th eigenvector through the value matrix V, which contains the original information used for feature fusion.

[0064] A i : The attention feature output of the i-th position is the vector of all values v j Based on the attention weight α ij The weighted sum of .

[0065] This formula realizes the adaptive fusion of multimodal features through weighted summation. Each value vector v j Carrying original feature information, attention weight α ij Determined v j For the current feature A icontribution. For example, when identifying moderate icing, the model may assign higher weight to the frequency band energy distribution characteristics of the vibration signal, while appropriately reducing the weight of the temperature field gradient characteristics. This dynamic weighting mechanism enables the model to automatically adjust the importance of each mode according to the specific characteristics of the input data, avoiding the dominance of a single mode or interference from irrelevant features, thereby constructing a more discriminative fusion feature representation. Compared with traditional splicing or average fusion methods, the advantage of the attention mechanism is that it focuses on key information: through end-to-end learning, the model can autonomously discover the core feature combination under different icing conditions, enhance the semantic connection between features, provide better input for the subsequent classification decision layer, and fundamentally improve the accuracy of multi-classification tasks.

[0066] It is worth mentioning that this embodiment describes the specific calculation process of the attention mechanism layer in the deep neural network. The input feature vector H is linearly transformed through the query matrix Q, key matrix K and value matrix V to obtain the query vector q i , key vector k j Sum value vector v j . Calculate the attention weight α using the query vector and key vector ij , normalized using the softmax function so that the sum of the weights is 1, thus measuring the degree of correlation between different feature vectors. Finally, the value vectors are weighted and summed according to the attention weights to obtain the output attention feature A. This achieves adaptive weighted fusion of features from different modalities and highlights features that are important for icing state recognition.

[0067] The technical effects achieved by the above-mentioned embodiments include: the attention mechanism layer can adaptively assign weights to different feature vectors by calculating attention weights, so that the model pays more attention to features closely related to the icing state and suppresses the influence of irrelevant or minor features. This adaptive weighted fusion method effectively integrates key information in multimodal data and enhances the model's ability to express complex icing state features. When processing multimodal data, it can give full play to the advantages of each modality and improve the model's ability to distinguish different icing states, thereby improving the accuracy and reliability of multi-classification recognition of icing states of wind turbine blades, so that the model can still accurately identify the corresponding icing state when faced with complex and changeable icing conditions.

[0068] For example, traditional technical solutions have the following technical problems: Traditional deep learning-based classification models usually only use a single classification loss function. When dealing with multi-classification problems, it is easy to have unclear distinctions between categories, resulting in low recognition accuracy for similar icing states. Based on this, the loss function of the multi-task learning strategy is constructed as follows: the classification loss function is defined as cross entropy loss:

[0069]

[0070] Where N is the number of samples, C is the number of icing state categories, and y ic is the true label (0 or 1) of sample i belonging to category c, is the category probability predicted by the model; the inter-class discrimination loss function is defined as the triplet loss:

[0071]

[0072] Among them, f i is the feature encoding vector of sample i, is the feature encoding vector of the positive sample of the same type as sample i, is the feature encoding vector of negative samples of different classes from sample i, d(·,·) is the Euclidean distance, and m is the preset inter-class interval threshold; the total loss function is the weighted sum of the two:

[0073] L=L class +λL triplet ;

[0074] In the formula, λ is the balance coefficient, and the balance optimization of 0 classification accuracy and inter-class discrimination is achieved by adjusting λ.

[0075] middle:

[0076] N: Number of training samples, that is, the total number of wind turbine blade data samples used for model training.

[0077] C: Icing status category number, such as no icing, light, moderate, and heavy, a total of 4 categories.

[0078] y ic : The true label of sample i belonging to category c (one-hot encoding, 1 if it belongs to the category, otherwise 0).

[0079] The model predicts the probability that sample i belongs to category c, which is output through the softmax layer and satisfies

[0080]

[0081] Cross entropy loss is a classic loss function for multi-classification tasks, which measures the difference between the model's predicted distribution and the true distribution. Close to the true label y icWhen , the loss value approaches 0; conversely, if the prediction is wrong, the loss value increases. In this embodiment, the loss function forces the model to learn the basic distinction boundaries between categories, such as the temperature field gradient threshold for distinguishing between no icing and light icing, the range of change in the point cloud curvature and other basic feature differences. However, a single cross-entropy loss has limitations: for samples with subtle differences between classes, such as the overlapping area of moderate and severe icing in the peak frequency of the vibration signal, it is difficult to effectively separate them by classification loss alone, which may cause the model to misjudge in critical states. Therefore, the patent introduces triplet loss as a supplement to construct a multi-task learning framework.

[0082] Triplet loss function:

[0083] f i : The feature encoding vector output by the feature encoding layer and the attention mechanism layer of sample i has a dimension of d′ (usually smaller than the original feature dimension d, which is the compressed discriminant feature).

[0084] The feature encoding vectors of positive samples similar to sample i are randomly selected or screened through the hard example mining strategy.

[0085] The feature encoding vectors of negative samples that are different from sample i must belong to different freezing state categories.

[0086] d(·,·): Euclidean distance, which calculates the distance between feature vectors in space.

[0087] m: The preset inter-class interval threshold (such as 0.5), which is used to force the inter-class distance to be greater than the intra-class distance m.

[0088] The core of triplet loss is to shorten the distance between samples of the same type and to extend the distance between samples of different types. For each sample i, its distance from the positive sample is required to be As small as possible, the distance from the negative sample As large as possible, and the difference between the two is at least m. When , the loss is 0, otherwise the loss is In the recognition of icing states, this loss function solves the shortcomings of cross-entropy loss: for example, moderate and severe icing may have partial overlap in temperature field and point cloud features, which are difficult to distinguish with traditional classification loss. However, triple loss forces the model to learn more discriminative features by enforcing inter-class intervals, such as the energy entropy difference of vibration signals, so that the feature encodings of different icing states form obvious clustering areas in space. This mechanism is especially effective for small sample scenarios. Even if the number of similar samples is small, the model's recognition ability of boundary samples can be enhanced through the difficult example mining strategy. The total loss function L = L class +λL triplet

[0089] λ: Balance coefficient, used to adjust the weights of classification loss and triplet loss, and the optimal value is determined through cross-validation.

[0090] The total loss function combines the accuracy of the classification task with the discrimination of metric learning. When λ is small, such as λ = 0.3, the model focuses more on classification accuracy and is suitable for processing samples with clear class boundaries. When λ is large, such as λ = 1.0, it enforces separation between classes and is suitable for optimizing scenarios with subtle differences between classes, such as light and moderate icing. Take actual wind turbine operating data as an example: when ice forms locally on the blade surface, infrared thermal imaging may only detect temperature anomalies in the edge area, while the curvature change in the lidar point cloud data is not obvious. In this case, the model may mistakenly classify it as non-icing. Using the triplet loss, the model learns that even if the curvature change is small, if the temperature gradient exceeds a threshold and high-frequency components are present in the vibration signal, it should still be classified as light icing, thereby capturing the joint discriminative pattern of multimodal features. This multi-task learning strategy enables the model to maintain high recognition rate even in complex operating conditions, such as progressive icing caused by low temperature and high humidity environments, significantly improving the robustness of the system.

[0091] It is worth mentioning that this embodiment constructs a loss function for a multi-task learning strategy. The classification loss function is defined as cross entropy loss, which is used to measure the difference between the model prediction results and the true label, and guide the model to learn the correct classification decision. At the same time, the triplet loss function is introduced. By calculating the distance between the sample feature encoding vector and the positive samples of the same class and the negative samples of different classes, the inter-class distance is maximized, the intra-class distance is minimized, and the model's ability to distinguish between different classes is enhanced. The classification loss and the triplet loss are summed according to a certain weighting coefficient λ to obtain the total loss function. During the training process, the classification accuracy and inter-class discrimination are balanced by optimizing the total loss function.

[0092] The technical effects achieved by the above embodiments include: the cross entropy loss function ensures that the model can accurately classify different icing states, so that the model prediction results are as close to the true label as possible. The triplet loss function further enhances the model's ability to learn the differences between different categories, and significantly improves the model's inter-class discrimination by widening the distance between different icing state categories and narrowing the distance between samples of the same icing state. By adjusting the weighting coefficient λ to balance the two loss functions, the model takes into account both classification accuracy and inter-class discrimination during training, effectively avoiding the model's misjudgment of certain similar icing states, improving the model's recognition accuracy and robustness for various icing states, and providing more reliable guarantees for accurate monitoring and timely warning of the icing state of wind turbine blades.

[0093] Traditional technical solutions have the following technical problems: training a high-precision deep neural network model usually requires a large amount of labeled data. However, in actual applications, labeled sample data for the icing status of wind turbine blades is often limited. Directly training the model is prone to overfitting, and the training time is long and inefficient. Based on this, the deep neural network training process adopts a transfer learning strategy. The network is first pre-trained on a large-scale general mechanical equipment fault dataset, and then fine-tuned on a sample dataset of wind turbine blade icing status. During the pre-training process, the classification decision layer is frozen, and only the parameters of the feature encoding layer and attention mechanism layer are updated. During the fine-tuning process, the parameters of the entire network are optimized.

[0094] It is worth mentioning that this embodiment proposes a transfer learning strategy during deep neural network training. The network is first pre-trained on a large-scale dataset of general mechanical equipment failures. During this stage, the classification decision layer is frozen, and only the parameters of the feature encoding layer and attention mechanism layer are updated, allowing the network to learn general mechanical equipment failure feature representations. The network is then fine-tuned on a dataset of wind turbine blade icing samples, freeing up optimization of all network parameters to adapt the network to the specific blade icing status recognition task. The knowledge learned during pre-training is leveraged to achieve rapid convergence, improving model training efficiency and performance.

[0095] The technical effects achieved by the above embodiments include: pre-training on a large-scale general mechanical equipment failure data set, so that the network learns a wealth of feature representations and knowledge related to mechanical equipment failures, which has certain universality and transfer value for the identification of the icing state of wind turbine blades. Pre-training by freezing the classification decision layer can prevent the network from over-adapting to the classification task of the general data set during the pre-training stage, and retain the network's ability to learn features. When fine-tuning on the sample data set of the icing state of wind turbine blades, based on the knowledge learned in pre-training, the network can adapt to new tasks more quickly, reduce dependence on a large amount of target data, and effectively improve training efficiency. At the same time, the transfer learning strategy reduces the risk of overfitting due to insufficient data, enhances the generalization ability of the model, and enables the model to accurately identify the icing state of wind turbine blades under limited data conditions, thereby improving the practicality and reliability of the model.

[0096] See also Figure 2, as the traditional technical solutions have the following technical problems: the simple identification method lacks a practical application carrier, and it is difficult to achieve real-time monitoring and management of the icing status of wind turbine blades. Based on this, the present embodiment provides a multi-classification intelligent identification system for the icing status of wind turbine blades, which is applied to the multi-classification intelligent identification method for the icing status of wind turbine blades as described in any one of the above, including a multi-modal data acquisition module, a data preprocessing module, a feature extraction and fusion module, a multi-classification recognition model and a state output module; the multi-modal data acquisition module includes an infrared thermal imager, a laser radar, a vibration acceleration sensor and a synchronous triggering device, which is used to collect blade surface temperature field data, contour point cloud data, vibration signals and realize synchronous acquisition of multi-sensor data in real time; the data preprocessing module includes noise suppression The unit, the spatiotemporal alignment unit and the normalization unit are respectively used to remove noise, align time and space and perform scale normalization on the multimodal data; the feature extraction and fusion module includes a temperature field feature extraction unit, a point cloud feature extraction unit, a vibration signal feature extraction unit and an adaptive feature selection unit, which are respectively used to extract the features of each modal data and perform fusion and dimensionality reduction; the multi-classification recognition model includes a model based on a deep neural network, which is used to classify the icing status of the fused key feature vectors; the status output module is used to display the classification results in a visual manner and send them to the wind turbine monitoring system through the communication interface.

[0097] It is worth mentioning that this embodiment proposes a multi-classification intelligent recognition system for wind turbine blade icing status. The system consists of a multimodal data acquisition module, a data preprocessing module, a feature extraction and fusion module, a multi-classification recognition model, and a status output module. The multimodal data acquisition module integrates multiple sensors and a synchronous trigger device to achieve real-time acquisition and synchronization of multimodal data. The data preprocessing module performs noise suppression, spatiotemporal alignment, and normalization on the collected data. The feature extraction and fusion module extracts features from each modal data and fuses them for dimensionality reduction. The multi-classification recognition model uses the deep neural network-based model described in the above embodiment to classify icing status. The status output module visualizes the classification results and sends them to the monitoring system.

[0098] The technical effects achieved by the above embodiments include: the multimodal data acquisition module ensures that the system can obtain comprehensive and accurate blade operation data, providing a basis for subsequent analysis. The data preprocessing module and the feature extraction and fusion module perform in-depth processing and feature optimization on the data, thereby improving the data quality and feature validity. The multi-classification recognition model based on deep neural networks ensures the recognition accuracy and reliability of the system. The status output module presents the classification results in a visual manner and sends them to the monitoring system in a timely manner, so that the operation and maintenance personnel can intuitively and quickly understand the blade icing status, facilitating timely response measures. The integrated design of the entire system realizes the full process automation from data acquisition to result application, improves the efficiency and accuracy of wind turbine blade icing status monitoring, and provides strong technical support for ensuring the safe and stable operation of wind power generation.

[0099] For example, traditional technical solutions have the following technical problems: the performance parameters of the sensor directly affect the quality and reliability of the collected data. If the sensor performance is insufficient, the collected data may not accurately reflect the actual state of the blade, resulting in deviations in subsequent feature extraction and state recognition. Based on this, the infrared thermal imager resolution of the multimodal data acquisition module is not less than 320×240 pixels, the temperature measurement accuracy is ±2°C, and the point cloud density of the laser radar is not less than 100 points / m 2 The measuring range of the vibration acceleration sensor is ±50g, and the sampling frequency is not less than 10kHz.

[0100] It is worth mentioning that this embodiment limits the sensor performance parameters of the multimodal data acquisition module in the above embodiment. It clearly stipulates that the resolution of the infrared thermal imager shall not be less than 320×240 pixels and the temperature measurement accuracy shall be ±2°C to ensure that the temperature field distribution on the blade surface can be clearly obtained and the temperature measurement is accurate; the point cloud density of the laser radar shall not be less than 100 points / m 2 , ensuring that the acquired blade surface contour point cloud data has sufficient details; the vibration accelerometer has a range of ±50g and a sampling frequency of not less than 10kHz, ensuring that the dynamic changes of the blade vibration signal can be accurately captured.

[0101] The technical effects achieved by the above embodiments include: limiting the resolution and temperature measurement accuracy of the infrared thermal imager, which can clearly and accurately obtain the blade surface temperature information, and help detect abnormal temperature changes caused by icing. Specifying the point cloud density of the lidar makes the acquired blade surface profile data more accurate and can better reflect the shape changes of the blade after icing. Clarifying the range and sampling frequency of the vibration acceleration sensor to ensure that the blade vibration signal can be recorded completely and accurately, and the vibration characteristic changes caused by icing can be captured. The limitation of these sensor performance parameters ensures that the collected multimodal data has high quality and reliability, and provides a solid data foundation for subsequent data processing, feature extraction and icing state identification, thereby improving the accuracy and stability of the entire intelligent identification system in identifying the icing state of wind turbine blades.

[0102] Traditional solutions, for example, face the following technical challenges: as wind turbine operating environments and conditions change, the characteristics of blade icing may change. If the model isn't updated promptly, its recognition accuracy will gradually decline, failing to meet actual application requirements. Meanwhile, effective storage and management of historical data and model parameters can help improve model training efficiency and optimize model performance.

[0103] Based on this, it also includes a data storage module and a model update module; the data storage module is used to store historically collected multimodal data, labeled icing state samples and trained deep neural network model parameters; the model update module regularly obtains new labeled data from the data storage module, and performs incremental training on the multi-classification recognition model. When the model classification accuracy is lower than the preset threshold, the full retraining process is triggered.

[0104] It is worth mentioning that, based on the above-mentioned embodiments, this embodiment adds a data storage module and a model update module to the intelligent recognition system. The data storage module is used to store historically collected multimodal data, annotated icing state samples, and trained deep neural network model parameters, providing data accumulation for model training and analysis. The model update module regularly obtains new annotated data from the data storage module and performs incremental training on the multi-classification recognition model. When the model classification accuracy falls below a preset threshold, a full retraining process is triggered to ensure that the model maintains high recognition performance.

[0105] The technical effects achieved by the above embodiments include: the historical data and labeled samples accumulated by the data storage module provide rich materials for model training, which helps the model learn more comprehensive icing state characteristics. The incremental training mechanism of the model update module enables the model to learn newly emerging icing state characteristics in a timely manner, continuously optimize the model performance, and improve the model's adaptability to environmental changes. The full retraining process triggered when the model accuracy drops can comprehensively optimize the model to avoid reduced recognition accuracy due to model aging. Through the collaborative work of the data storage module and the model update module, it is ensured that the multi-classification recognition model of the intelligent recognition system always maintains good performance, can accurately identify the icing state of wind turbine blades in different environments and working conditions, improves the long-term reliability and practicality of the system, and provides continuous protection for the stable operation of wind power generation.

[0106] The above are merely preferred embodiments of the present invention and do not limit the present invention in any other form. Any technician familiar with the profession may use the technical content disclosed above to change or modify it into an equivalent embodiment with equivalent changes and apply it to other fields. However, any simple modification, equivalent change and modification made to the above embodiment based on the technical essence of the present invention without departing from the content of the technical solution of the present invention shall still fall within the scope of protection of the technical solution of the present invention.

Claims

1. A multi-classification intelligent identification method for wind turbine blade icing status, characterized by: include: S1: Obtain blade operation data through a multimodal sensor array deployed on the wind turbine, wherein the multimodal sensor array includes at least an infrared thermal imager, a laser radar, and a vibration acceleration sensor, wherein the infrared thermal imager is used to collect blade surface temperature field distribution data, the laser radar is used to collect blade surface contour point cloud data, and the vibration acceleration sensor is used to collect blade vibration time domain signals; S2: Preprocessing of the acquired multimodal data, including noise suppression, spatiotemporal alignment, and normalization; S3: Extracting multidimensional features from the preprocessed multimodal data, the multidimensional features including temperature field gradient features, point cloud curvature features, and vibration signal time-frequency domain features, and performing dimensionality reduction and fusion on the multidimensional features using an adaptive feature selection algorithm to obtain key feature vectors for icing state identification; S4: Construct a multi-classification recognition model based on a deep neural network. The deep neural network includes a feature encoding layer, an attention mechanism layer, and a classification decision layer. The deep neural network is trained using labeled icing state sample data. During the training process, a multi-task learning strategy is used to simultaneously optimize the classification loss and inter-class discrimination. S5: Input the multimodal data acquired and preprocessed in real time into the trained deep neural network, and output the classification result of the current icing state of the blade, where the icing state includes at least four types: no icing, light icing, moderate icing, and heavy icing.

2. The multi-classification intelligent identification method for wind turbine blade icing status according to claim 1 is characterized in that: The noise suppression processing specifically includes: median filtering processing on infrared thermal imaging temperature data, outlier detection and repair of lidar point cloud data, and wavelet denoising based on adaptive threshold for vibration acceleration signals; the spatiotemporal alignment processing is to unify the timestamps of data collected by different sensors into the wind turbine main shaft rotation period coordinate system, specifically through a synchronous trigger device and an interpolation algorithm to achieve time synchronization of multimodal data, and through blade three-dimensional modeling coordinate transformation to achieve spatial alignment; the normalization processing is to map the eigenvalues of each modal data to the [-1,1] interval, specifically using the Min-Max normalization method.

3. The multi-classification intelligent identification method for wind turbine blade icing status according to claim 1 is characterized in that: The method for extracting the temperature field gradient feature is as follows: in the infrared thermal imaging temperature field image, the gradient value of each pixel in the horizontal and vertical directions is calculated, and the temperature gradient matrix is constructed by the Sobel operator; the method for extracting the point cloud curvature feature is as follows: local surface fitting is performed on the lidar point cloud data, and the Gaussian curvature and average curvature of each point are calculated; the method for extracting the time-frequency domain feature of the vibration signal is as follows: the vibration time domain signal is converted into a two-dimensional time-frequency image through short-time Fourier transform, and the energy entropy, peak frequency and frequency band energy distribution characteristics of the time-frequency image are extracted.

4. The multi-classification intelligent identification method for wind turbine blade icing status according to claim 1 is characterized in that: The specific steps of the adaptive feature selection algorithm are as follows: first, the mutual information value between each dimensional feature and the icing state label is calculated to obtain the initial feature importance ranking; then, a feature subset search algorithm is used to select feature combinations with high mutual information values and low mutual redundancy within a preset feature dimension threshold. The redundancy is calculated using the Pearson correlation coefficient. When the absolute value of the correlation coefficient of two features is greater than the preset threshold, the feature with the higher mutual information value is retained and the other feature is eliminated, ultimately obtaining the key feature vector.

5. The multi-classification intelligent identification method for wind turbine blade icing status according to claim 1 is characterized in that: The calculation process of the attention mechanism layer of the deep neural network is as follows: Assume that the feature vector input to the attention mechanism layer is: Where n is the number of eigenvectors and d is the feature dimension; by querying the matrix Bond Matrix Sum Matrix Calculate attention weights: Among them, q i =H i Q is the query vector, k j =H j K is the key vector, α ij Represents the attention weight of the i-th eigenvector to the j-th eigenvector; the final output attention feature for: Where, v j is a value vector, and the adaptive weighted fusion of different modal features is achieved through this attention mechanism.

6. The multi-classification intelligent identification method for wind turbine blade icing status according to claim 1 is characterized in that: The loss function of the multi-task learning strategy is constructed as follows: the classification loss function is defined as the cross entropy loss: Where N is the number of samples, C is the number of icing state categories, and y ic is the true label (0 or 1) of sample i belonging to category c, is the category probability predicted by the model; the inter-class discrimination loss function is defined as the triplet loss: Among them, f i is the feature encoding vector of sample i, f1 + is the feature encoding vector of the positive sample of the same type as sample i, f i - is the feature encoding vector of negative samples of different classes from sample i, d(·,·) is the Euclidean distance, and m is the preset inter-class interval threshold; the total loss function is the weighted sum of the two: L=L class +λL triplet ; In the formula, λ is the balance coefficient, and the balance optimization of 0 classification accuracy and inter-class discrimination is achieved by adjusting λ.

7. The multi-classification intelligent identification method for wind turbine blade icing status according to claim 1 is characterized in that: During the training process of the deep neural network, a transfer learning strategy is adopted. The network is first pre-trained on a large-scale general mechanical equipment fault dataset, and then fine-tuned on a sample dataset of wind turbine blade icing conditions. During the pre-training process, the classification decision layer is frozen, and only the parameters of the feature encoding layer and the attention mechanism layer are updated. During the fine-tuning process, the parameters of the entire network are optimized.

8. A wind turbine blade icing status multi-classification intelligent recognition system, applied to the wind turbine blade icing status multi-classification intelligent recognition method according to any one of claims 1 to 7, characterized in that: It includes a multimodal data acquisition module, a data preprocessing module, a feature extraction and fusion module, a multi-classification recognition model and a state output module; the multimodal data acquisition module includes an infrared thermal imager, a laser radar, a vibration acceleration sensor and a synchronous trigger device, which is used to collect blade surface temperature field data, contour point cloud data, vibration signals in real time and realize the synchronous acquisition of multi-sensor data; the data preprocessing module includes a noise suppression unit, a time-space alignment unit and a normalization unit, which are respectively used to remove noise, align time and space and scale normalization processing on multimodal data; the feature extraction and fusion module includes a temperature field feature extraction unit, a point cloud feature extraction unit, a vibration signal feature extraction unit and an adaptive feature selection unit, which are respectively used to extract features of each modal data and perform fusion and dimensionality reduction; the multi-classification recognition model includes a model based on a deep neural network, which is used to classify the icing state of the fused key feature vectors; the state output module is used to display the classification results in a visual manner and send them to the wind turbine monitoring system through the communication interface.

9. The wind turbine blade icing status multi-classification intelligent identification system according to claim 8, characterized in that: The infrared thermal imager of the multimodal data acquisition module has a resolution of no less than 320×240 pixels, a temperature measurement accuracy of ±2°C, and a point cloud density of no less than 100 points / m 2 The measuring range of the vibration acceleration sensor is ±50g, and the sampling frequency is not less than 10kHz.

10. The wind turbine blade icing status multi-classification intelligent identification system according to claim 8, characterized in that: It also includes a data storage module and a model update module; the data storage module is used to store historically collected multimodal data, labeled icing state samples and trained deep neural network model parameters; the model update module regularly obtains new labeled data from the data storage module, and performs incremental training on the multi-classification recognition model. When the model classification accuracy is lower than a preset threshold, a full retraining process is triggered.

Images