Multi-sensor fusion and image recognition combined blade fault detection method and system

Abstract

The invention discloses a multi-sensor fusion and image recognition combined blade fault detection method, which is applied to a multi-sensor fusion and image recognition combined blade fault detection system arranged outside a fan blade. The system at least comprises a signal acquisition unit formed by a plurality of groups of sensors, a data processing system and a signal fusion module, wherein the signal acquisition unit at least comprises a vibration sensor, an acoustic emission sensor, an electromagnetic sensor and an image data sensor. According to the method, data during blade operation are obtained through real-time signals, extracted features are obtained after processing, faults are recognized through the feature fusion and image recognition technology, and risk rating is carried out, so that the problem that a traditional fan blade fault detection method usually depends on a sensor to carry out data collection, but the methods have obvious limitation in application is solved. Complex damage characteristics of fan blades are difficult to comprehensively capture by using a sensor. And particularly, when the damage degree is evaluated, the performance is particularly insufficient.

Description

Technical Field

[0001] The present invention belongs to the technical field of fan blade fault diagnosis, and in particular relates to a blade fault detection method and system combining multi-sensor fusion with image recognition. Background Art

[0002] As one of the key components of wind power generation equipment, the operating status of wind turbine blades directly affects the power generation efficiency and equipment safety. However, wind turbine blades are exposed to harsh environments for a long time and are easily affected by external factors such as wind, sand, rain, temperature changes and lightning, resulting in damage, cracks and other faults. Therefore, how to effectively detect and diagnose wind turbine blade faults and perform relevant repairs according to the fault type has become an important problem in wind power operation and maintenance.

[0003] Traditional wind turbine blade fault detection methods usually rely on sensors for data collection, but these methods have obvious limitations in application. It is difficult to fully capture the complex damage characteristics of wind turbine blades using sensors. Especially when assessing the degree of damage, the performance is particularly insufficient. Summary of the invention

[0004] The purpose of the present invention is to provide a blade fault detection method and system combining multi-sensor fusion and image recognition, so as to identify faults and evaluate their severity, obtain data of blade operation through real-time signals, extract features after processing, and solve the above technical problems by identifying faults and risk rating through feature fusion and image recognition technology.

[0005] In order to solve the above technical problems, the technical solution of the present invention is: a blade fault detection method combining multi-sensor fusion and image recognition is applied to a blade fault detection system combining multi-sensor fusion and image recognition, which is arranged outside the wind turbine blade, and the system at least includes a signal acquisition unit formed by multiple groups of sensors, a data processing system and a signal fusion module; wherein the signal acquisition unit at least includes a vibration sensor, an acoustic emission sensor, an electromagnetic sensor and an image data sensor; the method includes the following steps:

[0006] S1. Collect image data, vibration data, acoustic emission data and electromagnetic data through a signal acquisition unit and perform feature extraction, perform aggregation operations on each data feature and assign weights to each feature, and then perform feature fusion on each data feature to form a data set;

[0007] S2, build a model based on the ultrametric covariance structure, randomly divide the data set into several clusters, and initialize the model parameters;

[0008] S3, determine the constraints of the model;

[0009] S4, estimating model parameters by EM algorithm, and updating model parameters to maximize the log-likelihood function;

[0010] S5, repeat S4 until the increment of the log-likelihood function is less than a preset threshold, and convergence is completed;

[0011] S6. After convergence, the data samples are classified using the preset responsibility values, and each data sample is assigned to the cluster with the maximum responsibility value;

[0012] S7. Match the final cluster center with the preset risk level, divide the corresponding data samples, and complete the risk prediction.

[0013] The specific steps of aggregating each data feature in S1 are:

[0014] Constructing a neighborhood graph p in high-dimensional space using the UMAP method ij , the similarity between the corresponding data points is expressed by fuzzy set theory. Assuming that two points x i and x j , expressed as:

[0015]

[0016] Among them, d(x i , x j ) is the point x i and x j The distance between i It is point x i The local density estimate is used to adjust the distance to make it more adaptable to the local structure; σ i is the scale parameter, which is used to control the neighborhood size;

[0017] Constructing a neighborhood graph q in low-dimensional space using the UMAP method ij , expressed as:

[0018]

[0019] Among them, point y i and j is a data point in low-dimensional space; d(y i ,y j ) is the midpoint y in space i With point y j The Euclidean distance between them; α is the scaling parameter used to control the distance scaling;

[0020] The optimization goal of the UMAP method is to minimize the cross entropy between two fuzzy sets so that the neighborhood relationship in the high-dimensional space and the low-dimensional space remains consistent, thereby achieving data dimensionality reduction. The optimization goal is expressed as:

[0021]

[0022] Its reason is the loss function, and the aggregation operation is achieved through repeated iterative optimization.

[0023] In S1, each data feature after the aggregation operation is weighted by the standard effect removal method.

[0024] The supermetric covariance structure in S2 is expressed as:

[0025] The ultrametric covariance structure ∑ g Decompose into within-group and between-group:

[0026]

[0027] Among them, V g is a p×m matrix used to divide the variables into m groups (where m≤p); element v jq Determine whether the jth variable belongs to the qth group; is within-group, indicating the difference between variables in the same group; is the between-group difference, which indicates the difference between different groups; is the group variance, which represents the variance of each group.

[0028] The constraints in S3 include:

[0029] Ultrametric Inequality: Between Groups The off-diagonal elements of must satisfy the triangle inequality:

[0030]

[0031] Positive definiteness constraint: To ensure that ∑ g is positive, within the group The smallest diagonal element of must be greater than the inter-group The largest off-diagonal element of , that is:

[0032]

[0033] Variance Constraints: Group Variance Each diagonal element of must be greater than the The absolute value of is:

[0034]

[0035] Among them, σ qh (B) represents the inter-group difference matrix The element in the qth row and hth column, σ qs (B) and σ hs(B) represents the covariances corresponding to other rows and columns in the matrix. Here q, h, and s are the row and column indices of the matrix, which are used to represent different elements.

[0036] The method for estimating model internal parameters using the EM algorithm in S4 is:

[0037] Calculate the posterior probability that the data sample belongs to each cluster, that is, the responsibility value γ ng , defined as:

[0038]

[0039] in, Represents the g-th Gaussian component for sample x n The probability density function of g and∑ g are the mean vectors of the g-th component respectively; k represents the k-th Gaussian cluster.

[0040] The model parameters are updated to maximize the log-likelihood function, and the grouped ultrametric covariance structure is updated as:

[0041]

[0042] The features extracted from image data include at least corner points, oriented gradient histograms, local binary patterns, scale-invariant features and sparsity; the features extracted from vibration data include at least vibration frequency, amplitude spectrum, peak value and root mean square; the features extracted from acoustic emission data include at least maximum amplitude, average amplitude and energy; the features extracted from electromagnetic data include at least amplitude change and spectrum characteristics.

[0043] A blade fault detection system combining multi-sensor fusion and image recognition is also provided. The system is arranged outside the wind turbine blade. The system at least includes a signal acquisition unit formed by multiple groups of sensors, a data processing system and a signal fusion module; the signal acquisition unit at least includes a vibration sensor, an acoustic emission sensor, an electromagnetic sensor and an image data sensor; wherein,

[0044] The vibration sensor is a laser Doppler vibration sensor, which is specifically arranged outside the cabin to monitor the vibration behavior during operation, and is used to collect vibration data and transmit it to the signal data processing system;

[0045] An acoustic emission sensor is specifically arranged at the root of the blade or a key part that bears a large load, and is used to detect acoustic emission data caused by cracks, fatigue and delamination. It is used to collect acoustic emission data and transmit it to a signal data preprocessing module;

[0046] Electromagnetic sensors are specifically arranged at the root of the blade or at key locations prone to structural defects, and are used to detect electromagnetic data caused by cracks, corrosion, and metal fatigue, and are used to collect electromagnetic data and transmit them to a signal data preprocessing module;

[0047] The image data sensor is a camera, which is specifically arranged on the outside of the cabin and is used to detect surface cracks and damage.

[0048] A computer device is also provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of any of the above methods when executing the computer program.

[0049] A computer-readable storage medium is also provided, on which a computer program is stored, and when the computer program is executed by a processor, the steps of any of the above methods are implemented.

[0050] Compared with the prior art, the present invention has the following beneficial effects:

[0051] The present invention collects the operation data of the fan blades in real time to more accurately reflect the actual operation of the fan. By eliminating interference data, extracting features, and fusing multi-dimensional features, the present invention uses advanced fault identification algorithms to analyze the fused features and evaluate the health of the fan. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Figure 1 It is a schematic diagram of a flow chart of an embodiment of the present invention;

[0053] Figure 2 Schematic diagram of the arrangement positions of the sensors in the embodiment of the present invention;

[0054] In the figure, 201 is the setting position of the laser Doppler vibration sensor, 202 is the setting position of the acoustic emission sensor, 203 is the setting position of the electromagnetic sensor, and 204 is the setting position of the camera. DETAILED DESCRIPTION

[0055] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention. In addition, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0056] The technical solution of the present invention is: a fan blade fault detection system based on the combination of multi-sensor fusion and image recognition, such as Figure 2 As shown, it includes a signal acquisition unit, a data processing system, and a signal fusion module, including:

[0057] Laser Doppler vibration sensor, installed outside the cabin, monitors vibration behavior during operation. It is responsible for collecting vibration signals and transmitting them to the signal data processing system.

[0058] Acoustic emission sensors are installed at the root of the blade or at key locations that are subject to heavy loads. AE can detect acoustic emission signals caused by cracks, fatigue, and delamination. It is responsible for collecting acoustic emission signals and transmitting them to the signal data preprocessing module.

[0059] Electromagnetic sensors are installed at the root of the blade or at key locations prone to structural defects. Electromagnetic sensors can detect electromagnetic signals caused by cracks, corrosion and metal fatigue. They are responsible for collecting electromagnetic signals and transmitting them to the signal data preprocessing module.

[0060] The camera is installed on the outside of the cabin and is suitable for detecting surface cracks, damage and other problems.

[0061] S1. The collected data will be transmitted to the signal data processing system through the 5G network. Different methods are used to process different signals.

[0062] The present invention firstly extracts relevant features from image data, and extracts a series of features such as corner points, histogram of oriented gradients HOG, local binary pattern LBP, scale invariant feature SIFT and sparseness.

[0063] Vibration features include frequency domain features, such as vibration frequency (VF) and amplitude spectrum (AS), peak value (PV) and root mean square (RMS). For acoustic emission (AE), the main focus is on maximum amplitude (MA), average amplitude (AA) and energy (SE), while electromagnetic signals extract amplitude change (AC) and frequency spectrum features (SF).

[0064] Considering that there are many selected features and the types of data collected vary greatly, it is necessary to perform dimensionality reduction processing on each feature data before feature fusion. The present invention adopts the UMAP method to perform aggregation operations on different sensor-related features. The specific steps are as follows:

[0065] Constructing a neighborhood graph p in high-dimensional space ij , using fuzzy set theory to represent the similarity between data points. Given two points x i and x j , expressed by the following formula:

[0066]

[0067] Where d(x i , x j ) is the point x i and xj The distance between i It is point x i The local density estimate of σ is used to adjust the distance to make it more adaptive to the local structure. i is a scale parameter that controls the size of the neighborhood.

[0068] In low-dimensional space, UMAP also constructs a neighborhood graph q ij , defined by the following formula:

[0069]

[0070] Where d(y i ,y j ) is the Euclidean distance between points in space. Point y i and j is a data point in low-dimensional space. The parameter α controls the distance scaling.

[0071] The goal of UMAP is to keep the neighborhood relationship in high-dimensional and low-dimensional space consistent by minimizing the cross entropy between two fuzzy sets, thereby achieving data dimensionality reduction. The optimization goal can be expressed as:

[0072]

[0073] Its reason is the loss function. Through repeated iterative optimization, UMAP finally obtains the representation of the data.

[0074] The processed signal features and image feature data are transmitted to the data fusion module through optical fiber for risk rating operation. Before analysis, the extracted features are first weighted. The weighting method based on standard removal effect (MEREC) is used. The following steps are used to calculate the target weight:

[0075] Step 1: Normalize the decision matrix (), using simple linear normalization to scale the elements

[0076]

[0077] Step 2: Calculate the overall effect of the alternatives

[0078]

[0079] Step 3: Calculate the performance of the alternatives by removing each criterion.

[0080]

[0081] Step 4: Calculate the sum of the absolute deviations to determine the final weight of the criteria.

[0082]

[0083] After finally obtaining the weights of various features, the next step of fusion can be carried out.

[0084] The present invention utilizes the extended ultrametric covariance structure (EUCovS) to reduce the parameter complexity of the model. Figure 1 As shown, the specific steps of this method are as follows:

[0085] S2. Randomly divide the data set into k clusters and randomly initialize the parameters, namely the mean, covariance matrix and mixing coefficients (weights).

[0086] The ultrametric structure ∑ g , decomposed into within-group and between-group:

[0087]

[0088] Among them, V g is a p×m matrix used to divide the variables into m groups (where m≤p), and the element v jq Determine whether the j-th variable belongs to the q-th group. is within-group, indicating the differences between variables in the same group. is the between-group difference, which indicates the difference between different groups. is the group variance, which represents the variance of each group.

[0089] S3. Determine the constraints of the algorithm. In order to ensure ultrametricity and positive definiteness, EUCovS needs to meet the following constraints:

[0090] Ultrametric Inequality: Between Groups The off-diagonal elements of must satisfy the triangle inequality:

[0091]

[0092] Positive definiteness constraint: To ensure that ∑ g is positive, within the group The smallest diagonal element of must be greater than the inter-group The largest off-diagonal element of , that is:

[0093]

[0094] Variance Constraints: Group Variance Each diagonal element of must be greater than the The absolute value of is:

[0095]

[0096] Among them, σ qh (B) represents the inter-group difference matrix The element in the qth row and hth column, σ qs (B) and σ hs (B) represents the covariances corresponding to other rows and columns in the matrix. Here q, h, and s are the row and column indices of the matrix, which are used to represent different elements.

[0097] S4. Parameter estimation is achieved using the EM algorithm. Each step of the algorithm must also consider the ultrametric and positive definiteness constraints.

[0098] Calculate the posterior probability (i.e. responsibility value) γ that the sample belongs to each cluster ng , defined as:

[0099]

[0100] Its reason Represents the g-th Gaussian component for sample x n The probability density function, μ g and∑ g are the mean vectors of the gth components respectively. k represents the kth Gaussian cluster.

[0101] Update the model parameters to maximize the log-likelihood function. Update using the grouped EUCovS structure:

[0102]

[0103] S5. Repeat S4 until the increment of the log-likelihood function is less than a preset threshold.

[0104] S6. After convergence, use the responsibility value γ ng Classify the samples and convert each sample x n Assigned to the cluster g with the largest responsibility value.

[0105] S7. Match the final cluster center with the risk level, divide the sample points, and complete the risk prediction.

[0106] When the data processing server detects a fault in a wind turbine blade, the system immediately sends out a warning signal to notify the relevant staff, who will then determine whether maintenance measures are needed based on the severity of the alarm information.

[0107] It will be easily understood by those skilled in the art that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A blade fault detection method combining multi-sensor fusion and image recognition, characterized in that: A blade fault detection system combining multi-sensor fusion and image recognition is applied, the system is arranged outside the wind turbine blade, the system at least includes a signal acquisition unit formed by multiple groups of sensors, a data processing system and a signal fusion module; wherein the signal acquisition unit at least includes a vibration sensor, an acoustic emission sensor, an electromagnetic sensor and an image data sensor; the method comprises the following steps: S1. Collect image data, vibration data, acoustic emission data and electromagnetic data through a signal acquisition unit and perform feature extraction, perform aggregation operations on each data feature and assign weights to each feature, and then perform feature fusion on each data feature to form a data set; S2, build a model based on the ultrametric covariance structure, randomly divide the data set into several clusters, and initialize the model parameters; S3, determine the constraints of the model; S4, estimating model parameters by EM algorithm, and updating model parameters to maximize the log-likelihood function; S5, repeat S4 until the increment of the log-likelihood function is less than a preset threshold, and convergence is completed; S6. After convergence, the data samples are classified using the preset responsibility values, and each data sample is assigned to the cluster with the maximum responsibility value; S7. Match the final cluster center with the preset risk level, divide the corresponding data samples, and complete the risk prediction.

2. The blade fault detection method combining multi-sensor fusion and image recognition according to claim 1 is characterized in that: The specific steps of aggregating each data feature in S1 are: Constructing a neighborhood graph p in high-dimensional space using the UMAP method ij , the similarity between the corresponding data points is expressed by fuzzy set theory. Assuming that two points x i and x j , expressed as: Among them, d(x i , x j ) is the point x i and x j The distance between i It is point x i The local density estimate is used to adjust the distance to make it more adaptable to the local structure; σ i is the scale parameter, which is used to control the neighborhood size; Constructing a neighborhood graph q in low-dimensional space using the UMAP method ij , expressed as: Among them, point y i and j is a data point in low-dimensional space; d(y i ,y j ) is the midpoint y in space i With point y j The Euclidean distance between them; α is the scaling parameter used to control the distance scaling; The optimization goal of the UMAP method is to minimize the cross entropy between two fuzzy sets so that the neighborhood relationship in the high-dimensional space and the low-dimensional space remains consistent, thereby achieving data dimensionality reduction. The optimization goal is expressed as: in, is the loss function, and the aggregation operation is achieved through repeated iterative optimization.

3. The blade fault detection method combining multi-sensor fusion and image recognition according to claim 1 is characterized in that: In S1, each data feature after the aggregation operation is weighted by the standard effect removal method.

4. The blade fault detection method combining multi-sensor fusion and image recognition according to claim 1 is characterized in that: The supermetric covariance structure in S2 is expressed as: The ultrametric covariance structure ∑ g Decompose into within-group and between-group: Among them, V g is a p×m matrix used to divide the variables into m groups (where m≤p); element v jq Determine whether the jth variable belongs to the qth group; is within-group, indicating the difference between variables in the same group; is the between-group difference, which indicates the difference between different groups; is the group variance, which represents the variance of each group.

5. The blade fault detection method combining multi-sensor fusion and image recognition according to claim 4 is characterized in that: The constraints in S3 include: Ultrametric Inequality: Between Groups The off-diagonal elements of must satisfy the triangle inequality: Positive definiteness constraint: To ensure that ∑ g is positive, within the group The smallest diagonal element of must be greater than the inter-group The largest off-diagonal element of , that is: Variance Constraints: Group Variance Each diagonal element of must be greater than the The absolute value of is: Among them, σ qh (B) represents the inter-group difference matrix The element in the qth row and hth column, σ qs (B) and σ hs (B) represents the covariance corresponding to other rows and columns in the matrix respectively; q, h, s represent the row and column indices of the matrix, which are used to represent different elements.

6. The blade fault detection method combining multi-sensor fusion and image recognition according to claim 5 is characterized in that: The method for estimating model internal parameters using the EM algorithm in S4 is: Calculate the posterior probability that the data sample belongs to each cluster, that is, the responsibility value γ ng , defined as: in, Represents the g-th Gaussian component for sample x n The probability density function of g and∑ g are the mean vectors of the g-th component respectively; k represents the k-th Gaussian cluster; The model parameters are updated to maximize the log-likelihood function, and the grouped ultrametric covariance structure is updated as:

7. The blade fault detection method combining multi-sensor fusion and image recognition according to claim 1 is characterized in that: The features extracted from image data include at least corner points, oriented gradient histograms, local binary patterns, scale-invariant features and sparsity; the features extracted from vibration data include at least vibration frequency, amplitude spectrum, peak value and root mean square; the features extracted from acoustic emission data include at least maximum amplitude, average amplitude and energy; the features extracted from electromagnetic data include at least amplitude change and spectrum characteristics.

8. The blade fault detection system combining multi-sensor fusion and image recognition is characterized by: The system is arranged outside the wind turbine blades, and the system at least includes a signal acquisition unit formed by multiple groups of sensors, a data processing system and a signal fusion module; the signal acquisition unit at least includes a vibration sensor, an acoustic emission sensor, an electromagnetic sensor and an image data sensor; wherein, The vibration sensor is a laser Doppler vibration sensor, which is specifically arranged outside the cabin to monitor the vibration behavior during operation, and is used to collect vibration data and transmit it to the signal data processing system; An acoustic emission sensor is specifically arranged at the root of the blade or a key part that bears a large load, and is used to detect acoustic emission data caused by cracks, fatigue and delamination. It is used to collect acoustic emission data and transmit it to a signal data preprocessing module; Electromagnetic sensors are specifically arranged at the root of the blade or at key locations prone to structural defects, and are used to detect electromagnetic data caused by cracks, corrosion, and metal fatigue, and are used to collect electromagnetic data and transmit them to a signal data preprocessing module; The image data sensor is a camera, which is specifically arranged on the outside of the cabin and is used to detect surface cracks and damage.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images