Wind farm fan major component status monitoring system and method thereof

Abstract

The present application discloses a state monitoring system and method for large components of wind turbines in a wind farm. First, the Gramian angular field image obtained by performing Gramian angular field transformation on the acoustic emission signal of the foundation structure of an offshore wind turbine is passed through a first convolutional neural network to obtain a Gramian angular field feature matrix. Next, multiple frequency-domain statistical feature vectors extracted from the vibration signal of the foundation structure of the offshore wind turbine are passed through a time series encoder to obtain frequency-domain statistical feature vectors. Then, the waveform diagram of the vibration signal is passed through an image encoder to obtain an image waveform feature vector. Next, the vibration feature matrix obtained by fusing the image waveform feature vector and the frequency-domain statistical feature vectors is fused with the Gramian angular field feature matrix to obtain a classification feature matrix. Finally, the classification feature matrix is passed through a classifier to obtain a classification result. In this way, the structural state of the offshore wind turbine can be more accurately evaluated, and the response time can be shortened.

Description

Technical Field

[0001] This application relates to the field of intelligent monitoring technology, and more specifically, to a status monitoring system and method for large components of wind turbines in a wind farm. Background Art

[0002] By the end of 2013, the total global installed wind power reached 318 GW, of which offshore wind power was 6.8 GW, the total installed wind power in China was 91.4 GW, and the installed offshore wind power was 428 MW. As time goes by, the safety accidents of wind turbine operation also show an upward trend. Among various wind power accidents, structural failure is second only to fire and blade failure. Therefore, it is of great significance to monitor the state of the wind turbine structure system.

[0003] Compared with onshore, the load environment of offshore wind turbines is more complex. The variable wind, wave, and current, and even the load excitations such as ice, typhoon, and earthquake in extreme cases have a more complex influence mechanism on the structure. At the same time, because offshore wind turbines are far from land, the management staff of wind farms cannot regularly evaluate and detect the structure, and the response time to accidents is also much longer than that of onshore wind turbines.

[0004] Therefore, an optimized status monitoring solution for offshore wind turbine structures is expected. Summary of the Invention

[0005] To solve the above technical problems, this application is proposed. The embodiments of this application provide a status monitoring system and method for large components of wind turbines in a wind farm. First, the acoustic emission signal of the foundation structure of an offshore wind turbine is subjected to Gramian Angular and Field Transformation (GAF) to obtain a GAF image, which is then passed through a first convolutional neural network to obtain a GAF feature matrix. Next, multiple frequency-domain statistical feature vectors extracted from the vibration signal of the foundation structure of the offshore wind turbine are passed through a time series encoder to obtain frequency-domain statistical feature vectors. Then, the waveform diagram of the vibration signal is passed through an image encoder to obtain an image waveform feature vector. Next, the vibration feature matrix obtained by fusing the image waveform feature vector and the frequency-domain statistical feature vector is fused with the GAF feature matrix to obtain a classification feature matrix. Finally, the classification feature matrix is passed through a classifier to obtain a classification result. In this way, the structural state of the offshore wind turbine can be more accurately evaluated, and the response time can be shortened.

[0006] According to one aspect of this application, a status monitoring system for large components of wind turbines in a wind farm is provided, which includes:

[0007] A monitoring data acquisition unit for acquiring the acoustic emission signal and vibration signal of the foundation structure of the offshore wind turbine to be detected;

[0008] A domain conversion unit for performing Gramian Angular and Field Transformation on the acoustic emission signal to obtain a GAF image;

[0009] Gram angle and field image encoding unit, configured to obtain a Gram angle and field feature matrix by passing the Gram angle and field image through a first convolutional neural network using a spatial attention mechanism that has been trained and completed;

[0010] Frequency-domain statistical feature extraction unit, configured to extract a plurality of frequency-domain statistical feature vectors from the vibration signal;

[0011] Frequency-domain time series encoding unit, configured to arrange the plurality of frequency-domain statistical feature vectors into a frequency-domain statistical input vector and then pass it through the time series encoder of the Clip model that has been trained and completed to obtain a frequency-domain statistical feature vector;

[0012] Vibration waveform image encoding unit, configured to obtain an image waveform feature vector by passing the waveform image of the vibration signal through the image encoder of the Clip model that has been trained and completed;

[0013] Joint encoding unit, configured to use the joint encoder of the Clip model that has been trained and completed to fuse the image waveform feature vector and the frequency-domain statistical feature vector to obtain a vibration feature matrix;

[0014] Feature fusion unit, configured to fuse the Gram angle and field feature matrix and the vibration feature matrix to obtain a classification feature matrix; and

[0015] Monitoring result generation unit, configured to pass the classification feature matrix through a classifier to obtain a classification result, and the classification result is used to indicate whether the state of the foundation structure of the offshore wind turbine to be detected is normal.

[0016] According to another aspect of the present application, there is provided a method for monitoring the state of large components of a wind farm fan, which includes:

[0017] Obtain the acoustic emission signal and vibration signal of the foundation structure of the offshore wind turbine to be detected;

[0018] Perform Gram angle and field transformation on the acoustic emission signal to obtain a Gram angle and field image;

[0019] Pass the Gram angle and field image through a first convolutional neural network using a spatial attention mechanism that has been trained and completed to obtain a Gram angle and field feature matrix;

[0020] Extract a plurality of frequency-domain statistical feature vectors from the vibration signal;

[0021] Arrange the plurality of frequency-domain statistical feature vectors into a frequency-domain statistical input vector and then pass it through the time series encoder of the Clip model that has been trained and completed to obtain a frequency-domain statistical feature vector;

[0022] Obtain an image waveform feature vector by passing the waveform diagram of the vibration signal through the image encoder of the trained Clip model;

[0023] Use the joint encoder of the trained Clip model to fuse the image waveform feature vector and the frequency-domain statistical feature vector to obtain a vibration feature matrix;

[0024] Fuse the Gramian angular and field feature matrix and the vibration feature matrix to obtain a classification feature matrix; and

[0025] Pass the classification feature matrix through a classifier to obtain a classification result, and the classification result is used to indicate whether the state of the foundation structure of the offshore wind turbine to be detected is normal.

[0026] Compared with the prior art, the wind farm wind turbine major component status monitoring system and method provided by the present application. First, the Gramian angular and field image obtained by performing Gramian angular and field transformation on the acoustic emission signal of the foundation structure of the offshore wind turbine is passed through a first convolutional neural network to obtain a Gramian angular and field feature matrix. Then, multiple frequency-domain statistical feature vectors extracted from the vibration signal of the foundation structure of the offshore wind turbine are passed through a time series encoder to obtain frequency-domain statistical feature vectors. Then, the waveform diagram of the vibration signal is passed through an image encoder to obtain an image waveform feature vector. Then, the vibration feature matrix obtained by fusing the image waveform feature vector and the frequency-domain statistical feature vector is fused with the Gramian angular and field feature matrix to obtain a classification feature matrix. Finally, the classification feature matrix is passed through a classifier to obtain a classification result. In this way, the structural state of the offshore wind turbine can be evaluated more accurately, and the response time can be shortened. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] By describing the embodiments of the present application in more detail in conjunction with the drawings, the above and other objects, features and advantages of the present application will become more obvious. The drawings are used to provide a further understanding of the embodiments of the present application, and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the present application and do not constitute a limitation to the present application. In the drawings, the same reference numerals generally represent the same components or steps.

[0028] Figure 1 Illustrates an application scenario diagram of a wind farm wind turbine major component status monitoring system according to an embodiment of the present application.

[0029] Figure 2 Illustrates a block diagram schematic of a wind farm wind turbine major component status monitoring system according to an embodiment of the present application.

[0030] Figure 3 Illustrates a block diagram schematic of the frequency-domain statistical feature extraction unit in a wind farm wind turbine major component status monitoring system according to an embodiment of the present application.

[0031] Figure 4 The block diagram of the frequency-domain time-series encoding unit in the large-component status monitoring system of a wind farm fan according to an embodiment of the present application is illustrated.

[0032] Figure 5 The block diagram of the training module further included in the large-component status monitoring system of a wind farm fan according to an embodiment of the present application is illustrated.

[0033] Figure 6 The flowchart of the method for monitoring the status of large components of a wind farm fan according to an embodiment of the present application is illustrated.

[0034] Figure 7 The schematic diagram of the system architecture of the method for monitoring the status of large components of a wind farm fan according to an embodiment of the present application is illustrated. Detailed implementation manners

[0035] Next, exemplary embodiments according to the present application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all embodiments of the present application. It should be understood that the present application is not limited by the exemplary embodiments described herein.

[0036] Scenario Overview

[0037] Currently, due to the relatively complex environment where the offshore wind turbines are located, the structural status monitoring of the offshore wind turbines is not intelligent and convenient enough, which will make the response time to accidents much longer than that of onshore wind turbines, thus causing unnecessary losses. Based on this, the inventors of the present application considered that when the offshore wind turbines are operating normally, the vibration signals generated by them will be conducted in a specific form in the basic structure. Therefore, it is expected to monitor the structural status of the offshore wind turbines through the vibration law of the basic structure of the offshore wind turbines. The inventors of the present application also found that different structures of the offshore wind turbines have different vibration tolerance ranges. Therefore, when monitoring the status of the basic structure of the offshore wind turbines, the vibration tolerance characteristics of each detected object should also be considered to more accurately evaluate the structural status of the offshore wind turbines.

[0038] Specifically, in the technical solution of the present application, first, acoustic emission signals and vibration signals of the foundation structure of the offshore wind turbine to be detected are acquired through various sensors. It should be understood that the generation of acoustic emission signals is due to the distortion of the molecular lattice, the aggravation of cracks, and a kind of ultra-high-frequency stress wave pulse signal released during plastic deformation in metal processing. It can extract the structural information of the object to be detected, while the vibration signal can extract the vibration law of the object to be detected. By collecting the signal data of both, it is convenient to perform implicit feature extraction subsequently. Furthermore, based on the object to be detected, that is, the implicit features of the foundation structure of the offshore wind turbine and the implicit features of the vibration law, the structural state assessment and monitoring of the offshore wind turbine are comprehensively carried out.

[0039] Then, for the acoustic emission signal, first, it is subjected to Gramian angular field transformation to obtain a Gramian angular sum field image. It should be understood that since the Gramian angular field (GAF) is based on the Gram principle, it can transfer the time series of the acoustic emission signal in the classical Cartesian coordinate system to the polar coordinate system for representation. GAF can well retain the dependence and correlation of the original acoustic emission time series signal and has similar time series characteristics to the original acoustic emission signal. In particular, GAF can obtain the Gramian angular sum field (GASF) and the Gramian angular difference field (GADF) according to different trigonometric functions used for encoding. After the GADF transformation, it is irreversible. Therefore, in the technical solution of the present application, the GASF transformation method that can be inversely transformed is selected to encode the acoustic emission signal. That is, the acoustic emission signal is subjected to Gramian angular field transformation to obtain the Gramian angular sum field image of the acoustic emission signal. Correspondingly, in a specific example, the encoding steps of the acoustic emission signal to the GASF image are as follows: For a time series of the acoustic emission signal with C dimensions = {Q1, Q2,..., QC}, where each dimension contains n sampling points Qi = {qi1, qi2,..., qin}, first, the data of each dimension is normalized. Then, all the values in the data are integrated into [-1, 1]. After integration, the normalized values are replaced with the Cos values of trigonometric functions, and the polar coordinates are used to replace the Cartesian coordinates, thereby retaining the absolute time relationship of the sequence.

[0040] Furthermore, a convolutional neural network with excellent performance in local implicit feature extraction of images is used to conduct deep feature mining on the Gram angle and field images. Considering that the acoustic emission signal has special implicit features in terms of spatial position, that is, the implicit feature information in the acoustic emission signal is different in different spatial positions. Therefore, in order to accurately extract the high-dimensional implicit feature distribution information in the acoustic emission signal to mine the structural features of the object to be detected, more focus on the position information in space is required when using the convolutional neural network. That is, specifically, in the technical solution of the present application, a first convolutional neural network using a spatial attention mechanism is used to process the Gram angle and field images to obtain a Gram angle and field feature matrix.

[0041] Regarding the vibration law of the offshore wind turbine foundation structure, since the vibration signal is a time-domain signal, although the time-domain signal is more intuitive for the manifestation of features in time correlation, under the influence of a strong noise environment, such as the application effect in the complex offshore environmental factors, it is not ideal. Therefore, when monitoring the structural state of the offshore wind turbine, only whether a fault occurs can be judged, but the type and location of the fault cannot be judged. The feature analysis of the frequency-domain signal is different from that of the time-domain signal. Converting the vibration signal into the frequency domain can determine the type of fault through the implicit feature distribution information of the vibration signal in the frequency domain, but its manifestation of the features of the vibration signal is not intuitive, ignoring the time-related features. Therefore, in the technical solution of the present application, the combination of the implicit features of the vibration signal in the time domain and the frequency domain is adopted. That is, specifically, in the technical solution of the present application, first, the vibration signal is subjected to Fourier transform to obtain a frequency-domain signal. Then, considering that in the frequency-domain signal, since the vibration signal in the frequency domain has more feature information, in order to be able to mine the global implicit correlation features in the frequency-domain statistical features to characterize the vibration law of the offshore wind turbine foundation structure, a plurality of frequency-domain statistical feature vectors are further extracted from the frequency-domain signal.

[0042] Furthermore, considering that the vibration signal has dynamic regular characteristics in the time series dimension, in order to more fully extract the dynamic implicit characteristics of the vibration signal in the frequency domain to express the vibration law of the detected object and accurately monitor the foundation structure of the offshore wind turbine, after arranging the multiple frequency domain statistical feature vectors into a frequency domain statistical input vector, the time series encoder of the Clip model is used to encode the frequency domain statistical input vector to extract the change characteristics of the implicit characteristics of the vibration signal in the time series dimension. In a specific example of the present application, the time series encoder of the Clip model consists of fully connected layers and one-dimensional convolutional layers arranged alternately, which extracts the correlation of the vibration signal in the time series dimension through one-dimensional convolutional encoding and extracts the high-dimensional implicit characteristics of the vibration signal through fully connected encoding.

[0043] For the time domain characteristics of the vibration signal, the waveform diagram of the vibration signal is passed through the image encoder of the Clip model to obtain an image waveform feature vector. Here, the image encoder can deeply excavate the local high-dimensional implicit characteristics in the waveform diagram of the vibration signal through a convolutional neural network. Then, the image waveform feature vector and the frequency domain statistical feature vector can be fused by using the joint encoder of the Clip model. In a specific example of the present application, the joint encoder uses the method of vector multiplication to fuse the features.

[0044] Then, considering that the structural characteristics of the detected object are extracted from the acoustic emission signal, and the vibration law characteristics of the detected object are extracted from the vibration signal. For the offshore wind turbine, the detected objects with different structures have different vibration tolerance ranges and capabilities. Therefore, the structural characteristics of the acoustic emission signal and the vibration law characteristics of the vibration signal are further fused to evaluate the foundation structure state of the offshore wind turbine. Correspondingly, in a specific example of the present application, the Gramian angle and field feature matrix and the vibration feature matrix can be fused in a cascaded manner to obtain a classification feature matrix, and then the classification feature matrix is passed through a classifier to obtain a classification result indicating whether the state of the foundation structure of the offshore wind turbine to be detected is normal.

[0045] Specifically, in the technical solution of the present application, when fusing the vibration feature matrix and the Gramian angle and field feature matrix, since the feature patterns of the vibration feature matrix and the Gramian angle and field feature matrix are quite different, when classifying them through a classifier after fusion, in the backpropagation process during training, the pattern expressed by the features may be eliminated due to abnormal gradient branches.

[0046] Therefore, a classification pattern cancellation and suppression loss is further introduced for the vibration feature matrix and the Gram angular sum and field feature matrix, expressed as:

[0047]

[0048] where V1 and V2 respectively represent the feature vectors obtained after projecting the vibration feature matrix and the Gram angular sum and field feature matrix, M1 and M2 are respectively the weight matrices of the classifier for the feature vectors obtained after projecting the vibration feature matrix and the Gram angular sum and field feature matrix, ||·|| F represents the Frobenius norm of the matrix, represents the square of the two-norm of the vector, represents the difference by position, exp(·) represents the exponential operation of the matrix and the exponential operation of the vector. The exponential operation of the matrix represents calculating the natural exponential function values with the eigenvalues at each position in the matrix as the exponents, and the exponential operation of the vector represents calculating the natural exponential function values with the eigenvalues at each position in the vector as the exponents.

[0049] Here, by introducing the classification pattern cancellation and suppression loss function, the pseudo-difference of the classifier weights can be pushed towards the feature distribution difference between the real features to be fused, so as to ensure that the directional derivative during gradient backpropagation is regularized near the gradient branch point, that is, over-weighting the gradient between patterns. In this way, the cancellation of the classification pattern of the features is suppressed, and thus the classification accuracy is improved. In this way, the abnormal state of the foundation structure of the offshore wind turbine can be accurately evaluated and monitored to avoid unnecessary losses caused by accidents.

[0050] Based on this, the present application provides a wind farm wind turbine major component condition monitoring system, which includes: a monitoring data acquisition unit, used to obtain acoustic emission signals and vibration signals of the foundation structure of the offshore wind turbine to be detected; a domain conversion unit, used to perform Gram angle and field transformation on the acoustic emission signal to obtain a Gram angle and field image; a Gram angle and field image encoding unit, used to pass the Gram angle and field image through a trained first convolutional neural network using a spatial attention mechanism to obtain a Gram angle and field feature matrix; a frequency domain statistical feature extraction unit, used to extract multiple frequency domain statistical feature vectors from the vibration signal; a frequency domain time series encoding unit, used to arrange the multiple frequency domain statistical feature vectors into a frequency domain statistical input vector and then pass through a trained Cl The invention relates to a monitoring result generating unit, which is used to pass the classification feature matrix through a classifier to obtain a classification result, and a monitoring result generating unit, which is used to pass the classification feature matrix through a classifier to obtain a classification result, and the classification result is used to indicate whether the state of the foundation structure of the offshore wind turbine to be detected is normal.

[0051] Figure 1 The figure shows an application scenario diagram of a wind farm wind turbine major component status monitoring system according to an embodiment of the present application. Figure 1 As shown, in this application scenario, multiple sensors (for example, Figure 1 C1, C2) to obtain the offshore wind turbine to be tested (for example, Figure 1 Then, the acquired acoustic emission signal and the acquired vibration signal are input into a server (for example, Figure 1 S) as shown in the figure, wherein the server is capable of using the wind farm wind turbine large component condition monitoring algorithm to process the acoustic emission signal and the vibration signal to generate a classification result indicating whether the condition of the foundation structure of the offshore wind turbine to be inspected is normal.

[0052] In a specific example, the plurality of sensors may include an acoustic sensor (e.g., an acoustic sensor) for acquiring an acoustic emission signal of a foundation structure of an offshore wind turbine to be detected. Figure 1 C1) and a vibration sensor (e.g., as shown in FIG. Figure 1As schematically shown in C2). In another specific example, the multiple sensors may further include other sensors for auxiliary sensing. It should be noted that the number of the multiple sensors may be more than just Figure 1 the two exemplified in it, and the number may be more.

[0053] After introducing the basic principle of the present application, various non-limiting embodiments of the present application will be specifically introduced with reference to the accompanying drawings.

[0054] Exemplary System

[0055] Figure 2 The block diagram of the large component status monitoring system of the wind farm fan according to the embodiment of the present application is shown. As Figure 2 shown, the large component status monitoring system 100 of the wind farm fan according to the embodiment of the present application includes: a monitoring data acquisition unit 110 for acquiring the acoustic emission signal and vibration signal of the basic structure of the offshore fan to be detected; a domain conversion unit 120 for performing Gram angle and field transformation on the acoustic emission signal to obtain a Gram angle and field image; a Gram angle and field image encoding unit 130 for passing the Gram angle and field image through a first convolutional neural network using a spatial attention mechanism that has been trained to obtain a Gram angle and field feature matrix; a frequency domain statistical feature extraction unit 140 for extracting a plurality of frequency domain statistical feature vectors from the vibration signal; a frequency domain time series encoding unit 150 for arranging the plurality of frequency domain statistical feature vectors into a frequency domain statistical input vector and then passing it through the time series encoder of the Clip model that has been trained to obtain a frequency domain statistical feature vector; a vibration waveform image encoding unit 160 for passing the waveform image of the vibration signal through the image encoder of the Clip model that has been trained to obtain an image waveform feature vector; a joint encoding unit 170 for using the joint encoder of the Clip model that has been trained to fuse the image waveform feature vector and the frequency domain statistical feature vector to obtain a vibration feature matrix; a feature fusion unit 180 for fusing the Gram angle and field feature matrix and the vibration feature matrix to obtain a classification feature matrix; and a monitoring result generation unit 190 for passing the classification feature matrix through a classifier to obtain a classification result, and the classification result is used to indicate whether the status of the basic structure of the offshore fan to be detected is normal.

[0056] More specifically, in the embodiments of the present application, the monitoring data acquisition unit 110 is configured to acquire the acoustic emission signal and the vibration signal of the foundation structure of the offshore wind turbine to be detected. It should be understood that the generation of the acoustic emission signal is a kind of ultra-high frequency stress wave pulse signal released due to the distortion of the molecular lattice, the aggravation of cracks, and the plastic deformation of the material during metal processing. It can extract the structural information of the object to be detected, and the vibration signal can extract the vibration law of the object to be detected. By collecting the signal data of both, it is convenient for subsequent implicit feature extraction, and then based on the implicit features of the object to be detected, that is, the implicit features of the foundation structure of the offshore wind turbine and the vibration law, the structural state evaluation and monitoring of the offshore wind turbine are comprehensively carried out. In other words, the acoustic emission signal can extract the structure of the object to be detected, and the vibration law of the object to be detected can be extracted from the vibration signal. It should be understood that objects to be detected with different structures have different vibration tolerance ranges and capabilities. Therefore, fusing the two can more accurately evaluate whether the state of the object to be detected is normal.

[0057] More specifically, in the embodiments of the present application, the domain conversion unit 120 is configured to perform Gramian angular field transformation on the acoustic emission signal to obtain a Gramian angular sum field image. It should be understood that since the Gramian angular field (GAF) can well retain the dependence and correlation of the original acoustic emission time series signal and has similar time series characteristics to the original acoustic emission signal. In particular, GAF can obtain the Gramian angular sum field (GASF) and the Gramian angular difference field (GADF) according to different trigonometric functions used for encoding. After the GADF transformation, it is irreversible. Therefore, in the technical solution of the present application, the GASF transformation method that can be inversely transformed is selected to encode the acoustic emission signal. That is, the Gramian angular field transformation is performed on the acoustic emission signal to obtain the Gramian angular sum field image of the acoustic emission signal.

[0058] Correspondingly, in a specific example, the encoding steps of the acoustic emission signal to the GASF image are as follows: For a time series of the acoustic emission signal with C dimensions = {Q1, Q2,..., QC}, where each dimension contains n sampling points Qi = {qi1, qi2,..., qin}, first perform a normalization operation on the data of each dimension. Then, integrate all the values in the data into [-1, 1]. After integration, replace the normalized values with the Cos values of the trigonometric function values, and use polar coordinates to replace the Cartesian coordinates to retain the absolute time relationship of the sequence.

[0059] More specifically, in the embodiments of the present application, the Gram angle and field image encoding unit 130 is configured to obtain a Gram angle and field feature matrix by passing the Gram angle and field image through a first convolutional neural network using a spatial attention mechanism that has been trained. It can be understood that a convolutional neural network, which has excellent performance in extracting local implicit features of images, is used to perform deep feature mining on the Gram angle and field image. Considering that the acoustic emission signal has special implicit features in terms of spatial position, that is, the implicit feature information in the acoustic emission signal is different in different spatial positions. Therefore, in order to accurately extract the high-dimensional implicit feature distribution information in the acoustic emission signal to mine the structural features of the object to be detected, it is necessary to focus more on the position information in space when using the convolutional neural network. That is, specifically, in the technical solution of the present application, the Gram angle and field image is processed by a first convolutional neural network using a spatial attention mechanism to obtain a Gram angle and field feature matrix.

[0060] Correspondingly, in a specific example, the Gram angle and field image encoding unit 130 is further configured to: in the forward propagation process of each layer of the trained first convolutional neural network using a spatial attention mechanism, perform the following operations on the input data respectively: perform convolutional processing on the input data to generate a convolutional feature map; perform pooling processing on the convolutional feature map to generate a pooled feature map; perform non-linear activation on the pooled feature map to generate an activation feature map; calculate the mean value of each position of the activation feature map along the channel dimension to generate a spatial feature matrix; calculate the class Softmax function value of each position in the spatial feature matrix to obtain a spatial score matrix; and calculate the element-wise multiplication of the spatial feature matrix and the spatial score map to obtain a feature matrix; where the feature matrix output by the last layer of the trained first convolutional neural network using a spatial attention mechanism is the Gram angle and field feature matrix.

[0061] More specifically, in the embodiments of the present application, the frequency-domain statistical feature extraction unit 140 is configured to extract a plurality of frequency-domain statistical feature vectors from the vibration signal. The vibration signal is a time-domain signal. Although the time-domain signal is more intuitive for the manifestation of features in the time correlation, under the influence of a strong noise environment, for example, the application effect in a complex marine environment is not ideal. Therefore, when monitoring the structural state of the offshore wind turbine, only whether a fault occurs can be judged, but the type and location of the fault cannot be determined. The feature analysis of the frequency-domain signal is different from that of the time-domain signal. By converting the vibration signal into the frequency domain, the type of the fault can be determined through the implicit feature distribution information of the vibration signal in the frequency domain, but its manifestation of the features of the vibration signal is not intuitive, and the correlation features in time are ignored. Therefore, in the technical solution of the present application, a combination of the implicit features of the vibration signal in the time domain and the frequency domain is adopted, that is, specifically, in the technical solution of the present application, first, the vibration signal is subjected to Fourier transform to obtain a frequency-domain signal. Then, considering that in the frequency-domain signal, since the vibration signal in the frequency domain has more feature information, in order to be able to extract the global implicit correlation features in the frequency-domain statistics features to characterize the vibration law of the basic structure of the offshore wind turbine, the plurality of frequency-domain statistical feature vectors are further extracted from the frequency-domain signal.

[0062] Correspondingly, in a specific example, as Figure 3 shown, the frequency-domain statistical feature extraction unit 140 includes: a Fourier transform sub-unit 141, configured to perform Fourier transform on the vibration signal to obtain a frequency-domain signal; a sampling sub-unit 142, configured to extract the plurality of frequency-domain statistical feature vectors from the frequency-domain signal.

[0063] More specifically, in the embodiments of the present application, the frequency-domain time-series encoding unit 150 is configured to arrange the multiple frequency-domain statistical feature vectors into a frequency-domain statistical input vector and then pass it through the time-series encoder of the trained Clip model to obtain the frequency-domain statistical feature vectors. Considering that the vibration signal has dynamic regular features in the time series dimension, therefore, in order to more fully extract this dynamic implicit feature of the vibration signal in the frequency domain to express the vibration law of the detected object and accurately monitor the basic structure of the offshore wind turbine, after further arranging the multiple frequency-domain statistical feature vectors into a frequency-domain statistical input vector, the time-series encoder of the Clip model is used to encode the frequency-domain statistical input vector to extract the change features of the implicit features of the vibration signal in the time series dimension. In a specific example of the present application, the time-series encoder of the Clip model is composed of a fully connected layer and a one-dimensional convolutional layer arranged alternately, and it extracts the correlation of the vibration signal in the time series dimension through one-dimensional convolutional encoding and extracts the high-dimensional implicit features of the vibration signal through fully connected encoding.

[0064] Correspondingly, in a specific example, as Figure 4 shown, the frequency-domain time-series encoding unit 150 includes: a vector arrangement subunit 151 configured to arrange the multiple frequency-domain statistical feature vectors into a frequency-domain statistical input vector; a fully connected encoding subunit 152 configured to use the fully connected layer of the time-series encoder of the trained Clip model to perform fully connected encoding on the frequency-domain statistical input vector according to the following formula to extract the high-dimensional implicit features of the eigenvalue at each position in the frequency-domain statistical input vector, where the formula is: where X is the frequency-domain statistical input vector, Y is the output vector, W is the weight matrix, and B is the bias vector, represents matrix multiplication; a one-dimensional convolutional encoding subunit 153 configured to use the one-dimensional convolutional layer of the time-series encoder of the trained Clip model to perform one-dimensional convolutional encoding on the frequency-domain statistical input vector according to the following formula to extract the high-dimensional implicit correlation features between the eigenvalues at each position in the frequency-domain statistical input vector, where the formula is:

[0065]

[0066] where a is the width of the convolutional kernel in the x direction, F(a) is the convolutional kernel parameter vector, G(x - a) is the local vector matrix operated with the convolutional kernel function, w is the size of the convolutional kernel, and X represents the frequency-domain statistical input vector.

[0067] More specifically, in the embodiments of the present application, the vibration waveform diagram encoding unit 160 is configured to obtain an image waveform feature vector by passing the waveform diagram of the vibration signal through the image encoder of the Clip model that has been trained. For the time-domain features of the vibration signal, the waveform diagram of the vibration signal is passed through the image encoder of the Clip model to obtain an image waveform feature vector. Here, the image encoder can deeply excavate the local high-dimensional implicit features in the waveform diagram of the vibration signal through a convolutional neural network.

[0068] Correspondingly, in a specific example, the vibration waveform diagram encoding unit 160 is further configured to: each layer of the convolutional neural network of the image encoder of the Clip model that has been trained performs the following operations on the input data during the forward pass of the layer: performing convolutional processing on the input data to obtain a convolutional feature map; performing mean pooling based on a local feature matrix on the convolutional feature map to obtain a pooled feature map; and performing non-linear activation on the pooled feature map to obtain an activated feature map; wherein, the output of the last layer of the convolutional neural network is the image waveform feature vector, and the input of the first layer of the convolutional neural network is the waveform diagram of the vibration signal.

[0069] More specifically, in the embodiments of the present application, the joint encoding unit 170 is configured to use the joint encoder of the Clip model that has been trained to fuse the image waveform feature vector and the frequency-domain statistical feature vector to obtain a vibration feature matrix. The image waveform feature vector and the frequency-domain statistical feature vector are fused by using the joint encoder of the Clip model. In a specific example of the present application, the joint encoder uses the method of vector multiplication to perform feature fusion.

[0070] Correspondingly, in a specific example, the joint encoding unit 170 is further configured to: use the joint encoder of the Clip model that has been trained to fuse the image waveform feature vector and the frequency-domain statistical feature vector with the following formula to obtain the vibration feature matrix;

[0071] wherein, the formula is:

[0072]

[0073] where V1 represents the image waveform feature vector, represents the transposed vector of the image waveform feature vector, V2 represents the frequency-domain statistical feature vector, M represents the vibration feature matrix, represents vector multiplication.

[0074] More specifically, in the embodiment of the present application, the feature fusion unit 180 is configured to fuse the Gram angle and field feature matrix and the vibration feature matrix to obtain a classification feature matrix. Considering that the structural features of the object to be detected are extracted from the acoustic emission signal, and the vibration law features of the object to be detected are extracted from the vibration signal. For the offshore wind turbine, the objects to be detected with different structures have different vibration tolerance ranges and capabilities. Therefore, the structural features of the acoustic emission signal and the vibration law features of the vibration signal are further fused to evaluate the basic structural state of the offshore wind turbine.

[0075] Correspondingly, in a specific example, the feature fusion unit 180 is further configured to concatenate the Gram angle and field feature matrix and the vibration feature matrix to obtain the classification feature matrix.

[0076] More specifically, in the embodiment of the present application, the monitoring result generation unit 190 is configured to pass the classification feature matrix through a classifier to obtain a classification result, and the classification result is used to indicate whether the state of the basic structure of the offshore wind turbine to be detected is normal. Passing the classification feature matrix through a classifier to obtain a classification result for indicating whether the state of the basic structure of the offshore wind turbine to be detected is normal, and through this classification result, the structural state of the offshore wind turbine can be evaluated more accurately, shortening the response time.

[0077] Correspondingly, in a specific example, the monitoring result generation unit 190 is further configured to: use the classifier to process the classification feature matrix according to the following formula to generate a classification result, where the formula is:

[0078] softmax{(M2,B2):…:(M1,B1)|Project(F)},

[0079] where Project(F) represents projecting the classification feature matrix into a vector, M1 and M2 are the weight matrices of each fully connected layer, and B1 and B2 represent the bias matrices of each fully connected layer.

[0080] More specifically, in the embodiment of the present application, the large component state monitoring system of the wind farm wind turbines further includes: a training module 200 for training the first convolutional neural network using the spatial attention mechanism and the Clip model; where, as Figure 5As shown, the training module 200 includes: a training data acquisition unit 201 for acquiring training data, where the training data includes acoustic emission signals and vibration signals of the foundation structure of the offshore wind turbine to be detected within a predetermined time period, and the true value of whether the state of the foundation structure of the offshore wind turbine to be detected is abnormal within the predetermined time period; a training domain conversion unit 202 for performing Gram angle and field transformation on the acoustic emission signals in the training data to obtain training Gram angle and field images; a training Gram angle and field image encoding unit 203 for passing the training Gram angle and field images through the first convolutional neural network using the spatial attention mechanism to obtain a training Gram angle and field feature matrix; a training frequency-domain statistical feature extraction unit 204 for extracting a plurality of training frequency-domain statistical feature vectors from the vibration signals in the training data; a training frequency-domain time-series encoding unit 205 for arranging the plurality of training frequency-domain statistical feature vectors into a training frequency-domain statistical input vector and then passing it through the time-series encoder of the Clip model to obtain a training frequency-domain statistical feature vector; a training vibration waveform graph encoding unit 206 for passing the waveform graph of the vibration signals in the training data through the image encoder of the Clip model to obtain a training image waveform feature vector; a training joint encoding unit 207 for using the joint encoder of the Clip model to fuse the training image waveform feature vector and the training frequency-domain statistical feature vector to obtain a training vibration feature matrix; a training feature fusion unit 208 for fusing the training Gram angle and field feature matrix and the training vibration feature matrix to obtain a training classification feature matrix; a classification loss unit 209 for passing the training classification feature matrix through the classifier to obtain a classification loss function value; a classification mode cancellation and suppression loss calculation unit 210 for calculating the classification mode cancellation and suppression loss value of the classifier, where the classification mode cancellation and suppression loss value is related to the square of the two-norm of the differential feature vector between the feature vectors projected from the vibration feature matrix and the Gram angle and field feature matrix; and a training unit 211 for training the first convolutional neural network using the spatial attention mechanism and the Clip model with the weighted sum of the classification mode cancellation and suppression loss value and the classification loss function value as the loss function value.

[0081] Correspondingly, in a specific example, the classification mode cancellation and suppression loss calculation unit 210 is further configured to: calculate the classification mode cancellation and suppression loss value of the classifier according to the following formula; where the formula is:

[0082]

[0083] Where V1 and V2 respectively represent the eigenvectors obtained after projection of the vibration feature matrix and the Gram angular sum and field feature matrix, and M1 and M2 are respectively the weight matrices of the classifier for the eigenvectors obtained after projection of the vibration feature matrix and the Gram angular sum and field feature matrix, ||·|| F represents the Frobenius norm of the matrix, represents the square of the two-norm of the vector, represents position-wise differencing, exp(·) represents the exponential operation of the matrix and the exponential operation of the vector. The exponential operation of the matrix represents calculating the values of the natural exponential function with the eigenvalues at each position in the matrix as the exponents, and the exponential operation of the vector represents calculating the values of the natural exponential function with the eigenvalues at each position in the vector as the exponents.

[0084] Here, by introducing the classification mode cancellation and suppression loss function, the pseudo-difference of the classifier weights can be pushed towards the feature distribution difference between the real features to be fused, thereby ensuring that the directional derivative during gradient backpropagation is regularized near the gradient branch point, that is, over-weighting the gradient between the modes. In this way, the classification mode cancellation of the features is suppressed, and thus the classification accuracy is improved. In this way, the abnormal state of the foundation structure of the offshore wind turbine can be accurately evaluated and monitored to avoid unnecessary losses caused by accidents.

[0085] In summary, the large component state monitoring system 100 of the wind farm fan based on the embodiment of the present application is elucidated. First, the Gram angular sum and field image obtained by performing the Gram angular sum and field transformation on the acoustic emission signal of the foundation structure of the offshore wind turbine is passed through the first convolutional neural network to obtain the Gram angular sum and field feature matrix. Then, a plurality of frequency-domain statistical feature vectors extracted from the vibration signal of the foundation structure of the offshore wind turbine are passed through the time series encoder to obtain the frequency-domain statistical feature vectors. Then, the waveform diagram of the vibration signal is passed through the image encoder to obtain the image waveform feature vector. Then, the vibration feature matrix obtained by fusing the image waveform feature vector and the frequency-domain statistical feature vector is fused with the Gram angular sum and field feature matrix to obtain the classification feature matrix. Finally, the classification feature matrix is passed through the classifier to obtain the classification result. In this way, the structural state of the offshore wind turbine can be evaluated more accurately, and the response time can be shortened.

[0086] As described above, the large component status monitoring system 100 of the wind farm turbines according to the embodiments of the present application can be implemented in various terminal devices, such as a server with the large component status monitoring algorithm of the wind farm turbines. In one example, the large component status monitoring system 100 of the wind farm turbines can be integrated into the terminal device as a software module and / or a hardware module. For example, the large component status monitoring system 100 of the wind farm turbines can be a software module in the operating system of the terminal device, or can be an application program developed for the terminal device; of course, the large component status monitoring system 100 of the wind farm turbines can also be one of the many hardware modules of the terminal device.

[0087] Alternatively, in another example, the large component status monitoring system 100 of the wind farm turbines and the terminal device can also be separate devices, and the large component status monitoring system 100 of the wind farm turbines can be connected to the terminal device through a wired and / or wireless network, and transmit and interact information according to a predefined data format.

[0088] Exemplary Method

[0089] Figure 6 The flowchart of the large component status monitoring method of the wind farm turbines according to the embodiments of the present application is illustrated. As Figure 6 shown, the large component status monitoring method of the wind farm turbines according to the embodiments of the present application includes: S110, acquiring the acoustic emission signal and the vibration signal of the basic structure of the offshore wind turbine to be detected; S120, performing Gramian angular field transformation on the acoustic emission signal to obtain a Gramian angular field image; S130, passing the Gramian angular field image through a first convolutional neural network using a spatial attention mechanism that has been trained to obtain a Gramian angular field feature matrix; S140, extracting a plurality of frequency domain statistical feature vectors from the vibration signal; S150, arranging the plurality of frequency domain statistical feature vectors into a frequency domain statistical input vector and then passing it through the time series encoder of the trained Clip model to obtain a frequency domain statistical feature vector; S160, passing the waveform diagram of the vibration signal through the image encoder of the trained Clip model to obtain an image waveform feature vector; S170, using the joint encoder of the trained Clip model to fuse the image waveform feature vector and the frequency domain statistical feature vector to obtain a vibration feature matrix; S180, fusing the Gramian angular field feature matrix and the vibration feature matrix to obtain a classification feature matrix; and S190, passing the classification feature matrix through a classifier to obtain a classification result, where the classification result is used to indicate whether the state of the basic structure of the offshore wind turbine to be detected is normal.

[0090] Figure 7The figure shows a schematic diagram of the system architecture of the method for monitoring the status of large components of a wind farm fan according to an embodiment of the present application. As Figure 7 shown, in the system architecture of the method for monitoring the status of large components of the wind farm fan, first, the acoustic emission signal and the vibration signal of the basic structure of the offshore fan to be detected are obtained; then, the Gram angle and field transformation is performed on the acoustic emission signal to obtain the Gram angle and field image; then, the Gram angle and field image is passed through the first convolutional neural network using the spatial attention mechanism that has been trained to obtain the Gram angle and field feature matrix; then, a plurality of frequency domain statistical feature vectors are extracted from the vibration signal; then, the plurality of frequency domain statistical feature vectors are arranged into a frequency domain statistical input vector and passed through the time series encoder of the trained Clip model to obtain the frequency domain statistical feature vectors; then, the waveform diagram of the vibration signal is passed through the image encoder of the trained Clip model to obtain the image waveform feature vectors; then, the trained joint encoder of the Clip model is used to fuse the image waveform feature vectors and the frequency domain statistical feature vectors to obtain the vibration feature matrix; then, the Gram angle and field feature matrix and the vibration feature matrix are fused to obtain the classification feature matrix; finally, the classification feature matrix is passed through a classifier to obtain a classification result, and the classification result is used to indicate whether the status of the basic structure of the offshore fan to be detected is normal.

[0091] In a specific example, in the above method for monitoring the status of large components of a wind farm fan, the passing the Gram angle and field image through the first convolutional neural network using the spatial attention mechanism that has been trained to obtain the Gram angle and field feature matrix further includes: each layer of the first convolutional neural network using the spatial attention mechanism that has been trained performs the following operations on the input data during the forward pass of the layer: performing convolutional processing on the input data to generate a convolutional feature map; performing pooling processing on the convolutional feature map to generate a pooling feature map; performing non-linear activation on the pooling feature map to generate an activation feature map; calculating the mean value of each position of the activation feature map along the channel dimension to generate a spatial feature matrix; calculating the class Softmax function value of each position in the spatial feature matrix to obtain a spatial score matrix; and, calculating the point-by-position multiplication of the spatial feature matrix and the spatial score map to obtain a feature matrix; wherein, the feature matrix output by the last layer of the first convolutional neural network using the spatial attention mechanism that has been trained is the Gram angle and field feature matrix.

[0092] In a specific example, in the above method for monitoring the status of large components of a wind farm fan, the extracting a plurality of frequency domain statistical feature vectors from the vibration signal includes: performing Fourier transform on the vibration signal to obtain a frequency domain signal; and extracting the plurality of frequency domain statistical feature vectors from the frequency domain signal.

[0093] In a specific example, in the above method for monitoring the status of large components of wind farm turbines, arranging the multiple frequency-domain statistical feature vectors into a frequency-domain statistical input vector and then obtaining the frequency-domain statistical feature vectors through the time series encoder of the trained Clip model includes: arranging the multiple frequency-domain statistical feature vectors into a frequency-domain statistical input vector; using the fully connected layer of the time series encoder of the trained Clip model to perform fully connected encoding on the frequency-domain statistical input vector according to the following formula to extract the high-dimensional implicit features of the eigenvalues at each position in the frequency-domain statistical input vector, where the formula is: where X is the frequency-domain statistical input vector, Y is the output vector, W is the weight matrix, and B is the bias vector, represents matrix multiplication; using the one-dimensional convolutional layer of the time series encoder of the trained Clip model to perform one-dimensional convolutional encoding on the frequency-domain statistical input vector according to the following formula to extract the high-dimensional implicit correlation features between the eigenvalues at each position in the frequency-domain statistical input vector, where the formula is:

[0094]

[0095] where a is the width of the convolutional kernel in the x direction, F(a) is the convolutional kernel parameter vector, G(x - a) is the local vector matrix for operating with the convolutional kernel function, w is the size of the convolutional kernel, and X represents the frequency-domain statistical input vector.

[0096] In a specific example, in the above method for monitoring the status of large components of wind farm turbines, further including obtaining the image waveform feature vector by passing the waveform diagram of the vibration signal through the image encoder of the trained Clip model: the image encoder of the trained Clip model uses each layer of the convolutional neural network to respectively perform the following operations on the input data during the forward pass of the layer: performing convolutional processing on the input data to obtain a convolutional feature map; performing mean pooling based on the local feature matrix on the convolutional feature map to obtain a pooled feature map; and performing non-linear activation on the pooled feature map to obtain an activated feature map; where the output of the last layer of the convolutional neural network is the image waveform feature vector, and the input of the first layer of the convolutional neural network is the waveform diagram of the vibration signal.

[0097] In a specific example, in the above-mentioned method for monitoring the state of large components of wind farm turbines, the use of the joint encoder of the trained Clip model to fuse the image waveform feature vector and the frequency domain statistical feature vector to obtain a vibration feature matrix further includes: using the joint encoder of the trained Clip model to fuse the image waveform feature vector and the frequency domain statistical feature vector according to the following formula to obtain the vibration feature matrix; where the formula is:

[0098]

[0099] where V1 represents the image waveform feature vector, represents the transpose vector of the image waveform feature vector, V2 represents the frequency domain statistical feature vector, M represents the vibration feature matrix, represents vector multiplication.

[0100] In a specific example, in the above-mentioned method for monitoring the state of large components of wind farm turbines, the fusion of the Gram angle and field feature matrix and the vibration feature matrix to obtain a classification feature matrix further includes concatenating the Gram angle and field feature matrix and the vibration feature matrix to obtain the classification feature matrix.

[0101] In a specific example, in the above-mentioned method for monitoring the state of large components of wind farm turbines, the use of the classifier for the classification feature matrix to obtain a classification result further includes: using the classifier to process the classification feature matrix according to the following formula to generate a classification result, where the formula is: softmax{(M2,B2):…:(M1,B1)|Project(F)}, where Project(F) represents projecting the classification feature matrix into a vector, and M1 and M2 are the weight matrices of each layer of the fully connected layer, and B1 and B are the bias matrices of each layer of the fully connected layer.

[0102] In a specific example, the above method for monitoring the status of large components of wind farm turbines further includes: training the first convolutional neural network using spatial attention mechanism and the Clip model; wherein, the training of the first convolutional neural network using spatial attention mechanism and the Clip model includes: obtaining training data, the training data including the acoustic emission signals and vibration signals of the foundation structure of the offshore wind turbine to be detected within a predetermined time period, and the true value of whether the status of the foundation structure of the offshore wind turbine to be detected is abnormal within the predetermined time period; performing Gram angle and field transformation on the acoustic emission signals in the training data to obtain training Gram angle and field images; passing the training Gram angle and field images through the first convolutional neural network using spatial attention mechanism to obtain a training Gram angle and field feature matrix; extracting a plurality of training frequency domain statistical feature vectors from the vibration signals in the training data; arranging the plurality of training frequency domain statistical feature vectors into a training frequency domain statistical input vector and passing it through the time series encoder of the Clip model to obtain a training frequency domain statistical feature vector; passing the waveform diagram of the vibration signal in the training data through the image encoder of the Clip model to obtain a training image waveform feature vector; using the joint encoder of the Clip model to fuse the training image waveform feature vector and the training frequency domain statistical feature vector to obtain a training vibration feature matrix; fusing the training Gram angle and field feature matrix and the training vibration feature matrix to obtain a training classification feature matrix; passing the training classification feature matrix through the classifier to obtain a classification loss function value; calculating the classification mode cancellation and suppression loss value of the classifier, wherein the classification mode cancellation and suppression loss value is related to the square of the two-norm of the differential feature vector between the vibration feature matrix and the feature vector projected from the Gram angle and field feature matrix; and using the weighted sum of the classification mode cancellation and suppression loss value and the classification loss function value as the loss function value to train the first convolutional neural network using spatial attention mechanism and the Clip model.

[0103] In a specific example, the calculating of the classification mode cancellation and suppression loss value of the classifier further includes: calculating the classification mode cancellation and suppression loss value of the classifier using the following formula;

[0104] wherein, the formula is:

[0105]

[0106] where V1 and V2 respectively represent the feature vectors obtained after projection of the vibration feature matrix and the Gram angle and field feature matrix, M1 and M2 are respectively the weight matrices of the classifier for the feature vectors obtained after projection of the vibration feature matrix and the Gram angle and field feature matrix, ||·|| pdenotes the Frobenius norm of a matrix, denotes the square of the two-norm of a vector, denotes position-wise difference, exp(·) denotes the exponential operation for a matrix and for a vector. The exponential operation for a matrix means calculating the values of the natural exponential function with the eigenvalues at each position in the matrix as the exponents, and the exponential operation for a vector means calculating the values of the natural exponential function with the eigenvalues at each position in the vector as the exponents.

[0107] Here, those skilled in the art can understand that the specific operations of each step in the above method for monitoring the states of large components of wind farm turbines have been introduced in detail in the description of the Figures 1 to 5 wind farm turbine large component state monitoring system above, and thus, the repeated description thereof will be omitted.

Claims

1. A large component status monitoring system for wind turbines in a wind farm, characterized in that, Including: A monitoring data acquisition unit for acquiring acoustic emission signals and vibration signals of the foundation structure of the offshore wind turbine to be detected; A domain transformation unit for performing Gram angle and field transformation on the acoustic emission signal to obtain a Gram angle and field image; A Gram angle and field image encoding unit for passing the Gram angle and field image through a first convolutional neural network using a spatial attention mechanism that has been trained to obtain a Gram angle and field feature matrix; A frequency-domain statistical feature extraction unit for extracting a plurality of frequency-domain statistical feature vectors from the vibration signal; A frequency-domain time-series encoding unit for arranging the plurality of frequency-domain statistical feature vectors into a frequency-domain statistical input vector and then passing it through the time-series encoder of the Clip model that has been trained to obtain a frequency-domain statistical feature vector; A vibration waveform image encoding unit for passing the waveform image of the vibration signal through the image encoder of the Clip model that has been trained to obtain an image waveform feature vector; A joint encoding unit for using the joint encoder of the Clip model that has been trained to fuse the image waveform feature vector and the frequency-domain statistical feature vector to obtain a vibration feature matrix; A feature fusion unit for fusing the Gram angle and field feature matrix and the vibration feature matrix to obtain a classification feature matrix; And A monitoring result generation unit for passing the classification feature matrix through a classifier to obtain a classification result, and the classification result is used to indicate whether the state of the foundation structure of the offshore wind turbine to be detected is normal.

2. The wind farm fan large component status monitoring system according to claim 1, characterized in that The Gram angle and field image encoding unit is further configured to: in the forward transmission process of each layer of the first convolutional neural network using a spatial attention mechanism that has been trained, the input data is respectively subjected to: Performing convolutional processing on the input data to generate a convolutional feature map; Performing pooling processing on the convolutional feature map to generate a pooling feature map; Performing non-linear activation on the pooling feature map to generate an activation feature map; Calculating the mean value of each position of the activation feature map along the channel dimension to generate a spatial feature matrix; Calculating the class Softmax function values of each position in the spatial feature matrix to obtain a spatial score matrix; and Calculating the element-wise multiplication of the spatial feature matrix and the spatial score map to obtain a feature matrix; Wherein, the feature matrix output by the last layer of the first convolutional neural network using a spatial attention mechanism that has been trained is the Gram angle and field feature matrix.

3. The condition monitoring system for large components of wind turbines in a wind farm according to claim 2, characterized in that, The frequency-domain statistical feature extraction unit includes: A Fourier transform sub-unit for performing Fourier transform on the vibration signal to obtain a frequency-domain signal; A sampling sub-unit for extracting the plurality of frequency-domain statistical feature vectors from the frequency-domain signal.

4. The state monitoring system for large components of wind turbines in a wind farm according to claim 3, wherein, The frequency-domain time-series encoding unit includes: A vector arrangement sub-unit for arranging the plurality of frequency-domain statistical feature vectors into a frequency-domain statistical input vector; A fully connected encoding subunit, which is used to perform fully connected encoding on the frequency-domain statistical input vector by using the fully connected layer of the temporal encoder of the trained Clip model to extract high-dimensional implicit features of the eigenvalue at each position in the frequency-domain statistical input vector, where the formula is: where X is the frequency-domain statistical input vector, Y is the output vector, W is the weight matrix, and B is the bias vector, represents matrix multiplication; A one-dimensional convolutional encoding subunit, which is used to perform one-dimensional convolutional encoding on the frequency-domain statistical input vector by using the one-dimensional convolutional layer of the temporal encoder of the trained Clip model according to the following formula to extract the high-dimensional implicit correlation features between the eigenvalue positions in the frequency-domain statistical input vector, where the formula is: where a is the width of the convolutional kernel in the x direction, F(a) is the convolutional kernel parameter vector, G(x - a) is the local vector matrix operated with the convolutional kernel function, w is the size of the convolutional kernel, and X represents the frequency-domain statistical input vector.

5. The condition monitoring system for large components of wind turbines in a wind farm according to claim 4, wherein The vibration waveform diagram encoding unit is further configured to: the image encoder of the trained Clip model uses each layer of the convolutional neural network to respectively perform the following operations on the input data during the forward pass of the layer: Perform convolutional processing on the input data to obtain a convolutional feature map; Perform mean pooling on the convolutional feature map based on the local feature matrix to obtain a pooled feature map; and Perform non-linear activation on the pooled feature map to obtain an activated feature map; where the output of the last layer of the convolutional neural network is the image waveform feature vector, and the input of the first layer of the convolutional neural network is the waveform diagram of the vibration signal.

6. The state monitoring system for large components of wind turbines in a wind farm according to claim 5, characterized in that, The joint encoding unit is further configured to: use the joint encoder of the trained Clip model to fuse the image waveform feature vector and the frequency-domain statistical feature vector according to the following formula to obtain the vibration feature matrix; where the formula is: where V1 represents the image waveform feature vector, represents the transposed vector of the image waveform feature vector, V2 represents the frequency domain statistical feature vector, and M represents the vibration feature matrix, represents vector multiplication.

7. The state monitoring system for large components of wind turbines in a wind farm according to claim 6, characterized in that, The feature fusion unit is further configured to concatenate the Gram angle and field feature matrix and the vibration feature matrix to obtain the classification feature matrix; where the monitoring result generation unit is further configured to: use the classifier to process the classification feature matrix according to the following formula to generate a classification result, where the formula is: softmax{(M2,B2):…:(M1,B1)|Project(F)}, where Project(F) represents projecting the classification feature matrix into a vector, and M1 and M2 are the weight matrices of each layer of the fully connected layer, and B1 and B2 represent the bias matrices of each layer of the fully connected layer.

8. The state monitoring system for large components of wind farm turbines according to claim 1, characterized in that, The large component state monitoring system of the wind farm fan further includes: a training module for training the first convolutional neural network using the spatial attention mechanism and the Clip model; where the training module includes: A training data acquisition unit, which is used to acquire training data, and the training data includes the acoustic emission signal and the vibration signal of the basic structure of the offshore fan to be detected within a predetermined time period, and the true value of whether the state of the basic structure of the offshore fan to be detected is abnormal within the predetermined time period; A training domain conversion unit, which is used to perform Gram angle and field transformation on the acoustic emission signal in the training data to obtain a training Gram angle and field image; A training Gram angle and field image encoding unit, which is used to pass the training Gram angle and field image through the first convolutional neural network using the spatial attention mechanism to obtain a training Gram angle and field feature matrix; A training frequency-domain statistical feature extraction unit for extracting a plurality of training frequency-domain statistical feature vectors from the vibration signals in the training data; A training frequency-domain time-series encoding unit for arranging the plurality of training frequency-domain statistical feature vectors into a training frequency-domain statistical input vector and then passing it through the time-series encoder of the Clip model to obtain a training frequency-domain statistical feature vector; A training vibration waveform graph encoding unit for passing the waveform graph of the vibration signal in the training data through the image encoder of the Clip model to obtain a training image waveform feature vector; A training joint encoding unit for using the joint encoder of the Clip model to fuse the training image waveform feature vector and the training frequency-domain statistical feature vector to obtain a training vibration feature matrix; A training feature fusion unit for fusing the training Gram angle and field feature matrix and the training vibration feature matrix to obtain a training classification feature matrix; A classification loss unit for passing the training classification feature matrix through the classifier to obtain a classification loss function value; A classification mode cancellation and suppression loss calculation unit for calculating the classification mode cancellation and suppression loss value of the classifier, where the classification mode cancellation and suppression loss value is related to the square of the two-norm of the differential feature vector between the vibration feature matrix and the feature vector projected from the Gram angle and field feature matrix; and A training unit for training the first convolutional neural network using the spatial attention mechanism and the Clip model with the weighted sum of the classification mode cancellation and suppression loss value and the classification loss function value as the loss function value.

9. The state monitoring system for large components of wind turbines in a wind farm according to claim 8, characterized in that, The classification mode cancellation and suppression loss calculation unit is further configured to calculate the classification mode cancellation and suppression loss value of the classifier according to the following formula; where the formula is: Where V1 and V2 respectively represent the eigenvectors obtained after projection of the vibration feature matrix and the Gram angular sum field feature matrix, M1 and M2 are respectively the weight matrices of the classifier for the eigenvectors obtained after projection of the vibration feature matrix and the Gram angular sum field feature matrix, and ||·|| F represents the Frobenius norm of the matrix, represents the square of the two-norm of the vector, θ represents the difference by position, exp(·) represents the exponential operation of the matrix and the exponential operation of the vector. The exponential operation of the matrix represents calculating the natural exponential function values with the eigenvalues at each position in the matrix as the exponents, and the exponential operation of the vector represents calculating the natural exponential function values with the eigenvalues at each position in the vector as the exponents.

10. A method for monitoring the status of large components of wind turbines in a wind farm, characterized in that, including: Obtaining the acoustic emission signal and vibration signal of the basic structure of the offshore wind turbine to be detected; Performing Gram angle and field transformation on the acoustic emission signal to obtain a Gram angle and field image; Passing the Gram angle and field image through the trained first convolutional neural network using the spatial attention mechanism to obtain a Gram angle and field feature matrix; Extracting a plurality of frequency-domain statistical feature vectors from the vibration signal; Arranging the plurality of frequency-domain statistical feature vectors into a frequency-domain statistical input vector and then passing it through the trained time-series encoder of the Clip model to obtain a frequency-domain statistical feature vector; Passing the waveform graph of the vibration signal through the trained image encoder of the Clip model to obtain an image waveform feature vector; Using the trained joint encoder of the Clip model to fuse the image waveform feature vector and the frequency-domain statistical feature vector to obtain a vibration feature matrix; Fusing the Gram angle and field feature matrix and the vibration feature matrix to obtain a classification feature matrix; and Passing the classification feature matrix through a classifier to obtain a classification result, where the classification result is used to indicate whether the state of the basic structure of the offshore wind turbine to be detected is normal.

Images