Cross-modal transmission framework construction method for unsupervised cross-domain fault diagnosis of wind turbine generator

Abstract

The invention belongs to the field of wind power, and discloses a cross-modal transmission framework construction method for unsupervised cross-domain fault diagnosis of a wind turbine generator, which comprises the following steps: step 1, collecting data of a target domain and a source domain, and carrying out data preprocessing and cross-modal feature alignment; 2, defining a feature alignment module, a domain discriminator, a feature extractor and a fault classifier, and constructing a domain adversarial framework based on pseudo tag enhancement; and step 3, according to classification loss, cross-modal comparison loss, domain discriminator loss and feature alignment loss involved in the training process, defining an overall loss function and performing model training. According to the method, the limitation caused by the scarcity of the marked data and the distribution difference among different operation conditions in the traditional method is solved.

Description

Technical Field

[0001] This invention relates to the field of wind power, specifically to a method for constructing a cross-modal transfer framework for unsupervised cross-domain fault diagnosis of wind turbine units. Background Technology

[0002] Intelligent monitoring and operation and maintenance technology for wind turbines has always been an indispensable part of ensuring the safe, stable, and efficient operation of large-scale wind farms. SCADA systems, due to their wide measurement range and strong real-time performance, have become one of the most important tools for wind farm operation and monitoring. Utilizing SCADA data for pattern recognition and fault diagnosis of wind turbines is a current hot research direction. However, due to the different hardware configurations, operation and maintenance strategies, and installation locations of different wind turbines, especially in wind farm projects still in the early stages of operation, the available samples are very limited. Therefore, how to learn domain-invariant features from the SCADA data of other operating wind farms and transfer this knowledge to new wind farms with limited and heterogeneous data is a pressing problem that needs to be solved.

[0003] In fact, wind turbine fault diagnosis technology based on deep learning / transfer learning has made some progress. Unsupervised domain adaptation (UDA) methods, which transfer domain-invariant fault features from the source domain to the target domain for auxiliary training, have received widespread attention. More specifically, Zhu et al. proposed a transfer learning architecture combining Long Short-Term Memory (LSTM) and fuzzy comprehensive; Li et al. proposed instance-weighted maximum mean difference to improve classification performance and ensure domain alignment; to advance fault diagnosis in practical applications, Yu et al. proposed a statistical identification-guided open-set domain adaptation method, which effectively identifies unknown class samples using an extended label classifier without requiring a threshold parameter during testing.

[0004] Despite the significant success of the methods described above in unsupervised cross-domain fault diagnosis (UCFD), current research still has the following problems or shortcomings:

[0005] 1) Insufficient model generalization and stability: Most existing studies assume that the fault type is uniform and rely heavily on rich source domain data to extract sufficient domain-invariant features for target domain detection. This reliance and assumption are often impractical in many industrial scenarios, limiting the generalization ability of the model and increasing the risk of overfitting too much source domain data, thereby reducing their effectiveness in the target domain.

[0006] 2) High dependence on labeled data and high annotation costs: Transfer learning typically requires a small amount of labeled data from the target generator to improve the model's adaptability. However, wind power fault data annotation is usually costly, and different generators have low fault occurrence frequencies, resulting in scarce labeled data. In addition, existing methods often lack efficient utilization of unlabeled data, necessitating the development of stronger unsupervised or weakly supervised transfer learning methods to reduce dependence on labels.

[0007] 3) Insufficient adaptability to dynamic environmental changes: The operating environment and load characteristics of wind turbines change over time. Most existing transfer learning methods are based on static features and struggle to adapt to dynamic environmental changes. This leads to model failures in long-term operation, lacking the ability to adapt to changes in data distribution and environmental drift. Summary of the Invention

[0008] The purpose of this invention is to provide a method for constructing a cross-modal transfer framework for unsupervised cross-domain fault diagnosis of wind turbine units, so as to solve the problems mentioned in the background art.

[0009] To achieve the above objectives, the present invention provides the following technical solution: A method for constructing a cross-modal transfer framework for unsupervised cross-domain fault diagnosis of wind turbine units includes: Step 1: Collect data from the target domain and source domain, perform data preprocessing and cross-modal feature alignment; Step 2: Define the feature alignment module, domain discriminator, feature extractor, and fault classifier to construct a domain adversarial framework based on pseudo-label enhancement; Step 3: Define the overall loss function and train the model based on the classification loss, cross-modal contrast loss, domain discriminator loss, and feature alignment loss involved in the training process.

[0010] Further, step 1 includes: Step 1.1: Collect data from the target domain and the source domain. Use the data from the new wind turbine as the target domain data and the SCADA data from the wind turbine that has been in operation for a certain period of time as the source domain data. Through an embedding layer containing linear transformation and multi-head self-attention mechanism, the multidimensional SCADA data collected by different sensors is mapped to the feature space of the same dimension as the pre-trained large language model to obtain the temporal embedding dataset. Step 1.2: Encode words or phrases in the text data using a pre-trained large language model to generate a corresponding word embedding database. Each word or phrase is represented as a high-dimensional vector. Use principal component analysis algorithm to reduce the dimensionality and cluster the word embedding database. By extracting the main components in the data, semantically similar word embeddings are grouped into a cluster, thereby obtaining a dimensionality-reduced word embedding dataset. Step 1.3: Input the time series embedding dataset and word embedding dataset prepared in Step 1.1 into a cross-modal matching module. Based on the similarity ranking, construct a set of positive and negative text samples for each time series, and select only positive samples for cross-modal fusion. Using the k-nearest pattern and distance function, a representative database is constructed, and the corresponding text space is embedded and encapsulated. Using a multi-head cross-modal attention mechanism, the temporal embedding dataset prepared in step 1.1 is used as the key and value, and the corresponding text space embedding is encapsulated as a query. The main word embeddings and time stamps are aligned to obtain the aligned embeddings.

[0011] Further, step 1.1 specifically includes: First, collect data from the target domain and the source domain, including data from the new wind turbine generator set. As the target domain data, the SCADA data of wind turbine units that have been in operation for a certain period of time will be used. As source domain data; among which... This represents unlabeled target domain data. Represents labels ( The source domain data, M, is the feature dimension of the data, which is the operating status collected by different sensors of SCADA. N represents all real numbers, T is the length of the sample set, i.e., the number of samples, and N is the number of samples. s and N t These represent the total number of samples from wind farms in the target and source domains, respectively; the SCADA data is then processed through an embedding layer containing linear transformation and a multi-head self-attention mechanism. Mapped to a feature space of the same dimension as the pre-trained large language model, the expression is: Among them, X time This represents a temporal embedding dataset, MSA represents a multi-head self-attention mechanism, Linear represents a linear layer, i.e., a fully connected neural network, and N is the latent space dimension of the pre-trained large language model.

[0012] Further, step 1.2 specifically includes: The text data is encoded using a pre-trained large language model, specifically GPT2, to generate a corresponding word embedding database. Each word or phrase is represented as a high-dimensional vector, where I represents the number of samples in the word embedding database. Principal component analysis (PCA) is used to reduce the dimensionality and cluster the word embedding database. By extracting the principal components from the data, semantically similar word embeddings are grouped into a cluster, thus obtaining a dimensionality-reduced word embedding dataset. : Where d is satisfied The predefined value is PCA(·), which represents the principal component analysis algorithm.

[0013] Furthermore, step 1.3 specifically includes: The time series embedding dataset X prepared in step 1.1 is used... time And the dimensionality-reduced word embedding dataset prepared in step 1.2 Both are input into a cross-modal matching module. Based on similarity ranking, a set of positive and negative text samples is constructed for each time series. Only positive samples are selected for cross-modal fusion. This selection process is expressed by the following formula: Among them, X t It is X time The samples in Representative and X t The corresponding selected text database, 'k' is a custom preset parameter, and 'Topk' means taking the first k elements from the set. Represents the magnitude of the vector; A representative database is constructed using the k-nearest model and distance function. The corresponding text space is embedded and encapsulated as follows: Where P represents the total text space. d j Represents embedding into X time The corresponding text space, where Softmax represents the softmax function operation applied to the input; Using a multi-head cross-modal attention mechanism, with X time Let P be the key and value, and P be the query. Align the keyword embeddings and time stamps to obtain the aligned embeddings. The formula is expressed as follows: in, , , W q W k and W v These represent the projection matrices of the query, key, and value, respectively.

[0014] Further, step 2 includes: Step 2.1: Use the maximum mean difference as a metric; in the cross-modal transfer framework based on a pre-trained large language model, deploy the maximum mean difference in two different layers, named shallow alignment and deep alignment. In shallow networks, the maximum mean difference is used to align cross-domain features; in deep networks, the maximum mean difference is used to align cross-domain features. The final maximum mean difference loss is defined as their weighted sum: Step 2.2: The domain discriminator is placed before the pre-trained large language model to determine whether the samples input to the pre-trained large language model come from the source domain or the target domain; the domain discriminator processes the embeddings after the cross-modal feature matching module. Step 2.3: Use the first five Transformer layers of the pre-trained large language model as feature extractors. Each Transformer layer contains a multi-head self-attention mechanism layer normalization, residual connections, and positional encoding. Step 2.4: Use focus loss instead of cross-entropy loss function to define classifier loss, adopt pseudo-labels, which include the probability of predicting target domain data, evaluate the confidence of each prediction, select high-confidence samples, assign pseudo-labels to these samples, and integrate them into the training process; pseudo-labels utilize unlabeled data to improve model performance; design diversity loss to ensure that the classes to which the target domain samples belong are uniformly distributed.

[0015] Further, step 2.1 specifically includes: The maximum mean difference is used as a metric. In the cross-modal transfer framework based on a pre-trained large language model, the maximum mean difference is deployed at two different levels, named shallow alignment and deep alignment. The formula for calculating the maximum mean difference is as follows: Where MMD represents the maximum mean difference. and Let represent the data from the source domain and the target domain, respectively, where n and m represent the number of samples in the source domain and the target domain, respectively. H represents the Hilbert space of the reconstruction kernel, and Φ represents the nonlinear mapping to H. The Hilbert space of the reconstruction kernel is induced by a Gaussian kernel; therefore, the maximum mean difference is redefined as: Where k(·) represents the Gaussian kernel function; Subsequently, the maximum mean difference is used in the shallow network to align cross-domain features, defined as... In deep networks, the maximum mean difference is used to align cross-domain features, defined as... The final maximum mean difference loss L MMD Then it is defined as their weighted sum: In the formula, β is the weight value.

[0016] Furthermore, step 2.2 specifically includes: The domain discriminator is strategically placed before the pre-trained large language model to determine whether the samples input to the large language model come from the source domain or the target domain; the domain discriminator processes the embeddings after the cross-modal feature matching module, and the loss function of the domain discriminator is... The definition is as follows: Where D represents the domain discriminator, and N represents the features of the source domain and the target domain, respectively. s and N t Represents the number of samples.

[0017] Furthermore, step 2.3 specifically includes: The first five Transformer layers of a pre-trained GPT-2 dataset are used as feature extractors. Each Transformer layer includes a multi-head self-attention mechanism for normalization, residual connections, and positional encoding. The feature extraction process can be represented by the following formula: Where h represents the extracted features, and GPT2 represents the pre-trained large language model GPT2; Step 2.4 specifically includes: Use focus loss instead of cross-entropy loss function; source domain classification loss. Defined as: in, It is the predicted probability of the correct class in the source domain, α. t It is a weighting factor that balances the importance of positive and negative samples, and γ is a focusing parameter that adjusts the reduction rate of easily weighted samples; To improve the classification ability of the target domain, pseudo-labels are used. Pseudo-labels consist of predicting the probability of data in the target domain, evaluating the confidence of each prediction, selecting high-confidence samples, assigning pseudo-labels to these samples, and integrating them into the training process. Pseudo-labels utilize unlabeled data to improve model performance, resolve class imbalance, and increase the robustness of the classifier. The classifier loss in the target domain with pseudo-labels... for: in, This represents the predicted probability of the correct category of the target domain data under the pseudo-label. Design diversity loss This ensures that the classes to which the samples in the target domain belong are uniformly distributed. Where K is the number of categories, δ k Indicates sample x t The probability of being classified as label k. This represents the mathematical expectation. To find the set x t Mathematical expectation, where C represents the classifier and G represents the feature extractor; The total loss of the fault classifier Defined as: .

[0018] Further, step 3 includes: The total loss of the model during training is defined as the classification loss. Cross-modal contrast loss Domain discriminator loss Feature alignment loss Weighted sum: In the formula, λ1, λ2, and λ3 are user-defined weighting coefficients.

[0019] Compared with existing technologies, the advantages of this invention are: it overcomes the limitations of traditional methods caused by the scarcity of labeled data and the distributional differences between different operational conditions. This invention leverages the powerful generalization ability of large language models to autonomously learn domain-invariant feature representations. Key technologies include integrating temporal and textual information through a cross-modal attention fusion module and aligning the input distribution using a cross-modal contrast alignment module. Furthermore, a pseudo-label-enhanced domain adversarial framework is designed to improve the generalization ability of the fault diagnosis model by jointly training labeled source data and unlabeled target data.

[0020] This invention has the potential to significantly improve the robustness and accuracy of fault diagnosis in complex and variable industrial environments, thereby contributing to more reliable and efficient wind farm operation and maintenance. Attached Figure Description

[0021] Figure 1 This is a flowchart of the cross-modal matching module.

[0022] Figure 2 A flowchart of the cross-modal transfer framework for unsupervised cross-domain fault diagnosis of wind turbine units. Detailed Implementation

[0023] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0024] Please see Figure 1 and Figure 2 A method for constructing a cross-modal transfer framework for unsupervised cross-domain fault diagnosis of wind turbine units, comprising:

[0025] Step 1: Collect data from the target and source domains, perform data preprocessing and cross-modal feature alignment, including:

[0026] Step 1.1, Timing Embedding

[0027] First, data from the target domain and source domain are collected. Data from new wind turbines is used as the target domain data, and SCADA data from wind turbines that have been in operation for a certain period of time is used as the source domain data. By using an embedding layer that includes linear transformation and multi-head self-attention mechanism, the multidimensional SCADA data collected by different sensors is mapped to a feature space of the same dimension as the pre-trained large language model.

[0028] Specifically, in this step, we first collect data from the target domain and the source domain, which is the new wind turbine unit (target domain). And wind turbine units (source area) that have been in operation for a certain period of time. SCADA data. Among them, This represents unlabeled target domain data. Represents labels ( The source domain data, M, is the feature dimension of the data, which is the operating status collected by different sensors of SCADA. N represents all real numbers, T is the length of the sample set, i.e., the number of samples, and N is the number of samples. s and N t These represent the total number of samples from the two wind farms. We use an embedding layer that includes linear transformation and a multi-head self-attention mechanism (MSA) to process the aforementioned time-series data (SCADA data). Mapping to the feature space of the same dimension (N-dimensional) as the pre-trained large language model, the expression is:

[0029]

[0030] Among them, X timeThis represents a temporal embedding dataset, MSA represents a multi-head self-attention mechanism, Linear represents a linear layer, i.e., a fully connected neural network, and N is the latent space dimension of the pre-trained large language model.

[0031] Step 1.2, Word Embedding and Clustering

[0032] We use pre-trained large language models to encode words or phrases in text data, generating corresponding word embedding databases. The preferred pre-trained large language model is GPT-2, where each word or phrase is represented as a high-dimensional vector. I represents the number of samples in the word embedding database, and N is the dimension of the latent space of the pre-trained large language model. Because word embeddings typically have high dimensionality (e.g., I=50257 in GPT-2), direct computation in the high-dimensional space leads to accuracy loss and inefficiency. Therefore, we employ Principal Component Analysis (PCA) to reduce the dimensionality and cluster the word embedding database. By extracting the principal components from the data, semantically similar word embeddings are grouped into clusters, resulting in a dimensionality-reduced word embedding dataset. :

[0033]

[0034] Where d is satisfied The predefined value is PCA(·), which represents the principal component analysis algorithm.

[0035] Step 1.3, Cross-modal feature matching

[0036] like Figure 1 As shown, the time-series embedding dataset X prepared in step 1.1 is... time And the dimensionality-reduced word embedding dataset prepared in step 1.2 These are input together into a cross-modal matching module. Despite the complex processing described above, many word clusters still fall short in effectively describing time series, and may mislead the model when cross-attention is used directly. To improve the efficiency of cross-modal fusion, we construct a set of positive and negative text samples for each time series based on similarity ranking, and only select positive samples for cross-modal fusion. This selection process can be expressed by the following formula:

[0037]

[0038] Among them, X t It is X time The samples in Representative and X t The corresponding selected text database, k is a user-defined preset parameter that indicates retrieving the first k elements from the set. Represents the magnitude of a vector.

[0039] Then, we use the k-nearest pattern and the distance function to construct a representative database. The corresponding text space is embedded and encapsulated as follows:

[0040]

[0041] Where P represents the total text space. d j Represents embedding into X time The corresponding text space, softmax represents the softmax function operation on the input.

[0042] Subsequently, we used a multi-head cross-modal attention mechanism to X time Let P be the key and value, and P be the query. Align the keyword embeddings and time stamps to obtain the aligned embeddings. The formula is expressed as follows:

[0043]

[0044]

[0045] in, , , W q W k and W v These represent the projection matrices of the query, key, and value, respectively.

[0046] In the cross-modal matching module, similarity between data from different modalities is matched through contrastive alignment, thereby achieving cross-modal feature matching between temporal and word modalities. These features are then integrated through a cross-attention layer. The cross-attention layer allows data from one modality to weight features of interest to data from another modality, thus capturing correlations between multiple modalities. This mechanism helps the model find meaningful connections between temporal and word modalities, ensuring information interaction and enhancement between features.

[0047] Step 2: Define the feature alignment module, domain discriminator, feature extractor, and fault classifier to construct a domain adversarial framework based on pseudo-label enhancement, including:

[0048] Step 2.1, Feature Alignment Module

[0049] To quantify the distributional differences between data points, we choose to use the nonparametric distance metric—Maximum Mean Difference (MMD)—as the evaluation standard. In the proposed Cross-Modal Transmission Framework (LLM-CTF), MMD is deployed at two different levels, named shallow alignment and deep alignment. The calculation formula for MMD is as follows:

[0050]

[0051] Where MMD represents the maximum mean difference. and Let represent the data from the source domain and the target domain, respectively, where n and m represent the number of samples in the source and target domains, respectively. H represents the Hilbert space of the reproduction kernel, and Φ represents the nonlinear mapping to H. In this work, RKHS is induced by a Gaussian kernel, therefore MMD can be redefined as:

[0052]

[0053] Where k(·) represents the Gaussian kernel function. Subsequently, the maximum mean difference is used in the shallow network to align cross-domain features, defined as... Aligning cross-domain features using the maximum mean difference in deep networks is defined as... Shallow layers are those closest to the input in a deep neural network, while deep layers are those closest to the output. The final maximum mean difference loss L... MMD Then it is defined as their weighted sum:

[0054]

[0055] Step 2.2, Domain Discriminator

[0056] A domain discriminator is strategically placed before the pre-trained large language model to determine whether the samples input to the large language model come from the source domain or the target domain. This placement helps ensure that the feature representations input to the large language model are domain-invariant, thus addressing differences early in the process and effectively reducing the discrepancies between features from the source and target domains. The domain discriminator processes embeddings after the cross-modal alignment layer, a step crucial for preventing domain-specific biases from affecting deeper layers of the model, thereby enhancing generalization and effective feature alignment. Furthermore, due to the inherent complexity and large parameter count of large language models, this early alignment not only simplifies the task of large language models but also promotes a more stable training process by fostering consistency in cross-domain input features. The loss function of the domain discriminator is defined as follows:

[0057]

[0058] Where D represents the domain discriminator, and N represents the features of the source domain and the target domain, respectively. s and N t Represents the number of samples.

[0059] Step 2.3, Feature Extractor Based on Pre-trained Large Language Model

[0060] We use the first five Transformer layers of a pre-trained GPT-2 dataset as our feature extractor. Each Transformer layer includes a multi-head self-attention (MSA) layer for normalization, residual connections, and positional encoding. The feature extraction process can be concisely represented by the following formula:

[0061]

[0062] Where h represents the extracted features, and GPT2 represents the pre-trained large language model GPT2 released by OpenAI.

[0063] Step 2.4, Introduce a fault classifier with pseudo-labels.

[0064] Due to the significant imbalance between faulty and normal samples in the dataset, we use focus loss instead of the traditional cross-entropy loss function. Focus loss addresses class imbalance by reducing the loss's contribution to well-classified samples and focusing more on challenging classifications. This modification significantly improves classification results for minority classes (such as faulty samples), thus promoting a fairer learning process. (Source domain classification loss) Defined as:

[0065]

[0066] in, It is the predicted probability of the correct class in the source domain, α. t It is a weighting factor that balances the importance of positive and negative samples, and γ is a focusing parameter that adjusts the reduction rate of easily weighted samples.

[0067] Due to the lack of labeled samples in the target domain, we employ pseudo-labeling to improve its classification ability. This technique involves predicting the probability of data in the target domain, evaluating the confidence of each prediction, selecting high-confidence samples, assigning pseudo-labels to these samples, and integrating them into the training process. Pseudo-labels leverage unlabeled data to improve model performance, address class imbalance, and increase the classifier's robustness. The classifier loss in the target domain with pseudo-labels is... for:

[0068]

[0069] in, This represents the predicted probability of the correct category for the target domain data under pseudo-labels. Additionally, since the model deals with an unlabeled target domain, we designed a diversity loss. This ensures that the classes to which the samples in the target domain belong are uniformly distributed.

[0070]

[0071] Where K is the number of categories, δ k Indicates sample x t The probability of being classified as label k. This represents the mathematical expectation. To find the set x t Mathematical expectation, where C represents the classifier and G represents the feature extractor.

[0072] Therefore, the total loss of this fault classifier Defined as:

[0073] .

[0074] Step 3: Based on the classification loss, cross-modal contrast loss, domain discriminator loss, and feature alignment loss involved in the training process, define the overall loss function and train the model, including:

[0075] This invention proposes a cross-modal transfer framework based on a large language model for unsupervised cross-domain fault diagnosis of wind turbines. Figure 2 As shown, the implementation steps of each part have been described in detail above. The total loss of the proposed model during training is defined as the classification loss. Cross-modal contrast loss Domain discriminator loss Feature alignment loss Weighted sum:

[0076]

[0077] In the formula, λ1, λ2, and λ3 are user-defined weighting coefficients.

[0078] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A method for constructing a cross-modal transfer framework for unsupervised cross-domain fault diagnosis of wind turbine units, characterized in that, include: Step 1: Collect data from the target domain and source domain, perform data preprocessing and cross-modal feature alignment; Step 2, Define a feature alignment module, a domain discriminator, a feature extractor, and a fault classifier to construct a domain adversarial framework based on pseudo-label enhancement; Step 3: Define the overall loss function and train the model based on the classification loss, cross-modal contrast loss, domain discriminator loss, and feature alignment loss involved in the training process.

2. The method for constructing a cross-modal transfer framework for unsupervised cross-domain fault diagnosis of wind turbine units according to claim 1, characterized in that, Step 1 includes: Step 1.1: Collect data from the target domain and the source domain. Use the data from the new wind turbine as the target domain data and the SCADA data from the wind turbine that has been in operation for a certain period of time as the source domain data. Through an embedding layer containing linear transformation and multi-head self-attention mechanism, the multidimensional SCADA data collected by different sensors is mapped to the feature space of the same dimension as the pre-trained large language model to obtain the temporal embedding dataset. Step 1.2: Encode words or phrases in the text data using a pre-trained large language model to generate a corresponding word embedding database. Each word or phrase is represented as a high-dimensional vector. Use principal component analysis algorithm to reduce the dimensionality and cluster the word embedding database. By extracting the main components in the data, semantically similar word embeddings are grouped into a cluster, thereby obtaining a dimensionality-reduced word embedding dataset. Step 1.3: Input the time series embedding dataset and word embedding dataset prepared in Step 1.1 into a cross-modal matching module. Based on the similarity ranking, construct a set of positive and negative text samples for each time series, and select only positive samples for cross-modal fusion. Using the k-nearest pattern and distance function, a representative database is constructed, and the corresponding text space is embedded and encapsulated. Using a multi-head cross-modal attention mechanism, the temporal embedding dataset prepared in step 1.1 is used as the key and value, and the corresponding text space embedding is encapsulated as a query. The main word embeddings and time stamps are aligned to obtain the aligned embeddings.

3. The method for constructing a cross-modal transfer framework for unsupervised cross-domain fault diagnosis of wind turbine units according to claim 2, characterized in that, Step 1.1 specifically includes: First, collect data from the target domain and the source domain, including data from the new wind turbine generator set. As the target domain data, the SCADA data of wind turbine units that have been in operation for a certain period of time will be used. As source domain data; among which... This represents unlabeled target domain data. Represents labels ( The source domain data, M, is the feature dimension of the data, which is the operating status collected by different sensors of SCADA. N represents all real numbers, T is the length of the sample set, i.e., the number of samples, and N is the number of samples. s and N t These represent the total number of samples from wind farms in the target and source domains, respectively; the SCADA data is then processed through an embedding layer containing linear transformation and a multi-head self-attention mechanism. Mapped to a feature space of the same dimension as the pre-trained large language model, the expression is: Among them, X time This represents a temporal embedding dataset, MSA represents a multi-head self-attention mechanism, Linear represents a linear layer, i.e., a fully connected neural network, and N is the latent space dimension of the pre-trained large language model.

4. The method for constructing a cross-modal transfer framework for unsupervised cross-domain fault diagnosis of wind turbine units according to claim 2, characterized in that, Step 1.2 specifically includes: The text data is encoded using a pre-trained large language model, specifically GPT2, to generate a corresponding word embedding database. Each word or phrase is represented as a high-dimensional vector, where I represents the number of samples in the word embedding database. Principal component analysis (PCA) is used to reduce the dimensionality and cluster the word embedding database. By extracting the principal components from the data, semantically similar word embeddings are grouped into a cluster, thus obtaining a dimensionality-reduced word embedding dataset. : Where d is satisfied The predefined value is PCA(·), which represents the principal component analysis algorithm.

5. The method for constructing a cross-modal transfer framework for unsupervised cross-domain fault diagnosis of wind turbine units according to claim 2, characterized in that, Step 1.3 specifically includes: The time series embedding dataset X prepared in step 1.1 is used... time And the dimensionality-reduced word embedding dataset prepared in step 1.2 Both are input into a cross-modal matching module. Based on similarity ranking, a set of positive and negative text samples is constructed for each time series. Only positive samples are selected for cross-modal fusion. This selection process is expressed by the following formula: Among them, X t It is X time The samples in Representative and X t The corresponding selected text database, 'k' is a custom preset parameter, and 'Topk' means taking the first k elements from the set. Represents the magnitude of the vector; A representative database is constructed using the k-nearest model and distance function. The corresponding text space is embedded and encapsulated as follows: Where P represents the total text space. d j Represents embedding into X time The corresponding text space, where Softmax represents the softmax function operation applied to the input; Using a multi-head cross-modal attention mechanism, with X time Let P be the key and value, and P be the query. Align the keyword embeddings and time stamps to obtain the aligned embeddings. The formula is expressed as follows: in, , , W q W k and W v These represent the projection matrices of the query, key, and value, respectively.

6. The method for constructing a cross-modal transfer framework for unsupervised cross-domain fault diagnosis of wind turbine units according to claim 1, characterized in that, Step 2 includes: Step 2.1: Use the maximum mean difference as a metric; in the cross-modal transfer framework based on a pre-trained large language model, deploy the maximum mean difference in two different layers, named shallow alignment and deep alignment. In shallow networks, the maximum mean difference is used to align cross-domain features; in deep networks, the maximum mean difference is used to align cross-domain features. The final maximum mean difference loss is defined as their weighted sum: Step 2.2: The domain discriminator is placed before the pre-trained large language model to determine whether the samples input to the pre-trained large language model come from the source domain or the target domain; the domain discriminator processes the embeddings after the cross-modal feature matching module. Step 2.3: Use the first five Transformer layers of the pre-trained large language model as feature extractors. Each Transformer layer contains a multi-head self-attention mechanism layer normalization, residual connections, and positional encoding. Step 2.4: Use focus loss instead of cross-entropy loss function to define classifier loss, adopt pseudo-labels, which include the probability of predicting target domain data, evaluate the confidence of each prediction, select high-confidence samples, assign pseudo-labels to these samples, and integrate them into the training process; pseudo-labels utilize unlabeled data to improve model performance; design diversity loss to ensure that the classes to which the target domain samples belong are uniformly distributed.

7. The method for constructing a cross-modal transfer framework for unsupervised cross-domain fault diagnosis of wind turbine units according to claim 6, characterized in that, Step 2.1 specifically includes: The maximum mean difference is used as a metric. In the cross-modal transfer framework based on a pre-trained large language model, the maximum mean difference is deployed at two different levels, named shallow alignment and deep alignment. The formula for calculating the maximum mean difference is as follows: Where MMD represents the maximum mean difference. and Let represent the data from the source domain and the target domain, respectively, where n and m represent the number of samples in the source domain and the target domain, respectively. H represents the Hilbert space of the reconstruction kernel, and Φ represents the nonlinear mapping to H. The Hilbert space of the reconstruction kernel is induced by a Gaussian kernel; therefore, the maximum mean difference is redefined as: Where k(·) represents the Gaussian kernel function; Subsequently, cross-domain features are aligned using the maximum mean difference in the shallow network, defined as... In deep networks, the maximum mean difference is used to align cross-domain features, defined as... The final maximum mean difference loss L MMD Then it is defined as their weighted sum: In the formula, β is the weight value.

8. The method for constructing a cross-modal transfer framework for unsupervised cross-domain fault diagnosis of wind turbine units according to claim 6, characterized in that, Step 2.2 specifically includes: The domain discriminator is strategically placed before the pre-trained large language model to determine whether the samples input to the large language model come from the source domain or the target domain; the domain discriminator processes the embeddings after the cross-modal feature matching module, and the loss function of the domain discriminator is... The definition is as follows: Where D represents the domain discriminator, and N represents the features of the source domain and the target domain, respectively. s and N t Represents the number of samples.

9. A method for constructing a cross-modal transfer framework for unsupervised cross-domain fault diagnosis of wind turbine units according to claim 6, characterized in that, Step 2.3 specifically includes: The first five Transformer layers of a pre-trained GPT-2 dataset are used as feature extractors. Each Transformer layer includes a multi-head self-attention mechanism for normalization, residual connections, and positional encoding. The feature extraction process can be represented by the following formula: Where h represents the extracted features, and GPT2 represents the pre-trained large language model GPT2; Step 2.4 specifically includes: Use focus loss instead of cross-entropy loss function; source domain classification loss. Defined as: in, It is the predicted probability of the correct class in the source domain, α. t It is a weighting factor that balances the importance of positive and negative samples, and γ is a focusing parameter that adjusts the reduction rate of easily weighted samples; Pseudo-labels are used, which include the probability of predicting data in the target domain. The confidence of each prediction is evaluated, high-confidence samples are selected, pseudo-labels are assigned to these samples, and they are integrated into the training process; the classifier loss in the target domain with pseudo-labels... for: in, This represents the predicted probability of the correct category of the target domain data under the pseudo-label. Design diversity loss This ensures that the classes to which the samples in the target domain belong are uniformly distributed. Where K is the number of categories, δ k Indicates sample x t The probability of being classified as label k. This represents the mathematical expectation. This means finding the set x. t Mathematical expectation, where C represents the classifier and G represents the feature extractor; The total loss of the fault classifier Defined as: 。 10. The method for constructing a cross-modal transfer framework for unsupervised cross-domain fault diagnosis of wind turbine units according to claim 1, characterized in that, Step 3 includes: The total loss of the model during training is defined as the classification loss. Cross-modal contrast loss Domain discriminator loss Feature alignment loss Weighted sum: In the formula, λ1, λ2, and λ3 are user-defined weighting coefficients.

Images