Rotating machinery fault prediction method and system based on time-frequency dual-domain cross-modal fusion

Abstract

The present application relates to the technical field of rotating machinery fault prediction, in particular to a rotating machinery fault prediction method and system based on time-frequency dual-domain cross-modal fusion. The present application obtains multi-modal monitoring sequences under a common time reference, divides each modal sequence into partially overlapping time slices, respectively performs time domain coding and frequency domain amplitude spectrum coding and fusion; before pooling, the time-frequency time slice sequence of the current modal is taken as the query, the time-frequency time slice sequence of other modal is taken as the key and value to perform cross-attention calculation, after pooling, joint context coding, modal scoring and weighted convergence are performed, the obtained fusion representation is output as a window-by-window fault probability, and fault warning information is generated by combining device threshold and consecutive windows; the corresponding system sets functional modules corresponding to the above processing. The present application can retain time slice level time-frequency information and cross-modal association, reduce information loss caused by premature pooling, and reduce false positives caused by device differences and transient disturbances.

Description

Technical Field

[0001] This invention relates to the field of rotating machinery fault prediction technology, specifically to a rotating machinery fault prediction method and system based on time-frequency dual-domain cross-modal fusion. Background Technology

[0002] In rotating machinery, such as wind turbine generators, which operate continuously for extended periods, the working status is typically reflected by monitoring signals including vibration, temperature, electrical power, rotational speed, environmental conditions, and overall operating status. Existing fault prediction methods usually organize monitoring data into time windows, extracting time-domain, frequency-domain, or local segment features from continuous sequences. Then, sequence models or classification models are used to generate window-by-window fault probabilities or fault determination results, enabling the identification of equipment status changes and providing a basis for maintenance before fault events occur.

[0003] When the rates of change and fault response patterns of different physical monitoring signals are inconsistent, fault samples are scarce, and equipment operating conditions change with installation location and environmental conditions, existing methods often rely on a unified feature space, pooled modal convergence, or fixed judgment thresholds. This processing mechanism struggles to simultaneously preserve temporal dependencies and spectral structures within the same local time range, and it is also difficult to utilize the current state of other modes to adjust the contributions of each mode. Fine-grained correlations are easily weakened during the convergence process. The resulting fault probability is also affected by equipment distribution offsets and transient conditions such as gusts, pitch control, or power grid fluctuations, generating isolated high-probability windows and increasing false alarms. Therefore, a rotating machinery fault prediction method and system based on time-frequency dual-domain cross-modal fusion is needed to address the aforementioned problems of easily weakened fine-grained time-frequency correlations and cross-modal conditional dependencies, as well as false alarms caused by equipment differences and transient disturbances. Summary of the Invention

[0004] In view of the above-mentioned problems in the prior art, the present invention provides the following technical solution: In a first aspect, the present invention provides a method for predicting faults in rotating machinery based on time-frequency dual-domain cross-modal fusion, comprising: Acquire a multimodal monitoring sequence of rotating machinery, wherein the multimodal monitoring sequence includes monitoring sequences of at least two physical monitoring modes under a common time reference; The monitoring sequence of each physical monitoring mode in the multimodal monitoring sequence is divided into time slices to obtain partially overlapping time slices. Time domain encoding and frequency domain amplitude spectrum encoding are performed on each time slice, and the two encoding results are fused to obtain a multimodal time-frequency representation composed of the time-frequency time slice sequences of each physical monitoring mode. Before pooling, for each time-frequency time slice sequence in the multimodal time-frequency representation, cross-attention is performed using the time-frequency time slice sequence as the query and other time-frequency time slice sequences in the multimodal time-frequency representation as the key and value to obtain an enhanced time slice sequence. Then, the enhanced time slice sequence is pooled to obtain a multimodal vector representation composed of each pooling result. Perform joint context encoding on the multimodal vector representation to obtain a joint context representation containing the context vectors of each physical monitoring mode; Modality scoring and normalization are performed on each context vector in the joint context representation to obtain a modality scoring result. Based on the modality scoring result, the joint context representation is weighted and converged to obtain a fused representation. The fusion representation outputs the rotating machinery fault prediction results.

[0005] Optionally, acquiring the multimodal monitoring sequence of the rotating machinery specifically includes: Aggregate SCADA data for the same wind turbine generator set in a ten-minute window; The aggregation results are divided into at least two of the following: vibration mode, temperature mode, electric power mode, speed and environment mode, and operating state mode, to form the multimodal monitoring sequence.

[0006] Optionally, the step of dividing the monitoring sequence for each physical monitoring mode in the multimodal monitoring sequence into time slices specifically includes: Divide the time along the time axis according to the time slice length, step size, and starting position shared by each physical monitoring mode, wherein the step size is less than the time slice length.

[0007] Optionally, the step of performing time-domain coding and frequency-domain amplitude spectrum coding on each time slice, and fusing the two coding results, specifically includes: Linear projection is performed on the time slice, and position encoding is superimposed on the projection result. The projection result after superimposed position encoding is input into the Transformer encoder to obtain the time domain representation. A real-valued fast Fourier transform is performed along the time axis of the time slice, the amplitude spectrum is preserved, and the frequency domain representation is obtained by projection through two layers of multilayer perceptron. By concatenating the time-domain representation and the frequency-domain representation of the same time slice and linearly projecting them, the time-frequency time slice sequence in the multimodal time-frequency representation is obtained.

[0008] Optionally, the step of performing cross-attention calculation using other time-frequency time slice sequences in the multimodal time-frequency representation as keys and values ​​specifically includes: Other time-frequency time-slice sequences are concatenated along the time-slice dimension, and the concatenation result is used as the key and value; Multiple attention heads perform attention calculations on the time-frequency time-slice sequence used as a query and the concatenated result used as a key and value to obtain an attention result. The attention result is then added to the time-frequency time-slice sequence used as a query and layer normalized to obtain the enhanced time-slice sequence.

[0009] Optionally, performing joint context encoding on the multimodal vector representation specifically includes: Each modal vector in the multimodal vector representation is combined with a learnable modal identity embedding of the same physical monitoring modality; The combined results are input into a single-layer Transformer to form the combined context representation.

[0010] Optionally, performing modality scoring and normalization on each context vector in the joint context representation specifically includes: Each of the aforementioned context vectors is mapped to a scalar score using a scoring network; The modal scores are obtained by performing Softmax normalization on each of the scalar scores.

[0011] Optionally, the rotating machinery fault prediction result includes a window-by-window fault probability, which is obtained by processing the fused representation using a multilayer perceptron and passing it through a Sigmoid function.

[0012] Optionally, the step of outputting the rotating machinery fault prediction result based on the fused representation specifically includes: The candidate threshold that maximizes the F1 value is selected from the candidate thresholds of the corresponding rotating machinery verification samples to obtain the equipment threshold; The window-by-window failure probabilities of K consecutive input windows are compared with the device threshold. A rotating machinery failure prediction result containing failure warning information is generated only when the window-by-window failure probabilities of the K consecutive input windows are all greater than the device threshold, and the rotating machinery failure prediction result is output, where K is an integer greater than 1.

[0013] Secondly, the present invention provides a rotating machinery fault prediction system based on time-frequency dual-domain cross-modal fusion, used to implement the above-mentioned rotating machinery fault prediction method based on time-frequency dual-domain cross-modal fusion, comprising: A monitoring sequence acquisition module is used to acquire multimodal monitoring sequences of rotating machinery, wherein the multimodal monitoring sequences include monitoring sequences of at least two physical monitoring modes under a common time reference; The time-frequency time-slice encoding module is used to divide the monitoring sequence of each physical monitoring mode in the multimodal monitoring sequence into time slices to obtain partially overlapping time slices. The module performs time-domain encoding and frequency-domain amplitude spectrum encoding on each time slice and fuses the two encoding results to obtain a multimodal time-frequency representation composed of the time-frequency time-slice sequences of each physical monitoring mode. The time-slice-level cross-modal interaction module is used to perform cross-attention calculation on each time-frequency time-slice sequence in the multimodal time-frequency representation before pooling, using the time-frequency time-slice sequence as the query and other time-frequency time-slice sequences in the multimodal time-frequency representation as the key and value, to obtain an enhanced time-slice sequence. Then, the enhanced time-slice sequence is pooled to obtain a multimodal vector representation composed of each pooling result. The joint context encoding module is used to perform joint context encoding processing on the multimodal vector representation to obtain a joint context representation containing the context vectors of each physical monitoring mode; The modality weighted fusion module is used to perform modality scoring and normalization processing on each context vector in the joint context representation to obtain a modality scoring result, and to perform weighted convergence on the joint context representation based on the modality scoring result to obtain a fused representation; The fault prediction module is used to output the fault prediction result of rotating machinery based on the fused representation.

[0014] Beneficial effects The technical solution provided by this invention has the following beneficial effects: 1. By dividing the multimodal monitoring sequence under a common time reference into partially overlapping time slices, and then fusing the time domain representation and frequency domain amplitude spectrum representation of the same time slice, it is possible to maintain the correspondence between time slices while taking into account the time dependence and periodic spectral structure of fault evolution, which is beneficial for characterizing weak fault precursors of rotating machinery.

[0015] 2. By performing cross-attention using the time-frequency time slice sequence of the current physical monitoring modality as the query and the time-frequency time slice sequences of other physical monitoring modalities as the key and value before pooling, and then using the joint context representation to score and weighted aggregate each physical monitoring modality, the fine-grained temporal correlation loss caused by early pooling can be reduced, and the contribution of each modality can be adjusted according to the context state of other modalities.

[0016] 3. By selecting verification samples for the corresponding rotating machinery to maximize the F1 value of the equipment threshold, and generating fault warning information only when the fault probability of each window in K consecutive input windows exceeds the equipment threshold, it can adapt to the probability distribution differences of different equipment and filter out isolated high-probability windows caused by transient operating conditions, thereby reducing the risk of false alarms. Attached Figure Description

[0017] Figure 1 A schematic diagram illustrating the steps of a rotating machinery fault prediction method based on time-frequency dual-domain cross-modal fusion provided by the present invention; Figure 2 The overall flowchart of a rotating machinery fault prediction method based on time-frequency dual-domain cross-modal fusion provided by the present invention; Figure 3 This is a schematic diagram of the time-frequency dual-domain coding relationship of the overlapping time slice provided by the present invention; Figure 4 This is a schematic diagram illustrating the relationship between time-slice-level cross-attention and joint context weighting provided by the present invention; Figure 5 A schematic diagram illustrating the steps for determining the device threshold and confirming the continuous window provided by this invention; Figure 6 A schematic diagram of the structure of a rotating machinery fault prediction system based on time-frequency dual-domain cross-modal fusion provided by the present invention; Figure 7 A comparison chart of the accuracy and F1 score of different inference strategies provided by this invention. Detailed Implementation

[0018] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0019] This implementation method is applied to early fault prediction of wind turbine generator sets, and can also be applied to gearboxes, generators, bearings, and other rotating machinery with at least two physical monitoring signals. The predicted object continuously generates vibration, temperature, electrical power, speed, environmental, and operating status data. The system organizes the monitoring records according to a common time base and generates window-by-window fault probability and fault early warning information within a predetermined prediction period, which is used by the operation and maintenance platform to determine the maintenance objects and early warning points.

[0020] This implementation can be deployed on a local server in a wind farm, an edge computing device, or a computing server capable of accessing a monitoring database. The server's memory stores monitoring data, training samples, validation samples, model parameters, device thresholds, and inference results. The processor or graphics processor executes feature construction, model training, and online inference programs. The SCADA acquisition interface writes monitoring records according to the rotating machinery identifier and timestamp. The online inference program only reads continuous records of the same rotating machinery and does not mix data from different rotating machinery in the same input window.

[0021] This embodiment provides a method for predicting rotating machinery faults based on time-frequency dual-domain cross-modal fusion, including: S1. Obtain a multimodal monitoring sequence of rotating machinery, wherein the multimodal monitoring sequence includes monitoring sequences of at least two physical monitoring modes under a common time reference; S2. Perform time-slice division on the monitoring sequence of each physical monitoring mode in the multimodal monitoring sequence to obtain partially overlapping time slices. Perform time-domain encoding and frequency-domain amplitude spectrum encoding on each time slice respectively, and fuse the two encoding results to obtain a multimodal time-frequency representation composed of the time-frequency time slice sequences of each physical monitoring mode. S3. Before pooling, for each time-frequency time slice sequence in the multimodal time-frequency representation, cross-attention calculation is performed using the time-frequency time slice sequence as the query and other time-frequency time slice sequences in the multimodal time-frequency representation as the key and value to obtain an enhanced time slice sequence. Then, the enhanced time slice sequence is pooled to obtain a multimodal vector representation composed of each pooling result. S4. Perform joint context encoding on the multimodal vector representation to obtain a joint context representation containing the context vectors of each physical monitoring mode; S5. Perform modal scoring and normalization processing on each context vector in the joint context representation to obtain a modal scoring result. Based on the modal scoring result, perform weighted aggregation on the joint context representation to obtain a fused representation. S6. Output the rotating machinery fault prediction result according to the fusion representation.

[0022] like Figure 1 As shown, S1 to S6 are executed in the above order, with the data output from the previous step directly used as the input for the next step. Figure 2 As shown, the main processing steps of this embodiment are as follows: acquiring a multimodal monitoring sequence under a common time reference, dividing partially overlapping time slices, forming time domain representations and frequency domain amplitude spectrum representations of each mode, fusing to form a multimodal time-frequency representation, performing time slice-level cross-modal attention before pooling, pooling to form a multimodal vector representation, performing joint context encoding, modal scoring and normalization, weighted convergence to form a fused representation, and outputting the rotating machinery fault prediction results. Figure 2 The modal patch time series in the text corresponds to the time-frequency time slice sequence referred to in this embodiment.

[0023] For S1, specifically, the acquisition program reads the rotating machinery identifier, sampling timestamp, and selected monitoring fields from the SCADA database or real-time acquisition interface. It first groups the data by rotating machinery identifier, then sorts them in ascending order by timestamp, and maps each field to the same 10-minute time boundary. Each time boundary corresponds to an aggregated record, and all physical monitoring modes are arranged according to the same time boundary and record number, thus establishing a common time reference. If a candidate input window lacks the necessary aggregated record for the selected mode, that candidate input window is marked as incomplete and not sent to the subsequent encoding process; other continuous and complete windows continue to be processed.

[0024] In some embodiments, acquiring the multimodal monitoring sequence of the rotating machinery specifically includes: Aggregate SCADA data for the same wind turbine generator set in a ten-minute window; The aggregation results are divided into at least two of the following: vibration mode, temperature mode, electric power mode, speed and environment mode, and operating state mode, to form the multimodal monitoring sequence.

[0025] Specifically, aggregation is performed separately for each wind turbine generator. For vibration signals, the mean, standard deviation, and maximum value of lateral and longitudinal vibrations are calculated within each 10-minute window, and then... Calculate the total vibrational energy, where, and These represent the transverse and longitudinal vibration values, respectively. After calculating the mean, standard deviation, and maximum value of the total vibration energy, a 9-dimensional vibration mode is obtained.

[0026] For the temperature signals, the average and maximum values ​​of the main bearing temperature, stator temperature, air temperature, converter temperature, and nacelle temperature are obtained respectively, and then... Calculate the generator temperature difference, where, Indicates stator temperature, This represents the air temperature. After calculating the mean and maximum values ​​of the generator temperature difference, a 12-dimensional temperature mode is obtained.

[0027] The electrical power mode includes the mean and maximum values ​​of active power, generator current, torque, and operating frequency. Pseudo-wind energy efficiency is based on... Calculate, where, Indicates active power. Indicates wind speed. This is a stable quantity greater than 0, used to avoid the denominator being 0 under low wind speed conditions. After calculating the mean and maximum values ​​of the pseudo-wind energy efficiency, a 10-dimensional electric power mode is obtained.

[0028] The rotational speed and environmental mode includes the mean and maximum values ​​of hub rotational speed, overspeed sensor rotational speed, and ambient temperature, as well as the mean, standard deviation, and maximum value of wind speed, resulting in a 9-dimensional rotational speed and environmental mode. The operating state mode includes the wind turbine generator state, hub state, and the mean and maximum values ​​of blade supercapacitor voltage, resulting in a 6-dimensional operating state mode. The aforementioned five modes total 46 dimensions, which is the total feature dimension. In actual deployment, at least two modalities should be selected, and the fields, field order, and units should be kept consistent during the training, validation, and inference phases.

[0029] This implementation method constructs training samples for each wind turbine generator set. Let the first... The starting time of each sample is Wind turbine generator set At any moment The feature vector is ,continuous The sample input consists of aggregated records, and the sample and its label satisfy:

[0030]

[0031] in, ; Indicated by time For predicting the start time of wind turbine generators The label indicating whether a fault occurred during the subsequent prediction period. Neighboring samples are ranked according to... slide, This is the input window step size. In this implementation, we take... The input window contains 72 aggregated records of 10 minutes each, corresponding to a 12-hour observation period. After the input window ends, there is a 12-aggregate record interval, corresponding to a 2-hour buffer period, before it is determined whether a fault occurs within the following week.

[0032] Each continuous feature is standardized using the mean and standard deviation calculated from the training samples. During the training phase, the standardization parameters, selected field versions, and modality order are saved. During the validation and inference phases, the same version of the data processing parameters is read to avoid using different units or field orders at different stages.

[0033] For S2, specifically, for each physical monitoring modality, a feature matrix arranged in ascending time order within an input window is read, and a time-slice sequence is formed according to common time-slice parameters. The time slices maintain the original monitoring field dimensions and temporal order; aggregated records in overlapping regions simultaneously enter two adjacent time slices, ensuring that state changes near time-slice boundaries can be encoded within at least one complete local interval. Each modality uses the same number of time slices and time-slice sequence numbers to enable cross-modal interaction at the same temporal granularity.

[0034] In some implementations, the step of dividing the monitoring sequence for each physical monitoring mode in the multimodal monitoring sequence into time slices specifically includes: Divide the time along the time axis according to the time slice length, step size, and starting position shared by each physical monitoring mode, wherein the step size is less than the time slice length.

[0035] Specifically, such as Figure 3 As shown, from the first Starting with the first aggregated record of the monitoring sequence for the physical monitoring modality, based on the time slice length... Continuous reading Each record is then moved along the time axis by a time slice step. Read the next time slice and repeat until the input window is covered. This yields a partially overlapping time-slice sequence. A representative configuration is taken as follows: , For a 72-step input window, the first time slice covers records 1 to 12, the second time slice covers records 7 to 18, and so on. Each time slice. Other configurations simply require that the time slice length be shared across all selected modes. Time slice step and starting position, and maintain This allows time slices with the same sequence number to cover the same time range.

[0036] Furthermore, the step of performing time-domain coding and frequency-domain amplitude spectrum coding on each time slice, and fusing the two coding results, specifically includes: Linear projection is performed on the time slice, and position encoding is superimposed on the projection result. The projection result after superimposed position encoding is input into the Transformer encoder to obtain the time domain representation. A real-valued fast Fourier transform is performed along the time axis of the time slice, the amplitude spectrum is preserved, and the frequency domain representation is obtained by projection through two layers of multilayer perceptron. By concatenating the time-domain representation and the frequency-domain representation of the same time slice and linearly projecting them, the time-frequency time slice sequence in the multimodal time-frequency representation is obtained.

[0037] Specifically, the time domain branch first divides the first... Each time slice of the modality is flattened according to a fixed field order, and then mapped through a linear projection in the time domain. The time slice is embedded and superimposed with the position code corresponding to the time slice number. Let the 1st time slice be... After linear projection and position encoding, each time slice is obtained By embedding time slices arranged in chronological order into the input multi-layer Transformer encoder, we obtain:

[0038] in, For the first Temporal representation of modes For the number of time slices, To achieve a unified representation dimension, this implementation takes... The Transformer encoder performs multi-head self-attention across different time slices, so that each time slice vector of the output contains state information from other time slices.

[0039] The frequency domain branch performs a real-valued Fast Fourier Transform along the time axis for each monitoring field of the same original time slice. Let the... Modal first The original time slice is The amplitude spectrum is represented as:

[0040] in, For the first Modal field dimensions, The time slice length is specified. Since a time slice covers a relatively short period, it can be approximated as a stationary sequence. Therefore, a real-valued Fast Fourier Transform (FFT) is used to extract its spectral structure, eliminating the need to introduce the window length and frequency resolution parameters of the Short-Time Fourier Transform (SFT). The frequency domain branch retains only the amplitude spectrum, omitting the phase, to reduce the impact of phase changes caused by sensor installation offset and measurement noise on feature comparisons between different devices; each field retains... The system flattens the amplitude spectrum according to a fixed field order, and inputs the flattened result into a two-layer multilayer perceptron; the first fully connected layer performs affine transformation and nonlinear activation, and the second fully connected layer projects the activation result onto the frequency point. Dimension, output the first Frequency domain representation of each time slice .

[0041] The system executes the concatenation of time slices with the same time slice number. The time-domain representation and the frequency-domain representation are concatenated along the feature dimension, and then fused and linearly projected.

[0042] in, This indicates splicing along the feature dimension. For the first The modal time-frequency time-slice sequence is obtained. The above process is repeated for all selected physical monitoring modes to obtain a multimodal time-frequency representation composed of the time-frequency time-slice sequences. The time-domain branch is used to characterize the state evolution between time-slices, and the frequency-domain branch is used to characterize the periodic structure within a time-slice; the two are fused according to the same original time-slice to maintain the correspondence between time position and spectral characteristics. Compared with element-wise addition, splicing followed by projection does not require the time-domain and frequency-domain representations to be in the same feature space beforehand, and can retain effective information in both types of representations.

[0043] For S3, specifically, a cross-attention update is performed once for each physical monitoring mode in the multimodal time-frequency representation. (Processing the...) In modal time, retain the first The time-frequency time-slice sequence of the modality is used as the query, and all time slices of the remaining modalities are used as the retrieveable information source; complete the first... After the modality is updated, other modalities are processed sequentially until an enhanced time-slice sequence for each modality is obtained. This processing is completed before average pooling, so the update result still corresponds one-to-one with each time slice of the current modality.

[0044] In some implementations, the step of performing cross-attention calculation using other time-frequency time slice sequences in the multimodal time-frequency representation as keys and values ​​specifically includes: Other time-frequency time-slice sequences are concatenated along the time-slice dimension, and the concatenation result is used as the key and value; Multiple attention heads perform attention calculations on the time-frequency time-slice sequence used as a query and the concatenated result used as a key and value to obtain an attention result. The attention result is then added to the time-frequency time-slice sequence used as a query and layer normalized to obtain the enhanced time-slice sequence.

[0045] Specifically, such as Figure 4 As shown, the first Modal time-frequency time slice sequence As a query Extract time-frequency time slice sequences of other modes according to a fixed modal order, maintain ascending time slice order within each mode, and concatenate them along the time slice dimension to form the key. Sum :

[0046] In the above construction relationship of query, key, and value, the part below the concatenation symbol... Indicates except the current number Other modalities besides the primary modality. Each attention head in the multi-head cross-attention algorithm computes the query separately. Time slice vector and key The attention weights are obtained by scaling the dot product of all vectors in the vector matrix, then normalizing them using Softmax, and finally adjusting the values ​​using these weights. The system performs a weighted summation of vectors. It then concatenates the outputs of each attention head and projects them onto the query result. Attention results with the same number of time slices. Residual connections and layer normalization are then performed:

[0047] in, For the first The modality is augmented with a time-slice sequence. The residual term uses the original time-frequency time-slice sequence before being projected into the query, ensuring that information from the current modality is retained after absorbing information from other modalities. The system performs average pooling along the time-slice dimension on the augmented time-slice sequence to form the... Modal vectors This average pooling is in Figure 2 The expression in the diagram represents global average pooling. All modalities are processed sequentially to form the modality vectors. If the online input lacks a modality required for model deployment, the system records the incomplete data state of the current window and does not output the failure probability of that window.

[0048] For S4, specifically, the modal vectors output by S3 are arranged according to the modal order fixed during the training phase, and each modal vector has the same feature dimension. The system sets a learnable modal identity embedding for each physical monitoring modality to label the modality source; after training, the modal identity embedding is saved together with the model parameters, and read according to the modal identifier during the inference phase.

[0049] In some implementations, performing joint context encoding on the multimodal vector representation specifically includes: Each modal vector in the multimodal vector representation is combined with a learnable modal identity embedding of the same physical monitoring modality; The combined results are input into a single-layer Transformer to form the combined context representation.

[0050] Specifically, each modality vector is element-wise added to its corresponding modality identity embedding, and then stacked in a fixed modality order to form a modality vector sequence. This sequence is input into a single-layer Transformer, which performs self-attention along the modality dimension, enabling each modality position to read information from other modality positions. The output of the single-layer Transformer is the first... Let the context vectors be denoted as This implementation uses There are several modalities, and the length of the modality sequence is shorter than the length of the aforementioned time slice sequence. A single-layer structure can be used to complete the information exchange between modalities, and reduce overfitting caused by excessive network depth when there are few fault samples. After this processing, each context vector retains the information of the corresponding modality, while also containing the states of other modalities. The subsequent scoring network uses this information to determine the weight of each modality in the current input window.

[0051] For S5, specifically, the system reads each context vector from the joint context representation according to modality location and outputs an unnormalized score for each modality using a parameter-shared scoring network. Normalization is performed across all modalities within the same input window, resulting in weights that are all not less than 0, and the sum of the weights is 1.

[0052] In some implementations, performing modality scoring and normalization on each of the context vectors in the joint context representation specifically includes: Each of the aforementioned context vectors is mapped to a scalar score using a scoring network; The modal scores are obtained by performing Softmax normalization on each of the scalar scores.

[0053] Specifically, the scoring network consists of a first fully connected layer, a non-linear activation layer, and a second fully connected layer connected sequentially. The first fully connected layer compresses the feature dimension of the context vector, and the second fully connected layer outputs a scalar score. Modal normalized weights We obtain it according to the following formula:

[0054] in, Represents a rating network. Indicates the number of physical monitoring modes. For the first Modality scoring results. The fusion representation is a weighted sum of the context vectors according to the weights of each modality:

[0055] in, This represents the fusion representation. Since the scoring network reads the context vector updated through inter-modal self-attention, the th... The weight of a mode changes with the state of other modes. For example, when the vibration mode fluctuates and the temperature mode increases at the same time, the scoring result can reflect the co-occurrence of these two changes, rather than scoring each mode in isolation.

[0056] For S6, specifically, the fault prediction head reads the fused representation output from S5, maps it to a scalar using a multilayer perceptron, and then uses the sigmoid function to convert this scalar into a window-by-window fault probability in the range of 0 to 1. This relationship is expressed as:

[0057] The online inference program saves the window-by-window fault probability according to the rotating machinery identifier and the end time of the input window, which is used for equipment threshold comparison and continuous window confirmation.

[0058] In some implementations, the rotating machinery fault prediction result includes a window-by-window fault probability, which is obtained by processing the fused representation using a multilayer perceptron and passing it through a Sigmoid function.

[0059] Specifically, during model training, samples from the same rotating machinery are divided into training samples, validation samples, and test samples in chronological order, with representative proportions of 70%, 15%, and 15%, respectively. Earlier data is used for training, and later data is used for validation and testing, respectively, to avoid future data entering the training phase. For cases where normal samples outnumber faulty samples, Focal Loss is used to calculate the... Loss per sample:

[0060] in, For the first Window-by-window failure probability for each sample For the corresponding tags; Used to balance faulty and normal samples. This is used to reduce the impact of easily classifiable samples on the total loss. These are loss function parameters, different from the modal weights represented in S5. This implementation method takes... ;when At this point, the loss degenerates into a binary cross-entropy with a class balance coefficient. The training procedure averages the batch loss and performs gradient backpropagation to update the parameters of the time-frequency time-slice coding module, the time-slice-level cross-modal interaction module, the joint context coding module, the modal weighted fusion module, and the fault prediction module. Validation samples are used to select the model version and device threshold, while test samples are used only to evaluate the frozen model and inference strategy.

[0061] Furthermore, the step of outputting the rotating machinery fault prediction result based on the fused representation specifically includes: The candidate threshold that maximizes the F1 value is selected from the candidate thresholds of the corresponding rotating machinery verification samples to obtain the equipment threshold; The window-by-window failure probabilities of K consecutive input windows are compared with the device threshold. A rotating machinery failure prediction result containing failure warning information is generated only when the window-by-window failure probabilities of the K consecutive input windows are all greater than the device threshold, and the rotating machinery failure prediction result is output, where K is an integer greater than 1.

[0062] Specifically, such as Figure 5 As shown, the threshold determination program reads the verification samples and candidate thresholds for the corresponding rotating machinery. Let the wind turbine generator set be identified as... , No. The true label of each validation sample is The probability of a window-by-window failure is Candidate threshold is The device threshold is selected according to the following formula:

[0063] in, This is the indicative function. Candidate thresholds can be iterated from 0 to 1 at intervals of 0.01, or the distinct window-by-window failure probabilities from the validation samples can be used. For each candidate threshold, the program records samples greater than the threshold as failure predictions and samples less than or equal to the threshold as normal predictions. Then, true positives, false positives, and false negatives are statistically analyzed based on the real labels, thereby calculating the F1 score corresponding to each candidate threshold. The program selects the device threshold corresponding to the largest F1 score; when multiple values ​​are tied, this implementation selects the candidate threshold with the largest value. The device threshold is associated with the rotating machinery identifier, model version, and validation data time range. During online inference, the device threshold matching the current model version is read. The original binary prediction for each window is:

[0064] The continuous verification procedure maintains a probability queue for each rotating machine, sorted in ascending order by the end time of the input window. For the... A set of input windows are used, and a final warning is generated only when the original binary predictions of the K most recent consecutive windows are all 1. Specifically:

[0065] in, For the first The final judgment result of each input window. Each time a new probability is obtained, the program writes it to the tail of the queue; if the queue length exceeds... The earliest probability is removed, and then the window-by-window fault probability of each of the K consecutive input windows is compared with the device threshold. If the window-by-window fault probability of each of the K consecutive input windows is greater than the device threshold, a fault warning is generated, including the rotating machinery identifier, confirmation time, and... Information on probabilities and equipment thresholds is output as the rotating machinery fault prediction result. If the queue is insufficient... If any window has a probability less than or equal to the device threshold, or the time interval between adjacent records is not equal to the input window step size, then continue receiving the next input window until the queue has a new input window. After determining the probability of consecutive time intervals, another judgment is made.

[0066] This is determined by a grid search on the validation samples, balancing noise suppression and confirmation delay. This implementation method... This means using a device-adaptive threshold and confirming for 5 consecutive windows. Other values ​​greater than 1... The value can be determined by comparing with the validation samples.

[0067] Furthermore, the system is used to implement the aforementioned method, and the system includes: A monitoring sequence acquisition module is used to acquire multimodal monitoring sequences of rotating machinery, wherein the multimodal monitoring sequences include monitoring sequences of at least two physical monitoring modes under a common time reference; The time-frequency time-slice encoding module is used to divide the monitoring sequence of each physical monitoring mode in the multimodal monitoring sequence into time slices to obtain partially overlapping time slices. The module performs time-domain encoding and frequency-domain amplitude spectrum encoding on each time slice and fuses the two encoding results to obtain a multimodal time-frequency representation composed of the time-frequency time-slice sequences of each physical monitoring mode. The time-slice-level cross-modal interaction module is used to perform cross-attention calculation on each time-frequency time-slice sequence in the multimodal time-frequency representation before pooling, using the time-frequency time-slice sequence as the query and other time-frequency time-slice sequences in the multimodal time-frequency representation as the key and value, to obtain an enhanced time-slice sequence. Then, the enhanced time-slice sequence is pooled to obtain a multimodal vector representation composed of each pooling result. The joint context encoding module is used to perform joint context encoding processing on the multimodal vector representation to obtain a joint context representation containing the context vectors of each physical monitoring mode; The modality weighted fusion module is used to perform modality scoring and normalization processing on each context vector in the joint context representation to obtain a modality scoring result, and to perform weighted convergence on the joint context representation based on the modality scoring result to obtain a fused representation; The fault prediction module is used to output the fault prediction result of rotating machinery based on the fused representation.

[0068] Specifically, such as Figure 6As shown, multimodal monitoring data of rotating machinery is input to the monitoring sequence acquisition module via the acquisition interface. The monitoring sequence acquisition module executes S1 and sends the multimodal monitoring sequence to the time-frequency time-slice encoding module. The time-frequency time-slice encoding module executes S2 and outputs the multimodal time-frequency representation to the time-slice-level cross-modal interaction module. The time-slice-level cross-modal interaction module executes S3 and outputs the multimodal vector representation to the joint context encoding module. The joint context encoding module executes S4 and outputs the joint context representation to the modal weighted fusion module. The modal weighted fusion module executes S5 and outputs the fused representation to the fault prediction module. The fault prediction module executes S6 and outputs the rotating machinery fault prediction result. Adjacent modules pass data through memory tensors, message queues, or file interfaces. Figure 6 Label the data objects and ensure that the rotating machinery identifier, input window end time, and model version are consistent.

[0069] From an overall technical perspective, different physical monitoring modes exhibit varying response speeds and manifestations to early faults. Vibration signals typically exhibit short-term fluctuations and periodic components, while temperature and power signals tend to show continuous changes. If all fields are directly treated as homogeneous sequences, the temporal and spectral characteristics of each mode can easily interfere with each other. This implementation first constructs modes according to their physical meaning, and then extracts time-domain and frequency-domain representations for common time slices, ensuring that the state changes and periodic structures within the same time slice remain consistent.

[0070] Subsequently, before average pooling, the time-slice representation of the current mode is updated with time-slice information from other modes, enabling interaction between different physical signals at the same temporal granularity. The enhanced time-slice sequence is then average-pooled to form modal vectors. A single-layer Transformer then jointly encodes these modal vectors, and the scoring network determines the modal weights based on the encoded states of all modes. The resulting fused representation retains information about each mode while also reflecting synchronous changes between different modes.

[0071] The fusion representation is converted into window-by-window fault probabilities using a multilayer perceptron and a sigmoid function. The device threshold is used to accommodate differences in probability distributions among different rotating machines, and is continuous. The window is used to filter out isolated high-probability events caused by gusts, pitch control, or grid fluctuations. The above processing sequentially connects time-slice-level time-frequency characteristics, pooling-based cross-modal interaction, inter-modal joint coding, and stability early warning, thereby reducing information loss caused by premature aggregation of heterogeneous monitoring data and reducing false alarms caused by equipment differences and transient disturbances.

[0072] To verify the effectiveness of the inference processing, data including 24 months of continuous SCADA records from 16 wind turbine generators, with a sampling interval of 10 minutes, was used for training and testing. The data was divided in chronological order, and the model output remained unchanged, only the inference strategy was modified. Figure 7 As shown, the horizontal axis represents the inference strategy, and the vertical axis represents the metric value. Dark bars represent precision, and blank bars represent the F1 score. A fixed threshold of 0.45 is used. The accuracy of the device adaptive threshold was 0.8500 and the F1 score was 0.9031; the accuracy of the device adaptive threshold and the F1 score was 0.9110 and the F1 score was 0.9061; the accuracy of the device adaptive threshold and the continuous 5-window confirmation was 0.9319 and the F1 score was 0.9201. These results indicate that, under the same test data and model output conditions, the device adaptive threshold and the continuous window confirmation can reduce false alarms caused by isolated high probability while maintaining the overall performance of fault identification.

[0073] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A rotating machinery fault prediction method based on time-frequency dual-domain cross-modal fusion, characterized in that, include: S1. Obtain a multimodal monitoring sequence of rotating machinery, wherein the multimodal monitoring sequence includes monitoring sequences of at least two physical monitoring modes under a common time reference; S2. Perform time-slice division on the monitoring sequence of each physical monitoring mode in the multimodal monitoring sequence to obtain partially overlapping time slices. Perform time-domain encoding and frequency-domain amplitude spectrum encoding on each time slice respectively, and fuse the two encoding results to obtain a multimodal time-frequency representation composed of the time-frequency time slice sequences of each physical monitoring mode. S3. Before pooling, for each time-frequency time slice sequence in the multimodal time-frequency representation, cross-attention calculation is performed using the time-frequency time slice sequence as the query and other time-frequency time slice sequences in the multimodal time-frequency representation as the key and value to obtain an enhanced time slice sequence. Then, the enhanced time slice sequence is pooled to obtain a multimodal vector representation composed of each pooling result. S4. Perform joint context encoding on the multimodal vector representation to obtain a joint context representation containing the context vectors of each physical monitoring mode; S5. Perform modal scoring and normalization processing on each context vector in the joint context representation to obtain a modal scoring result. Based on the modal scoring result, perform weighted aggregation on the joint context representation to obtain a fused representation. S6. Output the rotating machinery fault prediction result according to the fusion representation.

2. The method of claim 1, wherein, The acquisition of the multimodal monitoring sequence of the rotating machinery specifically includes: Aggregate SCADA data for the same wind turbine generator set in a ten-minute window; The aggregation results are divided into at least two of the following: vibration mode, temperature mode, electric power mode, speed and environment mode, and operating state mode, to form the multimodal monitoring sequence.

3. The method of claim 1, wherein, The step of dividing the monitoring sequence for each physical monitoring mode in the multimodal monitoring sequence into time slices specifically includes: Divide the time along the time axis according to the time slice length, step size, and starting position shared by each physical monitoring mode, wherein the step size is less than the time slice length.

4. The method of claim 1, wherein, The process of performing time-domain coding and frequency-domain amplitude spectrum coding on each time slice, and then fusing the two coding results, specifically includes: Linear projection is performed on the time slice, and position encoding is superimposed on the projection result. The projection result after superimposed position encoding is input into the Transformer encoder to obtain the time domain representation. A real-valued fast Fourier transform is performed along the time axis of the time slice, the amplitude spectrum is preserved, and the frequency domain representation is obtained by projection through two layers of multilayer perceptron. By concatenating the time-domain representation and the frequency-domain representation of the same time slice and linearly projecting them, the time-frequency time slice sequence in the multimodal time-frequency representation is obtained.

5. The method of claim 1, wherein, The cross-attention calculation, which uses other time-frequency time slice sequences in the multimodal time-frequency representation as keys and values, specifically includes: Other time-frequency time-slice sequences are concatenated along the time-slice dimension, and the concatenation result is used as the key and value; Multiple attention heads perform attention calculations on the time-frequency time-slice sequence used as a query and the concatenated result used as a key and value to obtain an attention result. The attention result is then added to the time-frequency time-slice sequence used as a query and layer normalized to obtain the enhanced time-slice sequence.

6. The method of claim 1, wherein, The joint context encoding process performed on the multimodal vector representation specifically includes: Each modal vector in the multimodal vector representation is combined with a learnable modal identity embedding of the same physical monitoring modality; The combined results are input into a single-layer Transformer to form the combined context representation.

7. The method of claim 1, wherein, The modality scoring and normalization process performed on each context vector in the joint context representation specifically includes: Each of the aforementioned context vectors is mapped to a scalar score using a scoring network; The modal scores are obtained by performing Softmax normalization on each of the scalar scores.

8. The method of claim 1, wherein, The rotating machinery fault prediction result includes window-by-window fault probabilities, which are obtained by processing the fused representation using a multilayer perceptron and then passing it through a Sigmoid function.

9. The method of claim 8, wherein, The step of outputting rotating machinery fault prediction results based on the fusion representation specifically includes: The candidate threshold that maximizes the F1 value is selected from the candidate thresholds of the corresponding rotating machinery verification samples to obtain the equipment threshold; The window-by-window failure probabilities of K consecutive input windows are compared with the device threshold. A rotating machinery failure prediction result containing failure warning information is generated only when the window-by-window failure probabilities of the K consecutive input windows are all greater than the device threshold, and the rotating machinery failure prediction result is output, where K is an integer greater than 1.

10. A rotating machinery fault prediction system based on time-frequency dual-domain cross-modal fusion, characterized in that, The system is used to implement the method of any one of claims 1 to 9, and the system comprises: A monitoring sequence acquisition module is used to acquire multimodal monitoring sequences of rotating machinery, wherein the multimodal monitoring sequences include monitoring sequences of at least two physical monitoring modes under a common time reference; The time-frequency time-slice encoding module is used to divide the monitoring sequence of each physical monitoring mode in the multimodal monitoring sequence into time slices to obtain partially overlapping time slices. The module performs time-domain encoding and frequency-domain amplitude spectrum encoding on each time slice and fuses the two encoding results to obtain a multimodal time-frequency representation composed of the time-frequency time-slice sequences of each physical monitoring mode. The time-slice-level cross-modal interaction module is used to perform cross-attention calculation on each time-frequency time-slice sequence in the multimodal time-frequency representation before pooling, using the time-frequency time-slice sequence as the query and other time-frequency time-slice sequences in the multimodal time-frequency representation as the key and value, to obtain an enhanced time-slice sequence. Then, the enhanced time-slice sequence is pooled to obtain a multimodal vector representation composed of each pooling result. The joint context encoding module is used to perform joint context encoding processing on the multimodal vector representation to obtain a joint context representation containing the context vectors of each physical monitoring mode; The modality weighted fusion module is used to perform modality scoring and normalization processing on each context vector in the joint context representation to obtain a modality scoring result, and to perform weighted convergence on the joint context representation based on the modality scoring result to obtain a fused representation; The fault prediction module is used to output the fault prediction result of rotating machinery based on the fused representation.

Images