Fault diagnosis method and system based on deep spatio-temporal convolutional network st-NN

Abstract

The application belongs to the technical field of fault diagnosis of marine diesel engines. A fault diagnosis method based on a deep space-time convolutional network ST-NN comprises: acquiring original oil spectrum and / or ferrography data and original vibration signals; performing dimension reduction on the original oil spectrum and / or ferrography data by using a rough set attribute reduction method to remove redundant features and obtaining a reduced oil and / or ferrography feature set; performing time-frequency conversion on the original vibration signals by using a continuous wavelet transform to obtain a time-frequency graph sequence; performing three-dimensional convolutional neural network fusion on the reduced oil and / or ferrography feature set as an additional feature channel and the time-frequency graph sequence, and introducing a self-attention mechanism to dynamically weight time steps to obtain a health index sequence; and performing fault trend prediction on the health index sequence by using a Gaussian process regression model to obtain a fault prediction result. The method effectively identifies the operating state of a diesel engine and improves the accuracy and reliability of fault diagnosis.

Description

Technical Field

[0001] This application belongs to the field of marine diesel engine fault diagnosis technology, and in particular relates to a fault diagnosis method and system based on deep spatiotemporal convolutional network ST-NN. Background Technology

[0002] In the field of marine diesel engine fault diagnosis, existing methods mainly rely on intelligent diagnostic models based on neural networks, such as backpropagation (BP) neural networks and extreme learning machines (ELM). However, these methods still have certain limitations when processing complex time-series data. For example, BP neural networks are prone to getting trapped in local optima and have slow convergence speeds, while ELM, although computationally efficient, may lead to large prediction errors due to its randomly initialized weights. In addition, existing methods still have shortcomings in feature extraction and data dimensionality reduction, making it difficult for models to fully utilize diesel engine oil monitoring data for accurate fault diagnosis. Summary of the Invention

[0003] Therefore, it is necessary to provide a fault diagnosis method and system based on the deep spatiotemporal convolutional network ST-NN to address the above-mentioned technical problems.

[0004] Firstly, this application provides a fault diagnosis method based on a deep spatiotemporal convolutional network (ST-NN), including:

[0005] Acquire raw oil spectral and / or ferrographic data, as well as raw vibration signals;

[0006] The original oil spectral and / or ferrographic data are reduced in dimensionality using a rough set attribute reduction method to remove redundant features and obtain a reduced oil and / or ferrographic feature set.

[0007] The original vibration signal is converted to time and frequency using continuous wavelet transform to obtain a time-frequency sequence.

[0008] The reduced oil and / or ferrographic feature set is used as an additional feature channel and fused with the time-frequency graph sequence using a three-dimensional convolutional neural network. A self-attention mechanism is introduced to dynamically weight the time steps to obtain a health indicator sequence.

[0009] Based on the health indicator sequence, a Gaussian process regression model is used to predict the fault trend and obtain the fault prediction results.

[0010] In some feasible methods, the step of performing dimensionality reduction on the original oil spectral and / or ferrographic data using a rough set attribute reduction method to remove redundant features and obtain a reduced oil and / or ferrographic feature set includes:

[0011] The continuous attributes of the original oil spectral and / or ferrographic data are discretized by equal-frequency binning to generate a discretized decision system S = (U, R, V, f), where U represents the set of objects, R = C ∪ D is the set of attributes, C is a subset of the conditional attributes of the oil spectral and / or ferrographic data, D is a subset of the fault state decision attributes, V represents the attribute range, and f represents the information function that determines the attribute value of the sample, f = U × R → V;

[0012] Calculate all single-attribute and key combined attribute subsets in the discretized decision system S to obtain the positive domain of all single-attribute and key combined attributes. The positive domain is obtained by dividing equivalence classes through indistinguishable relations.

[0013] Based on the positive domains of all single attributes and key combined attributes, perform heuristic attribute reduction to obtain the reduced oil and / or ferrographic feature sets.

[0014] In some feasible methods, the step of performing time-frequency conversion on the original vibration signal using continuous wavelet transform to obtain a time-frequency sequence includes:

[0015] Construct a basic wavelet function, wherein the scaling and translation formulas for the basic wavelet function ψ(t) are as follows:

[0016]

[0017] Where ψ represents the mother wavelet, k represents the scale parameter, l represents the translation parameter, and t represents time;

[0018] For the original vibration signal, a continuous wavelet transform is used for time-frequency conversion to obtain the calculation result. The calculation formula for the continuous wavelet transform is as follows:

[0019]

[0020] in, Let CWT(s)(k, l) represent the conjugate function of the wavelet function, let CWT(s)(k, l) represent the continuous wavelet transform coefficient matrix, let s(t) represent the original vibration signal, and let dt represent the integral variable.

[0021] The calculation results are normalized to obtain a time-frequency graph sequence.

[0022] In some feasible methods, the step of fusing the reduced oil and / or ferrographic feature set as an additional feature channel with the time-frequency graph sequence using a three-dimensional convolutional neural network, and introducing a self-attention mechanism to dynamically weight the time steps to obtain the health indicator sequence, includes:

[0023] The reduced oil and / or ferrographic feature set is used as an independent channel and expanded to the same resolution as the time-frequency plot sequence through spatial interpolation to form an additional feature channel;

[0024] The additional feature channels are fused with the time-frequency map sequence to generate a fused input tensor with shape (N, H, W, L, C), where N represents the number of samples, H represents the height of the time-frequency map, W represents the width of the time-frequency map, L represents the number of overlaps in the time-frequency map sequence, and C represents the number of channels in the time-frequency map sequence.

[0025] In some feasible methods, the step of fusing the reduced oil and / or ferrographic feature set as an additional feature channel with the time-frequency graph sequence using a three-dimensional convolutional neural network, and introducing a self-attention mechanism to dynamically weight the time steps to obtain the health indicator sequence, includes:

[0026] Construct at least three 3D convolutional layers, each using the ReLU activation function, to perform a unified nonlinear transformation on the input time-frequency plot sequence and the reduced oil and / or ferrographic features, and extract local correlation features in the time-frequency domain;

[0027] The fused input tensor is input into the three 3D convolutional layers. In each 3D convolutional layer, the ReLU activation function is used to perform a nonlinear transformation on the fused input tensor. The multi-source features of adjacent time-frequency maps are associated through the 3D convolutional layers in the time dimension to obtain the associated multi-source fused features. The multi-source fused features include vibration time-frequency maps, oil and ferrography features.

[0028] In some feasible methods, the step of fusing the reduced oil and / or ferrographic feature set as an additional feature channel with the time-frequency graph sequence using a three-dimensional convolutional neural network, and introducing a self-attention mechanism to dynamically weight the time steps to obtain the health indicator sequence, includes:

[0029] After the at least three 3D convolutional layers, two fully connected layers are constructed, and a Dropout operation is performed between the two fully connected layers. The fully connected layers include a first fully connected layer that uses the ReLU activation function and a second fully connected layer that uses the Sigmoid activation function to output health metrics.

[0030] The associated multi-source fusion features are input into the first fully connected layer, and the ReLU activation function is used for dimensionality reduction and nonlinear activation to obtain the features processed by the first fully connected layer.

[0031] The features processed by the first fully connected layer are subjected to a Dropout operation, in which neurons are randomly dropped with a preset probability to obtain Dropout-processed features that are not overfitted.

[0032] The features processed by Dropout are input into the second fully connected layer, and the Sigmoid activation function is used to map the features into a health indicator sequence to obtain the health indicator sequence.

[0033] In some feasible methods, the step of fusing the reduced oil and / or ferrographic feature set as an additional feature channel with the time-frequency graph sequence using a three-dimensional convolutional neural network, and introducing a self-attention mechanism to dynamically weight the time steps to obtain the health indicator sequence, includes:

[0034] The associated multi-source fusion features are linearly transformed using a trainable parameter matrix, mapping them to a Q / K / V matrix. The shape of the associated multi-source fusion features is (N, T, d), where N represents the number of samples, T represents the number of time steps, and d represents the feature dimension. The Q / K / V matrix includes a query matrix Q, a key matrix K, and a value matrix V. The formula for the Q / K / V matrix is:

[0035] Q = HI·W Q ;

[0036] K = HI·W K ;

[0037] V = HI·W v ;

[0038] in, W Q W K W v All of these are trainable parameters in the trainable parameter matrix;

[0039] By scaling the dot product, the influence weight of each time step on other time steps is calculated and normalized to generate a probability distribution weight matrix;

[0040] Using the probability distribution weight matrix, the value matrix V is weighted and summed to obtain the optimized feature representation, wherein the formula for the optimized feature representation is:

[0041] Z = A·V;

[0042] Where Z represents the optimized feature representation, and A represents the probability distribution weight matrix;

[0043] The optimized feature representation is input into a fully connected layer to generate a health indicator sequence, and the trainable parameters of the three-dimensional convolutional neural network and the self-attention mechanism are jointly optimized through backpropagation.

[0044] In some feasible methods, the step of using a Gaussian process regression model to predict fault trends based on the health indicator sequence and obtaining fault prediction results includes:

[0045] Establish a Gaussian process regression model;

[0046] The health indicator sequence is segmented according to a preset time window and normalized to obtain the segmentation results;

[0047] The segmentation results are input into a Gaussian process regression model to obtain fault prediction results.

[0048] Secondly, this application provides a fault diagnosis system based on a deep spatiotemporal convolutional network (ST-NN). Applying the aforementioned fault diagnosis method based on ST-NN, the system includes:

[0049] The acquisition unit is used to acquire raw oil spectral and / or ferrographic data, as well as raw vibration signals;

[0050] The rough set feature set is used to reduce the dimensionality of the original oil spectral and / or ferrographic data by using the rough set attribute reduction method to remove redundant features and obtain the reduced oil and / or ferrographic feature set.

[0051] The time-frequency unit is used to perform time-frequency conversion on the original vibration signal using continuous wavelet transform to obtain a time-frequency graph sequence;

[0052] The fusion unit is used to fuse the reduced oil and / or ferrographic feature set as an additional feature channel with the time-frequency graph sequence through a three-dimensional convolutional neural network, and introduce a self-attention mechanism to dynamically weight the time steps to obtain a health indicator sequence.

[0053] The result unit is used to perform fault trend prediction using a Gaussian process regression model based on the health indicator sequence, and obtain fault prediction results.

[0054] Thirdly, this application provides a computer device including a memory and a processor, wherein the memory stores a computer program, and the processor executes the steps of the aforementioned method.

[0055] Beneficial Effects: This application provides a fault diagnosis method based on a deep spatiotemporal convolutional neural network (ST-NN). The method acquires raw oil spectral and / or ferrographic data, as well as raw vibration signals. The raw oil spectral and / or ferrographic data are then dimensionality-reduced using a rough set attribute reduction method to remove redundant features, resulting in a reduced oil and / or ferrographic feature set. The raw vibration signals are then converted to time-frequency signals using continuous wavelet transform to obtain a time-frequency graph sequence. The reduced oil and / or ferrographic feature set is used as an additional feature channel and fused with the time-frequency graph sequence using a three-dimensional convolutional neural network. A self-attention mechanism is introduced to dynamically weight the time steps, resulting in a health indicator sequence. Based on the health indicator sequence, a Gaussian process regression model is used to predict fault trends, yielding fault prediction results. This method combines rough set attribute reduction and deep learning techniques, utilizing the rough set method to reduce the dimensionality of marine diesel engine oil spectral and ferrographic features, remove redundant information, optimize data input, and improve the model's computational efficiency. By extracting the spatiotemporal features of marine diesel engines using 3D-CNN and combining them with Gaussian process regression to further predict fault trends, this method can effectively identify the operating status of diesel engines and improve the accuracy and reliability of fault diagnosis. Attached Figure Description

[0056] To more clearly illustrate the technical solutions in the embodiments of this application or the conventional technology, the drawings used in the description of the embodiments or the conventional technology will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0057] Figure 1 This is a flowchart of a fault diagnosis method based on a deep spatiotemporal convolutional network (ST-NN) in one embodiment.

[0058] Figure 2 This is a schematic diagram of a fault diagnosis method based on a deep spatiotemporal convolutional network (ST-NN) in one embodiment. Detailed Implementation

[0059] To facilitate understanding of this application, a more complete description will be provided below with reference to the accompanying drawings, which illustrate embodiments of the present application. However, the present application can be implemented in many different forms and is not limited to the embodiments described herein. Rather, these embodiments are provided so that the disclosure of this application will be thorough and complete.

[0060] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein in the specification of this application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. The term "and / or" as used herein includes any and all couplings of one or more of the associated listed items.

[0061] It is understood that the terms “first,” “second,” etc., used in this application may be used herein to describe various elements, but these elements are not limited by these terms. These terms are only used to distinguish one element from another.

[0062] The following explanations of some terms used in this application are provided to aid in understanding the application:

[0063] Tensor is a fundamental concept in mathematics and computer science used to represent multidimensional data; it can be understood as a generalized form of multidimensional arrays.

[0064] Dropout is a regularization technique that randomly discards (zeros) some of the outputs of neurons during training (usually by a ratio of 0.2-0.5) to prevent the model from overfitting.

[0065] Spatial interpolation is a method for estimating numerical values ​​at unknown locations based on data from known discrete points.

[0066] ReLU (Rectured Linear Unit) is a widely used activation function that is very popular in deep learning due to its simplicity and efficiency.

[0067] A neuron is a mathematical model that simulates the behavior of biological neurons and is the basic computational unit of a network. It receives input signals and generates outputs through weighted summation and nonlinear transformation (activation function).

[0068] The Z-score is a statistical measure used to describe how much a data point deviates from the mean of a dataset, expressed in units of standard deviation. Its core purpose is to standardize values ​​in different datasets, making them easier to compare and analyze.

[0069] The scaled dot product is a core operation in the attention mechanism, primarily used to calculate the relevance score between a query and a key. Its core idea is to measure similarity through the dot product and introduce a scaling factor to stabilize the gradient, preventing the gradient vanishing problem in the softmax function caused by excessively large dot product values.

[0070] In practical applications, marine diesel engines operate under harsh environments such as high temperature, high load, and high vibration for extended periods, making their core components susceptible to corrosion and wear. Existing diagnostic methods primarily employ oil level testing and vibration signal analysis to monitor the engine's health. Oil level testing technology mainly relies on spectral analysis and ferrography, using the concentration of metal particles in the oil to determine mechanical wear. Vibration signal analysis, on the other hand, utilizes accelerometers to collect vibration data during engine operation and employs signal processing techniques such as Fast Fourier Transform (FFT) and wavelet transform to extract characteristic parameters.

[0071] However, these traditional methods have some significant shortcomings. First, BP neural network-based diagnostic methods are prone to getting trapped in local optima when dealing with complex nonlinear problems, and the gradient vanishing problem in the backpropagation algorithm results in slow convergence during training. Second, while the ELM method is computationally fast, its randomly initialized weights can lead to large prediction errors, resulting in poor model stability. Furthermore, traditional oil detection and vibration signal analysis methods rely on manually set feature extraction rules, lacking adaptability and generalization ability, making it difficult to accurately identify complex fault modes. In the face of complex and ever-changing combat environments, the diagnostic accuracy and real-time performance of these methods are insufficient to meet the requirements of marine diesel engine health monitoring.

[0072] like Figure 1 and Figure 2 As shown, in a first aspect, this application provides a fault diagnosis method based on a deep spatiotemporal convolutional network (ST-NN), comprising:

[0073] S100 acquires raw oil spectral and / or ferrographic data, as well as raw vibration signals.

[0074] Specifically, the acquisition of raw oil spectral and / or ferrographic data can be carried out at key nodes in the diesel engine lubricating oil circulation pipeline (main oil outlet / before the filter). For example, the spectral data is obtained by online monitoring of the concentration of 15 metal elements such as Fe, Cu, and Al (ppm level accuracy) using an atomic emission spectrometer; the ferrographic data is obtained by obtaining the concentration ratio (DL / DS) of large abrasive particles (>5μm) to small abrasive particles (1-2μm) using a direct-reading ferrometer; the sampling frequency is one complete set of oil data every 10 minutes.

[0075] For acquiring the original vibration signal, the acquisition device can be placed in a location such as the main bearing housing. This application does not limit how to obtain the original oil spectral and / or ferrographic data, or the original vibration signal; the location and acquisition can be arranged according to the actual situation.

[0076] S200, the original oil spectral and / or ferrographic data are reduced in dimensionality using a rough set attribute reduction method to remove redundant features and obtain the reduced oil and / or ferrographic feature set.

[0077] Specifically, obtaining the reduced oil and / or ferrographic feature set may include the following steps:

[0078] S201, the continuous attributes of the original oil spectral and / or ferrographic data are discretized by equal frequency binning to generate a discretized decision system S=(U,R,V,f).

[0079] Where U represents a set of objects, U = {x1, x2, ..., x} n}, corresponding to n oil samples x; R=C∪D={r1,r2,…r m} represents the attribute set, C represents the subset of oil spectral and / or ferrographic condition attributes, D represents the subset of fault state decision attributes; V represents V r Let f be the range of attribute r∈R, and it can be regarded as an information function between U and R to determine the attribute value of the sample, f: U×R→V, f(x,r)∈V r .

[0080] Specifically, firstly, the raw oil spectral and / or ferrographic data can be formed into a dataset (n×m matrix, n samples × m monitoring indicators). This method enables the raw data to be transformed into a computable, scalable, and interpretable standardized form.

[0081] Next, construct the decision table:

[0082] Conditional attribute subset C: Quantify the ferrographic characteristics of oil (Fe concentration, Cu concentration, DL / DS ratio, etc.).

[0083] Decision attribute subset D: Device status label (0 Normal / 1 Fault).

[0084] Discretization of continuous attributes:

[0085] An equal-frequency binning method (with the same number of samples in each bin) can be used to process each continuous attribute (such as Fe concentration) separately and divide it equally according to the sample frequency. For example, Fe concentration can be divided into 3 bins, with each bin containing approximately n / 3 samples.

[0086] It should be noted that the indistinguishability relation (IND) of rough sets requires that the attribute values ​​be finite discrete values. Continuous values ​​must be discretized into categorical variables before equivalence classes can be calculated.

[0087] After obtaining U, R, V, and f through the aforementioned steps, the discretized decision system S can be output.

[0088] S202, calculate all single attributes and key combined attributes subsets in the discretized decision system S to obtain the positive domain of all single attributes and key combined attributes.

[0089] The positive domain is obtained by dividing the equivalence classes through indistinguishable relations.

[0090] Specifically, if X is a subset of the set of objects U, then For each X and indistinct relation Define the lower approximation set B of X * (X), a set of objects in U that must belong to X according to knowledge B (knowledge B represents the ability to classify or distinguish objects in the universe of discourse U). X represents any subset of U. When B is a single attribute, the equivalence class and positive domain of a single conditional attribute are calculated. All conditional attribute equivalence relations include the results calculated independently for each single attribute. By combining key attribute subsets B, multiple attributes can be used to enhance classification capabilities.

[0091] The upper approximation set B of X * (X), a set of objects in U that may belong to X based on knowledge B. Y represents any one of the objects in U.

[0092] For conditional attributes, if the decision values ​​of objects in the indistinguishable set are the same, the decision system is compatible; otherwise, it is incompatible. Calculate all conditional attribute equivalence relations in the compatible decision system. For each equivalence relation, the upper and lower approximations relative to the decision values ​​are obtained through calculation, based on the heuristic function:

[0093] λ=(|B * (X)|-|B * (X)|) / |U|.

[0094] Where λ is the classification uncertainty measure of the current attribute subset B to the set X, the smaller the value, the more accurate the classification. By optimizing λ, the most critical features for fault diagnosis can be automatically selected, balancing model accuracy and efficiency. |U| is used to normalize the proportion of fuzzy regions, making λ a relative uncertainty measure (within the range [0,1]).

[0095] The attribute with the smallest λ value is retained. Given a universe of discourse, the lower approximation is used as the pruning criterion. Redundant features (such as irrelevant metal concentrations in oil spectra) are removed through reduction. For each candidate attribute subset B (each single attribute is the smallest unit of the candidate subset, and its λ value needs to be calculated separately as a benchmark for combined attributes), its λ value is calculated. The B that minimizes λ is selected as the reduction result, ensuring that the retained features can distinguish between "normal" and "faulty" states to the greatest extent. This reduces model complexity while retaining key diagnostic information. It also improves the training efficiency and generalization ability of subsequent 3D-CNN and other models.

[0096] In attribute reduction, the primary choice is to select the subset of attributes B that minimizes λ. When λ approaches 0, it indicates the disappearance of fuzzy regions, meaning B can accurately classify X. The λ value of the combined attributes must be significantly lower than that of the single attributes (e.g., λ ≥ 0.3 for a single attribute, while λ = 0.05 for the combined attribute), thus demonstrating the necessity of multi-attribute collaboration. When λ approaches 1, it indicates that the λ value of a single attribute is too high, fuzzy regions disappear, and B cannot provide effective classification information. This attribute has poor classification ability on its own and may be eliminated.

[0097] S203, based on the positive domains of all single attributes and key combined attributes, perform heuristic attribute reduction to obtain the reduced oil and / or ferrographic feature set.

[0098] Specifically, the attribute with the smallest λ value is retained. Given a set of objects, the following approximation is used as the pruning criterion:

[0099] U = U-POS red {D};

[0100] Among them, POS red {D} represents the set of objects whose conditional attributes for the reduced set red (the reduced oil and / or ferrographic feature set red) can be classified into the decision attribute D in the universe of discourse U. In other words, it represents the positive domain (union of lower approximation sets) of the reduced set red with respect to the decision attribute D. This process is repeated, selecting the remaining attributes. Each time, the retained attributes are combined with the attributes in the reduced set red to form new equivalence relations. This recursive operation continues until the given universe of discourse is empty. The recursion terminates when the remaining attributes cannot further reduce the value of λ, or when U is empty.

[0101] S300, the original vibration signal is converted to time and frequency using continuous wavelet transform to obtain a time-frequency sequence.

[0102] Specifically, obtaining the time-frequency graph sequence may include the following steps:

[0103] S301, construct the basic wavelet function.

[0104] The scaling and translation formulas for the basic wavelet function ψ(t) are as follows:

[0105]

[0106] Where ψ represents the mother wavelet, k represents the scaling parameter (e.g., k∈[2,256] logarithmically distributed), which can uniformly cover the signal frequency band, and the specific range is set according to the equipment fault characteristic frequency, l represents the translation parameter, and t represents time;

[0107] For example,

[0108] Where, π -1 / 4 , represents the normalization coefficient, e represents the base of the natural logarithm, i represents the imaginary unit, w o , where represents the center frequency of the wavelet, and t represents the time variable. This represents the Gaussian envelope function. For example, the Morlet wavelet center frequency w... o =6, which satisfies the allowable condition.

[0109] S302, For the original vibration signal, continuous wavelet transform is used for time-frequency conversion to obtain the calculation result.

[0110] The calculation formula for the continuous wavelet transform is as follows:

[0111]

[0112] in, Let CWT(s)(k,l) represent the conjugate function of the wavelet function, s(t) represent the original vibration signal, and dt represent the integral variable. The CWT continuous wavelet transform coefficient matrix is ​​converted into a grayscale image (CWT coefficients are complex numbers, such as taking the modulus |CWT(s)(k,l)|), for example, (size H×W×1, used as the temporal input channel for 3D-CNN spatiotemporal modeling).

[0113] S303, normalize the calculation results to obtain a time-frequency diagram sequence.

[0114] Normalization is performed independently for each scale k, and can be achieved using conventional normalization methods. This application does not specify how normalization is implemented. For example, the CWT coefficients can be normalized using a max-min normalization method, mapping them to the interval [0, 1].

[0115] Continuous wavelet transform (CWT) enables multi-scale analysis of vibration signals through extension and shift functions, thereby effectively extracting time-frequency domain information. CWT has low computational complexity and good time-frequency localization characteristics.

[0116] S400, the reduced oil and / or ferrographic feature set is used as an additional feature channel and fused with the time-frequency graph sequence using a three-dimensional convolutional neural network. A self-attention mechanism is introduced to dynamically weight the time steps to obtain a health indicator sequence.

[0117] Specifically, obtaining a sequence of health indicators may include the following steps:

[0118] S401, the reduced oil and / or ferrographic feature set is used as an independent channel, and through spatial interpolation, it is extended to the same resolution as the time-frequency graph sequence to form an additional feature channel.

[0119] Specifically, spatial interpolation extends additional features, such as conventional bilinear interpolation methods, which extend a one-dimensional time series into a two-dimensional spatial feature map.

[0120] For example, the temporal characteristics of the oil / ferrography are mapped to a time-frequency spectrum space to capture its spatial correlation with vibrational energy. For instance, regions with high metal concentrations may correspond to high-frequency energy concentration areas in the time-frequency spectrum.

[0121] Oil characteristics: shape (N, L, Cl);

[0122] Ferrographic characteristics: shape (N, L, C2);

[0123] After spatial interpolation:

[0124] Oil spread characteristics: (N, H, W, L, Cl);

[0125] Ferrographic extension characteristics: (N, H, W, L, C2).

[0126] S402, the additional feature channels are fused with the time-frequency map sequence to generate a fused input tensor with shape (N, H, W, L, C), where N represents the number of samples, H represents the height of the time-frequency map, W represents the width of the time-frequency map, L represents the number of overlaps in the time-frequency map sequence, and C represents the number of channels in the time-frequency map sequence.

[0127] Specifically, the input tensor can be generated by concatenating along the channel dimension. In the channel dimension, each channel represents the spatiotemporal features of a different mode (e.g., channel 1 represents vibrational energy, channels 2-4 represent oil metal concentration, and channels 5-6 represent ferrographic abrasive parameters). In the spatial dimension, the (H, W) of the time-frequency plot corresponds to the frequency-time resolution, and the expanded oil / ferrographic features are aligned with it, facilitating the convolution kernel to capture cross-modal local correlations.

[0128] S403, construct at least three 3D convolutional layers, each using the ReLU activation function, to perform a unified nonlinear transformation on the input time-frequency plot sequence and the reduced oil and / or ferrographic features, and extract local correlation features in the time-frequency domain.

[0129] For example, the kernel sizes of the 3D convolutional layers are 7×7×2, 3×3×2 and 3×3×2, and the stride is set to 2×2×1.

[0130] Spatial dimensions (H, W): The large core (7×7) captures global frequency domain patterns (such as broadband resonance of bearing failure), while the small core (3×3) extracts local details (such as high-frequency harmonics of gear meshing).

[0131] Time dimension (L): The kernel size is 2, which associates features of adjacent time steps to model the evolution of faults (such as the increase in vibration energy caused by gradual changes in oil concentration).

[0132] It should be noted that the ReLU activation function is applied immediately after the convolution operation in each 3D convolutional layer to perform an element-wise nonlinear transformation on the output feature map. This is to break the linear constraint, as the relationship between the time-frequency map (vibration signal) and oil / ferrographic features can be highly nonlinear. For example, a sudden increase in the metal concentration of the oil may be related to a nonlinear increase in the high-frequency energy of vibration, and simple linear convolution cannot capture such complex patterns. In addition, for multimodal feature fusion, ReLU allows the model to learn nonlinear interaction features across modes (vibration, oil, ferrography), such as the joint effect of oil parameter changes and vibration energy diffusion.

[0133] For example, suppose that during the early wear of a bearing, the concentration of iron particles in the oil (oil characteristics) is nonlinearly positively correlated with the high-frequency energy (vibration characteristics) in the vibration time-frequency diagram. ReLU, through nonlinear mapping, enables the model to fit this complex relationship, rather than relying solely on linear combinations.

[0134] It should also be noted that, through 3D convolutional layers and the ReLU activation function, the features of different modes (vibration time-frequency plots, oil, and ferrography) are uniformly encoded into the same nonlinear space. In this way, regardless of whether the input is vibration, oil, or ferrography features, the same nonlinear transformation (ReLU) is used to extract cross-modal correlated features, achieving unified processing. Features from all modes share the same set of convolutional kernel parameters and activation functions, ensuring consistent feature transformation rules rather than independent processing, facilitating subsequent computation steps.

[0135] S404, the fused input tensor is input into the three 3D convolutional layers. In each 3D convolutional layer, the ReLU activation function is used to perform a nonlinear transformation on the fused input tensor. The multi-source features of adjacent time-frequency maps are associated through the 3D convolutional layers in the time dimension to obtain the associated multi-source fused features.

[0136] The multi-source fusion features include vibration time-frequency diagrams, oil fluid and ferrographic features.

[0137] Specifically, by sliding a 3D convolutional kernel along the time axis, multimodal features from adjacent time steps are weighted and fused to capture the evolution of fault features. The ReLU function is used to perform a nonlinear transformation on the convolution results to suppress irrelevant responses and enhance cross-modal correlation features. For example, with a time kernel size of 2 (meaning each convolutional operation is associated with two consecutive time steps, such as t and t+1), if the oil concentration suddenly increases at time step t, and the vibration energy increases at t+1, the convolutional kernel automatically learns this causal relationship and outputs the tensor shape.

[0138] In a 3D convolutional kernel structure, the feature map of each convolutional layer is connected not only to the current time-frequency map (TFR) but also to multiple adjacent TFRs, thus enabling the extraction of more temporal information. In contrast, traditional 2D convolution can only extract local features from a single TFR, failing to fully utilize the temporal dependence and spatial features of diesel engine vibration signals. By constructing a TFR cube with continuous time steps, 3D convolution can not only extract the spatial features of a single TFR but also capture the temporal characteristics between adjacent TFRs, improving the accuracy of fault mode identification.

[0139] For example, after three layers of convolution processing, the fused input tensor (N, 128, 256, 10, 6) gradually evolves into (N, 64, 128, 9, 64), (N, 32, 64, 8, 128), and (N, 32, 64, 7, 256), and each channel in the final output represents a cross-modal correlation mode of vibration, oil, and ferrography features.

[0140] It should be noted that each 3D convolutional kernel contains learnable weight parameters, extracting local feature patterns in the spatial dimension (H×W) and learning the weight relationships between features at adjacent time steps in the temporal dimension (L). These weight parameters are automatically optimized during the training process, enabling the model to capture the spatiotemporal evolution of fault features.

[0141] During convolution, vibration time-frequency maps, oil characteristics, and ferrographic features are calculated uniformly using shared convolution kernel parameters, achieving cross-modal feature fusion. Each output channel comprehensively reflects the correlation pattern of multi-source features.

[0142] For example, when the first convolutional kernel is processed at time step t=5, it will simultaneously consider:

[0143] Oil characteristics at t=5 (e.g., iron concentration 0.9);

[0144] Vibrational characteristics at t=6 (e.g., high-frequency energy 0.8);

[0145] By training, strong correlation weights (such as 0.7) are established between the features of these two time steps, thereby encoding the delay effect of the fault.

[0146] S405, after the at least three 3D convolutional layers, construct two fully connected layers and perform a Dropout operation between the two fully connected layers.

[0147] The fully connected layer includes a first fully connected layer using the ReLU activation function and a second fully connected layer using the Sigmoid activation function to output health indicators. Each fully connected layer consists of multiple neurons (computational units), with each neuron receiving all the input from the previous layer and performing a linear transformation using a weight matrix and biases.

[0148] In the preceding steps, the associated multi-source fusion features were obtained, with an example shape of (N, 32, 64, 7, 256, where N is the number of samples). Global average pooling was then performed to compress the spatial dimension to a single point, while retaining the time step and channel information. The pooled shape is (N, 1, 1, 7, 256). Redundant spatial dimensions (H = 1, W = 1) were removed, and the output shape became (N, 7, 256). Next, the time step and channel dimensions were merged to obtain a two-dimensional feature matrix (with an example shape of (N, 7 × 256 = 1792)).

[0149] It should be noted that the role of the first fully connected layer is to reduce the features from 1792 dimensions to 512 dimensions and use the ReLU activation function to filter out nonlinear negative values.

[0150] ReLU activation function:

[0151] ReLU(v) = max(0, v);

[0152] Where v is the input value.

[0153] ReLU(v) output value: output v when v>0, otherwise output 0.

[0154] Sigmoid activation function:

[0155]

[0156] Where v is the input value, e is the natural constant, and σ(v) is the output value, ranging from (0, 1), representing the probability output of a binary classification problem.

[0157] v = W·X + b;

[0158] Where W is the weight of the fully connected layer, b is the bias, and X is the pooled feature (i.e., the two-dimensional feature result after compression of the multi-source fusion feature output by 3D convolution).

[0159] The purpose of the Dropout layer is to randomly block neurons with a 50% probability during training to prevent the model from overfitting.

[0160] The second fully connected layer maps the 512-dimensional features to the target dimension (such as predicting the next 5 time steps) and outputs health indicators (value range [0, 1]) through the Sigmoid function.

[0161] S406, the associated multi-source fusion features are input into the first fully connected layer, and the ReLU activation function is used for dimensionality reduction and nonlinear activation to obtain the features processed by the first fully connected layer.

[0162] Specifically, the reshaped two-dimensional features (e.g., (N, 1792)) are input and linearly transformed using a weight matrix and a bias term. The results are then non-linearly activated using the ReLU function to suppress negative responses, resulting in a 512-dimensional feature output (shape: (N, 512)). The aim is to extract high-order abstract features across modalities, such as the correlation between sudden increases in vibrational energy and oil metal concentration.

[0163] S407, Perform a Dropout operation on the features processed by the first fully connected layer, randomly discarding neurons with a preset probability to obtain Dropout-processed features that are not overfitted.

[0164] Specifically, during the training phase, the features processed by the first fully connected layer (composed of the outputs of multiple neurons) are randomly set to zero with a preset probability (e.g., 50%), allowing the model to learn redundant feature pathways. During the inference phase, all neurons are retained, but their weights are scaled probabilistically to maintain output stability. Example: If a neuron is dedicated to oil ferrographic concentration features, Dropout will force the model to simultaneously rely on vibrational features for prediction.

[0165] S408, the features processed by Dropout are input into the second fully connected layer, and the Sigmoid activation function is used to map the features into a health indicator sequence to obtain the health indicator sequence.

[0166] Specifically, the features processed by Dropout are input into the second fully connected layer, and the output is mapped to the [0, 1] interval by the Sigmoid function.

[0167] For example, the output is a matrix of shape (N, 5) representing the health indicators for the next 5 time steps, such as [0.1, 0.3, 0.5, 0.7, 0.9]. The closer the value is to 1, the higher the risk of failure. The health indicators comprehensively reflect the combined state of vibration, oil, and ferrography. For example, 0.9 indicates that immediate maintenance is required.

[0168] It should be noted that a dropout layer is added after the last convolutional layer to prevent overfitting. Furthermore, the last layer uses the Sigmoid activation function to fit the health metric (HI), while the other layers use the ReLU activation function to avoid the vanishing gradient problem.

[0169] Through the collaboration of fully connected layers and Dropout, the model gradually maps multi-source fusion features into an intuitive sequence of health indicators, capturing the complex patterns of fault evolution while avoiding over-reliance on training data, thus meeting the robustness requirements of industrial scenarios.

[0170] S500, based on the health indicator sequence, a Gaussian process regression model is used to predict the fault trend, and the fault trend prediction is obtained.

[0171] Specifically, obtaining a fault trend prediction may include the following steps:

[0172] S501, Establish a Gaussian process regression model.

[0173] The formula for the Gaussian process regression (GPR) model is as follows:

[0174] f(x)~GP(m(x),k(x,x'));

[0175] Where m(x) is the mean function, representing the baseline trend of the Health Indicator (HI), and k(x, x') is the covariance function, which measures the similarity of HI sequences across different time windows.

[0176] S502 divides the health indicator sequence into segments according to a preset time window and normalizes them to obtain the segmentation results.

[0177] Specifically, the health indicator sequence is divided according to a preset window. For example, the health indicator sequence is divided into input-output pairs according to a time window (e.g., window size k = 10, step size s = 1).

[0178] Next, the input-output pairs are standardized to eliminate differences and achieve normalization. The specific normalization method can be Z-Score or other conventional methods.

[0179] S503, input the segmentation result into the Gaussian process regression model to obtain the fault prediction result.

[0180] Specifically, by calculating the covariance matrix, the input-output pair is used to train the GPR model. The GPR model outputs predictions for several future time windows. The predictions for these time windows are then averaged to form a prediction mean, reflecting future trends. Additionally, the GPR model outputs prediction variance, which represents the prediction confidence level (the smaller the variance, the higher the confidence level).

[0181] The reliability of the prediction results is judged by the prediction variance, thereby obtaining the prediction results of the fault.

[0182] It should be noted that the input (training data) consists of segments of the historical Health Indicator (HI) sequence divided into fixed time windows. The step size is 1, and the sliding window moves forward by 1 time step at a time, generating multiple input-output pairs. The output (prediction target) is the HI value at the next m time steps.

[0183] In the Gaussian process regression model, the mean function m(x) describes the baseline trend of HI, and the covariance function k(x, x') measures the similarity between different time windows.

[0184] For example, the training phase of a Gaussian process regression model involves learning the parameters of the covariance function from historical input-output pairs to establish a time-dependent pattern for the HI sequence. The prediction phase involves inputting the HI sequence for the current time window, calculating the covariance matrix between the current window and historical windows, and generating predicted HI values ​​for future time steps using a weighted historical similarity pattern, such as [0.85, 0.90, 0.95, 0.98, 1.0], reflecting an upward trend in failure risk. Prediction variance (reliability) is also considered; a smaller variance indicates a more reliable prediction result, while a sudden increase in variance may indicate unknown interference. The application of the prediction results involves taking the moving average of the predicted mean sequence (e.g., a mean of 0.94 for the next 5 steps) as a comprehensive indicator of failure risk. If the mean consistently exceeds a threshold (e.g., 0.9), an early warning is triggered.

[0185] In one embodiment, the step of fusing the reduced oil and / or ferrographic feature set as an additional feature channel with the time-frequency graph sequence using a three-dimensional convolutional neural network, and introducing a self-attention mechanism to dynamically weight the time steps to obtain a health indicator sequence, includes:

[0186] In marine diesel engine fault diagnosis, data is input into feature engineering, which divides the data into test, validation, and training sets. The training set is used to train time-frequency analysis, 3D-CNN spatiotemporal modeling, and to construct a health indicator. This health indicator is also used as a self-attention mechanism in the validation set for fault prediction, and the parameters of the 3D-CNN are adjusted based on the fault prediction results. After the model is finally formed, the data from the test set is sent to the model for testing to obtain prediction results. The associated multi-source fusion features are linearly transformed into a Q / K / V matrix using a trainable parameter matrix. The shape of the associated multi-source fusion features is (N, T, d), where N represents the number of samples, T represents the number of time steps, and d represents the feature dimension. The Q / K / V matrix includes a query matrix Q, a key matrix K, and a value matrix V. The formula for the Q / K / V matrix is:

[0187] Q = HI·W Q ;

[0188] K = HI·W K ;

[0189] V = HI·W v ;

[0190] in, W Q W K W v All of these are trainable parameters in the trainable parameter matrix.

[0191] The associated multi-source fusion features are linearly transformed to generate a query matrix Q = HI·W Q The bond matrix K = HI·W K Value matrix V = HI·W v .

[0192] By scaling the dot product, the influence weight of each time step on other time steps is calculated and normalized to generate a probability distribution weight matrix.

[0193] The formulas for scaling the dot product and normalization are conventional formulas and methods, and this application does not limit them.

[0194] Using the probability distribution weight matrix, the value matrix V is weighted and summed to obtain the optimized feature representation, wherein the formula for the optimized feature representation is:

[0195] Z = A·V;

[0196] Where Z represents the optimized feature representation, and A represents the probability distribution weight matrix.

[0197] The value matrix is ​​weighted and summed using the probability distribution weight matrix to obtain the optimized feature representation Z, where the feature at each time step is a weighted combination of the features at other time steps.

[0198] The optimized feature representation is input into a fully connected layer to generate a health indicator sequence, and the trainable parameters of the three-dimensional convolutional neural network and the self-attention mechanism are jointly optimized through backpropagation.

[0199] The self-attention mechanism is modular, making it easy to adjust.

[0200] In summary, the core advantage of the ST-CNN structure lies in its ability to simultaneously extract feature information across both temporal and spatial dimensions. As a 3D-CNN spatiotemporal modeling approach, compared to traditional two-dimensional convolutional networks (2D-CNN), 3D-CNN can more comprehensively capture the operating status of marine diesel engines and accurately identify wear and tear fault modes. Furthermore, by incorporating a self-attention mechanism (SAM), this method can adaptively allocate weights at different time steps, improving the extraction capability of key features and avoiding the decline in diagnostic accuracy caused by improper feature selection in traditional methods. Finally, the GPR method is used to predict the health status of the diesel engine, quantifying the uncertainty of the prediction results and improving the reliability and robustness of fault diagnosis.

[0201] Secondly, this application provides a fault diagnosis system based on a deep spatiotemporal convolutional network (ST-NN). Applying the aforementioned fault diagnosis method based on ST-NN, the system includes:

[0202] The acquisition unit is used to acquire raw oil spectral and / or ferrographic data, as well as raw vibration signals;

[0203] The rough set feature set is used to reduce the dimensionality of the original oil spectral and / or ferrographic data by using the rough set attribute reduction method to remove redundant features and obtain the reduced oil and / or ferrographic feature set.

[0204] The time-frequency unit is used to perform time-frequency conversion on the original vibration signal using continuous wavelet transform to obtain a time-frequency graph sequence;

[0205] The fusion unit is used to fuse the reduced oil and / or ferrographic feature set as an additional feature channel with the time-frequency graph sequence through a three-dimensional convolutional neural network, and introduce a self-attention mechanism to dynamically weight the time steps to obtain a health indicator sequence.

[0206] The result unit is used to perform fault trend prediction using a Gaussian process regression model based on the health indicator sequence, and obtain fault prediction results.

[0207] Thirdly, this application provides a computer device including a memory and a processor, wherein the memory stores a computer program, and the processor executes the steps of the aforementioned method.

[0208] Example

[0209] For the information system DS=(U,C∪D,V,f), the heuristic attribute reduction algorithm is as follows:

[0210] 1) Initialize the reduced set red to (Empty set), C is initialized with all conditional attributes;

[0211] 2) Calculate the λ value of each conditional attribute in C using equation (1);

[0212] 3) Sort the λ values ​​in descending order and select the attribute with the smallest λ value. If attributes have the same λ value, select the attribute with the smallest value. If the input values ​​are the same and the attribute values ​​are also the same, randomly select an attribute.

[0213] 4) Select attribute a i Add the reduced set red, and remove attribute a from C. i Execution formula (2);

[0214] 5) If U is Proceed to step 7; otherwise, proceed to step 6.

[0215] 6) Combine the attributes in the reduced set red with the remaining attributes in C, and repeat steps 2 to 5.

[0216] 7) Output the reduced set red, and the reduction calculation ends.

[0217] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and RAMbus dynamic RAM (RDRAM), etc.

[0218] The various embodiments in this disclosure are described in a progressive manner. The same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on describing the differences from other embodiments.

[0219] The scope of protection of this disclosure is not limited to the embodiments described above. Obviously, those skilled in the art can make various modifications and variations to this disclosure without departing from its scope and spirit. If such modifications and variations fall within the scope of the claims of this disclosure and their equivalents, then the intent of this disclosure also includes such modifications and variations.

Claims

1. A fault diagnosis method based on a deep spatiotemporal convolutional network (ST-NN), characterized in that the method... include: Obtain raw oil spectral and ferrographic data, as well as raw vibration signals; The original oil spectral and ferrographic data were subjected to dimensionality reduction using the rough set attribute reduction method to remove redundant features, resulting in a reduced oil and ferrographic feature set. The original vibration signal is converted to time and frequency using continuous wavelet transform to obtain a time-frequency sequence. The reduced oil and ferrographic feature sets are used as additional feature channels and fused with the time-frequency graph sequence using a three-dimensional convolutional neural network. A self-attention mechanism is introduced to dynamically weight the time steps to obtain a health indicator sequence. Based on the health indicator sequence, a Gaussian process regression model is used to predict the fault trend, and the fault prediction results are obtained. The step of fusing the reduced oil and ferrographic feature sets as additional feature channels with the time-frequency graph sequence using a three-dimensional convolutional neural network, and introducing a self-attention mechanism to dynamically weight the time steps to obtain the health indicator sequence, includes: The reduced oil and ferrographic feature sets are used as independent channels, and spatial interpolation is used to extend them to the same resolution as the time-frequency plot sequence to form additional feature channels. The additional feature channels are fused with the time-frequency map sequence to generate a fused input tensor with shape (N, H, W, L, C), where N represents the number of samples, H represents the height of the time-frequency map, W represents the width of the time-frequency map, L represents the number of overlaps in the time-frequency map sequence, and C represents the number of channels in the time-frequency map sequence. The step of fusing the reduced oil and ferrographic feature sets as additional feature channels with the time-frequency graph sequence using a three-dimensional convolutional neural network, and introducing a self-attention mechanism to dynamically weight the time steps to obtain the health indicator sequence, includes: Construct at least three 3D convolutional layers, each using the ReLU activation function, to perform a unified nonlinear transformation on the input time-frequency plot sequence and the reduced oil and ferrographic features, thereby extracting local correlation features in the time-frequency domain; The fused input tensor is input into the three 3D convolutional layers. In each 3D convolutional layer, the ReLU activation function is used to perform a nonlinear transformation on the fused input tensor. The multi-source features of adjacent time-frequency maps are associated through the 3D convolutional layers in the time dimension to obtain the associated multi-source fused features. The multi-source fused features include vibration time-frequency maps, oil and ferrography features.

2. The fault diagnosis method based on a deep spatiotemporal convolutional network (ST-NN) according to claim 1, characterized in that, The step of performing dimensionality reduction on the original oil spectral and ferrographic data using a rough set attribute reduction method to remove redundant features and obtain the reduced oil and ferrographic feature sets includes: The continuous properties of the original oil spectral and ferrographic data are discretized using equal-frequency binning to generate a discretized decision system. ,in, , represents a collection of objects For a collection of attributes, It is a subset of the conditional properties of oil spectra and ferrograms. This is a subset of the fault state decision attributes. , indicating the attribute value range , representing an information function, determines the attribute values ​​of a sample. ; Calculate the discretized decision system The positive domain of all single attributes and key combined attributes is obtained by dividing the subset of all single attributes and key combined attributes into equivalence classes through indistinguishable relations. Based on the positive domains of all single attributes and key combined attributes, heuristic attribute reduction is performed to obtain the reduced oil and ferrographic feature sets.

3. The fault diagnosis method based on the deep spatiotemporal convolutional network ST-NN according to claim 1, characterized in that, The step of performing time-frequency conversion on the original vibration signal using continuous wavelet transform to obtain a time-frequency graph sequence includes: Construct a basic wavelet function, wherein the basic wavelet function The scaling and translation calculation formulas are as follows: ； in, , representing the mother wavelet, , representing the scale parameter, , represents the translation parameter, Indicates time; For the original vibration signal, a continuous wavelet transform is used for time-frequency conversion to obtain the calculation result. The calculation formula for the continuous wavelet transform is as follows: ； in, , represents the conjugate function of the wavelet function. , represents the continuous wavelet transform coefficient matrix. This represents the original vibration signal. , represents the integration variable; The calculation results are normalized to obtain a time-frequency graph sequence.

4. The fault diagnosis method based on a deep spatiotemporal convolutional network (ST-NN) according to claim 1, characterized in that, The step of fusing the reduced oil and ferrographic feature sets as additional feature channels with the time-frequency graph sequence using a three-dimensional convolutional neural network, and introducing a self-attention mechanism to dynamically weight the time steps to obtain the health indicator sequence, includes: After the at least three 3D convolutional layers, two fully connected layers are constructed, and a Dropout operation is performed between the two fully connected layers. The fully connected layers include a first fully connected layer that uses the ReLU activation function and a second fully connected layer that uses the Sigmoid activation function to output health metrics. The associated multi-source fusion features are input into the first fully connected layer, and the ReLU activation function is used for dimensionality reduction and nonlinear activation to obtain the features processed by the first fully connected layer. The features processed by the first fully connected layer are subjected to a Dropout operation, in which neurons are randomly dropped with a preset probability to obtain Dropout-processed features that are not overfitted. The features processed by Dropout are input into the second fully connected layer, and the Sigmoid activation function is used to map the features into a health indicator sequence to obtain the health indicator sequence.

5. The fault diagnosis method based on the deep spatiotemporal convolutional network ST-NN according to claim 4, characterized in that, The step of fusing the reduced oil and ferrographic feature sets as additional feature channels with the time-frequency graph sequence using a three-dimensional convolutional neural network, and introducing a self-attention mechanism to dynamically weight the time steps to obtain the health indicator sequence, includes: The associated multi-source fusion features are linearly transformed using a trainable parameter matrix, mapping them to a Q / K / V matrix. The shape of the associated multi-source fusion features is (N, T, d), where N represents the number of samples, T represents the number of time steps, and d represents the feature dimension. The Q / K / V matrix includes a query matrix Q, a key matrix K, and a value matrix V. The formula for the Q / K / V matrix is: ； ； ； in, , All of these are trainable parameters in the trainable parameter matrix; By scaling the dot product, the influence weight of each time step on other time steps is calculated and normalized to generate a probability distribution weight matrix; Using the probability distribution weight matrix, the value matrix Weighted summation is performed to obtain the optimized feature representation, wherein the formula for the optimized feature representation is: ； in, , representing the optimized feature representation, , represents the probability distribution weight matrix; The optimized feature representation is input into a fully connected layer to generate a health indicator sequence, and the trainable parameters of the three-dimensional convolutional neural network and the self-attention mechanism are jointly optimized through backpropagation.

6. The fault diagnosis method based on the deep spatiotemporal convolutional network ST-NN according to claim 5, characterized in that, The step of using a Gaussian process regression model to predict fault trends based on the health indicator sequence and obtaining fault prediction results includes: Establish a Gaussian process regression model; The health indicator sequence is segmented according to a preset time window and normalized to obtain the segmentation results; The segmentation results are input into a Gaussian process regression model to obtain fault prediction results.

7. A fault diagnosis system based on a deep spatiotemporal convolutional network (ST-NN), characterized in that, The fault diagnosis method based on a deep spatiotemporal convolutional network (ST-NN) according to any one of claims 1-6, the system comprising: The acquisition unit is used to acquire raw oil spectral and ferrographic data, as well as raw vibration signals; The rough set attribute reduction element is used to reduce the dimensionality of the original oil spectral and ferrographic data by using the rough set attribute reduction method, remove redundant features, and obtain the reduced oil and ferrographic feature sets. The time-frequency unit is used to perform time-frequency conversion on the original vibration signal using continuous wavelet transform to obtain a time-frequency graph sequence; The fusion unit is used to fuse the reduced oil and ferrographic feature sets as additional feature channels with the time-frequency graph sequence using a three-dimensional convolutional neural network, and introduces a self-attention mechanism to dynamically weight the time steps to obtain a health indicator sequence. The result unit is used to perform fault trend prediction using a Gaussian process regression model based on the health indicator sequence, and obtain fault prediction results.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1-6.

Images