Bearing diagnosis method and system based on variable working condition multi-modal data fusion

Abstract

The bearing diagnosis method and system based on variable working condition multi-modal data fusion provided in the application propose a solution of mutual verification of a defect decision level fusion diagnosis process and an existing signal analysis technology, so as to realize the design purpose of high diagnosis precision and better generalization ability for the actual running working condition of the bearing in a complex environment, thereby solving the defects of poor accuracy and robustness of the bearing data diagnosis in the prior art. Based on the multi-head self-attention mechanism in the Transformer and MoobileViT models, the representation of different positions is calculated at the same time, so that the model has better parallel computing capability, and the IECANet and parallel double attention mechanism are introduced into the Transformer and MoobileViT respectively, the data fusion features of the model in the spatial dimension are strengthened, and the training and reasoning processes are carried out. The performance on different tasks is good, which is not only limited to the processing of sequence data, but also can be applied to the field of image processing.

Description

TECHNICAL FIELD

[0001] The application relates to a bearing defect decision-level fusion diagnosis method based on variable working condition multi-modal data and a diagnosis system thereof, and belongs to the technical field of deep learning and fault diagnosis. BACKGROUND

[0002] Bearing is a necessary and core structure of a transmission device. Common faults caused by failure of a bearing carrying state include fatigue shedding, wear, fracture, etc., and such faults are more concentrated between an inner ring and an outer ring. The probability of bearing failure is higher under the conditions of long-time overloading, poor lubrication and incorrect installation. The rolling bearing has poor impact resistance and is prone to failure due to high rotating speed, corrosion and rust. The bearing of a vertical motor is important and vulnerable, and effective monitoring and diagnosis of the state of the bearing are particularly important due to the characteristics of the motor structure and the use mode, long-time high-load operation, high temperature, high humidity, high salt, strong vibration and other harsh working environments.

[0003] With the increasing requirement for bearing operation reliability, bearing lubrication state monitoring usually collects original vibration data of the motor bearing and components near the bearing seat by using a vibration sensor. However, the lubrication state signal is concentrated in high-frequency components, and multiple vibration sensors are often needed to obtain vibration original related data. The installation position of the vibration sensor may cause attenuation or interference of the vibration signal, thereby limiting effective monitoring of bearing lubrication and early weak faults, and the vibration signal is attenuated at the workpiece and the combination interface. On the one hand, the vibration signal is attenuated when passing through multiple groups of workpieces and the combination interface between different components. The reason is that the contact and transmission between the interfaces absorb part of the vibration energy, resulting in weakening of the vibration signal received by the sensor. On the other hand, there is strong vibration interference, such as vibration sensors installed outside the rotating machinery, which is easily affected by strong vibration interference caused by the bearing, the rotor or incorrect installation. These vibration interferences may cover the relevant information of bearing lubrication or early weak faults of the bearing, making the detection and diagnosis of faults difficult. Due to the above-mentioned adverse factors, the data obtained by the sensor is often affected by a large amount of noise and interference. How to obtain a signal with high signal-to-noise ratio and sensitive to lubrication state is the main solution. In addition, the cost of bearing acoustic emission high-frequency signal is high and the installation and maintenance are complex.

[0004] The prior art bearing diagnosis generally adopts single-mode information collection, so it is difficult to accurately characterize the comprehensiveness and complexity of fault characteristics, only one type of data is processed and analyzed, so the information is not comprehensive, and other types of data cannot be used to provide more comprehensive information; for multi-modal data, there may be certain correlation and semantic information between different modalities, which cannot be captured in the existing single-modal processing, which may result in the loss of some important semantic information.

[0005] In view of this, the present patent application is proposed. SUMMARY

[0006] The bearing diagnosis method and system based on variable working condition multi-modal data fusion provided by the present application solve the problems existing in the prior art by adopting a solution of mutual verification between a defect decision level fusion diagnosis process and existing signal analysis technology, so as to achieve the design purpose of high diagnosis precision and good generalization ability for the actual running working condition of bearings in complex environments, thereby solving the defects of poor accuracy and robustness of bearing data diagnosis in the prior art.

[0007] To achieve the above design purpose, the bearing diagnosis method based on variable working condition multi-modal data fusion proposed by the present application adopts an algorithm based on an attention mechanism architecture, which has many advantages compared with the traditional deep learning algorithms represented by CNN and RNN and the machine learning algorithms such as support vector machine (SVM) and random forest.

[0008] The Transformer architecture based on the attention mechanism introduces a self-attention mechanism, which can better capture long-range dependencies and overcome the problems of gradient disappearance or gradient explosion in RNN models and the disadvantages of difficulty in processing long-distance information transmission.

[0009] The multi-head self-attention mechanism in the Transformer and MoobileViT models simultaneously calculates the representations of different positions, so that the model has better parallel computing capability, and the IECANet and parallel double attention mechanism are introduced into the Transformer and MoobileViT respectively, to strengthen the data fusion features of the model in the spatial dimension, for training and reasoning process. The performance on different tasks is good, not only limited to the processing of sequence data, but also applicable to image processing and other fields using multi-head attention mechanism, which can pay attention to different levels and different types of features, showing good generalization ability and flexibility. Non-linear processing units are used in the deep structure to more accurately extract features and perform abstract modeling.

[0010] The bearing diagnosis method based on variable working condition multi-modal data fusion includes the following implementation steps:

[0011] Step 1, acquire multi-source heterogeneous signals under different working conditions and multiple fault types;

[0012] Step 2, perform data processing such as denoising, data alignment, and standardization on the multi-source heterogeneous signals, and perform data-level fusion on the signals according to the modal dimension to construct a feature vector matrix A1;

[0013] Step 3, apply Fourier transform to each feature vector of the feature vector matrix A1, and extend the dimension of the original feature vector with the Fourier transform result to finally obtain a new feature vector matrix A2;

[0014] Step 4, select different scales for wavelet transform and reconstruction of the constructed feature vector matrix A1 with Complex Morlet wavelet function, select the optimal scale by comparing errors, and calculate the features including amplitude and energy under the current optimal unique scale to generate a gray image data set I1 of wavelet transform;

[0015] Step 5, perform optimal scale STFT feature extraction on the feature vector matrix A1, select a Hamming window with different window sizes and a window overlap degree of 80%, calculate the spectral features under different scales, and perform IFFT reconstruction and error comparison with the original data; select the optimal window size and calculate the corresponding statistical features to generate a gray image data set I2 of the best scale STFT features;

[0016] Step 6, construct a Transformer network architecture;

[0017] The Transformer network model is as follows,

[0018] L1_trans=softmax(FC(Trans depth (PE((Embed(X df )) N )))

[0019] Wherein, the Transformer network architecture stacks the above Trans network on depth layers, X df is the feature vector data constructed from the preprocessed multi-source heterogeneous information in time domain and frequency domain, and the features represented by the N different modalities are fused;

[0020] Embed(·) is an embedding layer function that converts the input feature vector (x) into a representation with a dimension of d_model;

[0021] PE(·) is a position encoding function that adds position encoding to the embedded after embedding to preserve the position information of the sequence;

[0022] Trans depth (·) is a multi-layer self-attention mechanism function for encoding the input sequence;

[0023] FC(·) is a fully connected layer function that maps the encoded representation transformed to the output layer of the classification task;

[0024] The softmax function is used to activate the output layer and get the prediction result of the classification label;

[0025] Step 7, build an improved lightweight MoboileViT network model;

[0026] The MoboileViT network model is represented as,

[0027] L2_vit=softmax(MLP(Trans_Encoder depth (PE((PatchEmbed(X df )) N )))

[0028] In the formula, X df is a CWT and STFT grayscale image respectively after three-channel processing;

[0029] PatchEmbed(·) is a function that divides the input image (x) into image blocks and converts them into embedding vectors through linear projection;

[0030] Trans_Encoder(·) is a depth-layer Transformer encoder function for encoding the image block sequence;

[0031] MLP(·) is a multi-layer perceptron function composed of fully connected layers and activation functions, which maps the encoded representation transformed to the output layer of the classification task;

[0032] Step 8, align the feature vector matrices A2 and A1 obtained in steps 2 and 3 respectively, construct a multi-channel feature matrix and assign initial weights to the channels; apply position encoding to the multi-channel feature matrix, multiply the weight calculation output result with the original feature matrix, and input it to the subsequent decoding layer for training to achieve the preliminary fault classification result L1;

[0033] Step 9, image fusion is performed on the above mapping time-frequency image data I1 and I2, which are divided into two parts in the channel dimension, one part is input into the parallel double attention mechanism to extract local features, and the other part is input into the MoboileViT module. Finally, feature fusion is performed on both in the channel dimension; the encoded layer output feature is mapped to the classification label space of the decoding layer to obtain the fault classification result L2;

[0034] Step 10, based on the weighted coefficient and the aggregation operation method, the results L1, L2 output by the first fault classification model and the second fault classification model are decision-level fused, the probability after comprehensive decision-level fusion is taken as the final result, and the health state of the bearing is obtained;

[0035] The finally predicted bearing state Cpre is expressed as,

[0036] Cpre=FSE(Add(L1,L2))

[0037] Wherein, FSE(·) represents the fuzzy comprehensive evaluation theory, and Add(·) is a weighted fusion operation.

[0038] Further, in the step 1, multiple source signals in multiple different fault type states under different working conditions are obtained as signals of N modalities of the rolling bearing.

[0039] The different fault type settings are classified as data samples belonging to different faults, and the original data of different working conditions under the same fault type is grouped to construct data-level fusion.

[0040] Further, in the step 1, the different working conditions are to keep the temperature and humidity in the working environment unchanged, and to set the initial conditions of the bearing load, i.e. the size and direction of the axial and radial load borne by the bearing. The bearing speed is set in three different ways. Mainly, the stable constant speed, the uniform speed or the periodic change speed, the irregular speed simulation of the actual vertical motor operation condition, including but not limited to sudden acceleration, deceleration, and emergency turning of the output direction and torque of the motor. The change of the speed will cause the change of the frequency spectrum characteristics of the vibration signal.

[0041] Further, in the step 2, the multiple modal signals are subjected to feature extraction and information fusion enhancement processing, and the data after the features of different modalities are spliced, averaged, etc. are subjected to data-level fusion according to the modal dimension to construct a feature vector. The signal processed time series multi-source heterogeneous information is recorded as a feature vector matrix A1.

[0042] Further, in the step 3, the shape (M, N) of the feature vector matrix A1 is obtained, the A1[i] segment is intercepted, and FFT is applied to each A1[i] segment to expand the dimension shape of the original feature vector to (M, K).

[0043] The Fourier transform of the aperiodic continuous-time signal x(t) is expressed as,

[0044]

[0045] The discrete sampling value x(nT) of the continuous signal x(t) is obtained, and the following spectrum of the signal x(t) is calculated using the discrete signal x(nT),

[0046]

[0047] The new feature vector matrix A2 obtained should have the same time step as A1, and frequency interpolation is used for data with a longer step.

[0048] Further, in step 4, the wavelet transform function is as follows,

[0049]

[0050] In the formula, f(t) is a time domain signal, ψ(x) is a wavelet base function, a is a frequency positioning factor, and b is a time positioning factor;

[0051] The Complex Morlet wavelet function is selected;

[0052] The function expression of the Complex Morlet wavelet base is as follows,

[0053]

[0054] In the formula, a is a scale factor, and b is a time shift factor;

[0055] The wavelet transform of the wavelet base function is performed in different scales of 64, 32, and 16 to decompose and isomerize the heterogeneous signal, and the reconstruction error is calculated; a lower reconstruction error indicates a wavelet base function at the current scale; at the selected preferred scale, the required features are calculated; the calculated features are normalized, and the normalized feature values are mapped to a grayscale image;

[0056] A three-channel empty zero matrix is set, and single-channel data is copied to three channels to meet the data input condition of the RGB form of the ViT model in advance. The feature values are uniformly mapped to the grayscale range using linear mapping, larger feature values correspond to brighter pixels, and smaller feature values correspond to darker pixels. The generated image is used as the initial image data set I1.

[0057] Further, in step 5, the STFT basic operation formula is as follows,

[0058]

[0059] In the formula, x(t) is a time domain signal, and h(t-ω) is a window function.

[0060] The above grayscale image generation process includes the following steps:

[0061] S501, STFT adopts a smaller scale Hanning window function to provide a time-frequency image focusing on time domain features;

[0062] The mathematical expression of the Hanning window is as follows:

[0063]

[0064]

[0065] S502, three different window sizes are selected, and the overlap between the windows is set to 80%, and the selected window size and overlap are used for STFT calculation on the feature vector matrix A1;

[0066] For each window size, the FFT of each window is calculated to obtain the spectral information;

[0067] S503, IFFT reconstruction and comparison with original data;

[0068] The reconstructed signal is compared with the original data in terms of error, and the mean square error is used to evaluate the accuracy of the reconstruction, and the best scale is selected;

[0069] S504, calculate the spectral energy or other statistical features of the current scale, for the selected best scale, map the calculated features to a grayscale image, and use the generated image as the initial image data set I2.

[0070] Further, in step 6, the construction process of the Transformer network architecture is as follows,

[0071] S601, the constructed Transformer network architecture includes input sequence, position encoding, encoder / decoder, built-in components, and output layer;

[0072] S602, IECANet channel attention introduces a global maximum pooling operation based on global average pooling GAP to obtain a differentiated attention weighting effect;

[0073] S603, after the multi-head self-attention output, the IECANet channel attention mechanism is applied to weight the feature vectors of each channel; Specifically, before the residual connection, the IECANet channel attention is applied to weight the features of each channel; Then, the weighted features are connected with the input in residual connection, and then normalized.

[0074] Further, in step 8, the fault classification includes the following steps,

[0075] S801, slice the original time sequence information, and then apply FFT processing to each slice to obtain time sequence and frequency domain feature information;

[0076] S802, ensure that the time sequence and frequency domain feature information after time sequence slicing and FFT processing have the same time step; if the steps are inconsistent, interpolation or truncation is performed to make the two have the same time step;

[0077] S803, input the time sequence and frequency domain feature information after step alignment and processing as different channels to construct a multi-channel feature matrix; wherein one channel represents time sequence features and the other channel represents frequency domain features;

[0078] S804, use IECANet channel attention mechanism to calculate the channel weight of the multi-channel feature matrix; IECANet can adjust the weight of each channel according to the relationship and importance between channels, and obtain the weight parameter of each channel through learning;

[0079] S805, apply the weight parameter obtained after channel weight calculation to the multi-channel feature matrix, multiply the features of each channel by the corresponding weight to fuse the channel features, and obtain the final feature representation; the features after channel feature fusion are input into the subsequent network layer;

[0080] S806, according to the trained Transformer fault classification model, map the final feature representation output by the encoder to a specific fault category; through the feature representation and attention mechanism learned by the model, the fault frequency feature can be preliminarily distinguished, and the fault category to which it belongs can be inferred, to obtain a classification result L1.

[0081] Further, in step 9, the fault classification comprises the following steps,

[0082] S901, based on the two groups of mapping time-frequency image data with different emphases obtained in steps 4 and 5 as input tensors, create an empty multi-channel array after image scaling and size adjustment, and use the stacking function of Numpy to stack the image data I1 and I2 together in the form of multi-channel to form a two-dimensional image data set;

[0083] S902, split the channel dimension of the input through the channel attention mechanism; part of the input is sent to the channel space parallel attention mechanism to extract local features and enhance the interaction between different channels, and then a channel attention is connected after the output of the parallel attention to make it on the channel level. After splicing, further fuse its features using channel attention;

[0084] S903, the other part of the input is converted into the form of image block embedding, and position coding is applied to it to give initial weights to the channel attention mechanism; through the learned weight parameters, the input features are extracted and learned in the encoder built-in component, the local extraction fusion features obtained in step S902 are fused with the global features obtained at the encoder level in the form of residual link to capture the local and global correlation of each channel therebetween;

[0085] S904, input the above weighted feature image to the subsequent network layer to map the final classification label space; perform classification prediction to map the output features to the probability distribution of the fault category to realize the preliminary fault classification result L2.

[0086] The bearing diagnosis method based on the multi-modal and multi-scale fusion network is applied, and the following bearing diagnosis system based on the multi-modal and multi-scale fusion network is simultaneously proposed:

[0087] A plurality of lower computers are connected to the upper computer in parallel, or a single lower computer is connected to the upper computer;

[0088] The lower computer adopts a multi-channel data acquisition instrument, and a signal input line with a double-head BNC end and various adapters are used to collect signals of various data types, and various sensors are fixedly arranged at the vertical bearing part;

[0089] The upper computer comprises a rotating speed test module, a core processor module, a data acquisition module and an interactive communication module;

[0090] The memory is used for storing a computer executable program, and the processor reads the computer executable program from the memory and executes;

[0091] And a computer readable storage medium for storing the bearing diagnosis method based on the variable working condition multi-modal data fusion.

[0092] In summary, the bearing diagnosis method and system based on the variable working condition multi-modal data fusion have the following advantages:

[0093] 1. The bearing diagnosis method based on the variable working condition multi-modal data fusion is proposed, which helps to improve the accuracy and efficiency of fault diagnosis and accordingly improves the reliability and maintenance effect of the equipment.

[0094] 2. The multi-modal method has unique advantages in representing the fault state in multiple aspects, can obtain overlapping data to provide data redundancy and fault tolerance performance in data redundancy and fault tolerance, and can obtain multi-aspect feature information of the bearing running state in comprehensive feature extraction by using multiple sensors and measurement techniques.

[0095] 3、The application can capture more extensive context information by fusing features from different scales, and can provide more detailed representation capabilities on the basis of providing different granularities of visual information. Fusing these information can provide more comprehensive perception capabilities and help understand the bearing operation scene under complex working conditions. Therefore, the robustness of the model to installation centering, other noise, shaft stiffness deformation, selection and amount of lubricant, temperature changes of working environment and other interference factors is effectively improved.

[0096] 4、The application can provide more accurate target fault information based on multi-scale feature fusion from multi-modal features of different scales, so as to help the model more accurately locate the specific performance of the target, enhance the representation ability of the model and improve the generalization ability of the model, so that better results can be achieved in different scenes and data distributions.

[0097] 5、The application trains the Transformer network and the MoboileViT model based on the attention mechanism respectively, and obtains the final weighted result by fusing the diagnostic results of the neural network using the DS evidence theory, effectively solving the problem that signals of different dimensions cannot be combined.

[0098] 6、The application sets loads of different sizes or directions and different variable speed conditions, loads different fault bearings to verify the fault classification performance of the method, and comprehensively realizes multi-fault diagnosis and prediction under variable working condition. BRIEF DESCRIPTION OF DRAWINGS

[0099] Figure 1 is a bearing diagnosis method flowchart based on variable working condition multi-modal data fusion;

[0100] Figure 2 is a bearing diagnosis system schematic diagram based on variable working condition multi-modal data fusion;

[0101] Figure 3 is a bearing local defect diagnosis and monitoring parallel expansion system block diagram;

[0102] Figure 4 is a data acquisition module schematic diagram of a multi-channel data acquisition device;

[0103] Figure 5 is a multi-channel data construction schematic diagram and an IECA principle diagram;

[0104] Figure 6 is a Transformer encoder-decoder architecture schematic diagram;

[0105] Figure 7 is an STFT and CWT spectrum mapping fusion operation flowchart;

[0106] Figure 8 Figure 1 is a schematic diagram of a local feature extraction and MoboileViT global feature extraction encoder-decoder architecture. DETAILED DESCRIPTION

[0107] The technical solutions in the embodiments will be described below in conjunction with the drawings accompanying the present application. Obviously, the described solutions are only some embodiments of the present application but not all the embodiments. Based on the embodiments presented in the present application, those skilled in the art can make improvements or transformations without creative efforts, and all these improvements and transformations shall fall within the protection scope of the appended claims of the present application.

[0108] Embodiment 1, as shown in the present application, a bearing diagnosis method based on variable working condition multi-modal data fusion is proposed, which is suitable for real-time monitoring and diagnosis of decision-level fusion of vertical motor bearing defect. Figure 1

[0109] The present application uses a lower computer integrated signal acquisition module to synchronously collect various modal information such as sound, vibration, temperature, etc. during the operation of the bearing; these different types of signals can provide different angles of fault features, so as to more comprehensively evaluate the health status of the bearing. Different types of signals can complement each other in bearing fault detection, vibration signals can provide dynamic characteristics of the bearing, sound signals can provide sound wave patterns of the fault, and temperature signals can reflect the working state of the bearing.

[0110] Through comprehensive analysis of the above signal data, the sensitivity and accuracy of fault detection can be improved, including multi-modal signal acquisition which can improve the sensitivity and accuracy of fault detection, different types of signals can capture different characteristics of bearing faults and can cross-verify each other to reduce the possibility of misjudgment and omission; thereby helping to improve the reliability and precision of fault detection, through continuous acquisition and analysis of various types of signals, the abnormal behavior and trend of the bearing can be discovered in time and the occurrence of faults can be predicted.

[0111] The bearing diagnosis method based on variable working condition multi-modal data fusion comprises the following implementation steps:

[0112] Step 1, acquiring multi-source heterogeneous signals under different working conditions and various fault types;

[0113] Acquiring multi-source signals under different working conditions and various fault types as signals of N modalities of rolling bearings; signal types include sound, vibration, temperature, and rotating speed of the bearing, voltage, current, and power of the motor working state;

[0114] Setting different fault types as data sample classification of different faults, grouping the original data under different working conditions of the same fault type to construct data-level fusion;​

[0115] Adjusting different working condition settings, i.e. setting the bearing load under the constant quantitative conditions of keeping the temperature and humidity in the working environment unchanged, including the bearing bearing axial, radial different direction and force value load, three kinds and more than three kinds of bearing rotating speed, and simulating the actual vertical motor operation working condition (including but not limited to sudden acceleration, deceleration, and controlling the output direction and torque of the motor Emergency turn around, etc., the change of rotating speed will cause the change of frequency spectrum characteristics of vibration signal) with stable constant rotating speed, uniform variable speed or periodic change rotating speed, irregular variable speed;

[0116] Step 2, the multi-source heterogeneous signal is subjected to data processing such as denoising, data alignment and standardization, and data-level fusion is performed on the signal according to the modal dimension to construct a feature vector matrix A1;

[0117] The multi-modal signal is subjected to simple data processing such as feature extraction and information fusion enhancement according to the conventional processing method, and the features of different modalities are spliced and averaged to construct a feature vector mainly composed of vibration, temperature and rotating speed according to the modal dimension; the signal processed and containing time sequence multi-source heterogeneous information is recorded as a feature vector matrix A1;

[0118] The specific processing process includes:

[0119] S201, bearing original time sequence data preprocessing;

[0120] The original time sequence signal collected is preprocessed, and the multi-source modal signal subjected to filtering, denoising, time domain analysis, frequency domain analysis and other preprocessing is sampled, segmented, and time domain features are obtained by calculating features; FFT filtering, spectral analysis and other methods are used to obtain frequency domain features;

[0121] S202, constructing a feature vector matrix;

[0122] First, a series of feature parameters are extracted from the preprocessed multi-source signal, including time domain feature statistics such as kurtosis, kurtosis, margin, pulse index, and frequency domain feature statistics such as peak frequency, spectral energy, frequency band energy ratio, frequency purity, and spectral width;

[0123] The above feature parameters are obtained by time domain analysis, frequency domain analysis and other methods, and a feature vector is constructed by data-level fusion according to the modal dimension. The processed signal contains time sequence multi-source heterogeneous information, which is recorded as a feature vector matrix A1;

[0124] Step 3, for each feature vector of the feature vector matrix A1, Fourier transform is applied, and the Fourier transform result is used to expand the dimension of the original feature vector to finally obtain a new feature vector matrix A2;

[0125] Obtaining shape (M, N) of A1, where M represents the number of eigenvectors, and N represents the dimension of each eigenvector;

[0126] For each eigenvector A1[i], regarding it as a time-domain signal, applying FFT to each time-domain signal; for the FFT result of each time-domain signal, selecting to retain all or part of the frequency components, or applying further processing such as filtering, spectral subtraction, etc.; for each eigenvector, taking the FFT result as part of a new eigenvector to expand the dimension of the original eigenvector; finally obtaining a new eigenvector matrix A2 with shape (M, K), where K represents the expanded eigenvector dimension;

[0127] Further, as shown in Figure 5 For the eigenvector matrix A1, obtaining shape (M, N) of A1, cutting A1[i] segments, and applying FFT to each A1[i] segment to expand the dimension shape of the original eigenvector to (M, K);

[0128] The Fourier transform of the non-periodic continuous-time signal x(t) is represented as follows,

[0129]

[0130] The discrete sampling value x(nT) of the continuous signal x(t) is obtained, and the discrete signal x(nT) is used to calculate the spectrum of the signal x(t) as follows,

[0131]

[0132] The new eigenvector matrix A2 obtained should have the same time step as A1, and frequency interpolation is used for data with longer steps.

[0133] Step 4, the preliminary constructed eigenvector matrix A1 is subjected to wavelet transform and reconstruction by selecting different scales of Complex Morlet wavelet function; the optimal scale is selected by comparing errors, and the features including amplitude and energy are calculated under the current optimal unique scale to generate a gray image dataset I1 of wavelet transform;

[0134] Further, the basic operation formula of wavelet transform is as follows,

[0135]

[0136] In the formula, f(t) is a time-domain signal, ψ(x) is a wavelet base function, a is a frequency position, and b is a time position;

[0137] Further, based on the pulse signal that represents the periodic rolling body fault, which has obvious harmonic components and inner and outer ring fault characteristics with oscillation characteristics or medium-high frequency, the Complex Morlet wavelet function is selected, which is a complex wavelet basis function composed of a Gaussian function and a complex exponential function, and is particularly suitable for analyzing non-stationary signals with oscillation characteristics;

[0138] The function expression of the Complex Morlet wavelet basis is as follows,

[0139]

[0140] In the formula, a is a scale factor, and b is a time shift factor;

[0141] Further, the wavelet transform of the wavelet basis function is performed in different scales of 64, 32 and 16 to decompose and heterogenize the heterogeneous signal, and the reconstruction error is calculated; a lower reconstruction error indicates that the wavelet basis function at the current scale can more accurately restore the original signal, so this scale is selected;

[0142] At the selected preferred scale, the required features including amplitude, energy, variance, etc. are calculated;

[0143] Further, the calculated features are normalized to map the feature values to a suitable range of gray-scale images; normalization can eliminate the differences between different scales, and the normalized feature values are mapped to the gray-scale images.

[0144] A three-channel empty zero matrix is set, and the single-channel data is copied to the three channels to meet the data input condition of the RGB form of the ViT model in advance. The feature values are uniformly mapped to the gray-scale range using linear mapping, larger feature values correspond to brighter pixels, and smaller feature values correspond to darker pixels. The generated image is used as the initial image data set I1.

[0145] Step 5, multi-scale STFT feature extraction is performed on the feature vector matrix A1, different window sizes (such as time scale) are selected and the window overlap degree is set to 80%, the spectral features at different scales are calculated and IFFT reconstruction is performed, and the error is compared with the original data; the optimal scale is selected, the corresponding statistical features (such as spectral energy or others) are calculated, and the gray-scale image of the best scale STFT feature is generated;

[0146] The basic operation formula of STFT is as follows,

[0147]

[0148] In the formula, x(t) is a time-domain signal, and h(t-ω) is a window function;

[0149] Further, the above-mentioned gray image generation process includes the following steps:

[0150] S501, STFT adopts a smaller scale Hanning window function to provide a time-frequency image focusing on time domain features;

[0151] STFT focuses on the time-frequency characteristics of stationary signals, and the time resolution of STFT is determined by the length of the window. A shorter window can provide better time resolution to capture short time domain features in the signal.

[0152] The mathematical expression of the Hanning window is as follows:

[0153]

[0154]

[0155] S502, select three different window sizes, such as window size 64, 32 and 16; Set the overlap degree between the windows to 80%, and use the selected window size and overlap degree to calculate the STFT of the feature vector matrix A1;

[0156] For each window size, A1 is applied step by step, and the FFT (Fast Fourier Transform) of each window is calculated to obtain the frequency spectrum information;

[0157] S503, IFFT reconstruction and comparison with original data;

[0158] Compare the reconstructed signal with the original data for error, use mean square error (MSE) to evaluate the accuracy of reconstruction, and select the best scale;

[0159] S504, calculate the spectrum energy or other statistical features of the current scale, for the selected best scale, map the calculated features to the gray image, and the generated mapping image is taken as the initial image data set I2;

[0160] Step 6, build a Transformer network architecture;

[0161] The Transformer network model is as follows,

[0162] L1_trans = softmax (FC (Trans depth (PE ((Embed (X df )) N )))

[0163] Wherein, the Transformer network architecture stacks depth layers of the above-mentioned Trans network, X dfThe feature vector data constructed in the time domain and the frequency domain of the preprocessed multi-source heterogeneous information is fused with the features represented by the N different modalities;

[0164] Embed(·) is an embedding layer function that converts the input feature vector (x) into a representation with a dimension of d_model;

[0165] PE(·) is a position encoding function that adds position encoding to the embedded after embedding to preserve the position information of the sequence;

[0166] Trans depth (·) is a multi-layer self-attention mechanism function for encoding the input sequence;

[0167] FC(·) is a fully connected layer function that maps the encoded representation transformed to the output layer of the classification task;

[0168] The softmax function is used to activate the output layer and obtain the prediction result of the classification label;

[0169] Further, the construction process of the above-mentioned Transformer network architecture is as follows:

[0170] S601: As shown in the figure, the constructed Transformer network architecture includes an input sequence, position encoding, an encoder / decoder, built-in components (IECANet channel attention is introduced in the multi-head self-attention mechanism, residual connection, feedforward neural network, layer normalization), and an output layer. Figure 6

[0171] S602, IECANet channel attention introduces a global maximum pooling operation on the basis of the original global average pooling GAP to obtain a differentiated attention weighting effect; that is, it is composed of the original global average pooling operation, the global maximum pooling, the fully connected layer and the activation function.

[0172] As shown in the figure, the global average and maximum pooling operation performs a pooling operation on the input feature map in the channel dimension to obtain the global average and maximum features of each channel; after weighting, the global features are mapped to channel weights through a fully connected layer, which is used to weight each channel; finally, the activation function normalizes the channel weights to ensure that the weight range is within a reasonable range; learn the importance of each channel and weight the input feature image. Figure 5

[0173] ​​S603, after the multi-head self-attention output, the IECA channel attention mechanism is applied to weight the feature vectors of each channel; specifically, before the residual connection, the IECA channel attention is applied to weight the features of each channel; then, the weighted features are connected with the input in residual connection, and then the layer normalization is performed; in this way, it can be ensured that the influence of channel attention is considered before the residual connection, and the independence between channels is maintained;

[0174] Step 7, build an improved lightweight MoboileViT network model;

[0175] The MoboileViT network model is represented as,

[0176] L2_vit=softmax(MLP(Trans_Encoder depth (PE((PatchEmbed(X df )) N )))

[0177] In the formula, X df is a CWT and STFT gray image respectively subjected to three-channelization;

[0178] PatchEmbed(·) is a function of dividing the input image (x) into image blocks and converting it into an embedding vector through linear projection;

[0179] Trans_Encoder(·) is a depth layer Transformer encoder function for encoding the image block sequence;

[0180] MLP(·) is a multi-layer perceptron function composed of fully connected layers and activation functions, to map the encoded representation transformed to the output layer of the classification task;

[0181] Further, the MoboileViT network model construction process is as follows:

[0182] S701, as shown in Figure 8 , the MoboileViT network layer includes an input layer, a channel attention dimension segmentation, a local feature extraction (parallel double attention mechanism), an image block division, an image block embedding layer, a position encoding layer, an encoder (each encoder layer contains a multi-head self-attention mechanism (introducing IECA channel attention mechanism) and a feedforward neural network two sub-layers), a decoder (introducing IECA channel attention mechanism), an adaptive pooling layer, an MLP layer, and an output layer.

[0183] S702, the local feature extraction is composed of IECA channel attention, CBAM channel and spatial attention, and SA spatial attention modules.

[0184] S703, the spatial attention module is composed of two convolutional layers and a sigmoid activation function;

[0185] First, the input feature map is transformed by a convolutional layer to obtain an intermediate feature map; then, the intermediate feature map is mapped to a spatial weight map by another convolutional layer, which represents the importance of each spatial position; finally, the weight values in the spatial weight map are limited between 0 and 1 by a sigmoid activation function; thus, the spatial attention module can learn the importance of each spatial position and weight the input feature map;

[0186] S704, the weighted feature map obtained by the parallel double attention mechanism is fused with the features obtained by the MoboileViT module in the form of residual link to capture the local and global correlation between the two channels in the form of parallel double attention mechanism and transformer architecture combination; the input to the subsequent network layer is mapped to the final classification label space for classification prediction, and the gray mapping image is established in the form of fusion;

[0187] Step 8, align the feature vector matrices A2 and A1 obtained in steps 2 and 3 respectively, construct a multi-channel feature matrix and assign initial weights to the channels; apply position coding to the multi-channel feature matrix, multiply the weight calculation output result with the original feature matrix, input to the subsequent decoding layer for training, and realize the preliminary fault classification result L1;

[0188] Specifically, the feature vector matrices A2 and A1 are taken as time series and frequency domain characteristic vector matrices to ensure that they have the same time step; after alignment and construction of the multi-channel feature matrix, the multi-channel parameter input is applied, the IECA channel attention is applied to assign initial weights to each channel, and the position coding is applied; before the multi-head attention layer, the output result after IECA channel attention calculation is multiplied by the corresponding calculated weight of the original feature matrix, so that the time series information and the frequency domain information can be fused in independent encoders, and the interaction between different channels is enhanced; the first fault classification model of the transformer architecture is trained to realize the preliminary fault classification result L1;

[0189] Further, the above fault classification includes the following steps:

[0190] S801, as shown in Figure 5 The original time series information is sliced, and then FFT processing is applied to each slice to obtain time series and frequency domain feature information;

[0191] S802, ensure that the time sequence and frequency domain feature information after time sequence slicing and FFT processing have the same time step; if the steps are inconsistent, interpolation or truncation is performed to make the two have the same time step;

[0192] S803, input the time sequence and frequency domain feature information after step length alignment and processing as different channels to construct a multi-channel feature matrix; wherein one channel represents time sequence characteristics and the other channel represents frequency domain characteristics;

[0193] S804, use IECANet channel attention mechanism to calculate the channel weight of the multi-channel feature matrix; IECANet can adjust the weight of each channel according to the relationship and importance between channels, and obtain the weight parameter of each channel through learning;

[0194] S805, apply the weight parameter obtained after channel weight calculation to the multi-channel feature matrix, multiply the features of each channel by the corresponding weight to fuse the channel features, and obtain the final feature representation; the feature after channel feature fusion is input into the subsequent network layer;

[0195] S806, as shown in Figure 6 , according to the trained Transformer fault classification model, the final feature representation output by the encoder is mapped to a specific fault category; through the feature representation and channel attention mechanism learned by the model, the fault frequency feature can be preliminarily distinguished, and the fault category to which it belongs can be inferred, and the classification result L1 is obtained;

[0196] Step 9, image fusion is performed on the above mapping time-frequency image data I1, I2, which is divided into two parts in the channel dimension, one part is input into the parallel attention mechanism to extract local features, and the other part is input into the MoboileViT module. Finally, the features of the two are fused in the channel dimension; the output feature of the encoding layer is mapped to the classification label space of the decoding layer to obtain the fault classification result L2;

[0197] Further, the above fault classification includes the following steps:

[0198] S901, based on the two groups of mapping time-frequency image data with different emphases obtained in steps 4 and 5 as input tensors, an empty multi-channel array is created after the image data is scaled and resized, and the image data I1, I2 is stacked together in the form of multi-channel using the stacking function of Numpy to form a two-dimensional image data set;

[0199] S902, split the input channel dimension through the channel attention mechanism; part of the input is sent into the channel space parallel attention mechanism to extract local features, enhance the interaction between different channels, and then connect a channel attention after the output of the parallel attention to splice them at the channel level and further fuse their features using channel attention;

[0200] S903, another part of the input is converted into the form of image block embedding, and position encoding is applied to it to give the initial weight through the channel attention mechanism; through the learned weight parameters, the input features are extracted and learned in the encoder built-in component, the local fusion features obtained in step S902 are linked in the form of residual to the global features obtained at the encoder level for feature fusion to capture the local and global correlation of each channel;

[0201] S904, input the above weighted feature image into the subsequent network layer to map the final classification label space; perform classification prediction to map the output features to the probability distribution of the fault category, and realize the preliminary fault classification result L2;

[0202] Step 10, based on the weighting coefficient and the aggregation operation method, the results L1 and L2 output by the first fault classification model and the second fault classification model are fused at the decision level, the probability after the comprehensive decision level fusion is taken as the final result, and the health state of the bearing is obtained;

[0203] The final predicted bearing state Cpre expression is,

[0204] Cpre=FSE(Add(L1,L2))

[0205] Wherein, FSE(·) represents the fuzzy comprehensive evaluation theory, and Add(·) is a weighted fusion operation.

[0206] To realize the above bearing diagnosis method based on variable working condition multi-modal data fusion, the application also proposes the following bearing diagnosis system based on variable working condition multi-modal data fusion:

[0207] As shown in Figure 2 , the bearing diagnosis system includes a plurality of lower computers connected in parallel to the upper computer, or a single lower computer connected to the upper computer;

[0208] The lower computer can adopt a multi-channel data acquisition instrument, which is externally connected to a double-head BNC signal input line and various adapters for collecting signals of various data types, and various sensors are fixedly arranged at the vertical bearing position.

[0209] The host computer is composed of a server-side program of an integrated multi-modal information fusion bearing lubrication state monitoring algorithm and a monitoring interface; the server-side program is responsible for data processing, analysis and decision-making, and the monitoring interface provides a user-friendly display and operation interface.

[0210] The host computer includes a rotating speed test module, a core processor module, a data acquisition module and an interactive communication module; wherein,

[0211] The data acquisition module is used for acquiring original signals such as sound, vibration and temperature during the rotation of the bearing, and includes a sound sensor, a vibration acceleration sensor and a thermocouple temperature sensor; a DDS high-precision frequency synthesis technology is adopted to ensure that all channels realize clock parallel synchronous acquisition of multiple input channels, each channel is connected with a sensor, and is responsible for collecting and converting signals such as sound, vibration and temperature into digital signals. Through the input end connected with the processor module, the lower host computer realizes real-time data transmission through the interactive communication module; the monitored sound, vibration and temperature data are input into the host computer through the interactive module, and the host computer receives the original collected data and then performs signal data analysis means in the modern signal technology analysis field; the above overview can be used as an information processing part of multi-source heterogeneous data level fusion, and is stored as a database file; the data acquisition module is powered and grounded by the ARM core module, and the output signal of the acquisition module is an analog voltage signal. The sound sensor collects the start and stop frequencies of 0.1-20 kHz, and at the same time, in order to avoid the attenuation of ultrasonic signals, the sound sensor can be aimed at the center part of the inner and outer rings of the bearing to be tested, i.e. the pitch circle position of the bearing to be tested.

[0212] The interactive communication module is used for realizing the communication interaction between the data acquisition instrument and external devices; it realizes data transmission and control operation with external devices through different communication interfaces; in order to avoid signal interference in the working condition environment, wired communication can be adopted; usually one or more serial interfaces are provided for data exchange with serial devices (such as computers, sensors and actuators). It can be in asynchronous or synchronous mode, and the communication module is configured with ID information, the chip ID information is added in the data frame of the monitoring data for sending, which is used for the host computer to confirm the information source, so as to support the confidentiality and security of data transmission, remote monitoring and remote control functions;

[0213] The core processor module is used for signal analysis in the field of modern signal technology analysis after the upper computer receives the original collected data, and specifically for data storage, analysis, time / frequency domain visual display, and data analysis covering envelope, octave, mode, acoustics, wavelet transform and other data analysis means; the core processor module includes a minimum system module, a real-time clock module, a backup storage module, a debugging module and a signal conditioning module; the power module, reset module, clock module, program download module and start setting module in the minimum system module are connected with the microprocessor signal; the real-time clock module is connected with the microprocessor in the minimum system module, and is used for reading time information when collecting signals, which provides accurate clock information for time stamping and recording time information of collected data. The backup storage module is connected with the microprocessor in the minimum system module, and locally stores all monitoring data; in the current embodiment, the capacity of the backup storage module is 16 GB, and the data is stored in binary format, and the file name is named according to date information. The debugging module is connected with the microprocessor in the minimum system module, and is used for debugging the program of the lower computer; the input end of the signal conditioning module is connected with the output end of the signal collection module, and the output end of the signal conditioning module is connected with the microprocessor in the minimum system module; the input of the signal conditioning module is the analog voltage signal output by the signal collection module, and the analog voltage signal is converted into a digital signal output;

[0214] As described above, the application adopts multiple modal information for synchronous collection, including vibration signals, noise signals, temperature signals and the like during the operation of the bearing. In order to improve the signal quality and avoid interference, the sound sensor, the vibration sensor and the temperature sensor are integrated together and installed in the interior of the rotating machinery to obtain multiple modal information of the rolling bearing during operation. Finally, the application fuses the bearing lubrication state monitoring algorithm based on multiple modal information to accurately and efficiently judge the lubrication state of the current rolling bearing. Through comprehensive analysis and processing of information data of different modalities, accurate monitoring and diagnosis of the lubrication state of the bearing can be realized, and the reliability and service life of the bearing operation can be improved.

[0215] Although the above embodiments have shown and described the present application, it can be understood by those skilled in the art that various changes, modifications, replacements and variations can be made to the embodiments without departing from the principles and spirits of the present application, and the scope of the present application is defined by the appended claims and their equivalents.

Claims

1. A bearing diagnosis method based on variable working condition multi-modal data fusion, characterized in that: The embodiment comprises the following steps, Step 1, obtaining multi-source heterogeneous signals under different working conditions and multiple fault types; Step 2, performing data processing such as denoising, data alignment and standardization on the multi-source heterogeneous signals, and performing data-level fusion on the signals according to the modal dimension to construct a feature vector matrix A1; Step 3, applying Fourier transform to each feature vector of the feature vector matrix A1, extending the dimension of the original feature vector through the Fourier transform result, and finally obtaining a new feature vector matrix A2; Step 4, selecting different scales for wavelet transform and reconstruction of the constructed parametric feature vector matrix A1 through a Complex Morlet wavelet function; selecting the optimal scale by comparing the errors, and calculating the features including amplitude and energy under the current optimal unique scale to generate a gray image data set I1 of wavelet transform; Step 5, performing optimal scale STFT feature extraction on the feature vector matrix A1, selecting a Hamming window with different window sizes and setting the window overlap degree to 80%, calculating the spectral features under different scales, and performing IFFT reconstruction and error comparison with the original data; selecting the optimal window size and calculating the corresponding statistical features to generate a gray image data set I2 of the best scale STFT features; Step 6, constructing a Transformer network architecture; The Transformer network model is as follows, L1_trans = softmax(FC(Trans depth ( PE((Embed(X df )) N ) )) Wherein, the Transformer network architecture stacks the network of Trans above the depth layer, X df The feature vector data constructed for the time domain and the frequency domain of the preprocessed multi-source heterogeneous information fuses the features represented by N different modes. Embed(·) is an embedding layer function that converts the input feature vector (x) into a representation with a dimension of d_model; PE(·) is a position encoding function that adds position encoding to the embedded embedded to preserve the position information of the sequence; Trans depth (·) is a multi-layer self-attention mechanism function for encoding the input sequence; FC(·) is a fully connected layer function that maps the encoded representation transformed to the output layer of the classification task; The softmax function is used to activate the output layer and obtain the prediction result of the classification label; Step 7, constructing an improved lightweight MoboileViT network model; The MoboileViT network model is represented as, L2_vit = softmax(MLP(Trans_Encoder depth ( PE((PatchEmbed(X df )) N ) )) where X df are CWT and STFT grayscale images, respectively, which are tri-channeled. PatchEmbed(·) is a function that divides the input image (x) into image blocks and converts them into embedded vectors through linear projection; Trans_Encoder(·) is a depth layer of Transformer encoder function, which is used to encode the image block sequence; MLP(·) is a multi-layer perceptron function composed of fully connected layers and activation functions, which is used to map the encoded representation transformed to the output layer of the classification task; Step 8, aligning the feature vector matrices A2 and A1 obtained in steps 2 and 3, constructing a multi-channel feature matrix and assigning initial weights to the channels; applying position encoding to the multi-channel feature matrix, multiplying the weight calculation output result with the original feature matrix, and inputting it to the subsequent decoding layer for training to achieve the preliminary fault classification result L1; ​ Step 9, after fusing the images in the above mapping time-frequency image data sets I1, I2, the images are divided into two parts in the channel dimension, one part is sent into the parallel double attention mechanism to extract local features, and the other part is input into the MoboileViT module, and finally the features of the two are fused in the channel dimension; the encoded layer output feature is mapped to the classification label space of the decoding layer to obtain the fault classification result L2; Step 10, based on the weighted coefficient and the aggregation operation method, the results L1, L2 output by the first fault classification model and the second fault classification model are fused at the decision level, and the probability after the comprehensive decision level fusion is taken as the final result to obtain the health state of the bearing; The final predicted bearing state Cpre is expressed as, Cpre=FSE (Add(L1,L2)) Wherein, FSE (·) represents the fuzzy comprehensive evaluation theory, and Add (·) is a weighted fusion operation.

2. The bearing diagnostic method based on variable operating condition multi-modal data fusion of claim 1, wherein: In step 1, a plurality of multi-source signals under different working conditions and a plurality of different fault type states are obtained as signals of N modalities of rolling bearings; Different fault type settings are classified as data sample classification of different faults, and the original data of different working conditions under the same fault type is grouped to construct data level fusion.

3. The bearing diagnostic method based on variable operating condition multi-modal data fusion of claim 1, wherein: In step 2, after the multi-modal signals are denoised, smoothed, standardized and other operations, the statistical feature extraction is carried out, the feature vector of each modal signal is extracted, and the data after splicing and averaging is fused according to the modal dimension to construct a feature vector, and the signal processing is recorded as a feature vector matrix A1.

4. The bearing diagnostic method based on variable operating condition multi-modal data fusion of claim 1, wherein: In step 3, the shape (M, N) of the feature vector matrix A1 is obtained, the A1[i] segment is intercepted, and FFT is applied to each A1[i] segment to expand the dimension shape of the original feature vector to (M, K). The Fourier transform of the non-periodic continuous-time signal x(t) is expressed as, The discrete sampling value x(nT) of the continuous signal x(t) is obtained, and the following spectrum of the signal x(t) is calculated using the discrete signal x(nT), The new feature vector matrix A2 obtained should have the same time step as A1, and the frequency interpolation form is used for data with longer step.

5. The bearing diagnostic method based on variable operating condition multi-modal data fusion of claim 1, wherein: In step 4, the continuous wavelet transform function is as follows, In the formula, f(t) is a time domain signal, ψ(x) is a wavelet base function, a is a frequency positioning, and b is a time positioning; The Complex Morlet wavelet function is selected; The function expression of the Complex Morlet wavelet base is as follows, In the formula, a is a scale factor, and b is a time shift factor; The wavelet transform of the wavelet base function is carried out in different scales according to 64, 32 and 16 to decompose and heterogenize the heterogeneous signals at different scales, and the reconstruction error is calculated; A lower reconstruction error indicates that the wavelet base function at the current scale better represents the characteristics of the original signal; at the selected unique preferred scale, the required features are calculated; The calculated features are normalized, and the normalized feature values are mapped into a gray image. A three-channel empty zero matrix is set, and the data of a single channel is copied to three channels to make the grayscale image meet the data input condition of the RGB form of the ViT model in advance; The feature values are uniformly mapped to a grayscale range using a linear mapping, with larger feature values corresponding to brighter pixels and smaller feature values corresponding to darker pixels, and the generated image is taken as the initial image data set I1.

6. The bearing diagnostic method based on variable operating condition multi-modal data fusion of claim 1, wherein: In step 5, the STFT basic operation formula is as follows, In the formula, x(t) is a time domain signal, and h(t-ω) is a window function. The generation process of the grayscale image includes the following steps, S501, STFT adopts a smaller scale Hanning window function to provide a time-frequency image focusing on time domain features; The mathematical expression of the Hanning window is as follows, S502, three different window sizes are selected, the overlapping degree between the windows is set to 80%, and STFT calculation is performed on the feature vector matrix A1 using the selected window size and overlapping degree; For each window size, the FFT of each window is calculated to obtain the spectral information; S503, IFFT reconstruction and comparison with the original data; The reconstructed signal is compared with the original data in terms of error, and the mean square error is used to evaluate the accuracy of the reconstruction, and the best scale is selected; S504, the spectral energy or other statistical features of the unique selected scale are calculated, and for the selected best scale, the calculated features are mapped into a grayscale image, and the generated image is taken as the initial image data set I2.

7. The variable operating condition multi-modal data fusion based bearing diagnostic method of claim 1, wherein: In step 6, the construction process of the Transformer network architecture is as follows, S601, the constructed Transformer network architecture includes an input sequence, a position encoding, an encoder / decoder, built-in components, and an output layer; S602, IECANet channel attention introduces a global maximum pooling operation based on the original global average pooling GAP to obtain a differentiated attention weighting effect; S603, after the multi-head self-attention output, the IECANet channel attention mechanism is applied to weight the feature vectors of each channel; specifically, before the residual connection, the IECANet channel attention is applied to weight the features of each channel; Then, the weighted features are connected with the input in residual connection, and then normalized.

8. The bearing diagnostic method based on variable operating condition multi-modal data fusion of claim 1, wherein: In step 8, the fault classification includes the following steps, S801, the original time series information is sliced, and then the FFT processing is applied to each slice to obtain time series and frequency domain feature information; S802, ensure that the time series and frequency domain feature information after time series slicing and FFT processing have the same time step; If the steps are inconsistent, interpolation or truncation is performed to make them have the same time step; S803, the time series and frequency domain feature information after step alignment and processing are taken as different channel inputs to construct a multi-channel feature matrix; wherein one channel represents time series features, and the other channel represents frequency domain features; S804, using IECANet channel attention mechanism to calculate channel weight of multi-channel feature matrix; IECANet can adjust the weight of each channel according to the relationship and importance between channels, and obtain the weight parameter of each channel through learning; S805, applying the weight parameter obtained through channel weight calculation to the multi-channel feature matrix, multiplying the feature of each channel by the corresponding weight to fuse the channel features, obtaining the final feature representation, and inputting the feature after channel feature fusion into the subsequent network layer; S806, according to the trained Transformer fault classification model, the final feature representation output by the encoder is mapped to a specific fault category, and through the feature representation and attention mechanism learned by the model, the fault frequency feature can be preliminarily judged and the fault category to which it belongs can be inferred, and the classification result L1 is obtained.

9. The variable operating condition multi-modal data fusion based bearing diagnostic method of claim 1, wherein: The step 9, the fault classification comprises the following steps, S901, based on the two groups of mapping time-frequency image data with different emphases obtained in steps 4 and 5 as input tensors, an empty multi-channel array is created after the image data is scaled and adjusted in size, and the image data I1 and I2 are stacked together in the form of multi-channel using the stacking function of Numpy to form a two-dimensional image data set; S902, the channel dimension of the input is split through the channel attention mechanism; part of the input is sent into the parallel double attention mechanism to extract local features and enhance the interaction between different channels, and then a channel attention is connected to the output of the parallel attention to further fuse the features after the concat splicing operation in the channel level; S903, the other part of the input is converted into the form of image block embedding, and position encoding is applied to it to give an initial weight value through the channel attention mechanism; the weight parameter obtained through learning is used to extract and learn the input features in the encoder built-in component, and the local extraction and fusion features obtained in step S902 are connected in the form of residual to fuse the global features obtained in the encoder level to capture the local and global correlation of each channel; S904, the feature image obtained after the feature fusion in S903 is input into the subsequent network layer to map the final classification label space; the classification prediction is performed to map the output features to the probability distribution of the fault category, and the preliminary fault classification result L2 is realized.

10. A bearing diagnosis system applying the bearing diagnosis method based on variable working condition multi-modal data fusion according to any one of claims 1 to 9, characterized in that: The plurality of lower computers are connected to the upper computer in parallel, or a single lower computer is connected to the upper computer; The lower computer adopts a multi-channel data acquisition instrument, signal input lines of double-head BNC terminals and various adapters are externally connected for collecting signals of various data types, and various sensors are fixedly arranged at the vertical bearing part; The upper computer comprises a rotating speed test module, a core processor module, a data acquisition module and an interactive communication module; Furthermore, the computer executable program is stored in the memory, and the processor reads the computer executable program from the memory and executes the computer executable program. And a computer readable storage medium for storing the bearing diagnosis method based on variable working condition multi-modal data fusion.

Images