Bearing fault diagnosis method and system based on cross-modal feature alignment

Abstract

This invention discloses a bearing fault diagnosis method and system based on cross-modal feature alignment, belonging to the field of rotating machinery condition monitoring and fault diagnosis technology. The method includes: constructing sound feature vectors and vibration feature vectors based on the envelope spectrum of sound signals and the envelope spectrum of vibration signals to form a sound-vibration synchronization dataset; training the constructed cross-modal feature alignment diagnostic model using the training and validation sets in the sound-vibration synchronization dataset to obtain a trained cross-modal feature alignment diagnostic model, thereby obtaining a solidified cross-modal feature alignment diagnostic model; using the sound feature vectors from the test set in the sound-vibration synchronization dataset or the sound feature vectors to be diagnosed as input to the solidified cross-modal feature alignment diagnostic model to obtain the bearing diagnosis result. This invention can realize fault diagnosis under the same operating conditions based on sound vibration components, and further realize fault diagnosis across operating conditions.

Description

Technical Field

[0001] This invention relates to a bearing fault diagnosis method and system based on cross-modal feature alignment, belonging to the field of rotating machinery equipment condition monitoring and fault diagnosis technology. Background Technology

[0002] As a core rotating component of mechanical equipment, the operating condition of bearings directly determines the reliability, service life, and production safety of the equipment. In the field of mechanical equipment fault diagnosis, vibration signals have become the mainstream diagnostic basis because they can directly reflect the mechanical vibration characteristics of bearings and have high fault characteristic identification. However, in complex equipment such as machining centers and precision CNC machine tools, bearings are usually embedded in compact spindle boxes or transmission structures. The installation of vibration sensors requires damaging the original housing structure of the equipment, which not only increases the cost of equipment modification and deployment but may also affect the operating accuracy and sealing of the equipment. At the same time, the maintenance and calibration of sensors require downtime operation, which seriously affects production efficiency and limits their large-scale application in complex scenarios.

[0003] To address the challenges of deploying vibration sensors, the industry has experimented with using sound signals as an alternative diagnostic signal. Sound signals offer significant advantages such as non-contact acquisition, flexible deployment, and low cost (requiring only a microphone). However, they also have inherent drawbacks: sound signals are susceptible to environmental noise (such as background noise in the workshop and noise from other equipment) and air attenuation during propagation. Fault characteristic signals are often submerged by noise, resulting in a diagnostic accuracy rate of typically less than 80% for a single sound signal, which falls far short of the precision requirements for industrial-grade fault diagnosis.

[0004] Existing multimodal signal fusion technologies attempt to improve diagnostic performance by fusing sound and vibration signals, but they suffer from several key technical flaws: First, they often employ direct signal splicing or simple feature overlay without considering the differences in feature distribution between sound and vibration signals, resulting in poor cross-modal feature compatibility and the introduction of redundant information. Second, they fail to achieve deep alignment at the feature level, making it impossible to effectively transfer strong fault feature knowledge from vibration signals to the feature learning process of sound signals, resulting in no significant improvement in diagnostic performance when using sound signals alone. Third, traditional feature extraction methods often use full-dimensional feature input (such as a 4096-dimensional original signal), failing to utilize the sparse distribution characteristics of bearing fault features (fault features are concentrated only in a specific frequency range), leading to high computational complexity and the introduction of noise and redundant features, which affects the generalization ability of the diagnostic model.

[0005] Furthermore, under varying operating conditions (such as equipment speed and load fluctuations), the stability of fault characteristics of single-mode signals further decreases. Existing technologies lack effective cross-modal knowledge transfer mechanisms and are difficult to adapt to the dynamic operating scenarios of complex equipment.

[0006] In summary, the existing technology has the following pressing technical problems that need to be solved:

[0007] 1. The deployment of vibration sensors for bearings in complex mechanical equipment is limited, and diagnosis using a single sound signal is easily affected by noise, resulting in low accuracy;

[0008] 2. Multimodal signal fusion does not achieve deep alignment at the feature level, and fault feature knowledge of vibration signals cannot be effectively transferred to sound signals;

[0009] 3. Feature extraction does not utilize the sparsity of fault features, has high computational complexity, and redundant features affect the model's generalization ability.

[0010] 4. Lack of a robustness enhancement mechanism for cross-modal features to adapt to varying operating conditions.

[0011] In view of this, the present invention is hereby proposed. Summary of the Invention

[0012] To achieve effective transfer of vibration fault knowledge to sound signals, this invention provides a bearing fault diagnosis method based on cross-modal feature alignment, which is used to obtain a solidified cross-modal feature alignment diagnosis model. Based on the solidified cross-modal feature alignment diagnosis model, fault diagnosis under the same working condition is achieved based on sound vibration components, and further, fault diagnosis under different working conditions is achieved.

[0013] The technical solution of this invention is:

[0014] According to a first aspect of the present invention, a bearing fault diagnosis method based on cross-modal feature alignment is provided, comprising:

[0015] S1. Under the target working condition, synchronously collect sound and vibration signals of the bearing in different states for a preset duration; align and segment the sound and vibration signals of the preset duration in each state to form sound signal samples and vibration signal samples; wherein, the state includes normal state and fault state.

[0016] S2. Obtain the frequency domain representation of the sound signal and the frequency domain representation of the vibration signal based on the sound signal sample and the vibration signal sample, respectively; obtain the envelope spectrum of the sound signal and the envelope spectrum of the vibration signal for any single-sided spectrum of the frequency domain representation of the sound signal and the frequency domain representation of the vibration signal, respectively.

[0017] S3. Based on the characteristic frequency distribution of bearing fault theory, select the top two frequencies from the envelope spectrum of the sound signal and the envelope spectrum of the vibration signal, respectively. n The amplitude corresponding to each frequency point is used to construct sound feature vectors and vibration feature vectors; based on the sound feature vectors and vibration feature vectors of bearings in different states under various working conditions, a sound and vibration synchronization dataset is constructed; where n takes a positive integer.

[0018] S4. Divide the acoustic-vibration synchronization dataset into training set, validation set and test set according to a preset ratio;

[0019] S5. Train the constructed cross-modal feature alignment diagnostic model based on the training set and validation set to obtain the trained cross-modal feature alignment diagnostic model;

[0020] S6. Load the trained cross-modal feature alignment diagnostic model, perform the model parameter solidification operation, and obtain the solidified cross-modal feature alignment diagnostic model.

[0021] S7. Acquire the sound signal to be diagnosed, segment the sound signal to be diagnosed to form a sound signal diagnostic sample; obtain the frequency domain representation of the sound signal to be diagnosed based on the sound signal diagnostic sample; obtain the envelope spectrum of the sound signal to be diagnosed from any one-sided spectrum of the frequency domain representation of the sound signal to be diagnosed; select the top 2 from the envelope spectrum of the sound signal to be diagnosed. n The amplitude corresponding to each frequency point is used to construct the sound feature vector to be diagnosed; the sound feature vector in the test set or the sound feature vector to be diagnosed is used as the input of the solidified cross-modal feature alignment diagnostic model to obtain the bearing diagnosis result.

[0022] Further, in S7, using the sound feature vectors in the test set as input to the solidified cross-modal feature alignment diagnostic model specifically means: when the target working condition is the first working condition, the solidified cross-modal feature alignment diagnostic model obtained by training based on the training set and validation set in the acoustic-vibration synchronization dataset constructed under the first working condition is used as the first solidified model, and the sound feature vectors in the test set under the second working condition are used as input to the first solidified model to obtain the bearing diagnostic result; wherein the second working condition is the same as or different from the first working condition.

[0023] Furthermore, the cross-modal feature alignment diagnostic model includes:

[0024] The feature extraction module adopts a parallel dual-branch configuration: a sound branch residual self-attention feature dimensionality reduction module and a vibration branch residual self-attention feature dimensionality reduction module. The sound feature vector is used as the input of the sound branch residual self-attention feature dimensionality reduction module, and the output is the sound dimensionality reduction feature; the vibration feature vector is used as the input of the vibration branch residual self-attention feature dimensionality reduction module, and the output is the vibration dimensionality reduction feature.

[0025] The feature alignment module maps the reduced sound and vibration features output by the feature extraction module to a unified feature space through a shared projection layer. It eliminates the amplitude differences between modes through an L2 normalization layer, thereby achieving spatial unification and depth alignment of the sound and vibration features. It outputs spatially aligned sound and vibration features.

[0026] The classification module adopts a fully connected discriminant configuration, takes the fusion result of spatially aligned sound alignment features and vibration alignment features as input, and outputs a state classification logic value.

[0027] Furthermore, the sound branch residual self-attention feature dimensionality reduction module and the vibration branch residual self-attention feature dimensionality reduction module have the same structure. The sound branch residual self-attention feature dimensionality reduction module will be used as an example, specifically including:

[0028] Initial convolutional layer: used for preliminary extraction of sound signal features;

[0029] Batch normalization layer: used to take the sound signal features output from the initial convolutional layer as input and obtain a standardized distribution of the sound features;

[0030] GELU activation function: used to perform nonlinear mapping on the standardized distribution of sound features obtained from the batch normalization layer;

[0031] Stride Residual Block: The input is the sound features nonlinearly mapped by the GELU activation function. The input is passed through the main branch of the stride residual block and the shortcut connection branch of the stride residual block, respectively, and the output is the first sound backbone feature and the first sound jump feature. The first sound backbone feature and the first sound jump feature are added element-wise along the channel dimension and fused together, and then the nonlinear mapping transformation is completed by the GELU activation function.

[0032] Initial self-attention module: The input is the deep features after stride residual block transformation, which are fed into the initial self-attention module for global dependency modeling, and the output is the initial self-attention enhanced features;

[0033] Identical Residual Block: The input is the initial self-attention enhancement feature output by the initial self-attention module. It is then passed through the main branch of the identity residual block and the shortcut connection branch of the identity residual block to output the second sound backbone feature and the second sound jump feature. The second sound backbone feature and the second sound jump feature are then added element-wise along the channel dimension and fused together. Finally, the nonlinear mapping transformation is completed by the GELU activation function.

[0034] Enhanced Self-Attention Module: The input is the deep features mapped by the identity residual block, which are fed into the enhanced self-attention module for global dependency modeling, and the output is the enhanced self-attention enhanced features; the enhanced self-attention module has the same structure as the initial self-attention module.

[0035] Adaptive average pooling layer: The input is the enhanced self-attention enhancement feature output by the enhanced self-attention module, which is then normalized in terms of feature dimension by the adaptive average pooling layer, and the output is the normalized audio dimensionality reduction feature.

[0036] Furthermore, the stride residual block main branch includes a first convolutional layer, a BN layer, a GELU activation function, a second convolutional layer, and a BN layer connected in sequence, used to output the first sound backbone feature; the stride residual block shortcut connection branch includes a third convolutional layer and a BN layer connected in sequence, used to output the first sound jump feature.

[0037] Furthermore, the main branch of the identity residual block includes a fourth convolutional layer, a BN layer, a GELU activation function, a fifth convolutional layer, and a BN layer connected in sequence, used to output the second sound backbone features; the shortcut connection branch of the identity residual block is an identity mapping, which directly outputs the second sound jump features.

[0038] Furthermore, the classification module includes a feature fusion layer, a flattening layer, a first fully connected layer, a GELU activation function, a Dropout regularization layer, and a second fully connected layer connected in sequence.

[0039] Furthermore, the cross-modal feature alignment diagnostic model training process employs a cross-modal fusion loss function, which adopts a weighted fusion joint loss configuration consisting of two parts: classification loss and acoustic-vibration feature alignment loss. The classification loss uses a cross-entropy loss function. The acoustic-vibration feature alignment loss uses spatially aligned sound alignment features and vibration alignment features as inputs to calculate the mean square distance between the two types of features.

[0040] According to a second aspect of the present invention, a bearing fault diagnosis system based on cross-modal feature alignment is provided, comprising a module of the bearing fault diagnosis method based on cross-modal feature alignment as described above.

[0041] The beneficial effects of this invention are:

[0042] First, deployment flexibility is significantly improved. High-precision fault diagnosis is achieved through non-contact sound signals, eliminating the need to install vibration sensors on complex equipment bearings, thus avoiding damage to the equipment structure. Compared to current feature fusion models, deployment costs are reduced by more than 60%, and maintenance difficulty is significantly reduced, making it suitable for complex scenarios such as machining centers and CNC machine tools. Second, diagnostic accuracy is greatly improved. Cross-modal feature alignment enables effective transfer of vibration signal fault feature knowledge to sound signals. When trained using raw sound and vibration signals and tested with single sound data at different signal-to-noise ratios, the diagnostic accuracy remains close to 90% even at an SNR of -6dB, compared to single vibration... Compared to the failure of dynamic signal diagnostic models and single sound signal diagnostic models, the technology proposed in this invention can still meet the diagnostic accuracy requirements of industrial sites. Third, the computational efficiency is optimized. Based on the sparsity of fault features, 1024-dimensional core features are extracted. Compared with full-dimensional (4096-dimensional) feature input, the model training time is reduced by about 45%, the inference time is reduced by about 52%, and the single-sample inference time is only 0.015s, which lowers the hardware deployment threshold. Fourth, the feature robustness is enhanced. The proposed method of this invention also solves the gradient vanishing problem in deep networks, captures the global dependency relationship of fault features, and improves the model's adaptability to changing operating conditions (speed, load fluctuation). Attached Figure Description

[0043] Figure 1 This is a flowchart of the overall process of the method of the present invention.

[0044] Figure 2 A fault diagnosis model diagram aligned with complete cross-modal features.

[0045] Figure 3 This diagram illustrates the cross-modal feature alignment model structure where the feature extraction module only extracts the sound branch.

[0046] Figure 4 The training accuracy and validation accuracy are based on the model provided in Example 2.

[0047] Figure 5 This is a t-SNE visualization of the first epoch provided according to Example 2.

[0048] Figure 6 This is a t-SNE visualization of the 150th epoch provided according to Example 2.

[0049] Figure 7 This is a confusion matrix diagram of the model test results.

[0050] Figure 8 This is the t-SNE feature clustering diagram of the model test results.

[0051] Figure 9This is a comparison chart of intraclass Euclidean distances for similar acoustic-vibration faults under different signal-to-noise ratios.

[0052] Figure 10 This is a comparison chart of the mean Euclidean distances between different types of acoustic and vibration faults under different signal-to-noise ratios.

[0053] Figure 11 This is a comparison chart of the model's test accuracy under different signal-to-noise ratios. Detailed Implementation

[0054] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention. It should be noted that, unless otherwise specified, the embodiments and features in the embodiments of this application can be arbitrarily combined with each other.

[0055] Example 1: As Figures 1-3 As shown, according to a first aspect of the present invention, a bearing fault diagnosis method based on cross-modal feature alignment is provided, comprising:

[0056] S1. Under M target operating conditions, synchronously collect sound and vibration signals of preset duration for bearings in different states; align and segment the sound and vibration signals of preset duration for each state according to the sampling time to form sound signal samples and vibration signal samples; wherein, the state includes normal state and fault state, and M is a positive integer (e.g., M is 1, 2 or 3, etc.).

[0057] S2. Perform Fast Fourier Transform on the sound signal sample and vibration signal sample respectively to obtain the frequency domain representation of the sound signal and the frequency domain representation of the vibration signal; perform Hilbert Transform, envelope extraction and Fourier Transform on any single-sided spectrum of the frequency domain representation of the sound signal and the frequency domain representation of the vibration signal respectively to obtain the envelope spectrum of the sound signal and the envelope spectrum of the vibration signal.

[0058] S3. Based on the characteristic frequency distribution of bearing fault theory, select the top two frequencies from the envelope spectrum of the sound signal and the envelope spectrum of the vibration signal, respectively. n Based on the amplitude corresponding to each frequency point, construct sound feature vectors and vibration feature vectors; based on the sound feature vectors and vibration feature vectors of bearings in different states under M target working conditions, construct a sound and vibration synchronization dataset; where n takes positive integer values.

[0059] S4. Divide the acoustic-vibration synchronization dataset into training set, validation set and test set according to a preset ratio;

[0060] S5. Construct a cross-modal feature alignment diagnostic model. Train the cross-modal feature alignment diagnostic model based on the training set and validation set. Use the cross-modal fusion loss function during the training process to obtain the trained cross-modal feature alignment diagnostic model.

[0061] Furthermore, the cross-modal feature alignment diagnostic model includes:

[0062] The feature extraction module adopts a parallel dual-branch configuration: a sound branch residual self-attention feature dimensionality reduction module and a vibration branch residual self-attention feature dimensionality reduction module. The sound feature vector is used as the input of the sound branch residual self-attention feature dimensionality reduction module, and the output is the sound dimensionality reduction feature; the vibration feature vector is used as the input of the vibration branch residual self-attention feature dimensionality reduction module, and the output is the vibration dimensionality reduction feature.

[0063] The sound branch residual self-attention feature dimensionality reduction module and the vibration branch residual self-attention feature dimensionality reduction module have the same structure. The sound branch residual self-attention feature dimensionality reduction module is used as an example for description, specifically including:

[0064] Initial convolutional layer: used for preliminary extraction of sound signal features;

[0065] Batch Normalization (BN) layer: This layer takes the sound signal features output from the initial convolutional layer as input and obtains a standardized distribution of the sound features, thus accelerating training convergence. Since the mechanism of action of the batch normalization layer is consistent across all layers of the model, it will be referred to as BN layer in the following text.

[0066] GELU activation function: used to standardize the distribution of sound features obtained from the batch normalization layer and perform nonlinear mapping to improve feature representation ability;

[0067] Stride Residual Block: The input consists of sound features nonlinearly mapped using the GELU activation function. These features are processed through the main branch and shortcut connection branch of the stride residual block, outputting the first sound backbone feature and the first sound jump feature. The first sound backbone feature and the first sound jump feature are then element-wise added together along the channel dimension and fused, followed by a nonlinear mapping transformation using the GELU activation function. The main branch of the stride residual block comprises a first convolutional layer, a BN layer, a GELU activation function, a second convolutional layer, and a BN layer connected in sequence, used to output the first sound backbone feature. The shortcut connection branch of the stride residual block comprises a third convolutional layer and a BN layer connected in sequence, used to output the first sound jump feature.

[0068] Initial Self-Attention Module: The input consists of deep features transformed by stride residual blocks. These features are fed into the initial self-attention module for global dependency modeling, outputting initial self-attention enhanced features. The specific structure of the self-attention module is as follows: Eight attention heads are set, and the 128-dimensional input channels are evenly distributed among them. A 1×1 convolution is used to transform the features and generate three sets of features: Q, K, and V. The attention score distribution is constrained by a channel dimension scaling factor. The global attention weights are normalized using the Softmax function, completing the weighted aggregation of spatial and channel dimension features. Then, a 1×1 convolution is used for feature dimension projection and reconstruction, ultimately outputting the initial self-attention enhanced features. Residual connections are introduced to preserve original feature information and avoid gradient vanishing.

[0069] The identity residual block takes the initial self-attention enhancement features as input after global dependency modeling by the initial self-attention module. These features are then processed through the main branch and shortcut connection branch of the identity residual block, outputting the second voice backbone feature and the second voice jump feature. The second voice backbone feature and the second voice jump feature are then fused element-wise along the channel dimension, followed by a non-linear mapping transformation using the GELU activation function. The specific structure of the identity residual block's main branch includes a fourth convolutional layer, a BN layer, a GELU activation function, a fifth convolutional layer, and a BN layer connected in sequence, used to output the second voice backbone feature. The identity residual block's shortcut connection branch is an identity mapping, without convolution or normalization operations, directly outputting the second voice jump feature.

[0070] Enhanced Self-Attention Module: The input is the deep features mapped by the identity residual block, which are fed into the enhanced self-attention module for global dependency modeling, and the output is the enhanced self-attention feature. The structure of the enhanced self-attention module is the same as that of the initial self-attention module.

[0071] Adaptive Average Pooling Layer: The input is the enhanced self-attention feature output by the enhanced self-attention module. The adaptive average pooling layer performs feature dimension normalization and outputs the dimension-normalized audio dimensionality-reduced feature. The specific structure of the adaptive average pooling layer is as follows: It employs adaptive average pooling with a pooling parameter set to 1, i.e., keeping the height dimension of the feature map unchanged while compressing the width dimension to 1, achieving global average aggregation in the width dimension. This module has no additional convolution, normalization, or activation operations; it only performs feature dimension normalization through pooling, without changing the number of channels in the input feature (both input and output channels are 128). The final output is the dimension-normalized audio dimensionality-reduced feature.

[0072] The feature alignment module maps the reduced-dimensionality sound features and vibration features output by the feature extraction module to a unified feature space through a shared projection layer. It then eliminates amplitude differences between modes using an L2 normalization layer, achieving spatial unification and depth alignment of the acoustic and vibration features. The module outputs spatially aligned sound and vibration features. Specifically, it includes:

[0073] Shared projection layer: The input consists of the sound dimensionality reduction features and vibration dimensionality reduction features output by the feature extraction module. A 1×1 convolutional layer with shared weights is used to perform a unified spatial projection on the sound dimensionality reduction features and vibration dimensionality reduction features. The input channel is 128 and the output channel is 128. The sound dimensionality reduction features and vibration dimensionality reduction features are forced to be mapped to the same common feature space to obtain the sound cross-modal shared mapping features and vibration cross-modal shared mapping features.

[0074] L2 normalization layer: The sound cross-modal shared mapping features and vibration cross-modal shared mapping features output by the shared projection layer are normalized by L2 norm, constraining the features to the unit hypersphere space and eliminating the difference in feature dimensions.

[0075] The classification module adopts a fully connected discriminant configuration. It takes the fusion result of spatially aligned sound alignment features and vibration alignment features as input to achieve accurate classification of device status types and outputs a status classification logic value. The classification module includes, in sequence:

[0076] Feature fusion layer: The sound alignment features and vibration alignment features are fused element-wise along the channel dimension to obtain the joint sound and vibration discriminative three-dimensional feature tensor;

[0077] Flattening layer: Flattens the fused 3D feature tensor into a 1D feature vector to adapt to the input format requirements of the fully connected layer;

[0078] The first fully connected layer has an input dimension of 128 and an output dimension of 64, which realizes the mapping of high-dimensional joint features to low-dimensional discriminative features.

[0079] GELU activation function: performs non-linear mapping on the output features of the first fully connected layer to enhance the discriminative representation ability of classification features;

[0080] Dropout regularization layer: The inactivation probability is set to 0.3, which randomly suppresses the response of some neurons and effectively alleviates the overfitting problem of the model;

[0081] The second fully connected layer has an input dimension of 64 and an output dimension of 5 (corresponding to 5 bearing state types: normal NR, inner ring fault IF, outer ring fault OF, rolling element fault BF, and compound fault MF). It maps features to the state category dimension and outputs the final state classification logical value.

[0082] Furthermore, the cross-modal fusion loss function adopts a weighted fusion joint loss configuration, consisting of two parts: classification loss and acoustic-vibration feature alignment loss. A total loss function is constructed by weighted summation with fixed weights, simultaneously optimizing fault classification accuracy and constraining acoustic-vibration feature space alignment, outputting the model's total loss value; specifically as follows:

[0083] Classification loss: The cross-entropy loss function is used, taking the model's output state classification logical value and the true state label as input, to measure the difference in distribution between the classification prediction result and the true label, thus constraining the model to achieve accurate fault classification; the formula for calculating the classification loss is: ;in, This is the classification loss value; This represents the total number of samples in a single training batch. The total number of categories for state classification (in this invention) (Corresponding to normal, inner ring fault, outer ring fault, rolling element fault, and compound fault). For the first The true label one-hot encoding of the sample, when the sample belongs to the first... The value is 1 if the condition is met, and 0 otherwise. For the model to the first The predicted value for the nth sample is... The probability value of the class.

[0084] Acoustic and vibration feature alignment loss: An improved Wasserstein alignment loss function based on the L2 norm is adopted. Using spatially aligned acoustic and vibration alignment features as inputs, the mean square distance between the two types of features is calculated, constraining the acoustic and vibration features to map to the same feature space. The expression for the acoustic and vibration feature alignment loss is as follows: ,in, This represents the alignment loss value for acoustic vibration characteristics; The loss scaling factor (in this model) (used to amplify gradients and improve alignment constraint capabilities). This represents the total number of samples in a single training batch. For the first The sound alignment features of each sample after processing by the feature alignment module; For the first Vibration alignment features of a sample after processing by the feature alignment module; It is the squared L2 norm of the two-dimensional eigenvectors (mean square Euclidean distance).

[0085] Total loss function: The classification loss and alignment loss are fused using fixed weighting coefficients, and all trainable parameters of the model are optimized through backpropagation of the total loss; the expression for the total loss function is: ,in, This represents the total loss value of the model. Weighting coefficients for acoustic vibration feature alignment loss (in this invention) ); The weighting coefficients for classification loss (in this invention) ); This is the classification loss value; This represents the alignment loss value for acoustic vibration characteristics.

[0086] S6. Model Loading and Solidification: Load the trained cross-modal feature alignment diagnostic model and perform model parameter solidification to obtain the solidified cross-modal feature alignment diagnostic model. The model parameter solidification operation includes: freezing the vibration branch residual self-attention feature dimensionality reduction module and all subsequent vibration-related network parameters in the trained cross-modal feature alignment diagnostic model, and retaining only the weights of the sound branch residual self-attention feature dimensionality reduction module, feature alignment module, and classification module in a reasoning-ready state, thus eliminating the dependence on vibration signals during the inference stage.

[0087] S7. Acquire the sound signal to be diagnosed, segment the sound signal according to the sampling time to form a sound signal sample to be diagnosed; perform a Fast Fourier Transform on the sound signal sample to be diagnosed to obtain the frequency domain representation of the sound signal to be diagnosed; perform Hilbert Transform, envelope extraction, and Fourier Transform on any one-sided spectrum of the frequency domain representation of the sound signal to be diagnosed in sequence to obtain the envelope spectrum of the sound signal to be diagnosed; select the top 2 from the envelope spectrum of the sound signal to be diagnosed. n The amplitude corresponding to each frequency point is used to construct the sound feature vector to be diagnosed. The sound feature vector in the test set or the sound feature vector constructed based on the sound signal to be diagnosed is used as the input to the solidified cross-modal feature alignment diagnostic model to obtain the bearing diagnosis result (exemplarily, this invention uses 5 types of labels, and the cross-modal feature alignment diagnostic model outputs the probability distribution of the five states, taking the category with the highest probability as the diagnosis result). In the above, using the sound feature vector in the test set as the input to the solidified cross-modal feature alignment diagnostic model is regarded as the testing stage, and using the sound feature vector constructed based on the sound signal to be diagnosed as the input to the solidified cross-modal feature alignment diagnostic model is regarded as the inference stage.

[0088] Further, in S7, using the sound feature vectors in the test set as input to the solidified cross-modal feature alignment diagnostic model specifically means: when the target working condition is the first working condition, the solidified cross-modal feature alignment diagnostic model obtained by training based on the training set and validation set in the acoustic-vibration synchronization dataset constructed under the first working condition is used as the first solidified model, and the sound feature vectors in the test set under the second working condition are used as input to the first solidified model to obtain the bearing diagnostic result; wherein the second working condition is the same as or different from the first working condition.

[0089] According to a second aspect of the present invention, a bearing fault diagnosis system based on cross-modal feature alignment is provided, comprising modules of the bearing fault diagnosis method based on cross-modal feature alignment described above; specifically comprising: a first module, configured to perform S1: under target operating conditions, synchronously acquire sound signals and vibration signals of preset duration for bearings in different states; align and segment the sound signals and vibration signals of preset duration in each state to form sound signal samples and vibration signal samples; wherein, the states include normal state and fault state; a second module, configured to perform S2: obtain the frequency domain representation of the sound signal and the frequency domain representation of the vibration signal based on the sound signal samples and the vibration signal samples respectively; obtain the sound signal envelope spectrum and the vibration signal envelope spectrum for any single-sided spectrum of the sound signal frequency domain representation and the vibration signal frequency domain representation respectively; a third module, configured to perform S3: based on the theoretical characteristic frequency distribution of bearing faults, select the top 2 from the sound signal envelope spectrum and the vibration signal envelope spectrum respectively. n The amplitude corresponding to each frequency point is used to construct sound feature vectors and vibration feature vectors; based on the sound feature vectors and vibration feature vectors of bearings in different states under various working conditions, a sound-vibration synchronization dataset is constructed; where n is a positive integer; the fourth module is used to execute S4: divide the sound-vibration synchronization dataset into training set, validation set and test set according to a preset ratio; the fifth module is used to execute S5: train the constructed cross-modal feature alignment diagnostic model based on the training set and validation set to obtain the trained cross-modal feature alignment diagnostic model; the sixth module is used to execute S6: load the trained cross-modal feature alignment diagnostic model, perform model parameter solidification operation to obtain the solidified cross-modal feature alignment diagnostic model; the seventh module is used to execute S7: acquire the sound signal to be diagnosed, segment the sound signal to be diagnosed to form sound signal to be diagnosed samples; obtain the frequency domain representation of the sound signal to be diagnosed based on the sound signal to be diagnosed samples; obtain the envelope spectrum of the sound signal to be diagnosed from any single-sided spectrum of the frequency domain representation of the sound signal to be diagnosed; select the top 2 from the envelope spectrum of the sound signal to be diagnosed. n The amplitude corresponding to each frequency point is used to construct the sound feature vector to be diagnosed. The sound feature vector in the test set or the sound feature vector to be diagnosed is used as the input to the solidified cross-modal feature alignment diagnostic model to obtain the bearing diagnosis result. For the parts of each module not described in detail above, please refer to the relevant descriptions in this embodiment.

[0090] Example 2: As Figures 1-11 As shown, the following describes optional embodiments of the present invention in conjunction with data:

[0091] I. Experimental Environment and Dataset Construction

[0092] In the model training parameters used in this invention, a batch size of 32 was selected to balance memory usage frequency and model update cycle. The Adam optimizer was used for training, with 150 training cycles per cycle. The programming environment was Python 3.9, using the PyTorch 1.13.1 framework. Hardware environment: CPU: Intel Core i9-14900K, GPU: NVIDIA RTX 4090D (32GB VRAM), RAM: 128GB, Hard Drive: 2TB.

[0093] This embodiment uses a deep groove ball bearing mounted on a rotating machinery fault simulation test bench as the research object for verification. The rotating machinery fault simulation test bench mainly includes a drive motor, a coupling, a deep groove ball bearing shaft, and a bearing housing. The deep groove ball bearing installed in the bearing housing mates with the shaft. The drive motor serves as the power source, driving the shaft to rotate through the coupling, which in turn drives the inner ring of the deep groove ball bearing to move. A magnetic powder brake is installed at the rear end of the rotating machinery fault simulation test bench to apply a load to the bearing during the experiment. The deep groove ball bearing model is SKF6205-2RS (bearing geometric parameters: 11 rolling elements, rolling element diameter 7.94mm, pitch circle diameter 47mm, contact angle 0°). The deep groove ball bearing condition types include normal (NR), inner ring fault (IF, fault diameter 0.1mm), outer ring fault (OF, fault diameter 0.1mm), rolling element fault (BF, fault diameter 0.1mm), and combined fault (MF, inner ring + outer ring fault). The drive motor operates at a speed of 1800 r / min. To monitor the operating status of deep groove ball bearings, two vibration accelerometers and two acoustic sensors were used for data acquisition. The two vibration accelerometers were installed horizontally and vertically on the bearing housing, respectively. The two acoustic sensors were at a 45° angle to the bearing housing and approximately 30cm away. Synchronous acquisition of the vibration and acoustic signals was achieved using an NI9234 data acquisition card and LabVIEW. The acoustic signal acquired by the sensor located further from the motor was used, while the vibration signal acquired by the horizontally mounted vibration sensor was used. For example, both the vibration accelerometers and the acoustic sensors were from the same PCB brand, and the sampling rate was 51.2kHz.

[0094] During the experiment, for five types of bearing conditions, a rotational speed of 1800 r / min was selected and combined with three different loads (0 N·m, 3 N·m, and 6 N·m) to form three working conditions. Acoustic and vibration signals (sound signals and vibration signals) were simultaneously acquired under each of the three working conditions. The acquisition time for both sound and vibration signals under each working condition was 120 s. The 120 s of simultaneously acquired sound and vibration signals under each working condition were aligned and segmented using a sliding window length of 4096 and a window overlap rate of 0.5 to form sound signal samples and vibration signal samples. Each condition could be segmented into 1500 samples, resulting in a total of 7500 samples (7500 sound signal samples and 7500 vibration signal samples) across the five conditions. To meet the experimental requirements for model robustness verification, Gaussian white noise with different signal-to-noise ratios can be superimposed on the original acoustic vibration signal during the sliding window stage. Then, according to steps S2, S3, and S4 in Example 1, the noisy vibration synchronization dataset is divided and used as model input to achieve effective verification of model robustness.

[0095] Subsequently, Fast Fourier Transform (FFT) was performed on the segmented sound signal samples and vibration signal samples to obtain the frequency domain representations of the sound signal and vibration signal, respectively. Considering the symmetry of the frequency domain data, only any one-sided spectrum of the sound signal and vibration signal frequency domain representations was retained. Hilbert Transform, envelope extraction, and Fourier Transform were then sequentially performed on any one-sided spectrum of the sound signal and vibration signal frequency domain representations to obtain the sound signal envelope spectrum and vibration signal envelope spectrum, respectively (i.e., performing Hilbert Transform, envelope extraction, and Fourier Transform sequentially on any one-sided spectrum of the sound signal frequency domain representation to obtain the sound signal envelope spectrum, and performing Hilbert Transform, envelope extraction, and Fourier Transform sequentially on any one-sided spectrum of the vibration signal frequency domain representation to obtain the vibration signal envelope spectrum). Based on the characteristic frequency distribution of bearing fault theory, it was found that the upper limit of the three-fold characteristic frequency of the core bearing fault is below 1000Hz. The amplitudes corresponding to the first 1024 frequency points were selected from the sound signal envelope spectrum and vibration signal envelope spectrum, respectively, to construct the sound feature vector and vibration feature vector. In this embodiment, Hilbert transform, envelope extraction, and Fourier transform are sequentially performed on any single-sided spectrum of the frequency domain representation of the sound signal and the frequency domain representation of the vibration signal to obtain the envelope spectrum of the sound signal and the envelope spectrum of the vibration signal. Specifically, the Hilbert transform is performed using the signal.hilbert function in the SciPy library to obtain the analytic signal. Envelope extraction involves calculating the magnitude of the analytic signal to obtain the envelope signal. The Fourier transform is then performed on the envelope signal to obtain the envelope spectrum of the sound signal and the envelope spectrum of the vibration signal.

[0096] In the above, the first 1024 frequency points of the envelope spectrum were ultimately selected to construct the sound feature vector and vibration feature vector. This is because 1024 is a power of 2, and a power of 2 is more compatible with the operational rules of network layers such as convolution and pooling, preventing feature compression failure and loss of edge information due to dimensionality mismatch. Furthermore, the 1024 frequency points retain key features while also preserving a small number of high-frequency points, which can carry background information such as equipment operating noise and weak coupling vibrations. This helps the model learn a more comprehensive spectral distribution pattern, further enhancing the model's ability to extract fault features and its diagnostic robustness under complex operating conditions.

[0097] The acoustic and vibration synchronization datasets constructed based on bearings in different states under three working conditions are shown in Table 1 below; the acoustic and vibration synchronization datasets are divided into training set, validation set and test set in a ratio of 7:2:1.

[0098] Table 1 Datasets under different working conditions

[0099]

[0100] II. Constructing a cross-modal feature alignment diagnostic model

[0101] The cross-modal feature alignment diagnostic model constructed in this embodiment has the same structure as the model described in Embodiment 1, including a feature extraction module, a feature alignment module, and a classification module. The network structure, parameter settings, and computational logic of each module are as follows:

[0102] The feature extraction module adopts a parallel dual-branch configuration, including a sound branch residual self-attention feature dimensionality reduction module and a vibration branch residual self-attention feature dimensionality reduction module with completely identical structures and independently initialized parameters. These modules take sound feature vectors and vibration feature vectors as inputs, respectively, and output sound and vibration dimensionality reduction features with a unified dimension of 128×1. Taking the sound branch residual self-attention feature dimensionality reduction module as an example, its network structure is as follows:

[0103] Initial convolutional layer: 1 input channel, 64 output channels, 7×1 kernel size, 2 stride, 3 padding, used to perform preliminary extraction of the 1024×1 input feature vector, and the output feature dimension is 512×64;

[0104] Batch Normalization (BN) layer: Standardizes the features output by the initial convolutional layer, constraining the feature distribution to a normal distribution with 0 mean and 1 variance, thus accelerating model training convergence. The working mechanism of all BN layers in the following text is consistent with this.

[0105] GELU activation function: Nonlinearly maps the normalized distribution of the BN layer output, introduces nonlinear expressive power, and improves the discriminative power of fault features;

[0106] Stride Residual Block: The input is the feature processed by the GELU activation function, divided into a main branch and a shortcut connection branch. The main branch of the stride residual block consists of the first convolutional layer (64 input channels, 128 output channels, kernel size 5×1, stride 2, padding 2), a BN layer, a GELU activation function, the second convolutional layer (128 input channels, 128 output channels, kernel size 5×1, stride 1, padding 2), and a BN layer, outputting the first sound backbone feature. The shortcut connection branch of the stride residual block consists of the third convolutional layer (1×1 kernel size, stride 2, 64 input channels, 128 output channels) and a BN layer, outputting the first sound jump feature. The features output by the main branch and the shortcut connection branch of the stride residual block are summed element-wise along the channel dimension and then output through the GELU activation function, resulting in a feature dimension of 256×128.

[0107] Initial Self-Attention Module: The input is the deep features output by the stride residual block. The number of attention heads is set to 8, and the 128-dimensional input channels are evenly distributed to each attention head, with a single head dimension of 16. Three sets of features are generated through a 1×1 convolutional layer: query vector Q, key vector K, and value vector V. The attention score distribution is constrained by the scaling factor of the channel dimension. The global attention weights are solved by normalization using the Softmax function, completing the weighted aggregation of features in both spatial and channel dimensions. Then, a 1×1 convolutional layer is used to complete the feature dimension projection and reconstruction, outputting the initial self-attention enhanced features with the same dimension as the input. At the same time, residual connections are introduced to retain the original feature information and avoid gradient vanishing.

[0108] Identity Residual Block: The input is the initial self-attention enhancement feature output by the initial self-attention module, divided into a main branch and a shortcut connection branch. The main branch of the identity residual block consists of the fourth convolutional layer (128 input channels, 128 output channels, 3×1 kernel size, stride 1, padding 1), a BN layer, a GELU activation function, the fifth convolutional layer (128 input channels, 128 output channels, 3×1 kernel size, stride 1, padding 1), and a BN layer, outputting the second sound backbone feature. The shortcut connection branch of the identity residual block is an identity mapping, without convolution and normalization operations, directly outputting the original input as the second sound jump feature. The features output by the main branch and the shortcut connection branch of the identity residual block are summed element-wise along the channel dimension and then output through the GELU activation function, keeping the feature dimension unchanged at 256×128.

[0109] Enhanced self-attention module: The network structure and feature enhancement methods are completely consistent with the initial self-attention module, and the input and output dimensions remain unchanged at 256×128. It further captures the global dependencies of fault features and strengthens the robustness of features.

[0110] Adaptive average pooling layer: The input is the feature output by the enhanced self-attention module. The pooling parameter is set to (None, 1). The height dimension and number of channels of the feature remain unchanged, while the width dimension is compressed to 1. This completes the global information aggregation and dimension regularization of the width dimension, and finally outputs a 128×1 sound dimensionality reduction feature.

[0111] The feature alignment module takes the reduced sound and vibration features output by the feature extraction module as input, and achieves spatial unification and depth alignment of the sound and vibration features through shared projection space mapping and distribution constraints. The specific structure is as follows:

[0112] Shared feature projection layer: A 1×1 convolutional layer with shared weights is used to perform unified spatial projection on the dimensionality reduction features of sound and vibration. There are 128 input channels and 128 output channels. The weights are initialized using the Kaiming normal distribution to force the sound and vibration features to be mapped to the same common feature space, thus eliminating the semantic gap between modes.

[0113] L2 Normalization Layer: The L2 norm normalization process is applied to the cross-modal shared mapping features of sound and vibration output from the shared projection layer, constraining the features to a unit hypersphere space, eliminating differences in feature dimensions, and providing a unified metric for cross-modal feature alignment. The normalization formula is as follows: ,in, This refers to the original features output by the shared projection layer (i.e., sound cross-modal shared mapping features or vibration cross-modal shared mapping features). for L2 norm, This represents the alignment features after normalization.

[0114] The classification module adopts a fully connected discriminant architecture, using the fusion result of spatially aligned sound alignment features and vibration alignment features as input to achieve accurate classification of state types. The specific structure is as follows:

[0115] Feature Fusion Layer: During the training phase, the sound alignment features and vibration alignment features are fused element-wise along the channel dimension to obtain a joint sound-vibration discriminative 3D feature tensor; during the testing / inference phase, only the sound alignment features are input as the joint discriminative 3D feature tensor.

[0116] Flattening layer: Flattens the fused 3D feature tensor into a 128-dimensional 1D feature vector to adapt to the input format requirements of the fully connected layer;

[0117] The first fully connected layer has an input dimension of 128 and an output dimension of 64. The weights are initialized using a He normal distribution to achieve the mapping of high-dimensional joint features to low-dimensional discriminative features.

[0118] GELU activation function: performs non-linear mapping on the output features of the first fully connected layer to enhance the discriminative representation ability of classification features;

[0119] Dropout regularization layer: The inactivation probability is set to 0.3, which randomly suppresses the response of some neurons, effectively alleviating the overfitting problem and improving the generalization ability of the model;

[0120] The second fully connected layer has an input dimension of 64 and an output dimension of 5, corresponding to 5 types of bearing operating states. It maps features to the state category dimension and outputs the final state classification logical value.

[0121] The cross-modal fusion loss function adopts the same total loss function as in Example 1.

[0122] III. Model Training and Hyperparameter Setting

[0123] This embodiment completes model training based on the aforementioned acoustic-vibration synchronization dataset. The training process is divided into three stages: model initialization, iterative training, and model solidification. The specific settings and procedures are as follows:

[0124] The model was trained using the Adam optimizer with an initial learning rate of 0.0005, employing a fixed learning rate with no decay throughout. The batch size was set to 32, and the total number of epochs was 150. A weight decay factor of 1e-4 was used to suppress overfitting. Alignment loss scaling was also employed. Alignment loss weighting coefficients in the total loss function All convolutional layer weights are initialized using a uniform Xavier distribution, BN layer parameters are initialized using default settings, and fully connected layer biases are initialized to 0.

[0125] 1. Dataset Loading: All samples from working condition T1 (1800 r / min, 3 N·m) are used as the basic dataset, divided into training, validation, and test sets in a 7:2:1 ratio. The test set of samples from working conditions T2 and T3, after being divided according to the aforementioned ratio, is used as the variable working condition transfer test set, or the test set of working condition T1 is directly used for testing. That is, when the target working condition is T1, the solidified cross-modal feature alignment diagnostic model obtained by training on the training and validation sets in the acoustic-vibration synchronization dataset constructed under working condition T1 is used as the first solidified model, and the sound feature vectors in the test sets of working conditions T1, T2, and T3 are used as the input of the first solidified model for testing. When loading the training set, a random shuffling operation is performed, while the sample order of the validation and test sets remains unchanged. It should be noted that the same principle applies when using working conditions T2 and T3 as the basic dataset.

[0126] 2. Model forward propagation: Input the sound feature vector and vibration feature vector from the training set into the corresponding branches of the feature extraction module to obtain sound dimensionality reduction features and vibration dimensionality reduction features; perform shared projection and L2 normalization through the feature alignment module to obtain spatially aligned sound alignment features and vibration alignment features; send them into the classification module to output the state classification logic value; at the same time, calculate the classification loss and sound and vibration feature alignment loss, and sum them according to the weights to obtain the total loss;

[0127] 3. Backpropagation and parameter update: Backpropagation is performed based on the total loss value to calculate the gradient of each trainable parameter of the model, and the parameters are updated through the Adam optimizer; after each epoch of training, gradient calculation is turned off, model inference is performed on the validation set, and the diagnostic accuracy of the validation set is calculated.

[0128] 4. Model saving: During training, if the validation set accuracy of the current epoch is higher than the historical best value, the current model weights are saved as the optimal model; after training is completed, the full model weights of the last iteration are saved for subsequent model loading and solidification to obtain the solidified cross-modal feature alignment diagnostic model.

[0129] IV. Experimental Results and Analysis

[0130] Figure 4 The evolution of training and validation accuracy over 150 epochs under scenario T1 is illustrated. The model exhibits rapid convergence, with training and validation accuracy quickly soaring to nearly 100% in the early stages of training and remaining highly stable thereafter. Furthermore, the synchronous small fluctuations observed in the two curves indicate that the optimization algorithm is actively exploring the loss terrain to escape shallow local optima, further confirming the robustness of the training process.

[0131] To intuitively reveal the intrinsic mechanism by which the model processes multimodal inputs, Figure 5 , Figure 6The t-SNE visualization results of the latent representations learned in the 1st and 150th epochs under the T1 condition are presented respectively. In the visualization, A represents sound features, V represents vibration features, and NR, IF, OF, BF, and MF correspond to the five states (e.g., A-NR represents sound features in a healthy state, and so on). In the initial training phase, the feature space exhibits a fragmented and unaligned topology. Although samples of the same modality show a slight clustering trend, sound and vibration features belonging to the same diagnostic category are completely isolated in space. This obvious isolation indicates a significant semantic gap and cross-modal heterogeneity, suggesting that an insufficiently trained model cannot effectively associate these two distinct physical modalities. Conversely, after 150 epochs of joint training, the topology of the feature space undergoes a profound reconstruction. Sound and vibration features in the same healthy state achieve deep fusion and perfect alignment, converging into five highly distinct and mutually separated clusters. This stark contrast powerfully demonstrates that the proposed architecture successfully bridges the cross-modal semantic gap, effectively eliminating modal heterogeneity and extracting a unified, modality-invariant shared representation. The resulting feature space exhibits maximized intra-class compactness and inter-class separation, fundamentally explaining and supporting the model's excellent performance in achieving 100% diagnostic accuracy in complex multimodal scenarios.

[0132] Simultaneously, this embodiment performs single-modal inference testing on the solidified model, completely cutting off the vibration feature vector input and relying solely on the sound feature vector under the T1 condition to complete the test set diagnosis. Figure 7 Confusion matrix of model test results and Figure 8 The t-SNE feature clustering results show that the model achieves 100% diagnostic accuracy across all five bearing states, further validating the effectiveness of cross-modal knowledge transfer.

[0133] To verify the robustness and feature transfer capability of the model under varying load conditions, this embodiment designs 6 sets of cross-transfer tasks based on three load conditions: T1, T2, and T3. The accuracy of the task settings and test results is shown in Table 2. The training set and test set are completely from different load conditions to verify the model's cross-load generalization capability.

[0134] Table 2. Accuracy of test results under different task settings

[0135]

[0136] Experimental results show that the model exhibits excellent diagnostic performance in all six transfer tasks. The diagnostic accuracy rate of Task 1 (3N·m→0N·m) reached 96.34%, and even in Task 6 (6N·m→0N·m), which has the largest load span, the model still maintains a diagnostic accuracy rate of 93.48%, effectively demonstrating that the fault features extracted by the model can effectively adapt to complex industrial scenarios with load fluctuations.

[0137] To simulate the complex background noise environment of an industrial site, this embodiment superimposes Gaussian white noise with different signal-to-noise ratios (SNR) onto the sound signals corresponding to the test set. The SNR is gradually reduced from 0 dB to -6 dB to verify the model's noise resistance. The experimental results are as follows: Figures 9-11 As shown in the figure. Experimental results demonstrate that the model of this invention exhibits excellent noise robustness. With increasing noise intensity, the feature Euclidean distance of various states generally increases, especially the composite fault features, which are most significantly affected by strong noise interference, leading to distortion and aliasing in the feature space. Simultaneously, the model's diagnostic accuracy decreases steadily in a stepwise manner as noise intensifies. However, even under the harsh condition of extremely low signal-to-noise ratio (SNR=-6dB), although the model's diagnostic accuracy decreases from 100% of the original signal, it still maintains a relatively high level of 89.50%. This fully demonstrates that the method still possesses strong feature extraction capabilities and excellent noise robustness under complex and strong noise interference.

[0138] To further illustrate the superiority of the technology proposed in this invention, the experimental process was adjusted and compared with existing technologies again. All models were trained using data with a signal-to-noise ratio of 0dB and tested using data with a signal-to-noise ratio of -6dB. The invention method was compared with a single vibration signal model (i.e., including a vibration branch residual self-attention feature reduction module + classification module one, where classification module one removes the feature fusion layer based on the classification module of this invention), a single sound signal model (i.e., including a sound branch residual self-attention feature reduction module + classification module one, where the classification module removes the feature fusion layer based on the classification module of this invention), and a simple sound-vibration fusion model (feature extraction module + classification module, where the feature fusion layer of the classification module uses the sound dimensionality reduction features and vibration dimensionality reduction features output by the feature extraction module as input). The comparison results are shown in Table 3.

[0139] Table 3 Comparison of different methods

[0140]

[0141] The comparison results show that the diagnostic accuracy of the method of the present invention far exceeds that of the other three comparison methods when relying solely on sound signals for inference. At the same time, the single-sample inference time is only 0.015s, which is lower than the model that simply fuses the sound and vibration features. The above comparison results show that the cross-modal feature alignment realizes the effective transfer of vibration fault knowledge to sound signals, which significantly improves the diagnostic accuracy in strong noise environment while ensuring the inference speed.

[0142] The specific embodiments of the present invention have been described in detail above with reference to the accompanying drawings. However, the present invention is not limited to the above embodiments. Within the scope of knowledge possessed by those skilled in the art, various changes can be made without departing from the spirit of the present invention.

Claims

1. A bearing fault diagnosis method based on cross-modal feature alignment, characterized in that, include: S1. Under the target working condition, synchronously collect sound and vibration signals of bearings in different states for a preset duration; The sound and vibration signals of preset duration under each state are aligned and segmented to form sound signal samples and vibration signal samples; the states include normal state and fault state. S2. Obtain the frequency domain representation of the sound signal and the frequency domain representation of the vibration signal based on the sound signal sample and the vibration signal sample, respectively; obtain the envelope spectrum of the sound signal and the envelope spectrum of the vibration signal for any single-sided spectrum of the frequency domain representation of the sound signal and the frequency domain representation of the vibration signal, respectively. S3, based on the bearing fault theory characteristic frequency distribution, respectively, from the sound signal envelope spectrum, vibration signal envelope spectrum selects the amplitude corresponding to the first 2 n frequency points, constructs the sound feature vector and the vibration feature vector; according to the sound feature vector and the vibration feature vector of the bearing in different states under each working condition, a sound-vibration synchronous data set is constructed; wherein n is a positive integer; S4. Divide the acoustic-vibration synchronization dataset into training set, validation set and test set according to a preset ratio; S5. Train the constructed cross-modal feature alignment diagnostic model based on the training set and validation set to obtain the trained cross-modal feature alignment diagnostic model; S6. Load the trained cross-modal feature alignment diagnostic model, perform the model parameter solidification operation, and obtain the solidified cross-modal feature alignment diagnostic model. S7、obtaining the sound signal to be diagnosed, splitting the sound signal to be diagnosed to form a sound signal sample to be diagnosed; obtaining a frequency domain representation of the sound signal to be diagnosed according to the sound signal sample to be diagnosed; obtaining an envelope spectrum of the sound signal to be diagnosed from any single side spectrum of the frequency domain representation of the sound signal to be diagnosed; selecting the amplitudes corresponding to the first two n frequency points from the envelope spectrum of the sound signal to be diagnosed to construct a sound feature vector to be diagnosed; taking the sound feature vector in the test set or the sound feature vector to be diagnosed as the input of the solidified cross-modal feature alignment diagnosis model to obtain the bearing diagnosis result.

2. The bearing fault diagnosis method based on cross-modal feature alignment according to claim 1, characterized in that, In step S7, using the sound feature vectors in the test set as input to the solidified cross-modal feature alignment diagnostic model specifically means: when the target working condition is the first working condition, the solidified cross-modal feature alignment diagnostic model obtained by training based on the training set and validation set in the acoustic-vibration synchronization dataset constructed under the first working condition is used as the first solidified model, and the sound feature vectors in the test set under the second working condition are used as input to the first solidified model to obtain the bearing diagnostic result; wherein the second working condition is the same as or different from the first working condition.

3. The bearing fault diagnosis method based on cross-modal feature alignment according to claim 1, characterized in that, The cross-modal feature alignment diagnostic model includes: The feature extraction module adopts a parallel dual-branch configuration: a sound branch residual self-attention feature dimensionality reduction module and a vibration branch residual self-attention feature dimensionality reduction module. The sound feature vector is used as the input of the sound branch residual self-attention feature dimensionality reduction module, and the output is the sound dimensionality reduction feature; the vibration feature vector is used as the input of the vibration branch residual self-attention feature dimensionality reduction module, and the output is the vibration dimensionality reduction feature. The feature alignment module maps the reduced sound and vibration features output by the feature extraction module to a unified feature space through a shared projection layer. It eliminates the amplitude differences between modes through an L2 normalization layer, thereby achieving spatial unification and depth alignment of the sound and vibration features. It outputs spatially aligned sound and vibration features. The classification module adopts a fully connected discriminant configuration, takes the fusion result of spatially aligned sound alignment features and vibration alignment features as input, and outputs a state classification logic value.

4. The bearing fault diagnosis method based on cross-modal feature alignment according to claim 3, characterized in that, The sound branch residual self-attention feature dimensionality reduction module and the vibration branch residual self-attention feature dimensionality reduction module have the same structure. The sound branch residual self-attention feature dimensionality reduction module is used as an example for description, specifically including: Initial convolutional layer: used for preliminary extraction of sound signal features; Batch normalization layer: used to take the sound signal features output from the initial convolutional layer as input and obtain a standardized distribution of the sound features; GELU activation function: used to perform nonlinear mapping on the standardized distribution of sound features obtained from the batch normalization layer; Stride Residual Block: The input is the sound features nonlinearly mapped by the GELU activation function. The input is passed through the main branch of the stride residual block and the shortcut connection branch of the stride residual block, respectively, and the output is the first sound backbone feature and the first sound jump feature. The first sound backbone feature and the first sound jump feature are added element-wise along the channel dimension and fused together, and then the nonlinear mapping transformation is completed by the GELU activation function. Initial self-attention module: The input is the deep features after stride residual block transformation, which are fed into the initial self-attention module for global dependency modeling, and the output is the initial self-attention enhanced features; Identical Residual Block: The input is the initial self-attention enhancement feature output by the initial self-attention module. It is then passed through the main branch of the identity residual block and the shortcut connection branch of the identity residual block to output the second sound backbone feature and the second sound jump feature. The second sound backbone feature and the second sound jump feature are then added element-wise along the channel dimension and fused together. Finally, the nonlinear mapping transformation is completed by the GELU activation function. Enhanced Self-Attention Module: The input is the deep features mapped by the identity residual block, which are fed into the enhanced self-attention module for global dependency modeling, and the output is the enhanced self-attention enhanced features; the enhanced self-attention module has the same structure as the initial self-attention module. Adaptive average pooling layer: The input is the enhanced self-attention enhancement feature output by the enhanced self-attention module, which is then normalized in terms of feature dimension by the adaptive average pooling layer, and the output is the normalized audio dimensionality reduction feature.

5. The bearing fault diagnosis method based on cross-modal feature alignment according to claim 4, characterized in that, The stride residual block main branch includes a first convolutional layer, a BN layer, a GELU activation function, a second convolutional layer, and a BN layer connected in sequence, used to output the first sound backbone feature; the stride residual block shortcut connection branch includes a third convolutional layer and a BN layer connected in sequence, used to output the first sound jump feature.

6. The bearing fault diagnosis method based on cross-modal feature alignment according to claim 4, characterized in that, The main branch of the identity residual block includes a fourth convolutional layer, a BN layer, a GELU activation function, a fifth convolutional layer, and a BN layer connected in sequence, which are used to output the second sound backbone features; the shortcut connection branch of the identity residual block is an identity mapping, which directly outputs the second sound jump features.

7. The bearing fault diagnosis method based on cross-modal feature alignment according to claim 3, characterized in that, The classification module includes a feature fusion layer, a flattening layer, a first fully connected layer, a GELU activation function, a Dropout regularization layer, and a second fully connected layer connected in sequence.

8. The bearing fault diagnosis method based on cross-modal feature alignment according to claim 1, characterized in that, The cross-modal feature alignment diagnostic model is trained using a cross-modal fusion loss function, which is a weighted fusion joint loss configuration consisting of classification loss and acoustic-vibration feature alignment loss. The classification loss uses a cross-entropy loss function. The acoustic-vibration feature alignment loss is calculated by taking the spatially aligned sound alignment features and vibration alignment features as inputs and calculating the mean square distance between the two types of features.

9. A bearing fault diagnosis system based on cross-modal feature alignment, characterized in that, The module includes any one of the bearing fault diagnosis methods based on cross-modal feature alignment as described in claims 1-8.

Images