A rolling bearing anomaly detection method of multi-modal signal fusion and denoising diffusion probability model

Abstract

This invention relates to the field of bearing anomaly detection technology, and particularly to a rolling bearing anomaly detection method based on multimodal signal fusion and a denoising diffusion probability model. First, this invention proposes an attention-enhanced denoising diffusion model anomaly detection strategy (MFE-DDPM) based on vibration, acoustic, and temperature data for rolling bearing anomaly detection. Simultaneously, it proposes a multimodal signal data-level fusion method that can fuse vibration, acoustic, and temperature data. This invention embeds a channel attention module (SE-block) into the DDPM with U-net as the backbone network, which enhances the model's ability to perceive key anomaly regions and model multi-scale features. Compared with other single-modal signal and non-attention mechanism methods, the proposed method has the highest average accuracy.

Description

Technical Field

[0001] This invention relates to the field of bearing anomaly detection technology, and in particular to a rolling bearing anomaly detection method based on multimodal signal fusion and a denoising diffusion probability model. Background Technology

[0002] With the continuous improvement of intelligent manufacturing and industrial automation, equipment health management and fault diagnosis have become key links to ensure the stable operation of industrial systems. Traditional rolling bearing anomaly detection methods mainly rely on the analysis and modeling of single-mode sensor signals, which generally suffers from over-reliance on a single data source. They are easily affected by external factors such as operating condition fluctuations, background noise, and sensor placement errors, resulting in insufficient detection robustness and generalization ability.

[0003] Multi-source fusion methods have been extensively studied in recent years. Although some methods have alleviated the problem of limited information dimensionality in single-channel signals to some extent, they are essentially still limited to the fusion of homogeneous signals and have failed to fully explore the potential complementary features between heterogeneous modes. For multi-source heterogeneous sensor data fusion detection, the following problems still exist: 1. Existing methods mostly fuse signals from two different modes, and there is still room for improvement in constructing a more comprehensive representation of bearing fault information; 2. Most studies use supervised deep learning methods, which require labeled bearing fault data or a large amount of expert knowledge and experience as support. This poses a challenge to the current situation where bearing fault data is not easy to obtain and expert knowledge is insufficient in most cases. Summary of the Invention

[0004] The purpose of this invention is to provide a rolling bearing anomaly detection method based on multimodal signal fusion and a denoising diffusion probability model. This method proposes a fusion method for rolling bearing vibration, sound, and temperature signals, and improves the traditional denoising diffusion model to enable it to be used for rolling bearing signal fusion anomaly detection. By applying the model to publicly available rolling bearing fault data and comparing it with other methods, the superiority of the model can be verified.

[0005] To achieve the above objectives, the present invention adopts the following technical solution: A rolling bearing anomaly detection method based on multimodal signal fusion and a denoising diffusion probability model includes the following steps: S10: Acquire sample data signals through vibration sensors, sound sensors, and temperature sensors; S20: Construct corresponding image representation models for the non-stationary characteristics of vibration signals, the transient pulse characteristics of acoustic emission signals, and the thermodynamic degradation law of temperature fields, respectively. S30: Assign the three converted signals to the three color channels R (red), G (green), and B (blue) respectively, and realize cross-modal fusion at the data level in the RGB space; S40: Constructing a DDPM structure enhanced by SE attention mechanism: While keeping the basic framework of the original DDPM diffusion and anti-diffusion process unchanged, the key modules in U-Net were systematically improved to enhance the model's representation ability and information interaction ability. S50: The fused data is input into the improved MFE-DDPM model for training. The model is trained to perceive key feature regions and model multi-scale features. The input fused signal generates a reconstructed image after passing through the model. S60: An anomaly metric function is constructed by comparing the peak signal-to-noise ratio (PSNR) of the original input image and the reconstructed image of all sample data. This function is used to calculate the reconstruction error and thus achieve effective discrimination of anomalous samples. S70: Perform diagnostic performance testing on the trained model.

[0006] Furthermore, in step S20, constructing the corresponding image representation models includes the following steps: S21: Vibration data conversion: The one-dimensional time-series signal of the vibration signal is converted into a two-dimensional time-frequency image through the Adaptive Wavelet Transform (AWT); S22: Acoustic data encoding: Encoding the spectrum of acoustic data into the spectral Markov transition field (SMTF); S23: Temperature data conversion: Convert the temperature signal into a grayscale image through two-dimensional expansion.

[0007] Furthermore, in step S21, the one-dimensional time-series signal of the vibration signal is converted into a two-dimensional time-frequency image, which specifically includes the following steps: S211: Divide the original signal into multiple fixed-length time segments, and process each data segment... AWT is used for multi-level decomposition, with the wavelet basis function Daubechies4 used and a decomposition level of 4. This decomposition process decomposes the signal into coefficient subsets of multiple frequency bands.

[0008] in, The number of decomposition layers, For the first Wavelet coefficients for each frequency band; S212: The energy values ​​are normalized and copied along the vertical direction to form a two-dimensional frequency band energy matrix, which is finally mapped to a grayscale image, represented as:

[0009] in, This represents the normalized frequency band energy, with the number of rows corresponding to the image height and the number of columns representing the frequency bands.

[0010] Furthermore, in step S22, the specific steps for encoding the acoustic data into a spectral Markov transfer field are as follows: S221: Input signal preprocessing and normalization: The original sound signal is a one-dimensional discrete time series. First, perform min-max normalization to compress it to... Interval:

[0011] S222: State Discretization: Normalizing the sequence... Mapping to a set of discrete states K represents dividing a continuous signal into K levels of states. The discretization operation can be formalized as:

[0012] Discrete state ; S223: Construct the state transition matrix: Traverse the entire state sequence Statistically count the frequency of first-order Markov state transitions and construct the transition matrix. The definition is as follows:

[0013] in, Here is the characteristic function; to transform it into a probability matrix, row-by-row normalization is performed:

[0014] in To prevent tiny constants with a denominator of zero; S224: Constructing the spectral Markov transfer field image: transfer matrix Visualized as a two-dimensional grayscale image, where the intensity of each pixel represents the state. to state To improve image compatibility and network adaptability, the transition probability is adjusted using image interpolation to a fixed size. And save it as a grayscale image: .

[0015] Furthermore, in step S23, converting the temperature signal into a grayscale image through two-dimensional expansion specifically includes the following steps: compressing the one-dimensional temperature signal to a standard normalized range, and then performing row expansion in two-dimensional space by copying its amplitude vector, thereby generating a two-dimensional image representation: In the formula, This represents the original temperature signal.

[0016] Furthermore, in step S40, the key modules in U-Net are systematically improved, including the following steps: S41: The intermediate layer introduces the SE attention mechanism (Squeeze-and-Excitation Block): The SE module can capture the non-linear dependencies between channels through global average pooling and learn the recalibrated weights between channels through two fully connected layers, effectively improving the model's ability to focus on important feature channels. This module first processes the input feature map... Perform global average pooling in the spatial dimension:

[0017] Subsequently, a bottleneck structure is constructed using a fully connected layer, and the importance weights of each channel are learned through nonlinear transformation. ,in Indicates ReLU activation. This represents the Sigmoid activation function. Finally, the original feature map is reconstructed using weighted methods:

[0018] S42: Dilated Convolution Expands the Receptive Field: Dilated convolutions with a dilation rate of 2 are introduced into the convolutional operations of the intermediate layers of U-Net. This expands the receptive field without increasing the number of parameters, thereby enhancing the ability to model cross-regional features. It can be viewed as inserting holes into the standard convolutional kernel, and its calculation form is as follows: ,in This represents the dilation rate, a structure that allows for the aggregation of a wider range of contextual information while keeping computational complexity under control. S43: Introducing Residual Connections to Enhance Stability: To prevent information degradation or gradient vanishing problems during deep feature processing, lightweight residual connections are introduced after the SE module. These connections map the original feature map to the same number of channels through convolution and are then added element-wise to the SE output.

[0019] S44: Self-attention mechanism introduced in upsampling and downsampling paths: AttnDownBlock2D and AttnUpBlock2D modules are introduced into the downsampling and upsampling blocks of U-Net, respectively. This module integrates a multi-head self-attention mechanism to model the global dependency relationship between different regions in the spatial dimension and focus on abnormal feature regions.

[0020] Furthermore, in step S50, the process of inputting the fused data into the improved MFE-DDPM model for training includes the following: The evolution of noise in the fused data during the diffusion process is modeled. By adaptively adjusting the weight allocation of different feature channels, the expressive power of key feature information and the information interaction ability between features are enhanced, thereby achieving more accurate prediction of noise components. During the anti-diffusion process, noise interference is gradually removed based on the predicted noise information, and finally a reconstructed image corresponding to the input fused data is generated to obtain higher quality reconstruction results. The root mean square loss function for predicted noise and the Adam algorithm are used during the training process.

[0021] Furthermore, in step S60, the reconstruction error calculation, thereby achieving effective discrimination of abnormal samples, includes the following steps: S61: The mathematical definition of PSNR is:

[0022] in, The original input image, For the corresponding reconstructed image, Indicates the maximum value of a pixel. Indicates mean square error; S62: PSNR values ​​are further normalized, using... Define the normalized reconstruction error; S63: For all error values Calculate its mean with standard deviation The discrimination threshold is set according to the normal distribution probability and statistics theory. for: .

[0023] The beneficial effects of this invention are as follows: The rolling bearing anomaly detection method based on multimodal signal fusion and denoising diffusion probability model provided by this invention has the following characteristics and advantages: First, this invention proposes an attention-enhanced denoising diffusion model anomaly detection strategy (MFE-DDPM) based on vibration, acoustic and temperature data for rolling bearing anomaly detection. At the same time, it proposes a multimodal signal data-level fusion method that can fuse vibration, acoustic and temperature data. This invention embeds the channel attention module (SE-block) into the DDPM with U-net as the backbone network, which can enhance the model's ability to perceive key anomaly regions and model multi-scale features. Compared with other single-modal signals and methods without attention mechanisms, the proposed method has the highest average accuracy. Attached Figure Description

[0024] Figure 1 This is a flowchart of the rolling bearing anomaly detection method of the present invention; Figure 2 This is an overall framework diagram of the rolling bearing anomaly detection method of the present invention; Figure 3 This is the detection confusion matrix for different methods of the present invention; Figure 4 The fault diagnosis accuracy of different methods of the present invention; Figure 5 The distribution of F1 scores for different methods of this invention is shown. Detailed Implementation

[0025] The technical solutions of the present invention will be clearly and completely described below with reference to embodiments. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention. It should be noted that: in the present invention, unless otherwise specified, all embodiments and preferred methods mentioned herein can be combined to form new technical solutions. In the present invention, unless otherwise specified, all technical features and preferred features mentioned herein can be combined to form new technical solutions. The "scope" disclosed in the present invention can be in the form of a lower limit and an upper limit, and can be one or more lower limits and one or more upper limits, respectively. Specific Implementation Example 1: Reference Figure 1 As shown, this invention relates to a rolling bearing anomaly detection method based on multimodal signal fusion and a denoising diffusion probability model, comprising the following steps: S10: Acquire sample data signals through vibration sensors, sound sensors, and temperature sensors; S20: Construct corresponding image representation models for the non-stationary characteristics of vibration signals, the transient pulse characteristics of acoustic emission signals, and the thermodynamic degradation law of temperature fields, respectively. S30: Assign the three converted signals to the three color channels R (red), G (green), and B (blue) respectively, and realize cross-modal fusion at the data level in the RGB space; S40: Constructing a DDPM structure enhanced by SE attention mechanism: While keeping the basic framework of the original DDPM diffusion and anti-diffusion process unchanged, the key modules in U-Net were systematically improved to enhance the model's representation ability and information interaction ability. S50: The fused data is input into the improved MFE-DDPM model for training, which trains the model's ability to perceive key feature regions and model multi-scale features; the input fused signal generates a reconstructed image after passing through the model. S60: An anomaly metric function is constructed by comparing the peak signal-to-noise ratio (PSNR) of the original input image and the reconstructed image of all sample data. This function is used to calculate the reconstruction error and thus achieve effective discrimination of anomalous samples. S70: Perform diagnostic performance testing on the trained model; This detection method achieves multimodal signal fusion through RGB and improves the DDPM structure, giving it certain advantages in detecting anomalies in multimodal signal fusion of rolling bearings.

[0027] In embodiments of the present invention, as described in the appendix to the specification... Figure 1 As shown, in step S10, acquiring sample data signals includes the following steps: The Ottawa multimodal bearing data level was used for model training and validation, with 80% used for model training and the remaining 20% ​​used as a component of normal samples in the subsequent test set. In one embodiment of the present invention, as described in the appendix to the specification... Figure 2 As shown, in step S20, constructing the corresponding image representation models includes the following steps: S21: Vibration data conversion: The one-dimensional time-series signal of the vibration signal is converted into a two-dimensional time-frequency image through the Adaptive Wavelet Transform (AWT); S22: Acoustic data encoding: Encoding the spectrum of acoustic data into the spectral Markov transition field (SMTF); S23: Temperature data conversion: Convert the temperature signal into a grayscale image through two-dimensional expansion.

[0028] In embodiments of the present invention, as described in the appendix to the specification... Figure 2 As shown, step S21, converting the one-dimensional time-series signal of the vibration signal into a two-dimensional time-frequency image, specifically includes the following steps: S211: Divide the original signal into multiple fixed-length time segments, and process each data segment... AWT was used for multi-level decomposition, with the wavelet basis function being Daubechies 4 and the decomposition level being 4. This decomposition process decomposed the signal into coefficient subsets of multiple frequency bands:

[0029] in, The number of decomposition layers, For the first Wavelet coefficients for each frequency band; S212: The energy values ​​are normalized and copied along the vertical direction to form a two-dimensional frequency band energy matrix, which is finally mapped to a grayscale image, represented as:

[0030] in, This represents the normalized frequency band energy, with the number of rows corresponding to the image height and the number of columns representing the frequency bands.

[0031] In one embodiment of the present invention, as described in the appendix to the specification... Figure 2 As shown, in step S22, the specific steps for encoding the acoustic data into a spectral Markov transfer field are as follows: S221: Input signal preprocessing and normalization: The original sound signal is a one-dimensional discrete time series. First, perform min-max normalization to compress it to... Interval:

[0032] S222: State Discretization: Normalizing the sequence... Mapping to a set of discrete states K represents dividing the continuous signal into K levels of states. The discretization operation can be formalized as:

[0033] Discrete state ; S223: Construct the state transition matrix: Traverse the entire state sequence Statistically count the frequency of first-order Markov state transitions and construct the transition matrix. The definition is as follows:

[0034] in, Let be the indicator function. To transform it into a probability matrix, perform row-by-row normalization:

[0035] in To prevent tiny constants with a denominator of zero; S224: Constructing the spectral Markov transfer field image: transfer matrix Visualized as a two-dimensional grayscale image, where the intensity of each pixel represents the state. to state The transition probability. To improve image compatibility and network adaptability, image interpolation is used to adjust the image to a fixed size. And save it as a grayscale image:

[0036] In one embodiment of the present invention, such as Figure 2 As shown, step S23, converting the temperature signal into a grayscale image through two-dimensional expansion, specifically includes the following steps: The one-dimensional temperature signal is compressed to a standard normalized range, and then its amplitude vector is copied and row-expanded in two-dimensional space to generate a two-dimensional image representation.

[0037] In the formula, This represents the original temperature signal.

[0038] In embodiments of the present invention, as described in the appendix to the specification... Figure 2 As shown, in step S30, the data-level cross-modal fusion of the three converted signals in the RGB space includes the following steps: The three signals are assigned three color channels: red, green, and blue. Data-level fusion of multi-source information is completed in the RGB space to obtain 3-channel fused data for subsequent network training.

[0039] In one embodiment of the present invention, as described in the appendix to the specification... Figure 2 As shown, in step S40, while maintaining the basic framework of the original DDPM diffusion and de-diffusion process, the key modules in U-Net were systematically improved, including the following steps: Maintaining the DDPM forward diffusion process: from real samples Initially, scheduling is performed according to the pre-designed noise variance. Gradually inject noise into the data to form a... arrive The Markov chain. Specifically, the first... The forward transition probability of the step can be expressed as:

[0040] in It is the identity matrix. Controlling the noise amplitude. By iteratively applying the above process, the first step can be derived. Step given the original sample The edge distribution is as follows:

[0041] Here is defined , Therefore, at any time step samples It can be directly from and standard Gaussian noise get:

[0042] along with , , It approximately follows a standard normal distribution, meaning that the forward diffusion process gradually maps the data into pure noise; Maintain the DDPM reverse denoising process: DDPM constructs parameterized conditional probability distributions Let's learn the inverse process of the noise injection described above. This distribution is usually also assumed to be a Gaussian distribution, i.e.

[0043] In practice, covariance Fixed to a known value (such as with) (related diagonal matrix), neural network predicted mean Alternatively, the network can be made to directly output noise terms. To train this model, a loss function needs to be constructed such that... The goal is to match the conditional distribution of the forward process as closely as possible. A common approach is to derive the lower bound of the maximum likelihood (ELBO) using variational inference, which can be simplified to a mean squared error objective for noisy predictions: Let According to the forward process and real noise Generation, Network Prediction The training objective is to minimize This goal guides neural networks to perform tasks on known noisy samples. and time step At the same time, accurately estimate the added noise. This implicitly determines the mean of the reverse diffusion process. Through this training, the model can gradually learn how to remove noise at different noise levels and ultimately master the data distribution structure.

[0044] After training is complete, DDPM generates new samples through iterative sampling. Specifically, it samples an initial noise vector from a standard Gaussian distribution. Then, sequentially from Generate using formula :

[0045] At each step, based on the current sample Normally sampled from the learned network outputs (mean and variance) to obtain samples with lower noise levels. After several iterations, the final generated sample is obtained. The entire process is formally equivalent to applying the posterior distribution. Through progressive Markov chain sampling, images with the same distribution as the training data are gradually recovered from random noise. In this way, DDPM achieves a mapping from simple noise to complex data distribution through the construction of inverse denoising.

[0046] In the embodiments of the present invention, a convolutional network with a U-net structure is used as the generator. In view of the U-net network’s insufficient ability to model the multimodal spatiotemporal features, its difficulty in fully capturing the high-dimensional interaction relationship between the three modes of vibration, acoustic emission and temperature signals, and the limited receptive field of ordinary convolutional operations in the original U-Net, which limits the ability to model long-range dependent features, the SE attention mechanism is introduced to enhance the DDPM structure. S41: The intermediate layer introduces the Squeeze-and-Excitation Block (SE) attention mechanism: The SE module can capture the non-linear dependencies between channels through global average pooling and learn the recalibrated weights between channels through two fully connected layers, effectively improving the model's ability to focus on important feature channels. This module first processes the input feature map... Perform global average pooling in the spatial dimension:

[0047] Subsequently, a bottleneck structure is constructed using a fully connected layer, and the importance weights of each channel are learned through nonlinear transformation. ,in Indicates ReLU activation. This represents the Sigmoid activation function. Finally, the original feature map is reconstructed using weighted methods:

[0048] S42: Dilated Convolution Expands the Receptive Field: Dilated convolutions with a dilation rate of 2 are introduced into the convolutional operations of the intermediate layers of U-Net. This expands the receptive field without increasing the number of parameters, thereby enhancing the ability to model cross-regional features. It can be viewed as inserting holes into the standard convolutional kernel, and its calculation form is as follows:

[0049] in This represents the dilation rate, a structure that allows for the aggregation of a wider range of contextual information while keeping computational complexity under control. S43: Introducing Residual Connections to Enhance Stability: To prevent information degradation or gradient vanishing problems during deep feature processing, lightweight residual connections are introduced after the SE module. These connections map the original feature map to the same number of channels through convolution and are then added element-wise to the SE output.

[0050] S44: Self-attention mechanism introduced in upsampling and downsampling paths: AttnDownBlock2D and AttnUpBlock2D modules are introduced into the downsampling and upsampling blocks of U-Net, respectively. This module integrates a multi-head self-attention mechanism to model the global dependency relationship between different regions in the spatial dimension and focus on abnormal feature regions.

[0051] In one embodiment of the present invention, as described in the appendix to the specification... Figure 1 As shown, in step S50, the fused data is input into the improved MFE-DDPM model for training: the noise evolution law of the fused data during the diffusion process is modeled, and the weight allocation of different feature channels is adaptively adjusted to enhance the expressive power of key feature information and the information interaction ability between features, thereby achieving more accurate prediction of noise components. During the anti-diffusion process, noise interference is gradually removed based on the predicted noise information, and finally a reconstructed image corresponding to the input fused data is generated to obtain higher quality reconstruction results. The root mean square loss function for predicted noise and the Adam algorithm are used during training.

[0052] In one embodiment of the present invention, as described in the appendix to the specification... Figure 1 As shown, in step S60, the reconstruction error calculation, thereby achieving effective discrimination of abnormal samples, includes the following steps: S61: PSNR is an objective metric widely used in image and video quality evaluation to measure the similarity between a reconstructed image and an input image. PSNR is calculated based on Mean Squared Error (MSE) and is usually expressed in decibels (dB). The mathematical definition of PSNR is...

[0053] in, The original input image, For the corresponding reconstructed image, Indicates the maximum value of a pixel. Indicates mean square error; A higher PSNR value generally indicates better image reconstruction quality and lower noise or distortion. Although PSNR cannot fully reflect the subjective quality of human visual perception, it is still used as one of the benchmark indicators for evaluating the performance of image processing algorithms due to its ease of calculation and clear physical meaning. It is widely used in tasks such as image compression, denoising, and super-resolution reconstruction. S62: Further Normalization of PSNR Values: In this work, to enhance the comparability of errors between different images, PSNR values ​​are further normalized. Specifically, the PSNR value of each sample is compared with the maximum PSNR value in the training set. Perform ratio calculations and define the normalized reconstruction error in the following form: Thus, the reconstruction error value Larger values ​​indicate poorer reconstruction quality; S63: For all error values Calculate its mean with standard deviation The discrimination threshold is set according to the normal distribution probability and statistics theory. for: .

[0054] In one embodiment of the present invention, as described in the appendix to the specification... Figure 1 As shown, in step S70, a specific example of performing diagnostic performance testing on the trained model is as follows: The tests utilized the publicly available "Dataset of Vibration and Acoustic Fault Signals of Rolling Bearings under Constant Load and Speed" from the University of Ottawa, Canada. The test setup consisted of a single-phase induction motor mounted on a vibration-resistant base. The motor shaft was connected to an SKF E22206 spherical roller bearing via a bushing to withstand a constant radial load of 400 N applied by a cantilever beam. The motor operated at a constant speed of 1750 RPM. To accelerate the bearing degradation process, the drive-end bearings (NSK 6203ZZ for the first five experiments, and FAFNIR 203KD for subsequent experiments) were degreased before each test and replaced after each experiment to ensure the repeatability of the failure process and data consistency. This system integrates six sensors for multi-source data acquisition, including: an accelerometer (PCB623C01), a microphone (PCB 130F20), a force sensor (OMEGA LCM302), a Hall sensor (OMEGA OMDC-MPU-A), and two thermocouples (OMEGA KTSS-HH). The accelerometer is directly magnetically mounted inside the motor housing near the bearing being measured to minimize signal noise and avoid electromagnetic interference. The microphone is mounted 2 cm from the drive-end bearing and is independent of the motor body support. One thermocouple is mounted on the outer ring surface of the drive-end bearing to monitor the actual operating temperature of the bearing, while the other thermocouple is placed in a laboratory environment as a room temperature reference. The temperature difference between these two thermocouples reflects the changes in the bearing's thermal characteristics under operating conditions. The data acquisition system uses a NI USB-6212 data acquisition card, connected to the sensors via a PCB 482C signal conditioner, to simultaneously acquire multi-dimensional information such as voltage, vibration acceleration, sound pressure, load, rotational speed, and temperature. A total of 20 bearings were tested, covering five different failure modes (healthy, inner ring failure, outer ring failure, rolling element failure, and cage failure). Each failure type contained five independent samples, and Table 1 lists the five bearing states. During data acquisition, except for rolling element failure, all other failures were tested under a constant load of 400 N. Rolling element failure was accelerated under no-load conditions to promote natural failure. Each raw data point was sampled for 10 seconds at a sampling rate of 42 kHz, acquiring a total of 420,000 data points to ensure the capture of high-frequency mechanical fault characteristics. Table 1. Examples of five bearing conditions

[0055] 80% of the bearing health data was used for model training, and the remaining 20% ​​was used as components of the normal samples in the test set. To effectively evaluate the anomaly detection capability, the top 20% of the data from each type of fault data was extracted as anomaly samples in the test set. Specifically, inner ring fault data was selected from bearing 1, outer ring fault data from bearing 7, rolling element faults from bearing 14, and cage faults based on data from bearing 16, to cover four typical bearing fault types.

[0056] To verify the effectiveness of the proposed method in terms of fusion strategy and model improvement, comparative models include: MFE-DDPM (MFE-DDPM-vib) which uses only vibration signals, a model that uses only sound signals (MFE-DDPM-aco), a model based solely on temperature signals (MFE-DDPM-tem), and a detection model built under the condition of fusing multimodal data but without introducing an attention mechanism (DDPM-NoAtten). These four models represent the baseline performance of single-modal modeling capability and fusion without attention mechanism guidance, respectively.

[0057] Experiment 1: Accuracy comparison of this method with four other model detection methods (see attached instruction manual). Figure 3 The confusion matrices of MFE-DDPM, MFE-DDPM-vib, MFE-DDPM-aco, MFE-DDPM-tem, and DDPM-NoAtten after 2000 iterations in each state are shown. It is readily apparent from the figure that, compared to detection methods without an attention mechanism, the proposed MFE-DDPM exhibits stronger discriminative ability across all fault categories, achieving a maximum accuracy of 100%, fully demonstrating the crucial role of the attention mechanism in the detection process. Among the single-modal detection methods, sound-based and temperature-based methods exhibit poor detection capabilities with high false positive rates, reaching up to 70%. Notably, the vibration-based detection method demonstrates higher classification ability, achieving an accuracy of 96% for outer ring fault detection, indicating that the vibration signal contributes the most to the fused signal. However, overall, the detection performance of the fused signal surpasses all single-modal detection methods, strongly demonstrating the effectiveness and necessity of information fusion. Specifically, the proposed fusion strategy combines the advantages of multiple measurements to provide more representative fault information for detecting different fault states.

[0058] Experiment 2: Further quantifying the model's recognition performance under various fault conditions This experiment produced a bar chart showing the accuracy of each method after 10 repeated runs on the test set (as per the instruction manual). Figure 4As shown in the figure, compared with the other four methods, the proposed fusion MFE-DDPM maintained the best diagnostic performance in 10 repeated runs, with a maximum recognition accuracy of 100.00%, which is significantly higher than other methods based on single measurement. The satisfactory results strongly validate that the proposed fusion strategy can effectively detect bearing anomalies. The average accuracy in 10 independent experiments is shown in Table 2. The results show that MFE-DDPM, by jointly optimizing multimodal feature representation and adaptive attention allocation strategy, significantly improves the maximum, minimum, and average diagnostic accuracy, increasing it by 8.1 percentage points compared to the best single-modal scheme, and significantly enhances the model's robustness, fully validating the effectiveness of multimodal signal fusion. Although the DDPM-NoAtten model, which incorporates multimodal information, fuses multi-source data, its performance is lower than some single-modal models (Avg-acc=64.9%) due to the lack of an effective feature weighting mechanism, highlighting the importance of the attention mechanism. Table 2 Diagnostic results of different methods Methods Max-acc Min-acc Avg-acc MFE-DDPM 100% 98% 98.6% MFE-DDPM-vib 93% 88% 90.5% MFE-DDPM-aco 81% 72% 78.4% MFE-DDPM-tem 72% 67% 69.3% DDPM-NoAtten 66% 63% 64.9% Experiment 3: To further verify the performance stability of the proposed method in the task of detecting anomalies in rolling bearings. This experiment compared and analyzed the F1 score performance of this method with four control methods under four fault conditions. Each method was repeated 20 times under the same experimental configuration, and the corresponding F1 score box plots were plotted. The results are shown in the appendix to the instruction manual. Figure 5 As shown; As clearly observed from the figures, the proposed method achieves the best overall F1 score performance with a small interquartile range and well-converged extreme value distribution range, indicating that it not only excels in accuracy but also possesses stronger robustness and consistency. In contrast, sound-based and temperature-based detection methods perform the worst across all fault types, exhibiting problems such as a large number of low-scoring samples, demonstrating a certain degree of insufficient perception capability. While the vibration-based detection method performs well under outer race, rolling element, and cage fault conditions, it suffers from high sample dispersion and a large number of occasional low-scoring samples under inner race fault conditions, dragging down the overall model performance. Notably, although the multimodal fusion detection method without attention mechanism enhancement shows improvement in some scenarios, its overall F1 score distribution is still inferior to the model with attention mechanism, further illustrating the crucial role of attention mechanism in multi-source feature selection and weighting. These results demonstrate that the proposed method possesses high performance stability in anomaly detection tasks under complex working conditions, and the effectiveness of its multimodal fusion and feature selection mechanism structural design is fully proven.

[0059] In summary, both quantitative and visualization analysis results demonstrate that the multimodal information fusion strategy significantly enhances the model's ability to perceive multi-source fault information, while the introduction of the attention mechanism further improves the targeting and expression efficiency of feature selection, thereby achieving higher recognition accuracy and stronger robustness in the rolling bearing anomaly detection task.

[0060] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any way. Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make some modifications or alterations to the above-disclosed technical content to create equivalent embodiments without departing from the scope of the present invention. Any simple modifications, equivalent changes, and alterations made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the scope of the present invention.

Claims

1. A method for detecting anomalies in rolling bearings based on multimodal signal fusion and a denoising diffusion probability model, characterized in that, Includes the following steps: S10: Acquire sample data signals through vibration sensors, sound sensors, and temperature sensors; S20: Construct corresponding image representation models for the non-stationary characteristics of vibration signals, the transient pulse characteristics of acoustic emission signals, and the thermodynamic degradation law of temperature fields, respectively. S30: Assign the three converted signals to the three color channels R (red), G (green), and B (blue) respectively, and realize cross-modal fusion at the data level in the RGB space; S40: Constructing a DDPM structure enhanced by SE attention mechanism: While keeping the basic framework of the original DDPM diffusion and anti-diffusion process unchanged, the key modules in U-Net were systematically improved to enhance the model's representation ability and information interaction ability. S50: The fused data is input into the improved MFE-DDPM model for training. The model is trained to perceive key feature regions and model multi-scale features. The input fused signal generates a reconstructed image after passing through the model. S60: An anomaly metric function is constructed by comparing the peak signal-to-noise ratio (PSNR) of the original input image and the reconstructed image of all sample data. This function is used to calculate the reconstruction error and thus achieve effective discrimination of anomalous samples. S70: Perform diagnostic performance testing on the trained model.

2. The rolling bearing anomaly detection method based on multimodal signal fusion and denoising diffusion probability model according to claim 1, characterized in that, In step S20, constructing the corresponding image representation models includes the following steps: S21: Vibration data conversion: The one-dimensional time-series signal of the vibration signal is converted into a two-dimensional time-frequency image through the Adaptive Wavelet Transform (AWT); S22: Acoustic data encoding: Encoding the spectrum of acoustic data into the spectral Markov transition field (SMTF); S23: Temperature data conversion: Convert the temperature signal into a grayscale image through two-dimensional expansion.

3. The rolling bearing anomaly detection method based on multimodal signal fusion and denoising diffusion probability model according to claim 1, characterized in that, In step S21, the one-dimensional time-series signal of the vibration signal is converted into a two-dimensional time-frequency image, which specifically includes the following steps: S211: Divide the original signal into multiple fixed-length time segments, and process each data segment... AWT is used for multi-level decomposition, with the wavelet basis function Daubechies4 used and a decomposition level of 4. This decomposition process decomposes the signal into coefficient subsets of multiple frequency bands. in, The number of decomposition layers, For the first Wavelet coefficients for each frequency band; S212: The energy values ​​are normalized and copied along the vertical direction to form a two-dimensional frequency band energy matrix, which is finally mapped to a grayscale image, represented as: in, This represents the normalized frequency band energy, with the number of rows corresponding to the image height and the number of columns representing the frequency bands.

4. The rolling bearing anomaly detection method based on multimodal signal fusion and denoising diffusion probability model according to claim 1, characterized in that, In step S22, the specific steps for encoding the acoustic data into a spectral Markov transfer field are as follows: S221: Input signal preprocessing and normalization: The original sound signal is a one-dimensional discrete time series. First, perform min-max normalization to compress it to... Interval: S222: State Discretization: Normalizing the sequence... Mapping to a set of discrete states K represents dividing a continuous signal into K levels of states. The discretization operation can be formalized as: Discrete state ; S223: Construct the state transition matrix: Traverse the entire state sequence Statistically count the frequency of first-order Markov state transitions and construct the transition matrix. The definition is as follows: in, Here is the characteristic function; to transform it into a probability matrix, row-by-row normalization is performed: in To prevent tiny constants with a denominator of zero; S224: Constructing the spectral Markov transfer field image: transfer matrix Visualized as a two-dimensional grayscale image, where the intensity of each pixel represents the state. to state To improve image compatibility and network adaptability, the transition probability is adjusted using image interpolation to a fixed size. And save it as a grayscale image: 。 5. The rolling bearing anomaly detection method based on multimodal signal fusion and denoising diffusion probability model according to claim 1, characterized in that, In step S23, converting the temperature signal into a grayscale image through two-dimensional expansion specifically includes the following steps: compressing the one-dimensional temperature signal to a standard normalized range, and then performing row expansion in two-dimensional space by copying its amplitude vector, thereby generating a two-dimensional image representation: In the formula, This represents the original temperature signal.

6. The rolling bearing anomaly detection method based on multimodal signal fusion and denoising diffusion probability model according to claim 1, characterized in that, In step S40, the key modules in U-Net were systematically improved, including the following steps: S41: The intermediate layer introduces the SE attention mechanism (Squeeze-and-Excitation Block): The SE module can capture the non-linear dependencies between channels through global average pooling and learn the recalibrated weights between channels through two fully connected layers, effectively improving the model's ability to focus on important feature channels. This module first processes the input feature map... Perform global average pooling in the spatial dimension: Subsequently, a bottleneck structure is constructed using a fully connected layer, and the importance weights of each channel are learned through nonlinear transformation. ,in Indicates ReLU activation. This represents the Sigmoid activation function; finally, the original feature map is reconstructed using weighted methods. S42: Dilated Convolution Expands the Receptive Field: Dilated convolutions with a dilation rate of 2 are introduced into the convolutional operations of the intermediate layers of U-Net. This expands the receptive field without increasing the number of parameters, thereby enhancing the ability to model cross-regional features. It can be viewed as inserting holes into the standard convolutional kernel, and its calculation form is as follows: ,in This represents the dilation rate, a structure that allows for the aggregation of a wider range of contextual information while keeping computational complexity under control. S43: Introducing Residual Connections to Enhance Stability: To prevent information degradation or gradient vanishing problems during deep feature processing, lightweight residual connections are introduced after the SE module. These connections map the original feature map to the same number of channels through convolution and are then added element-wise to the SE output. S44: Self-attention mechanism introduced in upsampling and downsampling paths: AttnDownBlock2D and AttnUpBlock2D modules are introduced into the downsampling and upsampling blocks of U-Net, respectively. This module integrates a multi-head self-attention mechanism to model the global dependency relationship between different regions in the spatial dimension and focus on abnormal feature regions.

7. The rolling bearing anomaly detection method based on multimodal signal fusion and denoising diffusion probability model according to claim 1, characterized in that, In step S50, the process of inputting the fused data into the improved MFE-DDPM model for training includes the following: Modeling the noise evolution pattern of fused data during the diffusion process, and by adaptively adjusting the weight allocation of different feature channels, the expressive power of key feature information and the information interaction ability between features are enhanced, thereby achieving more accurate prediction of noise components. During the anti-diffusion process, noise interference is gradually removed based on the predicted noise information, and finally a reconstructed image corresponding to the input fused data is generated to obtain a higher quality reconstruction result. The root mean square loss function for predicted noise and the Adam algorithm are used during the training process.

8. The rolling bearing anomaly detection method based on multimodal signal fusion and denoising diffusion probability model according to claim 1, characterized in that, In step S60, the reconstruction error calculation, thereby achieving effective discrimination of abnormal samples, includes the following steps: S61: The mathematical definition of PSNR is: in, The original input image, For the corresponding reconstructed image, Indicates the maximum value of a pixel. Indicates mean square error; S62: PSNR values ​​are further normalized, using... Define the normalized reconstruction error; S63: For all error values Calculate its mean with standard deviation The discrimination threshold is set according to the normal distribution probability and statistics theory. for: .

Images