Roller fault monitoring method and system based on fault tree and multi-modal fusion

Abstract

The application discloses a kind of based on fault tree and multimodal fusion's roller fault monitoring method and system.Method includes: constructing roller fault tree model, carries out structural importance analysis to bottom event, determines static weight;Establish the feature sensitivity mapping of bottom event and multimodal;Control intelligent mobile device to collect visual, sound, vibration, temperature multimodal data;Wavelet transform is used to filter out the interference of device itself movement to sound and vibration data;Visual data target detection positioning roller area, and use positioning result to extract corresponding area data from temperature data;Each modality data is respectively carried out single-mode feature extraction and fault probability decision, obtains dynamic fault probability, and then based on static weight, feature sensitivity and dynamic fault probability carries out weighted decision fusion, calculates comprehensive risk index, assesses roller fault state.The application improves diagnostic accuracy, realizes intelligent, high safety roller fault monitoring.

Description

Technical Field

[0001] This invention belongs to the field of fault diagnosis and health management technology for belt conveyor equipment, specifically relating to a fault monitoring method and system for idler rollers based on fault tree and multimodal fusion. Background Technology

[0002] Belt conveyors are a primary means of material transport in industries such as coal, power, building materials, chemicals, machinery, and light industry, characterized by large conveying capacity and long conveying distances. Idler rollers are a key component of belt conveyors, their function being to support the conveyor belt and reduce the conveyor's running resistance. However, due to the long-term operation of belt conveyors at high speeds and under heavy loads, idler roller damage is frequent, thus affecting industrial production. In more than half of current transport disruptions, conveyor belt failure caused by idler roller malfunctions leads to unexpected transport interruptions.

[0003] Traditional belt conveyor inspections are primarily conducted manually, with staff checking all idlers along the conveyor belt by tapping, listening, and visually inspecting, or using specialized instruments for sampling. However, for long-distance conveyor belts, this method often fails to effectively detect most idler damage and is prone to misjudgments and omissions.

[0004] Existing intelligent inspection methods generally use single-modal data to identify faults. For example, patent CN202511287609 uses a distributed optical fiber sensing system (DAS) to collect vibration signals of idlers along the entire line, and uses wavelet decomposition to extract signal features and build a machine learning model to identify idler faults. However, this method requires dense fiber optic deployment for long-distance conveyor corridors, leading to a sharp increase in cost, and single-modal signals are easily affected by environmental interference such as conveyor belt vibration and personnel movement. In addition, multi-modal sensors are also used in the field of belt conveyor monitoring to monitor equipment faults, such as patent CN202211187479, which uses audio sensors, infrared temperature cameras, and smart cameras to collect multi-modal data of belts and idlers, and uses artificial intelligence algorithms for identification. However, this method can only monitor the entire belt conveyor in a certain area, and the monitoring accuracy is difficult to reduce to each idler, and it is not suitable for long-distance conveyor corridors. Summary of the Invention

[0005] To overcome the shortcomings of existing technologies, this invention provides a method and system for monitoring idler roller faults based on fault tree and multimodal fusion.

[0006] The technical solution adopted in this invention is as follows:

[0007] A method for monitoring idler roller faults based on fault tree and multimodal fusion includes the following steps:

[0008] S1. Construct a fault tree model for the idler roller, the fault tree model including top events, intermediate events and bottom events, perform structural importance analysis on the bottom events, and determine the static weights corresponding to each bottom event;

[0009] S2. Establish the feature sensitivity mapping relationship between the aforementioned bottom event and multiple monitoring modalities;

[0010] S3. Control the intelligent mobile device to move along the track and collect raw data of multiple monitoring modes during the operation of the idler roller, including visual mode, sound mode, vibration mode and temperature mode;

[0011] S4. For the collected sound mode and vibration mode data, wavelet transform is used to filter out the characteristic frequencies generated by the movement of the intelligent mobile device itself; for the collected sound mode data, spectral subtraction is used to filter out environmental noise, that is, environmental background noise is collected and spectral estimation is performed, and the environmental spectrum is subtracted from the original sound to obtain the target signal;

[0012] S5. Perform target detection on the acquired visual modal data, identify and locate the idler roller area in the image; and use the location result to extract the idler roller temperature data of the corresponding area from the temperature modal data; calculate the difference between the idler roller temperature and the acquired ambient temperature to obtain the relative temperature difference of the idler roller, thus eliminating the interference of ambient temperature.

[0013] S6. For the sound data and vibration data processed in step S4, and the visual data and temperature data processed in step S5, perform single-modal feature extraction and fault probability decision-making respectively to obtain dynamic fault probability; based on the static weight, feature sensitivity and dynamic fault probability, perform weighted decision fusion to calculate comprehensive risk index to assess the fault status of the idler roller.

[0014] Furthermore, the construction of the fault tree model for the idler roller specifically includes:

[0015] The idler roller failure is taken as the top event and decomposed layer by layer into multiple intermediate events, including cylinder failure, bearing failure, central shaft failure and seal failure. Each intermediate event is further subdivided into multiple bottom events, including cylinder skin wear, outer ring wear, inner ring fracture, rolling element damage, central shaft fracture, central shaft deformation, journal wear and seal cracking.

[0016] Furthermore, determining the static weights corresponding to each bottom event specifically includes:

[0017] Based on expert experience and historical fault statistics, the structural importance coefficients of each bottom event are assigned values ​​and normalized to obtain the static weights of each bottom event, as shown in the formula:

[0018] ,

[0019] in, , is the number of the base event, with a value range of 1, 2, ..., n, where n is the total number of base events; , They are respectively the i-th and the i-th The structural importance coefficient of each underlying event; Let be the static weight of the i-th bottom event, and satisfy . . It is the sum of the structural importance coefficients of all bottom events, used for normalization.

[0020] Furthermore, step S3 specifically includes: installing a travel track along the direction of the idler belt conveyor, wherein the intelligent mobile device is movably mounted on the track;

[0021] The intelligent mobile device is equipped with at least four types of sensors, including cameras, thermal imagers, microphones, and vibration meters.

[0022] The vibration meter captures the vibration characteristics of the idler roller through the vibration transmission of the roller frame and the track; the thermal imager is used to measure the temperature of the idler roller and the ambient temperature; the camera is used to collect three-dimensional contour data and two-dimensional image data of the idler roller surface; and the microphone is used to collect the sound data of the idler roller operation.

[0023] Furthermore, the step of using wavelet transform to filter out the characteristic frequencies generated by the motion of the intelligent mobile device itself specifically includes:

[0024] Select wavelet basis functions and determine the number of decomposition levels. Perform multi-scale wavelet decomposition on the original signal sequences of the acquired sound modes and / or vibration modes to obtain approximation coefficients and detail coefficients for different frequency bands.

[0025] Based on the motion characteristic frequency of the intelligent mobile device, a wavelet coefficient layer containing the motion characteristic frequency is identified;

[0026] Suppression processing is applied to the identified wavelet coefficient layers;

[0027] The wavelet coefficient layer after suppression is reconstructed by inverse wavelet transform to obtain the target signal after filtering out its own motion interference.

[0028] Furthermore, the wavelet basis function is a Daubechies series wavelet, and the number of decomposition levels N is based on the signal sampling frequency. And the preset motion characteristic frequency of the smart mobile device itself. Confirmed, the formula is as follows:

[0029] ;

[0030] The suppression process for the identified wavelet coefficient layers specifically includes:

[0031] If the motion characteristic frequency of the intelligent mobile device is mainly concentrated in the low frequency band, then the corresponding approximation coefficient is set to zero.

[0032] If the motion characteristic frequency distribution of the intelligent mobile device is in the high-frequency detail layer, then a soft thresholding function or a hard thresholding function is used to quantize the detail coefficients of the corresponding layer.

[0033] Furthermore, step S5 includes:

[0034] Construct and train a YOLOv5-based object detection network model;

[0035] The collected visual modal image data is input into the trained target detection network model, and the bounding box coordinate information and bounding box width and height information of the roller target are output.

[0036] Based on the bounding box coordinates and width / height information, the region of interest image is cropped from the original image and used as input for subsequent single-modal feature extraction.

[0037] Furthermore, the single-modal feature extraction and fault probability decision specifically include:

[0038] For the sound and vibration data processed in step S4, time-domain waveforms, spectrograms, envelope spectra, and time-frequency diagrams are generated respectively. The time-domain waveforms and envelope spectra are input into a one-dimensional convolutional neural network to extract time-domain impact features and envelope spectrum rotational speed-related features. The spectrograms and time-frequency diagrams are input into a two-dimensional convolutional neural network to extract frequency-domain high-frequency energy features and time-frequency-domain periodic impact features. The output features of the two networks are fused to obtain the dynamic fault probabilities of the sound and vibration modes.

[0039] The visual and temperature data processed in step S5 are input into a two-dimensional convolutional neural network to extract abnormal features and identify fault modes, thereby obtaining the dynamic fault probabilities of visual and temperature modes.

[0040] Furthermore, the step of calculating a comprehensive risk index based on the weighted decision fusion of the static weights, feature sensitivity, and dynamic failure probability specifically includes:

[0041] For the i-th type of low-event failure mode, the comprehensive risk index The calculation formula is:

[0042] ,

[0043] in, is the base event number, and j is the modal number; Let i be the sensitivity of the i-th base event to the j-th modality of features. Let be the dynamic failure probability of the i-th bottom event obtained based on the j-th modal data.

[0044] The present invention also provides a roller fault monitoring system based on fault tree and multimodal fusion, comprising:

[0045] Intelligent mobile devices for movement along tracks;

[0046] A multimodal sensor kit, mounted on the intelligent mobile device, includes a vision sensor, a sound sensor, a vibration sensor, and a temperature sensor, which are used to collect visual modal data, sound modal data, vibration modal data, and temperature modal data during the operation of the idler roller, respectively.

[0047] One or more processors;

[0048] Memory, which stores computer programs;

[0049] When the computer program is executed by the one or more processors, it implements the steps in the above method.

[0050] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0051] 1. Multi-dimensional perception aligns with the physical mechanism of the fault, resulting in more comprehensive diagnostic results.

[0052] This invention constructs a hierarchical model (S1) for idler roller faults based on fault tree analysis, clarifying the logical relationship between top, intermediate, and bottom events from a physical structure perspective, thus overcoming the limitations of single fault representation. Simultaneously, it employs a four-modal fusion approach (S2, S6) encompassing visual, auditory, vibration, and temperature characteristics to comprehensively capture the performance features of idler roller faults in different physical fields. For example, bearing wear is primarily manifested as abnormal vibration and temperature rise, while drum wear is more often reflected in changes in visual features. Multimodal fusion not only complements the perception blind spots of single sensors but also combines the structural importance and feature sensitivity of faults through a weighted fusion algorithm, making the diagnostic results more consistent with the physical nature of idler roller fault occurrence and improving the accuracy and robustness of fault identification.

[0053] 2. Intelligent devices replace manual inspections, making operation convenient and safe.

[0054] This invention utilizes a smart mobile device equipped with a multimodal sensor to collect data along a track (S3), transforming the traditional cumbersome operation mode of manually holding instruments or installing sensors at fixed points. Inspection personnel no longer need to be in close contact with high-temperature, high-dust, or high-speed conveyor components; they only need to control the smart mobile device to complete data collection. Furthermore, the YOLOv5 target detection algorithm automatically identifies and crops the idler roller area (S5), automating image data processing without the need for manual image selection and cropping. This significantly reduces the operational threshold and labor intensity, while improving operational safety.

[0055] 3. Automated processing significantly improves detection speed and quality.

[0056] This invention incorporates deep learning and signal processing technologies to automate the entire process from data acquisition to fault diagnosis. Specifically, S4 utilizes wavelet transform to effectively filter out vibration noise generated by the trolley's own movement, avoiding interference from invalid data and reducing the time spent manually troubleshooting interference signals; S5 uses a YOLOv5 network to quickly locate the idler roller target; and S6 uses a convolutional neural network to automatically extract features and perform decision fusion. Compared to the inefficient traditional manual inspection method of "seeing, hearing, touching, and measuring," this invention enables high-frequency, all-weather inspection of the idler roller at a constant speed and standard, significantly improving inspection efficiency and achieving early warning of faults. Attached Figure Description

[0057] The accompanying drawings, which are provided to further illustrate this application and form part of this application, illustrate exemplary embodiments of this application and are used to explain this application, but do not constitute an undue limitation of this application.

[0058] Figure 1 This is a flowchart illustrating the method in an embodiment of the present invention.

[0059] Figure 2 This is a schematic diagram of the fault tree structure in an embodiment of the present invention. Detailed Implementation

[0060] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0061] Example 1

[0062] This embodiment provides a method for monitoring idler roller faults based on fault tree and multimodal fusion. Figure 1This is a schematic flowchart of the method of the present invention. Figure 1 As shown, the method includes the following steps:

[0063] S1. Construct a fault tree model for the idler roller, which includes top events, intermediate events, and bottom events. Perform structural importance analysis on the bottom events to determine the static weights corresponding to each bottom event.

[0064] First, a fault tree model is constructed based on the physical structure of the idler roller. Idler roller failure is taken as the top event and decomposed layer by layer into four intermediate events: cylinder failure, bearing failure, central shaft failure, and seal failure. These intermediate events are further subdivided into eight bottom events: cylinder wear, outer ring wear, inner ring fracture, rolling element failure, central shaft fracture, central shaft deformation, journal wear, and seal cracking. Figure 2 The diagram shown is a fault tree structure diagram of this embodiment.

[0065] Then, a structural importance analysis was performed on each bottom event. A structural importance coefficient was assigned to each bottom event using a combination of expert experience and historical failure statistics. In this embodiment, considering that bearings and the central shaft are the core supporting components for the rotation of the idler roller, their failures will directly lead to conveyor shutdown or even belt tearing. Therefore, these two types of critical events are assigned a large structural importance coefficient. In contrast, the cylinder and seals are auxiliary or protective components, and their failures are usually gradual and will not lead to catastrophic consequences in the short term. Therefore, they are assigned a smaller structural importance coefficient. For example, the structural importance coefficient for bearing-related critical events (outer ring wear, inner ring fracture, rolling element failure) is assigned 0.4; the structural importance coefficient for central shaft-related critical events (central shaft fracture, central shaft deformation, journal wear) is assigned 0.3; the structural importance coefficient for cylinder-related critical events (cylinder skin wear) is assigned 0.2; and the structural importance coefficient for seal-related critical events (seal ring cracking) is assigned 0.2. It should be noted that the above values ​​are only examples and can be adjusted according to specific working conditions and statistical data in actual applications.

[0066] Next, the structural importance coefficients are normalized to obtain the static weights of each bottom event. :

[0067] ,

[0068] Where n=8, , The number of the bottom event, The static weights of each bottom event were calculated. satisfy These static weights will be used for weighted decision fusion in step S6.

[0069] S2. Establish the feature sensitivity mapping relationship between the base event and multiple monitoring modalities.

[0070] For each of the eight basic event failure modes, their characteristic sensitivities to four modes—visual, auditory, vibrational, and temperature—were determined. .in, As the bottom event number, These correspond to four modes: visual, auditory, vibration, and temperature. Feature sensitivity represents the significance of abnormal characteristics exhibited by the fault mode in the corresponding mode. Specifically, the feature sensitivity mapping relationship between each underlying event and the multi-source detection mode is determined based on the physical failure mechanism of the idler roller fault, the response characteristics of the monitoring signal, and the external observability of the fault. Let... Indicates the first The bottom event for the first The sensitivity coefficient for a monitoring mode ranges from [0,1]. A higher sensitivity coefficient is assigned when a certain type of fault can produce a direct, significant, and stable characteristic response in a particular monitoring mode. For example, 0.7~0.9; when the fault can only manifest in this mode through indirect effects or later degradation, a moderate value is assigned. For example, 0.4~0.6; when the fault characteristic is difficult to observe in this mode, a lower value is assigned. For example, rolling element failure, inner ring fracture, and outer ring wear are all internal bearing faults, mainly manifested as periodic impact, increased friction, and increased vibration energy, thus exhibiting high sensitivity in sound and vibration modes; central shaft deformation, central shaft fracture, and journal wear mainly cause rotational eccentricity, dynamic imbalance, and abnormal support, thus exhibiting the highest sensitivity in vibration modes, followed by sound modes; cylinder shell wear mainly manifests as degradation of the outer surface morphology and frictional temperature rise, thus exhibiting the highest sensitivity in visual modes, followed by temperature modes; seal cracking leads to loss of lubricating medium and intrusion of external impurities, causing lubrication failure and frictional temperature rise, thus exhibiting high sensitivity in temperature modes.

[0071] S3. Control the intelligent mobile device to move along the track and collect raw data of multiple monitoring modes during the operation of the idler roller. The multiple monitoring modes include visual mode, sound mode, vibration mode and temperature mode.

[0072] A travel track is installed along the direction of the belt conveyor, and this track is connected to the idler frame. An intelligent mobile device (inspection trolley) is movably mounted on the track. The trolley is equipped with four types of sensors: a camera, a thermal imager, a microphone, and a vibration meter. The vibration meter captures the vibration characteristics of the idler rollers through the vibration transmission between the idler frame and the track; the thermal imager simultaneously measures the temperature of the idler rollers and the ambient temperature, calculating the temperature rise of the idler rollers due to friction; the camera acquires three-dimensional contour data and two-dimensional image data of the idler roller surface; and the microphone collects the sound data of the idler rollers' operation.

[0073] The vehicle moves along the track at a constant speed (e.g., 0.3 m / s), and all sensors collect data synchronously. The sampling frequency for sound and vibration signals is set to 10 kHz, the image acquisition frequency is set to 5 frames / second, and the thermal imager sampling frequency is set to 2 frames / second. The collected data is transmitted in real time to the onboard edge computing unit or a remote server.

[0074] S4. Wavelet transform is used to filter out the characteristic frequencies generated by the movement of the intelligent mobile device itself from the collected sound mode and vibration mode data.

[0075] The vehicle's own movements, such as the interaction between the wheels and the track, and the vibration of the motor, introduce interference frequencies that need to be filtered out. The specific process is as follows:

[0076] (1) Select the wavelet basis function as the Daubechies series wavelet (e.g., db4). Based on the signal sampling frequency... and the preset motion characteristic frequency of the car itself Determine the minimum number of decomposition levels. 6.64, rounded up to 7 layers.

[0077] (2) The original vibration and sound signal sequences were decomposed into 7-level wavelet decomposition. The decomposition process includes a low-pass filter (h[k]) and a high-pass filter (g[k]). The original signal (x[n]) was convolved with the low-pass filter (h[k]) and the high-pass filter (g[k]) to obtain the approximation coefficients A[m] and detail coefficients D[m].

[0078] Taking the first-level wavelet decomposition as an example, the original signal x[n] is convolved with the db4 low-pass filter h[k] and the db4 high-pass filter g[k], respectively, and then downsampled by 2 times after convolution to obtain the first-level approximation coefficients A1[m] and the first-level detail coefficients D1[m]. The calculation formula is as follows:

[0079]

[0080]

[0081] Where m is the discrete-time index after downsampling; 2m represents the position of 2x downsampling.

[0082] (3) Since the characteristic frequency of the car's motion (100 Hz) is mainly distributed in the low frequency band, the approximate coefficient is identified as the target coefficient layer containing the interference.

[0083] (4) Set all approximation coefficients to zero to achieve interference suppression.

[0084] Furthermore, if the vehicle's own motion characteristic frequencies are distributed in the high-frequency detail layer, then a soft-thresholding function or a hard-thresholding function is used to threshold the target detail coefficient layer D7 containing these motion interference frequencies. The threshold λ7 is adaptively determined based on the noise level of the target detail coefficient layer, specifically as follows:

[0085]

[0086] Where N7 is the length of the detail coefficients at the 7th layer, and σ7 is the estimated noise standard deviation corresponding to the detail coefficients at the 7th layer. The noise standard deviation can be estimated from the median absolute deviation of the detail coefficients.

[0087]

[0088] in, 0.6745 is the median of D7, and 0.6745 is the inverse function value of the standard normal distribution at the 0.75 quantile.

[0089] The processing rule of the hard threshold function is as follows: when the absolute value of the detail coefficient is greater than or equal to the set threshold, the original value of the coefficient is retained; when the absolute value of the detail coefficient is less than the set threshold, the coefficient is set to zero.

[0090] The processing rule of the soft threshold function is as follows: when the absolute value of the detail coefficient is greater than or equal to the set threshold, the detail coefficient is shrunk towards zero to reduce the threshold size; when the absolute value of the detail coefficient is less than the set threshold, the coefficient is set to zero.

[0091] (5) The processed coefficients are reconstructed by wavelet inverse transform to obtain the target vibration and sound signals after filtering out their own motion interference.

[0092] S5. Perform target detection on the acquired visual modal data, identify and locate the idler roller area in the image; and use the location results to extract the idler roller temperature data of the corresponding area from the temperature modal data.

[0093] This step first involves building and training a YOLOv5-based object detection network model to automatically identify the position of idlers from visible light images.

[0094] A total of 2000 visible light images, including normal idler rollers and images showing different fault modes, were collected as the training dataset. The idler roller targets in the images were manually labeled using an image annotation tool to generate bounding box labels. The labeled dataset was then divided into training, validation, and test sets in an 8:1:1 ratio. During model training, the input image resolution was uniformly adjusted to 640×640 pixels, the learning rate was set to 0.01, momentum to 0.937, weight decay to 0.0005, batch size to 16, and the SGD optimizer was used. Training was conducted for 300 epochs with an early stopping mechanism (the model stopped when the mAP on the validation set did not improve for 10 consecutive epochs). After training, the model achieved an mAP of over 0.95 on the test set. It should be noted that the above parameters are only examples, and those skilled in the art can adjust them according to actual application scenarios.

[0095] During actual inspection, the current visible light image captured by the camera is input into the trained YOLOv5 model, and the model outputs the bounding box coordinates and width and height information of the roller target.

[0096] Based on the bounding box coordinates and the bounding box width and height information, a region of interest (ROI) image is cropped from the original visible light image. This ROI image contains only the roller surface, removing background interference, and is used for subsequent visual modality feature extraction.

[0097] Simultaneously, using the same bounding box coordinates, the corresponding region's idler roller temperature data matrix is ​​cropped from the thermal imaging image (temperature modal data) acquired by the thermal imager, and the cropped idler roller temperature data matrix is ​​then subjected to ambient temperature compensation processing. Specifically, the ambient temperature is subtracted from the temperature value of each pixel in the idler roller temperature data matrix to obtain the ambient-compensated relative temperature matrix of the idler roller, which serves as the input for subsequent temperature modal analysis.

[0098] S6. For the sound data and vibration data processed in step S4, and the visual data and temperature data processed in step S5, perform single-modal feature extraction and fault probability decision-making respectively to obtain dynamic fault probability; based on the static weight, feature sensitivity and dynamic fault probability, perform weighted decision fusion to calculate comprehensive risk index to assess the fault status of the idler roller.

[0099] (I) Single-modal feature extraction and fault probability decision

[0100] (1) Sound and vibration modes

[0101] The sound and vibration signals processed by S4 are each segmented into fixed-length segments (e.g., 1 second). For each segment, a time-domain waveform, spectrogram, envelope spectrum, and time-frequency graph are generated. The time-domain waveform and envelope spectrum serve as inputs to a one-dimensional convolutional neural network, while the spectrogram and time-frequency graph serve as inputs to a two-dimensional convolutional neural network.

[0102] The input to a one-dimensional convolutional neural network is a one-dimensional sequence X1∈R^(C×N), where C represents the number of input channels and N represents the number of sampling points for each signal segment. When the input includes a time-domain waveform and an envelope spectrum, C can be 2. This one-dimensional convolutional neural network can adopt the following structure: a first convolutional layer Conv1D with a kernel size of 7 and 32 output channels, followed by a batch normalization layer, a ReLU activation function, and a max-pooling layer; a second convolutional layer Conv1D with a kernel size of 5 and 64 output channels, followed by a batch normalization layer, a ReLU activation function, and a max-pooling layer; a third convolutional layer Conv1D with a kernel size of 3 and 128 output channels, followed by a batch normalization layer and a ReLU activation function; finally, a global average pooling layer is applied to obtain a one-dimensional feature vector f1. This network is used to extract time-domain impact features and rotational speed-related features in the envelope spectrum.

[0103] The input to a two-dimensional convolutional neural network (2D convolutional neural network) is a two-dimensional image X2∈R^(H×W×C), where H and W represent the image height and width, respectively, and C represents the number of image channels. When the input includes a spectrogram and a time-frequency graph, both can be used as dual-channel image inputs, or they can be input separately into a shared-structure 2D convolutional network and then concatenated for feature extraction. This 2D convolutional neural network can adopt a ResNet50 structure, including an initial convolutional layer, a batch normalization layer, a ReLU activation function, a max-pooling layer, and multiple residual convolutional modules. Finally, a global average pooling layer outputs a two-dimensional feature vector f2. This network is used to extract high-frequency energy features in the frequency domain and periodic impact features in the time-frequency domain.

[0104] The feature vector f1 output by the one-dimensional convolutional neural network is concatenated with the feature vector f2 output by the two-dimensional convolutional neural network to obtain a fused feature vector f=[f1,f2]. This fused feature vector is then sequentially input into a fully connected layer, a Dropout layer, and a Softmax classifier to output the dynamic failure probabilities of each basic event failure mode. .

[0105] (2) Feature extraction and fault probability output of visual modality

[0106] The visible light ROI image obtained by cropping using S5 is used as the visual modality input. The input image can be uniformly scaled to 126×126 pixels. The visual model can use a pre-trained VGG16 or ResNet50 network for transfer learning. Taking VGG16 as an example, it includes 13 convolutional layers, 5 max pooling layers, and a fully connected classification layer. During transfer learning, the pre-trained convolutional layers can be retained as the basic feature extractor, and the final classification layer can be replaced so that its output dimension is consistent with the number of basic event failure modes, i.e., outputting the probability of 8 types of failures. .

[0107] (3) Feature extraction and fault probability output of temperature modes

[0108] The thermal imaging ROI obtained by S5 cropping or the relative temperature matrix after ambient temperature compensation is used as the temperature mode input. The temperature model can use the same two-dimensional convolutional neural network structure as the visual model, such as VGG16 or ResNet50.

[0109] It should be noted that the above-mentioned one-dimensional convolutional neural network, two-dimensional convolutional neural network, VGG16, ResNet50, optimizer type, learning rate, batch size and training epochs are all exemplary implementations. Those skilled in the art can adjust them according to the sampling frequency, image resolution, number of samples, number of fault categories and hardware computing power.

[0110] (II) Weighted Decision Integration and Calculation of Comprehensive Risk Indicators

[0111] For the i-th type of low-event failure mode (i=1,……,8), the comprehensive risk index The calculation formula is:

[0112] ,

[0113] in, is the base event number, and j is the modal number; Let i be the sensitivity of the i-th base event to the j-th modality of features. Let be the dynamic failure probability of the i-th bottom event obtained based on the j-th modal data.

[0114] like Exceeding a preset threshold (e.g.) If 0.7), then the idler roller is determined to have a type i fault risk, and an alarm message (including fault type and location) is output. If all If all values ​​are below the threshold, the idler roller is considered to be operating normally.

[0115] In practical applications, the normal characteristics of idler rollers undergo long-term, slow changes due to seasons, load, or belt wear. Traditional static threshold methods are prone to false alarms or missed alarms. To adapt to the long-term changes in the normal characteristics of idler rollers, this invention also includes a dynamic correction process. By constructing a baseline model of normal characteristics, it adaptively identifies normal fluctuations caused by environmental changes and long-term, slow changes, effectively distinguishing normal changes from actual faults and significantly reducing the model's false alarm rate. Specifically, based on multimodal data such as vibration, temperature, sound, and images continuously collected during normal operation, modal features are extracted, such as the impact amplitude of vibration and sound, and the relative temperature difference of the temperature mode. The mean, standard deviation, and covariance matrix of each modal data are calculated to construct a baseline model of normal characteristics. During real-time operation, if the Mahalanobis distance and / or Euclidean distance of each feature relative to the normal feature baseline model do not exceed the preset threshold, or if the contribution of the deviation feature to the feature sensitivity is not significant, the idler is determined to be in a normal state, and its changes are normal fluctuations within the historical trend range. If the Mahalanobis distance or Euclidean distance of a feature exceeds the preset threshold, and the deviation contributes significantly to the feature sensitivity, the idler is determined to be abnormal. The trend baseline model for each modal feature established based on historical data is only used to exclude normal fluctuations caused by long-term slow changes, i.e., whether the abnormality belongs to the slow drift of normal operating conditions; it does not have the ability to identify fault types.

[0116] Example 2

[0117] A fault monitoring system for idler rollers based on fault tree and multimodal fusion includes:

[0118] Intelligent mobile devices for movement along tracks;

[0119] A multimodal sensor kit, mounted on the intelligent mobile device, includes a vision sensor, a sound sensor, a vibration sensor, and a temperature sensor, which are used to collect visual modal data, sound modal data, vibration modal data, and temperature modal data during the operation of the idler roller, respectively.

[0120] One or more processors;

[0121] Memory, which stores computer programs;

[0122] When the computer program is executed by the one or more processors, it implements the steps of the method in Embodiment 1.

[0123] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0124] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0125] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0126] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0127] The above description is merely a preferred embodiment of the present invention. Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make many possible variations and modifications to the technical solutions of the present invention using the methods and techniques disclosed above, or modify them into equivalent embodiments with equivalent changes, without departing from the scope of the technical solutions of the present invention. Therefore, any simple modifications, equivalent changes, and modifications made to the above embodiments based on the technical essence of the present invention without departing from the content of the technical solutions of the present invention shall still fall within the protection scope of the technical solutions of the present invention.

Claims

1. A method for monitoring idler roller faults based on fault tree and multimodal fusion, characterized in that, Includes the following steps: S1. Construct a fault tree model for the idler roller, the fault tree model including top events, intermediate events and bottom events, perform structural importance analysis on the bottom events, and determine the static weights corresponding to each bottom event; S2. Establish the feature sensitivity mapping relationship between the aforementioned bottom event and multiple monitoring modalities; S3. Control the intelligent mobile device to move along the track and collect raw data of multiple monitoring modes during the operation of the idler roller, including visual mode, sound mode, vibration mode and temperature mode; S4. For the collected vibration mode data, wavelet transform is used to filter out the characteristic frequencies generated by the movement of the intelligent mobile device itself; for the collected sound mode data, spectral subtraction is used to filter out environmental noise. S5. Perform target detection on the acquired visual modal data, identify and locate the idler roller area in the image; use the location result to extract the idler roller temperature data of the corresponding area from the temperature modal data, and calculate the difference between the idler roller temperature and the acquired ambient temperature to obtain the relative temperature difference of the idler roller, thus eliminating the interference of ambient temperature. S6. For the sound data and vibration data processed in step S4, and the visual data and temperature data processed in step S5, perform single-modal feature extraction and fault probability decision-making respectively to obtain dynamic fault probability; based on the static weight, feature sensitivity and dynamic fault probability, perform weighted decision fusion to calculate comprehensive risk index to assess the fault status of the idler roller.

2. The idler roller fault monitoring method according to claim 1, characterized in that, The construction of the fault tree model for the idler roller specifically includes: The idler roller failure is taken as the top event and decomposed layer by layer into multiple intermediate events, including cylinder failure, bearing failure, central shaft failure and seal failure. Each intermediate event is further subdivided into multiple bottom events, including cylinder skin wear, outer ring wear, inner ring fracture, rolling element damage, central shaft fracture, central shaft deformation, journal wear and seal cracking.

3. The idler roller fault monitoring method according to claim 1, characterized in that, The determination of the static weights corresponding to each bottom event specifically includes: Based on expert experience and historical fault statistics, the structural importance coefficients of each bottom event are assigned values ​​and normalized to obtain the static weights of each bottom event, as shown in the formula: ， in, , is the number of the base event, with a value range of 1, 2, ..., n, where n is the total number of base events; Let be the structural importance coefficient of the i-th bottom event; Let be the static weight of the i-th bottom event, and satisfy . .

4. The idler roller fault monitoring method according to claim 1, characterized in that, Step S3 specifically includes: installing a travel track along the direction of the idler belt conveyor, wherein the intelligent mobile device is movably mounted on the track; The intelligent mobile device is equipped with at least four types of sensors, including cameras, thermal imagers, microphones, and vibration meters. The vibration meter captures the vibration characteristics of the idler roller through the vibration transmission of the roller frame and the track; the thermal imager is used to measure the temperature of the idler roller and the ambient temperature; the camera is used to collect three-dimensional contour data and two-dimensional image data of the idler roller surface; and the microphone is used to collect the sound data of the idler roller operation.

5. The idler roller fault monitoring method according to claim 1, characterized in that, The step of using wavelet transform to filter out characteristic frequencies generated by the motion of the intelligent mobile device itself specifically includes: Select wavelet basis functions and determine the number of decomposition levels. Perform multi-scale wavelet decomposition on the original signal sequences of the acquired sound modes and / or vibration modes to obtain approximation coefficients and detail coefficients for different frequency bands. Based on the motion characteristic frequency of the intelligent mobile device, a wavelet coefficient layer containing the motion characteristic frequency is identified; Suppression processing is applied to the identified wavelet coefficient layers; The wavelet coefficient layer after suppression is reconstructed by inverse wavelet transform to obtain the target signal after filtering out its own motion interference.

6. The idler roller fault monitoring method according to claim 5, characterized in that, The wavelet basis functions are Daubechies wavelets, and the number of decomposition levels N is based on the signal sampling frequency. And the preset motion characteristic frequency of the smart mobile device itself. Confirmed, the formula is as follows: ； The suppression process for the identified wavelet coefficient layers specifically includes: If the motion characteristic frequency of the intelligent mobile device is mainly concentrated in the low frequency band, then the corresponding approximation coefficient is set to zero. If the motion characteristic frequency distribution of the intelligent mobile device is in the high-frequency detail layer, then a soft thresholding function or a hard thresholding function is used to quantize the detail coefficients of the corresponding layer.

7. The idler roller fault monitoring method according to claim 1, characterized in that, S5 further includes: Construct and train a YOLOv5-based object detection network model; The collected visual modal image data is input into the trained target detection network model, and the bounding box coordinate information and bounding box width and height information of the roller target are output. Based on the bounding box coordinates and width / height information, the region of interest image is cropped from the original image and used as input for subsequent single-modal feature extraction.

8. The idler roller fault monitoring method according to claim 1, characterized in that, The single-modal feature extraction and fault probability decision-making specifically include: For the sound and vibration data processed in step S4, time-domain waveforms, spectrograms, envelope spectra, and time-frequency diagrams are generated respectively. The time-domain waveforms and envelope spectra are input into a one-dimensional convolutional neural network to extract time-domain impact features and envelope spectrum rotational speed-related features. The spectrograms and time-frequency diagrams are input into a two-dimensional convolutional neural network to extract frequency-domain high-frequency energy features and time-frequency-domain periodic impact features. The output features of the two networks are fused to obtain the dynamic fault probabilities of the sound and vibration modes. The visual and temperature data processed in step S5 are input into a two-dimensional convolutional neural network to extract abnormal features and identify fault modes, thereby obtaining the dynamic fault probabilities of visual and temperature modes.

9. The idler roller fault monitoring method according to claim 1, characterized in that, The step of calculating a comprehensive risk index based on the weighted decision fusion of the static weights, feature sensitivity, and dynamic failure probability specifically includes: For the i-th type of low-event failure mode, the comprehensive risk index The calculation formula is: ， in, is the base event number, and j is the modal number; Let i be the sensitivity of the i-th base event to the j-th modality of features. Let be the dynamic failure probability of the i-th bottom event obtained based on the j-th modal data.

10. A fault monitoring system for idler rollers based on fault tree and multimodal fusion, characterized in that, include: Intelligent mobile devices for movement along tracks; A multimodal sensor kit, mounted on the intelligent mobile device, includes a vision sensor, a sound sensor, a vibration sensor, and a temperature sensor, which are used to collect visual modal data, sound modal data, vibration modal data, and temperature modal data during the operation of the idler roller, respectively. One or more processors; Memory, which stores computer programs; When the computer program is executed by the one or more processors, it implements the steps of the method according to any one of claims 1-9.

Images