A mechanical fault detection method based on multimodal analysis of engine vibration signals

Abstract

The present invention relates to a mechanical fault detection method based on multimodal analysis of engine vibration signals. For the extracted diesel engine vibration signal, a multimodal feature extraction network is first used to extract image features related to abnormal signals from one-dimensional amplitude data and two-dimensional image data. A mixed-channel feature fusion detection network is then used to split the feature graph into two groups. Spatial and channel attention mechanisms are used to perform weighted computations on the feature graphs in the spatial and channel domains, respectively. The calculated feature graphs are further grouped and merged to obtain multi-dimensional weighted feature graphs. Finally, a multi-scale detector is used to simultaneously detect the three feature graphs to determine whether the signal within the time period is abnormal. Compared with existing technologies, the present invention has the advantages of high accuracy and excellent noise resistance.

Description

Technical Field

[0001] The present invention relates to the field of machine learning technology, and in particular to a mechanical fault detection method based on multimodal analysis of engine vibration signals. Background Art

[0002] Fault detection of diesel engines can extend their service life and enhance their safety, which has important economic value and social benefits.

[0003] However, existing fault detection methods cannot cope with the high noise environments found in real-world applications and lack universal applicability under multiple operating conditions. Furthermore, existing fault diagnosis models employ single-data detection schemes, such as using only one-dimensional data as input for anomaly analysis and detection. These schemes often fail to consider the inherent correlations and distributional disparities between the same data sources in different formats, limiting their ability to explore multi-source data. Summary of the Invention

[0004] The purpose of the present invention is to overcome the defects of the above-mentioned prior art and provide a mechanical fault detection method based on multimodal analysis of engine vibration signals.

[0005] The purpose of the present invention can be achieved by the following technical solutions:

[0006] A mechanical fault detection method based on multimodal analysis of engine vibration signals comprises the following steps:

[0007] Collect vibration signals of generators;

[0008] Inputting the vibration signal into a multimodal feature extraction network to obtain multiple feature information;

[0009] Select p The feature information is input into the mixed channel feature fusion detection network, and the feature secondary processing and detection are performed to output the detection results.

[0010] The multimodal feature extraction network includes a shaping network, a convolution module, a fully connected module and a multimodal Transformer module;

[0011] The shaping network is used to convert the input engine vibration signal into a two-dimensional image, and the convolution module performs feature extraction based on the two-dimensional image to obtain a two-dimensional feature image;

[0012] The fully connected module is used to extract a one-dimensional feature vector of the input engine vibration signal; the multimodal Transformer module is used to integrate the one-dimensional feature vector and the two-dimensional feature image;

[0013] The mixed-channel feature fusion detection network includes a feature aggregation module, a feature mixing group module and a multi-scale detection module;

[0014] The feature aggregation module is used to p Aggregate the feature information to get the aggregated feature map FM ;

[0015] The feature mixing module is used to aggregate the feature map Perform feature mixing to obtain multiple feature maps of different sizes;

[0016] The multi-scale detection module is used to detect the mixed feature map, and to determine whether there is an abnormal vibration signal in the diesel engine cylinder during the current period by detecting whether there is an abnormal or irregular texture area in the feature map.

[0017] Furthermore, the multimodal feature extraction network includes q Layer structure, each layer includes a convolution module, a fully connected module and a multimodal Transformer module.

[0018] Furthermore, the multimodal feature extraction network performs feature extraction on the vibration signal to obtain a plurality of feature information, including the following steps:

[0019] S1: The vibration signal is input into the shaping network and the fully connected module respectively. The shaping network outputs a two-dimensional image, and the two-dimensional image is input into the convolution module for feature extraction;

[0020] S2, the fully connected module outputs a one-dimensional feature vector;

[0021] S3, the convolution module outputs a two-dimensional feature image;

[0022] S4, inputting the one-dimensional feature vector and the two-dimensional feature image into a multimodal Transformer module to obtain integrated feature information;

[0023] S5, inputting the integrated feature information and the two-dimensional feature image into the convolution module of the next layer, and inputting the one-dimensional feature vector into the fully connected module of the next layer;

[0024] S6. Repeat steps S2-S5 until the qth layer structure of the multimodal feature extraction network is reached, and perform feature extraction layer by layer to obtain multiple feature information.

[0025] Furthermore, the multimodal Transformer module includes two multi-head attention networks, which respectively correspond to the long-distance relationship interaction between the one-dimensional feature vector and the two-dimensional feature image.

[0026] Furthermore, the multimodal Transformer module integrates the one-dimensional feature vector and the two-dimensional feature image, including the following steps:

[0027] The one-dimensional feature vector is divided into tokens sub-vectors;

[0028] The two-dimensional feature image is divided into feature blocks, extending the feature blocks to tokens sub-vectors;

[0029] In the first multi-head attention network, the tokens corresponding to the two-dimensional feature image are input into the matrix In the example, the tokens corresponding to the one-dimensional feature vector are input into the matrix and In the calculation, the query matrix Q with image feature information and the one-dimensional amplitude feature The key matrix K of the information is used to calculate the matching degree, and the obtained matching degree is assigned to the corresponding eigenvalue matrix V to complete the operation of mapping the image features to the amplitude features;

[0030] In the second multi-head attention network, the tokens corresponding to the one-dimensional feature vector are input into the matrix In the example, the tokens corresponding to the two-dimensional feature image are input into the matrix and In the calculation, the query matrix Q with image feature information and the one-dimensional amplitude feature The key matrix K of the information is used to calculate the matching degree, and the obtained matching degree is assigned to the corresponding eigenvalue matrix V to complete the operation of mapping the image features to the amplitude features;

[0031] Among them, in the two multi-head attention networks, the corresponding two groups of Q, K, and V vectors are respectively 、 、 and 、 、 The matrix is ​​calculated;

[0032] The output one-dimensional feature vectors of the two multi-head attention networks are merged, and the merged features are activated using the fully connected layer and ReLU activation function. Perform integer calculation on the vector to get Dimensional feature maps.

[0033] Furthermore, the feature aggregation module combines the feature maps output by the three convolution modules at the bottom of the multimodal feature extraction network 、 and Performing aggregation includes the following steps:

[0034] Improve the feature information of each feature map;

[0035] The deconvolution method is used to double the size of the feature map and compress the number of channels to half of the original. The characteristics of Go up and get ;

[0036] right The feature map is upsampled and channel compressed, and The features of the feature map are fused to obtain ;

[0037] Use concat layer 、 and The three feature maps are merged. Downsample, Upsample and aggregate to obtain feature maps .

[0038] Furthermore, the feature mixing module aggregates the feature graphs Performing feature mixing includes the following steps:

[0039] Use 1x1 convolution layer to The number of feature channels is compressed and merged to Same, using group convolution Divide into and Two feature maps, whose size is Same, the number of channels is one-half;

[0040] Feature map after grouping and Perform feature extraction and obtain and ;

[0041] Using the same parallel convolution module from and Extract feature maps of different sizes and numbers of channels to form two paired groups;

[0042] The fusion module composed of concat layer and 1x1 convolution layer is used to merge the feature maps in the pairing group and obtain 、 and Three feature maps of different sizes.

[0043] Furthermore, the feature map Use spatial attention calculation to enhance the morphological feature information corresponding to abnormal signals ;right Use channel attention calculation to improve the semantic feature weights of abnormal signals ;

[0044] exist The extracted pairing groups contain the spatial attention weighted feature values, The pairing groups extracted from

[15] contain the channel attention weighted feature values.

[0045] Furthermore, the multi-scale detection module is used to detect the feature map output by the feature mixing module, and the cross entropy classification loss is used to adjust the classification module of the detector, and the position loss is calculated based on the CIoU position evaluation relationship.

[0046] Compared with the prior art, the present invention has the following beneficial effects:

[0047] The present invention takes into account the multimodality of the generator vibration signal and extracts the one-dimensional amplitude vector and two-dimensional feature vector from the engine vibration signal through a multimodal feature extraction network. The extracted features are then subjected to secondary feature processing and detection by a mixed-channel feature fusion detection network. The mixed-channel feature fusion module includes a spatial and channel dual-dimensional attention calculation mechanism, which performs weighted calculations on the feature graph in the spatial and channel domains to suppress vibration noise. This allows the network to resist the influence of environmental noise and changes in operating conditions on the final detection results. The method is suitable for strong noise environments in practical applications and has universal applicability under multiple working conditions.

[0048] The accuracy and noise resistance of the present invention are significantly better than those of the existing technology. On four data sets constructed under different working conditions, even when the signal-to-noise ratio is -4dB, the accuracy of the present invention reaches at least 99.008%, which is much higher than other methods. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 Schematic diagram of the structure of the multimodal feature extraction network of the present invention.

[0050] Figure 2 2 is a two-dimensional diagram of the waveform signal of the cylinder under normal and abnormal working conditions in an embodiment of the present invention, wherein (2a) represents the normal working condition and (2b) represents the abnormal working condition.

[0051] Figure 3 Schematic diagram of the structure of the multimodal Transformer module in an embodiment of the present invention.

[0052] Figure 4 Schematic diagram of the structure of the mixed-channel feature fusion detection network in an embodiment of the present invention.

[0053] Figure 5 Schematic diagram of the MITDCNN network structure in an embodiment of the present invention.

[0054] Figure 6 : This is a typical time domain signal of a single-cylinder misfire in a diesel engine at a running speed of 1800 rpm in an embodiment of the present invention.

[0055] Figure 7 1 is a schematic diagram of a two-dimensional image after converting a time domain signal of a single-cylinder misfire in a diesel engine at a running speed of 1800 rpm in an embodiment of the present invention.

[0056] Figure 8 It is the loss iteration curve of network 3 (MITDCNN) in the embodiment of the present invention.

[0057] Figure 9 This is the AP iteration curve of network 3 (MITDCNN) in the embodiment of the present invention. DETAILED DESCRIPTION

[0058] The present invention is described in detail below with reference to the accompanying drawings and specific embodiments. This embodiment is implemented based on the technical solution of the present invention, and provides a detailed implementation method and specific operation process, but the protection scope of the present invention is not limited to the following embodiments.

[0059] To solve the problems existing in the prior art, the present invention proposes a convolutional neural network (MITDCNN) based on multimodal Transformer feature extraction for diesel engine misfire diagnosis in strong environmental noise and different working conditions.

[0060] This example experimentally collects vibration signals from engine cylinder heads at different speeds, extracts one-dimensional amplitude vector features and two-dimensional image features, and feeds them into a multimodal feature extraction network. The extracted features are then processed and detected by a mixed-channel feature fusion detection network. The mixed-channel feature fusion module incorporates a dual-dimensional attention mechanism, spatial and channel-wise, to suppress vibration noise, making the network robust against the effects of environmental noise and varying operating conditions on the final detection results. The effectiveness of the proposed method was verified using experimentally collected datasets and compared with existing representative algorithms. Results demonstrate that the proposed MITDCNN significantly outperforms existing algorithms in accuracy and noise immunity. Across four datasets constructed under different operating conditions, the proposed method achieves an accuracy of at least 99.008%, even at a signal-to-noise ratio of -4dB, significantly exceeding other methods.

[0061] In order to enhance the sensitivity of the network to abnormal signals, the present invention designs a multimodal feature extraction network for detecting abnormal signals of diesel engines. Compared with the single-modal feature extraction network, the network designed by the present invention extracts features from both one-dimensional and two-dimensional data at the same time, improving the richness of the extracted feature information. The overall network structure is as follows: Figure 1 shown.

[0062] like Figure 1 As shown in the figure, the data information extracted from the diesel generator cylinder is a waveform signal. First, the waveform signal is converted from one-dimensional data into a two-dimensional image based on time and amplitude through the reshaping network (ReshapeNet). Secondly, the waveform signal is extracted from the corresponding amplitude according to the time unit to form a one-dimensional vector, thereby obtaining two different types of data in the same time period.

[0063] The waveform signal output by the cylinder under normal working conditions has a certain regularity, and the converted two-dimensional image is an image with certain texture regularity. When the cylinder is working abnormally, irregular areas appear in the converted image. The two-dimensional images converted under normal working and abnormal working conditions are as follows: Figure 2 shown.

[0064] Similarly, for one-dimensional vector data, when the cylinder is abnormal, its value will change irregularly. Figure 1 As shown, the network extracts and aggregates features in two directions, the x-axis and the y-axis. In the x-axis, the waveform signal is shaped to obtain a two-dimensional image. The convolutional module (ConvNet) then extracts features to obtain a two-dimensional feature image, which is then fed into the Multi-Modal Transformer module. The output features of the Multi-Modal Transformer module are aggregated with the output of the convolutional module to map the multimodal features into a feature map. In the backbone network, six of the aforementioned structures are stacked and combined to form a multimodal feature extraction network. On the y-axis, the extracted one-dimensional vector is then extracted using a fully connected module (FCNet) to obtain a one-dimensional feature vector, capturing the characteristic relationship between timing and amplitude. The resulting feature vector is then fed into the Multi-Modal Transformer module to complement the feature information of the two-dimensional image and learn the characteristic patterns of abnormal waveforms.

[0065] In summary, the multimodal feature extraction network primarily extracts feature information from two-dimensional images, while supplemented by the extraction and fusion of one-dimensional temporal features, expanding the network's feature richness. The core unit of the network is the multimodal Transformer module, which is responsible for building relationships between one-dimensional and two-dimensional input features.

[0066] For one-dimensional time series features, the multimodal Transformer module proposed in the present invention has better long-term dependency resolution capabilities than recurrent neural networks, and can obtain feature information in longer time series. And when the multimodal Transformer module is used to extract image features, it is possible to extract the correlation between features of each region of the image, that is, global features. In order to fuse one-dimensional and two-dimensional input information, the present invention designs a multimodal Transformer module, performs multi-head attention calculations on the two types of inputs at the same time, and fuses the extracted feature information. The structure of the multimodal Transformer module is as follows: Figure 3 shown.

[0067] like Figure 3 As shown in the figure, the multimodal Transformer module structure contains two multi-head attention networks, which correspond to the long-distance relationship interaction between the one-dimensional feature vector and the two-dimensional feature image.

[0068] The two types of data input into the multimodal Transformer module are preprocessed as follows: the one-dimensional amplitude vector is extracted into a one-dimensional feature vector through the fully connected layer, and the one-dimensional feature vector is divided into tokens sub-vectors; the two-dimensional feature image is extracted into the feature map by the convolution module and the feature map is evenly divided into feature blocks, and then extend the feature blocks to One-dimensional feature vector tokens.

[0069] In the two Transformer multi-head attention networks, the corresponding two sets of Q, K, and V vectors are respectively 、 、 and 、 、 Six matrices are calculated.

[0070] First, in the first Transformer module (Transformer-1 Multi Head Attention), the tokens corresponding to the two-dimensional feature image of the transformed image are input into In the matrix, the tokens corresponding to the one-dimensional magnitude vector are input into and In the matrix, the query matrix Q with image feature information and the one-dimensional amplitude feature are calculated. The key matrix K of the information is used to calculate the matching degree, and the obtained matching degree is assigned to the corresponding eigenvalue matrix V, thereby completing the operation of mapping the image features to the amplitude features;

[0071] Similarly, the tokens corresponding to the one-dimensional amplitude feature are input into the second Transformer module (Transformer-2 Multi Head Attention) In the , the tokens corresponding to the image features are input to and Matrix, thereby mapping the amplitude features to the feature map.

[0072] Then the output one-dimensional feature vectors of the multi-head attention network are merged, and the fully connected layer (FCLayer) and ReLU activation function are used to activate the merged features to enhance their nonlinearity. Perform integer calculation on the vector to get Dimensional feature maps.

[0073] After completing the extraction of multimodal features, the present invention designs a mixed channel feature fusion network to merge network features at different levels to improve the comprehensiveness of the features. Since the appearance time of abnormal signals is different, the abnormal image blocks displayed in the converted image are also different. Therefore, for the detection of abnormal signal images, the present invention adopts a multi-scale detection scheme to detect feature maps of different sizes. The network structure designed above is as follows: Figure 4 shown.

[0074] The mixed-channel feature fusion network can be divided into three parts: feature aggregation module, feature mixing module and multi-scale detection module. The workflow and function of each part are as follows:

[0075] First, in the feature aggregation module, the feature maps output by the three convolutional modules at the bottom of the multimodal feature extraction network are aggregated, such as Figure 4 As shown in the figure, the input to the feature aggregation module is three feature maps of different sizes, named 、 and (Feature Map, FM). Analyzing from the feature dimension, to The semantic features are enhanced step by step, so during the aggregation operation, the feature information of each feature map is first improved (UpSample, upsampling). The deconvolution method is used to double the size of the feature map and compress the number of channels to half of the original. The add fusion method is used to assign the features to Go up and get ; Similarly, for the fused The feature map is upsampled and channel compressed, and The features of the feature map are fused to obtain . Then the concat layer is used to 、 and The three feature maps are merged. Downsample (DownSample), Upsample and aggregate the feature maps Its size is the same , the feature map contains three levels of feature information, so it is better than And so on three feature maps.

[0076] Secondly, the feature mixing module is used to aggregate the feature maps Perform feature mixing. Use 1x1 convolution layer to The number of feature channels is compressed and merged to Same, then group convolution is used to Divide into and Two feature maps, whose size is Same, the number of channels is Perform feature extraction on the grouped feature graph to enhance the weight of the effective feature in the feature graph. Use spatial attention calculation (Spatial Attention model) to enhance the morphological feature information corresponding to abnormal signals ;right Adopt channel attention calculation (Channel Attention model) to improve the semantic feature weight of abnormal signals . Then the same parallel convolution module is used from and Extract feature maps of different sizes and numbers of channels. For example, using a 3x3 convolution with a sliding step size of 2 to get a size of Same as the feature map with half the number of channels ; Use 3x3 convolution with a step size of 1 to get the size of Same, the channel is half of its , and finally 3x3 deconvolution is used to obtain the size and Same, the number of channels is also half of its Similarly, The feature map is calculated using a parallel convolution module 、 and In terms of feature map parameters, and 、 and 、 and Similarly, the above three groups of feature maps are called pairing groups. Each pairing group contains the spatial attention weighted feature value and the channel attention weighted feature value. The fusion module composed of concat layer and 1x1 convolution layer is used to merge the two feature maps in the pairing group to obtain 、 and Three feature maps of different sizes.

[0077] Finally, the three mixed feature maps are tested using a multi-scale detection module. By detecting whether there are abnormal or irregular texture areas in the feature maps, it is determined whether there are abnormal vibration signals in the diesel engine cylinder during the current period. For the detector, the classification loss and regression loss used are as follows: In the detection task of the present invention, only the presence of abnormal signals in the converted graphic area is determined. Therefore, it is a binary classification problem. The cross-entropy classification loss is used to adjust the classification module of the detector. Its loss function formula is:

[0078]

[0079] In the calculation of position loss, the image block area generated by the abnormal information number is a regular rectangle without complex boundary information. Therefore, the position loss is based on the CIoU position evaluation relationship, and its loss formula can be expressed as:

[0080]

[0081] In summary, in order to more accurately detect the misfire signal of the diesel engine, the present invention constructs a MITDCNN network based on a multimodal feature extraction network and a mixed channel feature fusion detection network. The overall network structure is as follows: Figure 5 As shown. The overall working principle of the network is: for the extracted diesel engine vibration signal, first use the multimodal feature extraction network to extract the image features of the abnormal signal from the one-dimensional amplitude data and the two-dimensional image data, and then use the mixed channel feature fusion detection network to split the feature map into two groups, and use the spatial attention mechanism and the channel attention mechanism to perform weighted calculations on the feature map in the spatial domain and the channel domain respectively, and then group the calculated feature maps again, and obtain the multi-dimensional weighted feature maps through the merging operation, and finally use the multi-scale detector to detect the three feature maps at the same time to determine whether the signal in this period is in an abnormal state. The details of the constructed network are shown in Table 1 below:

[0082] Table 1 MITDCNN network layer structure

[0083]

[0084] Based on the above, this example studies diesel engine misfires at three different operating speeds. The three operating speeds are 1300 rpm, 1800 rpm, and 2200 rpm, corresponding to simulated low-speed, medium-speed, and high-speed operating conditions, respectively. Because misfires in three or more cylinders can cause severe engine vibration and are easily observable by the operator without requiring fault detection, this example focuses on single-cylinder misfires and dual-cylinder (mixed) misfires. As shown in Table 4, single-cylinder misfire detection was performed at 1300 rpm, 1800 rpm, and 2200 rpm, and dual-cylinder misfire detection was performed at 1800 rpm. Except for misfires, each group used normal operating conditions as a reference for comparison. Common engine misfires are generally listed in Table 2. Since the relevant data related to these misfires was collected relatively comprehensively during the experiment, the dataset used in this example is universally applicable.

[0085] Table 2 Diesel engine failure at different operating speeds

[0086]

[0087] In this embodiment, the vibration signal is collected at a sampling frequency of 25.6 kHz. The sampling time for each misfire type is 41 seconds. The sampling time includes at least 900 working cycles, and a total of 1,049,600 vibration sequence points are obtained. The typical time domain signals of single-cylinder misfire at different operating speeds are as follows: Figure 6 As shown in the figure. Since the difference between the time domain signals of different misfire types under the same working conditions is extremely small, it is difficult to directly diagnose the specific misfire type based on the time domain signal. Therefore, it is necessary to use computer vision to analyze the "texture structure" of the signal and then determine whether it is abnormal. At the same time, the sample after the vibration signal is converted into a two-dimensional image is as follows Figure 7 shown.

[0088] Table 3 shows the dataset division results for four different operating conditions. Datasets A, B, and C correspond to single-cylinder misfires at 1300 rpm, 1800 rpm, and 2200 rpm, respectively. Dataset D corresponds to mixed-cylinder misfires at 1800 rpm. Since 1,049,600 vibration sequence points were collected for each misfire fault at different speeds, 511 samples were obtained for each label. Each dataset has five labels: I, II, III, IV, and V, resulting in a total of 2555 samples per dataset. The training, test, and validation sets account for 80%, 10%, and 10% of these datasets, respectively.

[0089] Table 3 Dataset division under different working conditions

[0090]

[0091] Secondly, the environment used in this embodiment is shown in Table 4.

[0092] Table 4 Experimental environment

[0093]

[0094] In this example, the CPU used was an Intel i7 12100, and the GPU used was an Nvidia RTX 3080, which has 8704 CUDA cores and 184 Tensor cores, along with 10GB of video memory. In terms of software, PyTorch 1.11.0 was used as the deep learning framework API, along with the CUDA computing platform version 11.3 and the cuDNN computing acceleration library version 8.2.1.

[0095] This embodiment uses the following evaluation indicators to judge the performance of the network model:

[0096] Precision: evaluates the accuracy of the model's judgment on whether the signal is abnormal, that is, the precision rate, which is calculated as follows:

[0097] (3)

[0098] Recall: evaluates the model's sensitivity to abnormal signals, i.e., recall rate. Its calculation formula is:

[0099] (4)

[0100] F1-Score: The harmonic mean of Precision and Recall indicators, and its calculation formula is:

[0101] (5)

[0102] Average Precision (AP): Evaluates the average detection accuracy of the model. It reflects the overall performance of the model by calculating the area under the Precision-Recall indicator curve. The calculation formula is:

[0103] (6)

[0104] In the above formula 、 、 are the elements in the confusion matrix, and the meaning of each element is:

[0105] : The number of correctly detected targets, that is, the number of correctly detected abnormal signal image areas;

[0106] : The number of false detection targets, that is, the number of image areas with normal signals judged as abnormal signals;

[0107] : The number of missed targets, that is, the number of image areas where abnormal signals are not detected.

[0108] In the ablation experiment, the network MITDCNN designed by the present invention is first disassembled into the following three groups of networks:

[0109] Network 1: In the feature extraction network, only the convolution module is used to extract the 2D image features. The mixed channel feature fusion network is not used, and only the detection network is used.

[0110] Network 2: In the feature extraction network, a multimodal feature extraction network is used to extract one-dimensional and two-dimensional features. The mixed-channel feature fusion network is not used, and only the detection network is used.

[0111] Network 3: A multimodal feature extraction network and a mixed-channel feature fusion detection network, namely the MITDCNN network.

[0112] The above three networks were tested on the vibration signal data set of the engine at different working speeds using the above evaluation indicators. The comparative test results are shown in Tables 5-8.

[0113] Table 5 Comparison of 1300 rpm single-cylinder low-speed operating condition data sets

[0114]

[0115] Table 5 shows the comparative test results of different networks on Dataset A. Network 1 achieved Precision, Recall, F1-Score, and AP of 83.128%, 84.035%, 83.579%, and 88.268%, respectively, all within the [80%, 90%] range. Network 2 achieved Precision, Recall, F1-Score, and AP exceeding 90%, representing improvements of 9.229%, 11.017%, 10.106%, and 6.589%, respectively, compared to Network 1. Network 3 achieved Precision, Recall, F1-Score, and AP of 99.738%, 99.888%, 99.812%, and 99.926%, respectively, extremely close to 100%, with standard deviations within 1%.

[0116] Table 6 Comparison of 1800 rpm single-cylinder medium-speed operating condition data sets

[0117]

[0118] Table 6 shows the comparative test results of different networks on Dataset B. Network 1 achieves a Precision of 83.571% and a Recall of 84.268%. Networks 2 and 3 achieve F1-Score and AP of 93.724% and 94.968% and 99.735% and 99.853%, respectively, all exceeding 90%.

[0119] Table 7 Comparison of 2200 rpm single-cylinder high-speed working condition data sets

[0120]

[0121] Table 7 shows the comparative test results of different networks on Dataset C. Network 1 achieved Precision and Recall of 84.287% and 85.937%, respectively, exceeding the corresponding Precision and Recall values ​​of Network 1 in Datasets A and B. Network 2 achieved Recall, F1-Score, and AP all exceeding 95%, at 96.872%, 95.201%, and 95.587%, respectively. Network 3 achieved Precision, Recall, F1-Score, and AP of 99.184%, 99.179%, 99.181%, and 99.217%, respectively, representing improvements of 14.897%, 13.242%, 14.077%, and 11.292%, compared to Network 1.

[0122] Table 8 Comparison of 1800 rpm mixing cylinder medium speed condition data sets

[0123]

[0124] Table 8 shows the comparative test results of different networks on Dataset D. Network 1 achieved Precision, Recall, F1-Score, and AP of 81.918%, 84.268%, 83.076%, and 86.687%, respectively, with standard deviations within 4%. Network 2 achieved Precision, Recall, F1-Score, and AP improvements of 8.619%, 11.225%, 9.872%, and 4.571%, respectively, compared to Network 1. Network 3 achieved Precision, Recall, F1-Score, and AP all exceeding 99%, representing improvements of 8.473%, 4.089%, 6.347%, and 8.481%, respectively, compared to Network 2.

[0125] Based on the comparisons in Tables 5-8, Network 1 uses only convolutional modules to extract relevant features from the transformed image. However, the grayscale texture map generated by signal conversion lacks clear differentiation between normal and abnormal signals. Consequently, the precision and recall metrics achieved on all four datasets are low, resulting in lower F1-Score and AP overall performance. Network 2, compared to Network 1, employs multimodal feature extraction. The backbone network fuses 1-dimensional amplitude features, significantly improving feature richness. Furthermore, the Transformer module extracts global features that complement the local features of the convolutional module, further enhancing the feature hierarchy. Consequently, the comparison results in the table above show significant improvements across all metrics compared to Networks 1 and 2. Finally, Network 3 differs significantly from Network 2 in the inclusion of a mixed-channel feature fusion detection network. This network aggregates extracted features from different layers and weights them in both the spatial and channel domains, enhancing the significance of abnormal signal features. From the test and comparison results, all the data of Network 3 are the best, and the improvement is also greater than that of Network 2. This also shows that the mixed-channel feature fusion detection network is very helpful in improving network performance.

[0126] Secondly, in order to simulate the misfire fault signal detection performance in different noise environments, noise with different signal-to-noise ratios is added to the original data set. and are the signal and noise energies respectively, then the signal-to-noise ratio SNR is defined as follows:

[0127] (7)

[0128] In the ablation experiment in a noisy environment, Gaussian white noise with different signal-to-noise ratios (-4, -2, 0, 2, 4, 6, 8, and 10 dB) was added to the original signals of the engine at different operating speeds and cylinder numbers. The detection performance of each network for abnormal signals under different working conditions and noise environments was compared and tested. The test results of the three networks are shown in Tables 9-11 respectively.

[0129] Table 9 Detection performance test of network 1 under various working environments with different signal-to-noise ratios

[0130]

[0131] Table 9 shows the detection performance metrics of Network 1 across different datasets and under varying noise intensities. In Dataset A, its average precision, recall, F1-Score (a measure of the model's ability to find positive examples), and average precision (AP) are 76.636%, 78.058%, 77.340%, and 82.038%. Its precision values ​​are very similar in Datasets B and C, at 75.616% and 75.278%, respectively. In Dataset D, its precision is 73.350%, the lowest of the four datasets. Under -4dB noise, the average precision, recall, F1-Score, and AP of Network 1 across the four datasets fluctuate within the range of [70% to 80%], reaching 70.760%, 72.757%, 71.743%, and 76.995%, respectively.

[0132] Table 10 Detection performance test of network 2 under various working environments with different signal-to-noise ratios

[0133]

[0134] Table 10 shows the detection performance metrics of Network 2 in different datasets and under different noise intensities. In Dataset A, its Recall value is above 90%, at 91.786%, 13.728% higher than the Recall value of Network 1 in Dataset A. Its average AP values ​​in Datasets B, C, and D are 92.482%, 92.965%, and 89.345%, respectively, with a standard deviation of 1.414% in Dataset C. Under -2dB noise interference, the average Recall, F1-Score, and AP values ​​of Network 2 in all four datasets are all above 90%, at 91.940%, 90.845%, and 92.225%, respectively. These values ​​are 1.934%, 1.587%, and 1.455% higher than the average Recall, F1-Score, and AP values ​​under 8dB noise interference, respectively.

[0135] Table 11 Detection performance test of network 3 under different signal-to-noise ratios in various working environments

[0136]

[0137] Table 11 shows the detection performance of Network 3 on different datasets and under varying noise intensities. In Dataset A, its average Precision, Recall, F1-Score, and AP all exceed 99.7%, reaching 99.711%, 99.754%, 99.732%, and 99.856%, respectively. In Datasets B and C, its average Precision, Recall, F1-Score, and AP fluctuate within the range of [99.1%-99.9%], extremely close to 100%. In Dataset D, its average Precision, Recall, F1-Score, and AP are 98.995%, 99.304%, 99.149%, and 99.699%, significantly exceeding the average Precision, Recall, F1-Score, and AP of Networks 1 and 2 in Dataset D.

[0138] By comparing the test results of the three networks under different working conditions and noise environments, we can draw the following conclusions: for Network 1, it only uses the convolution module to extract features, and the image texture features converted after adding noise are greatly disturbed. Therefore, in the test results, the performance indicators of Network 1 fluctuate greatly; while Network 2 adopts a multimodal feature extraction solution. The noise resistance of the network is improved to a certain extent by adding one-dimensional features. Compared with the other two networks, Network 3 adds a mixed channel feature fusion module, which includes spatial attention and channel attention calculations, and can suppress noise in the spatial domain and channel domain. Therefore, in the test results, the fluctuations caused by noise in various indicators of Network 3 are relatively small, reflecting strong robustness. Secondly, the iterative curves of the loss function value and accuracy value during the training of the Network 3 (MITDCNN) model are as follows. Figure 8 、 Figure 9 shown.

[0139] In this embodiment, after multiple experimental tests, the following optimal hyperparameter settings are obtained: the number of epoch iterations is 200, the initial learning rate is set to 0.001, and gradually converges to 0.00001 after training, the momentum parameter momentum is set to 0.954, and the weight decay rate weight decay is set to 0.0013. Figure 8-9 As can be seen in the figure, after 100 epochs, both the loss and model accuracy remain stable. The loss function value quickly decreases and converges in a short period of time, while the AP value also increases rapidly, indicating that the network has a strong ability to learn the target features.

[0140] The above describes in detail the preferred embodiments of the present invention. It should be understood that those skilled in the art can make numerous modifications and variations based on the concepts of the present invention without inventive effort. Therefore, any technical solutions that can be derived by those skilled in the art through logical analysis, reasoning, or limited experimentation based on the concepts of the present invention and the prior art should be within the scope of protection defined by the claims.

Claims

1. A mechanical fault detection method based on multimodal analysis of engine vibration signals, characterized in that: The following steps are involved: Collect vibration signals of generators; Inputting the vibration signal into a multimodal feature extraction network to obtain multiple feature information; Select p The feature information is input into the mixed channel feature fusion detection network, and the feature secondary processing and detection are performed to output the detection results. The multimodal feature extraction network includes a shaping network, a convolution module, a fully connected module and a multimodal Transformer module; The shaping network is used to convert the input engine vibration signal into a two-dimensional image, and the convolution module performs feature extraction based on the two-dimensional image to obtain a two-dimensional feature image; The fully connected module is used to extract a one-dimensional feature vector of the input engine vibration signal; the multimodal Transformer module is used to integrate the one-dimensional feature vector and the two-dimensional feature image; The mixed-channel feature fusion detection network includes a feature aggregation module, a feature mixing group module and a multi-scale detection module; The feature aggregation module is used to p Aggregate the feature information to get the aggregated feature map FM ; The feature mixing module is used to aggregate the feature map Perform feature mixing to obtain multiple feature maps of different sizes; The multi-scale detection module is used to detect the mixed feature map and determine whether there is an abnormal vibration signal in the diesel engine cylinder during the current period by detecting whether there is an abnormal or irregular texture area in the feature map; The multimodal Transformer module includes two multi-head attention networks, corresponding to the long-range relationship interaction between the one-dimensional feature vector and the two-dimensional feature image; The multimodal Transformer module integrates the one-dimensional feature vector and the two-dimensional feature image, including the following steps: The one-dimensional feature vector is divided into tokens sub-vectors; The two-dimensional feature image is divided into feature blocks, extending the feature blocks to tokens sub-vectors; In the first multi-head attention network, the tokens corresponding to the two-dimensional feature image are input into the matrix In the example, the tokens corresponding to the one-dimensional feature vector are input into the matrix and In the calculation, the query matrix Q with image feature information and the one-dimensional amplitude feature The key matrix K of the information is used to calculate the matching degree, and the obtained matching degree is assigned to the corresponding eigenvalue matrix V to complete the operation of mapping the image features to the amplitude features; In the second multi-head attention network, the tokens corresponding to the one-dimensional feature vector are input into the matrix In the example, the tokens corresponding to the two-dimensional feature image are input into the matrix and In the calculation, the query matrix Q with image feature information and the one-dimensional amplitude feature The key matrix K of the information is used to calculate the matching degree, and the obtained matching degree is assigned to the corresponding eigenvalue matrix V to complete the operation of mapping the image features to the amplitude features; Among them, in the two multi-head attention networks, the corresponding two groups of Q, K, and V vectors are respectively 、 、 and 、 、 The matrix is ​​calculated; The output one-dimensional feature vectors of the two multi-head attention networks are merged, and the merged features are activated using the fully connected layer and ReLU activation function. Perform integer calculation on the vector to get Dimensional feature maps.

2. A mechanical fault detection method based on multimodal analysis of engine vibration signals according to claim 1, characterized in that: The multimodal feature extraction network includes q Layer structure, each layer includes a convolution module, a fully connected module and a multimodal Transformer module.

3. The mechanical fault detection method based on multimodal analysis of engine vibration signals according to claim 2, characterized in that: The multimodal feature extraction network extracts features from the vibration signal to obtain multiple feature information, including the following steps: S1: The vibration signal is input into the shaping network and the fully connected module respectively. The shaping network outputs a two-dimensional image, and the two-dimensional image is input into the convolution module for feature extraction; S2, the fully connected module outputs a one-dimensional feature vector; S3, the convolution module outputs a two-dimensional feature image; S4, inputting the one-dimensional feature vector and the two-dimensional feature image into a multimodal Transformer module to obtain integrated feature information; S5, inputting the integrated feature information and the two-dimensional feature image into the convolution module of the next layer, and inputting the one-dimensional feature vector into the fully connected module of the next layer; S6. Repeat steps S2-S5 until the qth layer structure of the multimodal feature extraction network is reached, and perform feature extraction layer by layer to obtain multiple feature information.

4. The mechanical fault detection method based on multimodal analysis of engine vibration signals according to claim 1, characterized in that: The feature aggregation module combines the feature maps output by the three convolution modules at the bottom of the multimodal feature extraction network 、 and Performing aggregation includes the following steps: Improve the feature information of each feature map; The deconvolution method is used to double the size of the feature map and compress the number of channels to half of the original. The characteristics of Go up and get ; right The feature map is upsampled and channel compressed, and The features of the feature map are fused to obtain ; Use concat layer 、 and The three feature maps are merged. Downsample, Upsample and aggregate to obtain feature maps .

5. The mechanical fault detection method based on multimodal analysis of engine vibration signals according to claim 4, characterized in that: The feature mixing module aggregates the feature map Performing feature mixing includes the following steps: Use 1x1 convolution layer to The number of feature channels is compressed and merged to Same, using group convolution Divide into and Two feature maps, whose size is Same, the number of channels is one-half; Feature map after grouping and Perform feature extraction and obtain and ; Using the same parallel convolution module from and Extract feature maps of different sizes and numbers of channels to form two paired groups; The fusion module composed of concat layer and 1x1 convolution layer is used to merge the feature maps in the pairing group and obtain 、 and Three feature maps of different sizes.

6. The mechanical fault detection method based on multimodal analysis of engine vibration signals according to claim 5, characterized in that: Feature Map Use spatial attention calculation to enhance the morphological feature information corresponding to abnormal signals ;right Use channel attention calculation to improve the semantic feature weights of abnormal signals ; exist The extracted pairing groups contain the spatial attention weighted feature values, The pairing groups extracted from [15] contain the channel attention weighted feature values.

7. The mechanical fault detection method based on multimodal analysis of engine vibration signals according to claim 1, characterized in that: The multi-scale detection module is used to detect the feature map output by the feature mixing module, and the cross entropy classification loss is used to adjust the classification module of the detector.

8. The mechanical fault detection method based on multimodal analysis of engine vibration signals according to claim 1, characterized in that: The multi-scale detection module is used to detect the feature map output by the feature mixing module, and the position loss is calculated based on the CIoU position evaluation relationship.

Images