Hub motor fault diagnosis method based on multi-modal data fusion
Through multimodal data fusion technology, the time domain, frequency domain and time frequency domain characteristics of the hub motor are extracted, and the neural network model is constructed, which realizes intelligent diagnosis of hub motor failure, and solves the problem that the diagnosis system in the existing technology cannot fully utilize data. The problem improves the accuracy and adaptability of the diagnosis.
Patent Information
- Application Number
- CN202411169867.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-26
- Publication Date
- 2024-12-03
AI Technical Summary
Existing hub motor diagnostic systems cannot fully utilize status data and make it difficult to conduct intelligent and accurate fault diagnosis, especially in complex environments.
Using a multimodal data fusion method, by obtaining vibration signals at the motor and bearing ends, extracting time domain, frequency domain and time frequency domain features, constructing multimodal advanced feature extraction layer, multimodal advanced feature fusion layer and fault diagnosis The layer model uses neural networks to extract and fusion features to achieve accurate identification of fault types.
It improves the accuracy and robustness of fault diagnosis, can more accurately judge the working status of the hub motor and identify potential faults, and adapt to complex and changeable working environments.
Smart Images

Figure CN119066392A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the field of wheel hub motor health management, and in particular relates to a wheel hub motor fault diagnosis method based on multimodal data fusion. Background Art
[0002] As concerns about environmental issues and energy consumption continue to increase, electric vehicles are gradually emerging as a clean energy vehicle to replace traditional fuel vehicles. Due to the distributed design of the hub motor, the motor is directly installed on the wheel or hub, making the vehicle's interior space more flexible and able to more flexibly control the power output of each wheel, reducing the energy loss of the transmission system and improving the vehicle's stability and handling, making the hub motor more and more popular in the field of electric vehicles.
[0003] Since the in-wheel motor is directly mounted on the wheel, it is susceptible to road vibration and impact. This may cause the internal parts of the motor to be damaged or loose. Current in-wheel motors are usually equipped with automatic diagnostic systems to monitor their performance and detect potential faults. However, the on-board automatic diagnostic system generally uses a threshold monitoring method, which can only diagnose simple faults, such as abnormal faults of current, voltage, oil pressure, sensors and other parameters, but cannot make full use of the existing status data to further intelligently and automatically diagnose faults accurately.
[0004] In summary, developing a real-time and efficient intelligent fault diagnosis method for hub motors has great practical significance for improving the reliability, safety and maintenance efficiency of electric vehicles and reducing maintenance costs. Summary of the invention
[0005] 1. Technical issues to be resolved
[0006] The technical problem to be solved by the present invention is: how to provide a real-time and efficient intelligent fault diagnosis method for a hub motor.
[0007] (II) Technical solution
[0008] In order to solve the above technical problems, the present invention provides a hub motor fault diagnosis method based on multimodal data fusion, the method comprising the following steps:
[0009] Step 1: Obtain the status data of the hub motor, including the vibration signal S1 at the motor end and the vibration signal S2 at the bearing end, to represent the original operating state of the motor;
[0010] Step 2: Preliminarily extract the features of the original vibration signals S1 and S2, highlight the features related to the fault in the original vibration signals, and establish a multimodal fault feature set to reduce the interference of non-fault related information; the multimodal fault feature set includes a time domain modal feature set, a frequency domain modal feature set, and a time-frequency domain modal feature set;
[0011] Step 3: Preprocess the data, perform standardization operations on each feature set, and group the data of the feature set for model training and testing, including training data set, validation data set, and test data set;
[0012] Step 4: Construct a hub motor fault diagnosis model, including a multimodal advanced feature extraction layer, a multimodal advanced feature fusion layer, and a fault diagnosis layer. The model input is a multimodal feature and the output is a fault type.
[0013] Step 5: Train the hub motor fault diagnosis model to optimize the model parameters. The process includes training parameter configuration, forward propagation calculation, loss calculation and back propagation update process.
[0014] Step 6: Test the hub motor fault diagnosis model and use the test set to evaluate the generalization performance of the hub motor fault diagnosis model to ensure that the model can show high fault identification accuracy on unseen data.
[0015] Among them, in the step 1, an acceleration sensor deployed near the motor drive end and the bearing is used to collect vibration signals; wherein, the vibration signal of the motor drive end is S1, and the vibration signal of the bearing end is S2, the original data of different types of fault data are recorded, different fault data sets are labeled, and the unique hot encoding technology is used for fault encoding.
[0016] The time domain modal features in step 2 include mean, standard deviation std, kurtosis kt and skewness sn, which constitute the time domain modal feature set F of the hub motor. time ={mean,std,kt,sn};
[0017] The frequency domain modal features include spectrum peak pkt, spectrum entropy phf, spectrum mean meanf, median frequency medf, frequency band power bp and power bandwidth pbw, and the frequency domain modal feature set of the hub motor is F freq ={pkt,phf,meanf,medf,bp,pbw}.
[0018] The time-frequency domain modal feature is the time-frequency spectrum spect TF , the time-frequency domain modal feature set of the hub motor is F time-freq = {spect TF}.
[0019] Wherein, the standardization operation in step 3 adopts the Z-score algorithm;
[0020] In step 3, the data of the feature set are grouped by first randomly shuffling the data of the feature set and regrouping them, and then dividing the data of the feature set into a training data set, a validation data set, and a test data set in a ratio of 8:1:1.
[0021] Among them, in step 4, the structure of the multimodal high-level feature extraction layer includes a statistical feature extraction layer, a sequence feature extraction layer, a graph feature extraction layer and a feature splicing layer, which utilizes the adaptive ability of the neural network to automatically extract high-level features from the primary feature data and learn the complex feature representation of the fault.
[0022] Among them, in the statistical feature extraction layer, for the time domain modal feature F time and frequency domain modal characteristics F freq The statistical feature parameters in the algorithm are used for advanced feature extraction, including {mean, std, kt, sn, meanf, medf, bp, pbw}. Considering that there are two signal sources, a total of 16 statistical feature parameters are involved in the calculation. The statistical feature extraction layer uses a 4-layer linear neural network to extract advanced features. The number of neural nodes in the input layer is consistent with the number of statistical parameters.
[0023] In the sequence feature extraction layer, for the frequency domain modal feature F freq The sequence feature parameters {pkt, phf} in the algorithm are used for high-level feature extraction. Considering the inconsistency of the dimensions of the two parameters, two long short-term memory neural networks (LSTMs) are used to extract the feature dependencies between the nodes of different frequencies of the two feature sequences. The outputs of the two LSTM networks are then concatenated and converted into a one-dimensional vector through a flatten operation. The linear layer is then used to transform the feature dimension to the specified dimension.
[0024] In the graph feature extraction layer, for the time-frequency domain modal feature F time-freq The graph tensor spect in TF To extract high-level features, the convolutional neural network (conv) is used to extract image features, then it is converted into a one-dimensional vector through a flattening operation, and then the linear layer (linear) is used to transform the feature dimension to the specified dimension;
[0025] In the feature concatenation layer, the three feature vectors extracted by the statistical feature extraction layer, the sequence feature extraction layer, and the graph feature extraction layer are concatenated to form a two-dimensional multimodal high-level feature.
[0026] Among them, in step 4, the multimodal high-level feature fusion layer includes a multi-head attention layer, a residual layer and a feedforward network, a residual connection layer and a normalization layer. The multi-head attention mechanism is used to capture key features, realize the fusion of multimodal high-level features in the modal dimension, form multimodal fusion features, and reduce the uncertainty of fault diagnosis.
[0027] Among them, in step 4, the fault diagnosis layer includes a linear layer and a logistic regression layer; the multimodal fusion features output by the multimodal fusion layer are flattened and input into the linear layer, the linear layer is used to transform the feature dimension to the same dimension as the number of fault categories to be identified, and the logistic regression layer is used to calculate the probability of the fault type, and the fault category with the highest probability is the fault type identified by diagnosis.
[0028] The training process of the hub motor fault diagnosis model in step 5 includes the following steps:
[0029] Step 51: Import the data required for the training process, including the training data set and the validation data set after feature extraction in step 3, and the hub motor fault diagnosis model defined in step 4;
[0030] Step 52: Configure training parameters, including learning rate, training cycle, mini-batch size, loss function, and optimization method;
[0031] Step 53: Execute the training process, including gradient calculation and reverse gradient propagation to update model parameters, and stop training when the maximum training cycle is reached or the verification error increases for 5 consecutive training cycles;
[0032] Step 54: The output trained diagnostic model is saved in onnx format.
[0033] Among them, the test hub motor fault diagnosis model in step 6 uses the test data set after feature extraction in step 3 to input the hub motor fault diagnosis model, obtains the predicted fault type corresponding to the test input data, and obtains the test fault recognition accuracy by comparing and calculating with the correct fault type label, and draws a confusion matrix diagram based on the test results to observe the test accuracy of each fault.
[0034] (III) Beneficial effects
[0035] Compared with the prior art, the beneficial technical effects of the present invention are:
[0036] The present invention makes full use of multimodal feature data from different sensors and obtains more comprehensive and multi-dimensional fault information by fusing data from multiple modes. It has significant advantages in improving accuracy, robustness and real-time performance, making the fault diagnosis system more adaptable to complex and changeable working environments, and helps to more accurately judge the working status of the hub motor and identify potential faults. BRIEF DESCRIPTION OF THE DRAWINGS
[0037] Figure 1 A flow chart is constructed for the fault diagnosis method of the present invention.
[0038] Figure 2 This is the hub motor fault diagnosis model diagram.
[0039] Figure 3 This is the structural diagram of the multimodal high-level feature extraction layer.
[0040] Figure 4 This is the structural diagram of the multimodal high-level feature fusion layer.
[0041] Figure 5 This is the structural diagram of the fault diagnosis layer.
[0042] Figure 6 Diagram of the training process of the hub motor fault diagnosis model.
[0043] Figure 7 Confusion matrix plot for model testing. DETAILED DESCRIPTION
[0044] In order to make the purpose, content, and advantages of the present invention more clear, the specific implementation methods of the present invention are further described in detail below in conjunction with the accompanying drawings and examples.
[0045] Example 1
[0046] In order to solve the above technical problems, this embodiment provides a hub motor intelligent fault diagnosis method based on multimodal data fusion, comprising the following steps:
[0047] Step 1: Obtain the status data of the hub motor, including the signal S1 at the motor end and the signal S2 at the bearing end, to represent the original operating state of the motor;
[0048] Step 2: Preliminarily extract the features of the original state signals S1 and S2, highlight the fault-related features in the original vibration signal, and establish a multi-modal fault feature set to reduce the interference of non-fault-related information, including a time domain modal feature set, a frequency domain modal feature set, and a time-frequency domain modal feature set;
[0049] Step 3: Preprocess the data, perform standardization operations on each feature set, and group the data sets for model training and testing, including training data sets, validation data sets, and test data sets;
[0050] Step 4: Construct a hub motor fault diagnosis model, including a multimodal advanced feature extraction layer, a multimodal advanced feature fusion layer, and a fault diagnosis layer. The model input is a multimodal feature and the output is a fault type.
[0051] Step 5: Train the hub motor fault diagnosis model to optimize the model parameters. The process includes training parameter configuration, forward propagation calculation, loss calculation, and back propagation update.
[0052] Step 6: Test the hub motor fault diagnosis model and use the test set to evaluate the generalization performance of the hub motor fault diagnosis model to ensure that the model can show high fault identification accuracy on unseen data.
[0053] Based on the above steps, the method further includes:
[0054] 1. In step 1, the acceleration sensor deployed near the motor drive end and the bearing is used to collect vibration signals with a sampling frequency of 51.2kHz. The vibration signal at the motor drive end is S1, and the vibration signal at the bearing end is S2. The original data of different types of fault data are recorded, and different fault data sets are labeled. The unique hot encoding technology is used for fault encoding.
[0055] 2. The time domain modal features described in step 2 include mean, standard deviation std, kurtosis kt and skewness sn, where
[0056] The mean represents the overall state of the hub motor over a period of time and is calculated as follows:
[0057]
[0058] The length of signal S is N.
[0059] The standard deviation std indicates the degree of deviation of the state of the hub motor from the overall state, and is calculated as follows:
[0060]
[0061] The kurtosis kt indicates the sharpness of the wheel hub motor signal distribution and is calculated as follows:
[0062]
[0063] The skewness sn indicates the asymmetry of the wheel hub motor signal distribution and is calculated as follows:
[0064]
[0065] Therefore, the time domain modal feature set of the hub motor is F time ={mean,std,kt,sn}.
[0066] 3. The frequency domain modal features described in step 2 include spectrum steepness pkt, spectrum entropy phf, spectrum mean meanf, median frequency medf, frequency band power bp and power bandwidth pbw.
[0067] The spectrum peak pkt represents the spectrum shape of the motor signal and is calculated as follows:
[0068]
[0069] Where K is the number of frequency resolutions, f i is the frequency resolution f i The frequency at which P(f i ) is the frequency resolution f i The power spectral density at is the mean frequency of the spectrum.
[0070] The spectral entropy phf represents the spectral complexity of the motor signal and is calculated as follows:
[0071]
[0072] The spectral mean value meanf represents the average position of the motor signal in the frequency domain and is calculated as follows:
[0073]
[0074] Among them, f min and f max are the minimum and maximum values of the frequency range of the integration, respectively.
[0075] The median frequency medf indicates that 50% of the energy in the spectrum is concentrated below this frequency and is calculated as follows:
[0076]
[0077] Among them, f m represents the median frequency.
[0078] The band power bp represents the power within a specific frequency band and is calculated as follows:
[0079]
[0080] The power bandwidth pbw represents the frequency bandwidth where the power spectrum density drops to half of the peak power and is calculated as follows:
[0081] pbw=f2-f1
[0082] Where f1 and f2 are the two frequency points where the power spectrum density drops to half of the peak power. Therefore, the frequency domain modal feature set of the hub motor is
[0083] F freq ={pkt,phf,meanf,medf,bp,pbw}.
[0084] 4. The time-frequency domain modal features described in step 2 are the time-frequency spectrum spect TF , calculated as follows:
[0085]
[0086] Among them, spect TF (t,f) is the short-time Fourier transform result at time t and frequency f, x(τ) is the original signal, w(τ-t) is the window function, e -j2πfτ This patent uses the Hanning window function, the signal length is 512, and the number of overlapping samples is 120, so as to obtain the appropriate frequency and time resolution.
[0087] Therefore, the time-frequency domain modal feature set of the hub motor is F time-freq = {spect TF}.
[0088] 5. The standardization operation described in step 3 uses the Z-score algorithm, and the calculation formula is as follows:
[0089]
[0090] Among them, μ is the mean of each feature, and σ is the standard deviation of each feature. Here, the normalization operation is performed independently on each feature to prevent the drowning effect caused by inconsistent scales of each feature.
[0091] 6. In step 3, the data grouping requires first randomly shuffling the data set and regrouping it, and then dividing the data into a training data set, a validation data set, and a test data set in a ratio of 8:1:1.
[0092] 7. In step 4, the structure of the multimodal high-level feature extraction layer includes a statistical feature extraction layer, a sequence feature extraction layer and a graph feature extraction layer, which fully utilizes the adaptive ability of the neural network to automatically extract high-level features from the primary feature data and learn the complex feature representation of the fault.
[0093] 8. In the above statistical feature extraction layer, for the time domain modal feature F time and frequency domain modal characteristics F freq The statistical feature parameters in the statistic feature extraction include {mean, std, kt, sn, meanf, medf, bp, pbw}. Considering that there are two signal sources, a total of 16 statistical feature parameters are involved in the calculation. The statistical feature extraction layer uses a 4-layer linear neural network to extract advanced features. The number of neural nodes in the input layer is consistent with the number of statistical parameters.
[0094] 9. In the above sequence feature extraction layer, for the frequency domain modal feature F freqThe sequence feature parameters {pkt, phf} in the algorithm are used for high-level feature extraction. Considering the inconsistency of the dimensions of the two parameters, two long short-term memory neural networks (LSTMs) are used to extract the feature dependencies between different frequency nodes of the two feature sequences respectively. Then, the outputs of the two LSTM networks are concatenated and converted into a one-dimensional vector through a flatten operation. Finally, a linear layer is used to transform the feature dimension to the specified dimension.
[0095] 10. In the above-mentioned graph feature extraction layer, for the time-frequency domain modal feature F time-freq The graph tensor spect in TF To extract high-level features, the convolutional neural network (conv) is first used to extract image features, then it is converted into a one-dimensional vector through a flattening operation, and then the linear layer (linear) is used to transform the feature dimension to the specified dimension.
[0096] 11. In the above feature concatenation layer, the three extracted feature vectors are concatenated to form a 2-dimensional multimodal high-level feature.
[0097] 12. In step 4, the structure of the multimodal high-level feature fusion layer mainly includes a multi-head attention layer, a residual layer and a feedforward network, a residual connection layer and a normalization layer. The multi-head attention mechanism is used to capture key features, and the fusion of multimodal high-level features in the modal dimension can be achieved to form multimodal fusion features and reduce the uncertainty of fault diagnosis.
[0098] The multi-head attention layer is used to extract the relationship between different modal features. Each attention mechanism can extract different high-order feature representations. The calculation method of attention is as follows:
[0099]
[0100] Q=X*W Q
[0101] K=X*W K
[0102] V=X*W V
[0103] Where X is the multimodal high-level feature matrix, W Q , W K and W V are the transformation matrices of Q, K, and V respectively, d k is the scaling factor.
[0104] 13. In step 4, the structure of the fault diagnosis layer includes a linear layer and a logistic regression layer. The multimodal fusion features output by the multimodal fusion layer are flattened and then input into the linear layer. The linear layer is used to transform the feature dimension to the same dimension as the number of fault categories to be identified. The logistic regression layer is used to calculate the probability of the fault type. The fault category with the highest probability is the fault type identified by diagnosis.
[0105] The calculation process of the logistic regression layer is as follows:
[0106]
[0107] Output y j (x) is the input x belonging to fault category y j probability.
[0108] 14. Step 5: The training process of the hub motor fault diagnosis model mainly includes the following steps:
[0109] S51, importing data required for the training process, including the training data set and the verification data set after feature extraction in step 3, and the hub motor fault diagnosis model defined in step 4;
[0110] S52, configuring training parameters, including learning rate, training cycle, mini-batch size, loss function and optimization method;
[0111] S53, executing the training process, including gradient calculation and reverse gradient propagation to update model parameters, and stopping the training when the maximum training cycle is reached or the verification error increases for 5 consecutive training cycles;
[0112] S54, the output trained diagnostic model is saved in onnx format.
[0113] 15. In step 6, the test hub motor fault diagnosis model uses the test data set after feature extraction in step 3 to input the hub motor fault diagnosis model to obtain the predicted fault type corresponding to the test input data, and obtains the test fault recognition accuracy by comparing with the correct fault type label, and draws a confusion matrix diagram based on the test results to observe the test accuracy of each fault.
[0114] Example 2
[0115] This embodiment discloses a hub motor fault diagnosis method based on multimodal data fusion, which mainly includes the following steps:
[0116] Step 1: Obtain the status data of the hub motor, including the signal at the motor end and the signal at the bearing end, to represent the original operating status of the motor;
[0117] Specifically, the vibration signal is collected by using acceleration sensors deployed near the motor drive end and the bearing, with a sampling frequency of 51.2kHz. The vibration signal at the motor drive end is S1, and the vibration signal at the bearing end is S2. The original data of 8 different types of fault data are recorded, including the normal state under low-speed and medium-speed conditions, bearing rolling element defects, bearing outer ring defects, and bearing inner ring defects. The vibration signal collected every 0.1 second is taken as a sample, and the sample includes the original vibration data of S1 and S2. Each original vibration data is a one-dimensional data consisting of 51,200 time series points. Each type of fault data set consists of 1,000 samples, so the entire data set includes 8,000 samples. As shown in Table 1, different fault data sets are labeled, and the unique hot encoding technology is used for fault encoding.
[0118] Table 1 Fault data set and coding
[0119]
[0120]
[0121] Step 2: Preliminarily extract the features of the original state signals S1 and S2, highlight the fault-related features in the original vibration signal, and establish a multi-modal fault feature set to reduce the interference of non-fault-related information, including a time domain modal feature set, a frequency domain modal feature set, and a time-frequency domain modal feature set;
[0122] Specifically, the time domain modal features include mean, standard deviation std, kurtosis kt and skewness sn, where:
[0123] The mean represents the overall state of the hub motor over a period of time and is calculated as follows:
[0124]
[0125] The length N of signal S is 51200.
[0126] The standard deviation std indicates the degree of deviation of the state of the hub motor from the overall state, and is calculated as follows:
[0127]
[0128] The kurtosis kt indicates the sharpness of the wheel hub motor signal distribution and is calculated as follows:
[0129]
[0130] The skewness sn indicates the asymmetry of the wheel hub motor signal distribution and is calculated as follows:
[0131]
[0132] Therefore, the time domain modal feature set of the hub motor is F time ={mean,std,kt,sn}. After preliminary feature extraction, in this example, the dimension of each feature parameter set is 1000*2*1.
[0133] Specifically, the frequency domain modal features that need to be extracted include spectrum steepness pkt, spectrum entropy phf, spectrum mean meanf, median frequency medf, frequency band power bp and power bandwidth pbw.
[0134] The spectrum peak pkt represents the spectrum shape of the motor signal and is calculated as follows:
[0135]
[0136] Where K is the number of frequency resolutions, f i is the frequency resolution f i The frequency at which P(f i ) is the frequency resolution f i The power spectral density at is the mean frequency of the spectrum.
[0137] The spectral entropy phf represents the spectral complexity of the motor signal and is calculated as follows:
[0138]
[0139] The spectral mean value meanf represents the average position of the motor signal in the frequency domain and is calculated as follows:
[0140]
[0141] Among them, f min and f max are the minimum and maximum values of the frequency range of the integration, respectively.
[0142] The median frequency medf indicates that 50% of the energy in the spectrum is concentrated below this frequency and is calculated as follows:
[0143]
[0144] Among them, f m represents the median frequency.
[0145] The band power bp represents the power within a specific frequency band and is calculated as follows:
[0146]
[0147] The power bandwidth pbw represents the frequency bandwidth where the power spectrum density drops to half of the peak power and is calculated as follows:
[0148] pbw=f2-f1
[0149] Where f1 and f2 are the two frequency points where the power spectral density drops to half of the peak power.
[0150] Therefore, the frequency domain modal feature set of the hub motor is F freq ={pkt,ph f ,meanf,medf,bp,pbw}, after preliminary feature extraction, in this example, the dimension of the feature parameter set pkt is 1000*2*641, the dimension of the feature parameter set phf is 1000*2*129, and the dimension of the feature parameter sets meanf, medf, bp and pbw is 1000*2*1.
[0151] Specifically, the time-frequency domain modal features to be extracted are the time-frequency spectrum spect TF , calculated as follows:
[0152]
[0153] Among them, spect TF (t,f) is the short-time Fourier transform result at time t and frequency f, x(τ) is the original signal, w(τ-t) is the window function, e -j2πfτ is a complex exponential term. In this case, the Hanning window function is used, the signal length is 512, and the number of overlapping samples is 120 to obtain appropriate frequency and time resolution. The transformed time spectrum spect TF The dimension is 2*512*36.
[0154] Therefore, the time-frequency domain modal feature set of the hub motor can be obtained as F time-freq = {spect TF}.
[0155] Step 3: Preprocess the data, perform standardization operations on each feature set, and group the data sets for model training and testing, including training data sets, validation data sets, and test data sets;
[0156] Among them, the standardization operation adopts the Z-score algorithm, and the calculation formula is as follows:
[0157]
[0158] Among them, μ is the mean of each feature, and σ is the standard deviation of each feature.
[0159] The data set grouping process requires first randomly shuffling the data set and regrouping it, and then dividing the data into training data set, validation data set, and test data set in a ratio of 8:1:1. Finally, the number of samples in the training data set, validation data set, and test data set are 6400, 800, and 800, respectively.
[0160] Step 4: Construct a hub motor fault diagnosis model. The model structure is as follows: Figure 2 As shown, it includes a multimodal high-level feature extraction layer, a multimodal high-level feature fusion layer and a fault diagnosis layer. The model input is the multimodal feature and the output is the fault type.
[0161] Among them, the structure of the multimodal high-level feature extraction layer is as follows: Figure 3 As shown in the figure, for the characteristic variables of different modes, high-level feature extraction networks are constructed respectively. The network includes statistical feature extraction layer, sequence feature extraction layer and graph feature extraction layer. The adaptive ability of neural network is fully utilized to automatically extract high-level features from primary feature data and learn the complex feature representation of faults.
[0162] In the statistical feature extraction layer, for the time domain modal feature F time and frequency domain modal characteristics F freq The statistical feature parameters in the statistic feature extraction include {mean, std, kt, sn, meanf, medf, bp, pbw}. Considering that there are two signal sources, a total of 16 statistical feature parameters are involved in the calculation. The statistical feature extraction layer uses a 4-layer linear neural network to extract advanced features. The number of neural nodes in the input layer is consistent with the number of statistical parameters, which is defined as 16. The number of neural nodes in the two intermediate layers is 20, and the number of nodes in the output layer is 20.
[0163] linear_output=linear(linear_in; size={1*16,1*20})
[0164] Among them, linear_in is an input vector composed of 16 scalars, and linear_output is an output vector with a dimension of 1*20.
[0165] Specifically, in the sequence feature extraction layer, for the frequency domain modal feature F freq The sequence feature parameters {pkt, phf} in the algorithm are used for high-level feature extraction, where the dimension of pkt is 2*641 and the dimension of phf is 2*129. Considering the inconsistency of the dimensions of the two parameters, two long short-term memory neural networks (LSTMs) are used to extract the feature dependencies between different frequency nodes of the two feature sequences respectively. Then, the outputs of the two LSTM networks are concatenated and converted into a one-dimensional vector through a flatten operation. Finally, a linear layer is used to transform the feature dimension to the specified dimension.
[0166] lstm_output_pkt=lstm(pkt,size={2*641,2*40})
[0167] lstm_output_phf=lstm(phf,size={2*129,2*40})
[0168] lstm_output=linear(flatten([lstm_output_pkt,lstm_output_phf]); size={1*160,1*20})
[0169] Among them, the dimensions of the two LSTM output layers are 2*40, the dimension of the one-dimensional vector after flattening is 1*160, and the output feature dimension after linear layer transformation is 1*20.
[0170] Specifically, in the graph feature extraction layer, for the time-frequency domain modal feature F time-freq The graph tensor spect in TF To extract high-level features, the convolutional neural network (conv) is first used to extract image features, then it is converted into a one-dimensional vector through a flattening operation, and then the linear layer (linear) is used to transform the feature dimension to the specified dimension.
[0171] conv_output1 = Conv(spect TF ,size={2*512*36,2*256*18})
[0172] conv_output=linear(flatten(conv_output1),size={1*9216,1*20})
[0173] Among them, the convolution kernel size is 3*3, the stride is 2, the padding is 2, the Conv output layer dimension is 2*256*18, the one-dimensional vector dimension after flattening is 1*9216, and the output feature dimension after linear layer transformation is 1*20.
[0174] Specifically, in the feature concatenation layer, the three extracted feature vectors with dimensions of 1*20 are concatenated to form a multimodal high-level feature with a dimension of 3*20.
[0175] The structure of the multimodal high-level feature fusion layer described in step 4 is as follows Figure 4As shown in Figure 2, it mainly includes a multi-head attention layer, a residual layer and a feedforward network, a residual connection layer and a normalization layer. The multi-head attention mechanism is used to capture key features, and the fusion of multi-modal advanced features in the modal dimension can be realized to form multi-modal fusion features and reduce the uncertainty of fault diagnosis. The parameter configuration of the multi-modal advanced feature fusion layer is shown in Table 2.
[0176] Table 2 Parameter configuration of multimodal high-level feature fusion layer
[0177] Network Layer Parameter configuration Multi-Head Attention Layer Number of heads: 4, input size: 1*3*20, output size: 1*3*20 Residual Connection Layer Input size: 1*3*20, output size: 1*3*20 Normalization layer Normalization method: batch normalization Feedforward Neural Network Number of layers: 2, input size: 1*3*20, output size: 1*3*20
[0178] The multi-head attention layer is used to extract the relationship between different modal features. Each attention mechanism can extract different high-order feature representations. The calculation method of attention is as follows:
[0179]
[0180] Q=X*W Q
[0181] K=X*W K
[0182] V=X*W V
[0183] Where X is the multimodal high-level feature matrix, W Q , W K and W V are the transformation matrices of Q, K, and V respectively, d k is the scaling factor.
[0184] The structure of the fault diagnosis layer in step 4 is as follows Figure 5 As shown in Figure 3, it includes a linear layer and a logistic regression layer, and the network parameter configuration of each layer is shown in Table 3. The multimodal fusion features output by the multimodal fusion layer are flattened and input into the linear layer. The linear layer is used to transform the feature dimension to the same dimension as the number of fault categories to be identified. The logistic regression layer is used to calculate the probability of the fault type. The fault category with the highest probability is the fault type identified by diagnosis.
[0185] Table 3 Parameter configuration of fault diagnosis layer
[0186] Network Layer Parameter configuration Linear Layer Input dimension: 1*60, output dimension: 1*8 Logistic Regression Layer Input dimension: 1*8, output dimension: 1*8
[0187] The calculation process of the logistic regression layer is as follows:
[0188]
[0189] Output y j (x) is the input x belonging to fault category y j probability.
[0190] Step 5: Train the hub motor fault diagnosis model to optimize the model parameters. The process includes training parameter configuration, forward propagation calculation, loss calculation, and back propagation update.
[0191] Specifically, the training process of the hub motor fault diagnosis model is as follows: Figure 6 As shown, it mainly includes the following steps:
[0192] S51, importing the data required for the training process, including the 6400 sets of training data sets and 800 sets of verification data sets after feature extraction in step 3, and the hub motor fault diagnosis model defined in step 4;
[0193] S52, configure the training parameters, including learning rate, training cycle, mini-batch size, loss function and optimization method, which are defined as 1e-5, 100, 64, cross entropy function and Adamw respectively;
[0194] S53, executing the training process, including gradient calculation and reverse gradient propagation to update model parameters, and stopping the training when the maximum training cycle is reached or the verification error increases for 5 consecutive training cycles;
[0195] S54, the output trained diagnostic model is saved in onnx format.
[0196] Step 6: Test the hub motor fault diagnosis model and use the test set to evaluate the generalization performance of the hub motor fault diagnosis model to ensure that the model can show high fault identification accuracy on unseen data.
[0197] Specifically, the test data set of 800 sets of data after feature extraction in step 3 is used to input the hub motor fault diagnosis model, and the test input data corresponds to 800 sets of predicted fault types. The test fault recognition accuracy is calculated by comparing with the correct fault type label. The test accuracy of this test case is 98.62%, and the following is drawn according to the test results: Figure 7 The confusion matrix diagram shown above shows the test accuracy of each fault.
[0198] In summary, the present invention belongs to the field of wheel hub motor health management, and specifically relates to a wheel hub motor fault diagnosis method based on multimodal data fusion, including: obtaining the state data of the wheel hub motor to characterize the original operating state of the motor; preliminarily extracting the features of the original state signal, highlighting the features related to the fault in the original vibration signal, and establishing a multimodal fault feature set, including a time domain modal feature set, a frequency domain modal feature set, and a time-frequency domain modal feature set; preprocessing the data, and grouping the data set for model training and testing; constructing a wheel hub motor fault diagnosis model, the model input is a multimodal feature, and the output is a fault type; training the wheel hub motor fault diagnosis model to achieve model parameter optimization; testing the wheel hub motor fault diagnosis model, evaluating the generalization performance of the wheel hub motor fault diagnosis model, and ensuring that the model can show a high fault recognition accuracy on unseen data. The wheel hub motor fault diagnosis model constructed by the present invention based on the multimodal data fusion method has great practical significance for improving the reliability, safety and maintenance efficiency of electric vehicles, and reducing maintenance costs.
[0199] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the technical principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.
Claims
1. A hub motor fault diagnosis method based on multimodal data fusion, It is characterized in that The method comprises the following steps: Step 1: Obtain the status data of the hub motor, including the vibration signal S at the motor end 1 and the vibration signal S at the bearing end 2 , characterizing the original operating state of the motor; Step 2: Preliminary extraction of the original vibration signal S 1 and S 2 The features related to the fault in the original vibration signal are highlighted, and a multi-modal fault feature set is established to reduce the interference of non-fault related information; the multi-modal fault feature set includes a time domain modal feature set, a frequency domain modal feature set, and a time-frequency domain modal feature set; Step 3: Preprocess the data, perform standardization operations on each feature set, and group the data of the feature set for model training and testing, including training data set, validation data set, and test data set; Step 4: Construct a hub motor fault diagnosis model, including a multimodal advanced feature extraction layer, a multimodal advanced feature fusion layer, and a fault diagnosis layer. The model input is a multimodal feature and the output is a fault type. Step 5: Train the hub motor fault diagnosis model to optimize the model parameters. The process includes training parameter configuration, forward propagation calculation, loss calculation and back propagation update process. Step 6: Test the hub motor fault diagnosis model and use the test set to evaluate the generalization performance of the hub motor fault diagnosis model to ensure that the model can show high fault identification accuracy on unseen data.
2. The hub motor fault diagnosis method based on multimodal data fusion as claimed in claim 1, It is characterized in that In step 1, an acceleration sensor deployed near the motor drive end and the bearing is used to collect vibration signals; wherein the vibration signal at the motor drive end is S 1 , the vibration signal at the bearing end is S 2 , record the original data of different types of fault data, label different fault data sets, and use the one-hot encoding technology to encode the faults.
3. The hub motor fault diagnosis method based on multimodal data fusion as claimed in claim 1, It is characterized in that The time domain modal features in step 2 include mean, standard deviation std, kurtosis kt and skewness sn, which constitute the time domain modal feature set F of the hub motor. time ={mean,std,kt,sn}; The frequency domain modal features include spectrum peak pkt, spectrum entropy phf, spectrum mean meanf, median frequency medf, frequency band power bp and power bandwidth pbw, and the frequency domain modal feature set of the hub motor is F freq ={pkt,phf,meanf,medf,bp,pbw}. The time-frequency domain modal feature is the time-frequency spectrum spect TF , the time-frequency domain modal feature set of the hub motor is F time-freq = {spect TF }.
4. The hub motor fault diagnosis method based on multimodal data fusion as claimed in claim 3, It is characterized in that The standardization operation in step 3 adopts the Z-score algorithm; In step 3, the data of the feature set are grouped by first randomly shuffling the data of the feature set and regrouping them, and then dividing the data of the feature set into a training data set, a validation data set, and a test data set in a ratio of 8:1:
1.
5. The hub motor fault diagnosis method based on multimodal data fusion as claimed in claim 4, It is characterized in that In step 4, the structure of the multimodal high-level feature extraction layer includes a statistical feature extraction layer, a sequence feature extraction layer, a graph feature extraction layer and a feature splicing layer, which utilizes the adaptive ability of the neural network to automatically extract high-level features from the primary feature data and learn the complex feature representation of the fault.
6. The hub motor fault diagnosis method based on multimodal data fusion as claimed in claim 5, It is characterized in that In the statistical feature extraction layer, for the time domain modal feature F time and frequency domain modal characteristics F freq Advanced feature extraction is performed based on the statistical feature parameters in, including {mean, std, kt, sn, meanf, medf, bp, pbw}. Considering that there are two signal sources, a total of 16 statistical feature parameters are involved in the calculation; The statistical feature extraction layer uses a 4-layer linear neural network to extract high-level features. The number of neural nodes in the input layer is consistent with the number of statistical parameters. In the sequence feature extraction layer, for the frequency domain modal feature F freq The sequence feature parameters {pkt, phf} in the vector are used for high-level feature extraction. Considering the inconsistency of the dimensions of the two parameters, two long short-term memory neural networks (LSTMs) are used to extract the feature dependencies between the nodes of different frequencies of the two feature sequences. The outputs of the two LSTM networks are then concatenated and converted into a one-dimensional vector through a flattening operation. The linear layer is then used to transform the feature dimension to the specified dimension. In the graph feature extraction layer, for the time-frequency domain modal feature F time-freq The graph tensor spect in TF To extract high-level features, the convolutional neural network (conv) is used to extract image features, then it is converted into a one-dimensional vector through a flattening operation, and then the linear layer (linear) is used to transform the feature dimension to the specified dimension; In the feature concatenation layer, the three feature vectors extracted by the statistical feature extraction layer, the sequence feature extraction layer, and the graph feature extraction layer are concatenated to form a two-dimensional multimodal high-level feature.
7. The hub motor fault diagnosis method based on multimodal data fusion as claimed in claim 6, It is characterized in that In step 4, the multimodal high-level feature fusion layer includes a multi-head attention layer, a residual layer and a feedforward network, a residual connection layer and a normalization layer. The multi-head attention mechanism is used to capture key features, realize the fusion of multimodal high-level features in the modal dimension, form multimodal fusion features, and reduce the uncertainty of fault diagnosis.
8. The hub motor fault diagnosis method based on multimodal data fusion as claimed in claim 7, It is characterized in that In step 4, the fault diagnosis layer includes a linear layer and a logistic regression layer; the multimodal fusion features output by the multimodal fusion layer are flattened and input into the linear layer, the linear layer is used to transform the feature dimension to the same dimension as the number of fault categories to be identified, and the logistic regression layer is used to calculate the probability of the fault type, and the fault category with the highest probability is the fault type identified by diagnosis.
9. The hub motor fault diagnosis method based on multimodal data fusion as claimed in claim 8, It is characterized in that The training process of the hub motor fault diagnosis model in step 5 includes the following steps: Step 51: Import the data required for the training process, including the training data set and the validation data set after feature extraction in step 3, and the hub motor fault diagnosis model defined in step 4; Step 52: Configure training parameters, including learning rate, training cycle, mini-batch size, loss function, and optimization method; Step 53: Execute the training process, including gradient calculation and reverse gradient propagation to update model parameters, and stop training when the maximum training cycle is reached or the verification error increases for 5 consecutive training cycles; Step 54: The output trained diagnostic model is saved in onnx format.
10. The hub motor fault diagnosis method based on multimodal data fusion as claimed in claim 9, It is characterized in that The test hub motor fault diagnosis model of step 6 uses the test data set after feature extraction in step 3 to input the hub motor fault diagnosis model, obtains the predicted fault type corresponding to the test input data, and obtains the test fault recognition accuracy by comparing and calculating with the correct fault type label, and draws a confusion matrix diagram based on the test results to observe the test accuracy of each fault.
Citation Information
Cited By
Zero sample prototype extension method and system for relieving hubness problem and composite fault diagnosis
CN119779677A
Marine diesel engine fault detection method, device and equipment and storage medium
CN119783024A
Time-frequency characteristic deep fusion method for secondary power supply of high-temperature aging test equipment
CN119903328A
State monitoring method and system for injection molding equipment
CN120492819A
Method and system for diagnosing health state of power equipment based on multi-modal data fusion
CN120508909A
