Gear milling machine cutter fault diagnosis method and system based on continuous wavelet transform and deep learning of multi-modal signal fusion

Abstract

The invention provides a numerical control machine gear milling cutter fault diagnosis method and system based on continuous wavelet plate change and deep learning of multi-modal signal fusion, and relates to the field of numerical control gear milling cutter data monitoring and data fusion set fault detection. The method mainly comprises the steps that (1) an acceleration sensor and a noise sensor are adopted to collect vibration signals and sound signals of gradual abrasion in the working process of the tool respectively; 2) preprocessing each signal, and converting a one-dimensional signal into a two-dimensional image by adopting continuous wavelet transform to serve as an original sample set for fault diagnosis; 3) inputting the images of the two signals into a convolutional neural network to carry out time-frequency image feature extraction and multi-modal feature fusion; 4) dividing the fused features into a training set, a test set and a verification set according to 70%, 20% and 10%; and 5) carrying out model establishment, establishing a ConvNext model to carry out fault diagnosis on the data set, carrying out classification by adopting a softmax classifier, and identifying the tool wear state reflected by the multi-modal fusion signal. According to the method, the multi-mode sensor data is processed in an image processing mode, and the tool wear state can be reflected more effectively through feature fusion.

Description

Technical Field

[0001] The present invention relates to a method and system for diagnosing tool faults of a CNC gear milling machine, and in particular ... Background Art

[0002] With the advancement of industrial automation, CNC gear milling tools are playing an increasingly important role in the manufacturing industry. As cutting tools, their performance directly impacts machining quality and efficiency. However, tool fault diagnosis faces numerous challenges. First, the tool is affected by a variety of factors during the cutting process, such as material properties, cutting parameters, and environmental conditions. These factors lead to diverse tool wear and fault manifestations, increasing the complexity of diagnosis. Second, the tool's operating environment is often accompanied by strong vibrations and noise. These interference signals often mask the true fault characteristics, posing challenges to signal analysis and processing.

[0003] Against this backdrop, acoustic and vibration signals are particularly important in tool fault diagnosis. Monitoring and analyzing these two signals effectively captures tool status changes and provides early warnings. These signals not only provide dynamic information about the tool's operation but also reveal issues like tool wear and failure, thus providing a basis for maintenance decisions.

[0004] However, existing technologies still have shortcomings in tool fault diagnosis. Traditional shallow learning methods rely on manual feature extraction, which is inefficient when processing large amounts of data and has difficulty extracting higher-level information from raw data. While the rise of deep learning has alleviated this problem to some extent, most research relies on a single model and fails to fully utilize the advantages of multiple models, resulting in widespread defects such as poor robustness and low generalization ability when faced with complex data. In addition, the choice of hyperparameters has a significant impact on the model's ultimate diagnostic accuracy. However, current research often relies on manual parameter adjustment based on experience, which is not only time-consuming and labor-intensive, but also difficult to find the optimal solution. Summary of the Invention

[0005] The present invention addresses the above-mentioned technical deficiencies and proposes a gear milling tool fault diagnosis method and system based on continuous wavelet transform and deep learning for multimodal signal fusion. First, the vibration and sound signals are processed. Time-frequency conversion is performed on the vibration and sound signals using continuous wavelet transform, converting the time-domain signals into time-frequency representations to more comprehensively capture signal features. In continuous wavelet transform, the selection of the mother wavelet is crucial, and the present invention employs the Morlet wavelet. The Morlet wavelet has excellent frequency-domain centralization, enabling more accurate analysis of signal characteristics at different frequencies. This allows for better preservation of key signal information during the time-frequency conversion process, improving the accuracy of subsequent analysis. Next, a deep learning model is used to extract and fuse features from the time-frequency converted multimodal signals. To ensure that signals of different modalities play an appropriate role in the final diagnosis, an adaptive weight allocation mechanism is introduced into the multimodal feature fusion, allowing the proportion of each modality's contribution to the final diagnosis to adjust according to task requirements. Dynamic weight allocation is achieved through the Attention layer, adjusting weights in real time based on the importance of signal features to further optimize feature fusion and effectively improve the accuracy of fault diagnosis in complex environments.

[0006] The present invention is achieved through the following technologies:

[0007] A fault diagnosis method and system based on continuous wavelet transform and deep learning of multimodal signal fusion, comprising the following steps:

[0008] Step S1: Acquire the vibration and sound signals of the gear milling machine tool using an acceleration sensor and a noise sensor and preprocess the signals. For data with a lower sampling frequency, linear interpolation is used. Linear interpolation calculates a new data point between two adjacent data points using a linear function. The value of the missing point in the middle is calculated based on the values of the two adjacent points. In this way, the low-sampling-frequency data can be aligned with the high-sampling-frequency signal after processing.

[0009] Step S2: using a continuous wavelet transform algorithm to convert the one-dimensional vibration signal and sound signal into a two-dimensional time-frequency image, cropping the wavelet image of the vibration signal and the wavelet image of the sound signal to the same size, and inputting them into a convolutional neural network (CNN) model;

[0010] Step S3: Feature extraction and feature fusion. The deep learning model convolutional neural network (CNN) is used to extract the time-frequency features of the vibration signal and the sound signal. The extracted features are then input into the multimodal network and fused in the fusion layer through the attention mechanism.

[0011] Step S4: Divide the data set into a training set, a test set, and a validation set in a ratio of 7:2:1.

[0012] Step S5: Use the ConvNext model combined with the softmax classifier to complete the fault classification of the gear milling machine tool;

[0013] Step S6: Input the test set into the trained model to verify whether the model is feasible.

[0014] In step S1, the specific steps are as follows:

[0015] Step S1.2: After acquiring the one-dimensional vibration and sound signals from the test bench, when processing the data collected by the acceleration sensor and noise sensor, due to their different sampling frequencies, linear interpolation must first be performed on the data with the lower sampling frequency. Linear interpolation is to calculate a new data point between two adjacent data points using a linear function. The value of the missing point in the middle is calculated based on the values of the two adjacent points. In this way, the low-sampling-frequency data can be aligned with the high-sampling-frequency signal after processing.

[0016] Linear interpolation formula:

[0017] Where: Assume that two adjacent data points are known to be (x0, y0) and (x1, y1); x is the position to be interpolated; y is the value of the position obtained by linear interpolation;

[0018] Step S1.2: filtering and denoising (e.g., wavelet threshold denoising) the one-dimensional vibration signal and the sound signal after the acquisition frequency alignment to remove high-frequency noise; and normalizing the signal to enhance feature consistency;

[0019]

[0020] Where: x(t) is a vibration or sound signal, which can be a time series signal; μ is the mean of the data set, and σ is the standard deviation of the data set.

[0021] In step S2, the specific steps are as follows:

[0022] Step S2.1: Perform a continuous wavelet transform on the preprocessed vibration and sound signals. The continuous wavelet transform (CWT) is a technique used to decompose a continuous time function into several wavelets. The CWT can construct a time-frequency signal with good localization in the time and frequency domains. The wavelet transform is used to decompose the signal into different frequency bands to obtain the signal's time-frequency localization characteristics. The continuous wavelet transform (CWT) formula is:

[0023]

[0024] Where: x(t) is the vibration signal, is the complex conjugate form of the wavelet function, defined as: a is the scale parameter (controlling frequency resolution), b is the translation parameter (controlling time resolution), and ψ(t) is the mother wavelet function. The result W(a,b) represents the components of the signal at different scales a and times b, forming a two-dimensional time-frequency plot.

[0025] Step S2.2: Use the imshow function to draw a time-frequency graph. Draw the transformation coefficients as a two-dimensional image, with the x-axis representing time and the y-axis representing frequency. The drawing method is as follows:

[0026] Time-frequency plot = imshow(|W(a,b)|, extent = [t min ,t max ,f min ,f max ])

[0027] The extracted energy features, statistical features, spectral entropy, etc. are combined into a feature vector F = [E1, μ1, σ1, ..., H].

[0028] Step S2.3: Try different scale ranges and color mappings to ensure that the frequency features are clearly visible.

[0029] Step S2.4: When the resolution requirement is high, increasing the number of scales can improve the frequency resolution, but the amount of calculation will also increase.

[0030] In step S3, the specific steps are as follows:

[0031] Step S3.1, construct a CNN convolutional neural network. The CNN convolutional neural network module includes a convolution layer, a pooling layer, and a dropout layer.

[0032] Step S3.2: Use a convolutional neural network to process the vibration signal and the sound signal, and extract complex features such as local, frequency band, and nonlinear features through layer-by-layer convolution in the time-frequency domain of the signal.

[0033] Step S3.3: Concatenate the feature vectors of the vibration and sound signals. Introducing an attention layer into the fusion layer enables the model to assign different weights to each modal feature, improving the representation capability of the fused feature.

[0034] In step S4, the specific steps are as follows:

[0035] Step S4.1: Select random partitioning and use the Knuth shuffle algorithm to shuffle the data. This algorithm randomly selects elements from the data and swaps their positions. Its time complexity is O(n), ensuring fairness in the shuffle.

[0036] Step S4.2: Divide the data into a ratio of 7:2:1 based on the total size of the dataset.

[0037] Step S4.3: Stratified sampling is used to ensure that the labels of each subset are evenly distributed.

[0038] In step S5, the specific steps are as follows:

[0039] Step S5.1: Build the basic architecture of the ConvNext model by adding two convolutional layers, each followed by a pooling layer and a dropout layer. Increase the number of filters in each convolutional layer to enhance feature extraction, while keeping other parameters such as kernel size and stride unchanged.

[0040] Step S5.2: After convolution and pooling, use the flattening layer to convert the feature map into a one-dimensional vector for input to the fully connected layer.

[0041] Step S5.3: After the flattening layer, add a fully connected layer to enhance the representation capability of the model and introduce nonlinear transformations. Add a dropout layer after the fully connected layer to reduce the risk of overfitting.

[0042] Step S5.4: Select categorical_crossentropy as the loss function, use the Adma optimizer to accelerate convergence, and specify accuracy as the evaluation metric.

[0043] Step S5.5: When training the model, you can use EarlyStopping to dynamically monitor the loss of the validation set to avoid overfitting.

[0044] Step S5.6: Use the softmax activation function to accurately classify the four types of tool wear faults;

[0045] In step S6, the test set is input into the trained model to verify whether the model is feasible.

[0046] The present invention has the following benefits:

[0047] This paper proposes a method and system for diagnosing gear milling tool faults based on continuous wavelet transform (CWT) and deep learning. This method fully combines the advantages of each technology to achieve high-precision classification of equipment faults. Specifically, this method converts a one-dimensional signal into a two-dimensional time-frequency image using CWT. The generated time-frequency image is cropped to a certain size and then input into a convolutional neural network (CNN) to extract features from the time-frequency image. The extracted features are then deeply fused using multimodal features. The fusion result is further optimized by introducing an attention mechanism, enhancing the feature expression capability. The fused feature image is then proportionally divided into datasets, which are then sequentially input into the ConvNext network. Combining its powerful feature extraction capabilities and deep structural characteristics, high-level semantic features are further learned. Finally, a classifier is used to complete the task of accurately classifying faults. The entire process is designed as an end-to-end structure, which can maximize the preservation of the effective information of the original signal while reducing the impact of noise interference on the diagnosis results. Through multimodal signal fusion, the present invention effectively integrates the information of vibration and sound signals, significantly improving the comprehensiveness of the diagnosis. Combined with dynamic learning rate adjustment and early stopping strategy, the model performs well in terms of convergence speed, generalization performance and robustness. In addition, a reasonable evaluation system, including multi-index analysis such as accuracy, precision, recall rate and F1 score, further ensures the stability and practicality of the model. Overall, the present invention achieves accurate diagnosis of equipment failures in complex industrial scenarios by introducing the integrated design of multidimensional feature extraction, multimodal fusion and end-to-end deep learning model, taking into account the efficiency, accuracy and interpretability of the model, and has important application value. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the implementation of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or descriptions of the prior art. Obviously, the drawings described below are only some embodiments. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0049] Figure 1 This is a schematic diagram of the signal acquisition module of the present invention.

[0050] Figure 2 This is an overall implementation flow chart of the fault diagnosis of the present invention.

[0051] Figure 3 This is the CNN convolutional neural network structure diagram of the present invention.

[0052] Figure 4 A flow chart is established for the ConvNext diagnostic model of the present invention.

[0053] Figure 5 This is the overall framework diagram of ConvNext of the present invention. DETAILED DESCRIPTION

[0054] In order to make the technical solution of the present invention clearer, the present invention is described in more detail below with reference to the accompanying drawings.

[0055] Refer to the attached Figure 1 A fault diagnosis method and system based on continuous wavelet transform and deep learning of multimodal signal fusion includes the following steps:

[0056] S1, obtaining the vibration signal and sound signal of the gear milling machine tool through the acceleration sensor and the noise sensor and preprocessing the signals;

[0057] S2. Use the continuous wavelet transform algorithm to convert the one-dimensional vibration signal and sound signal into a two-dimensional time-frequency image. Then, crop the wavelet image of the vibration signal and the wavelet image of the sound signal to the same size and input them into the convolutional neural network (CNN) model.

[0058] Step S3: Feature extraction and feature fusion. The deep learning model convolutional neural network (CNN) is used to extract the time-frequency features of the vibration signal and the sound signal. The extracted features are then input into the multimodal network and fused in the fusion layer through the attention mechanism.

[0059] Step S4: Divide the data set into a training set, a test set, and a validation set in a ratio of 7:2:1.

[0060] Step S5: Use the ConvNext model combined with the softmax classifier to complete the fault classification of the gear milling machine tool;

[0061] Step S6: Input the test set into the trained model to verify whether the model is feasible.

[0062] Refer to the attached Figure 2 , step S1 specifically includes:

[0063] (1) Build a test system, install acceleration sensors on the spindle box and workpiece of the CNC gear milling machine, and install noise sensors in the processing area to measure the vibration and sound signals during the processing.

[0064] (2) After obtaining the one-dimensional vibration signal and sound signal from the test bench, when processing the data collected by the acceleration sensor and noise sensor, due to their different sampling frequencies, it is necessary to first use linear interpolation on the data with lower sampling frequency. Linear interpolation is to calculate a new data point between two adjacent data points through a linear function, and calculate the value of the missing point in the middle based on the values of the two adjacent points. In this way, the data with lower sampling frequency can be aligned with the signal with higher sampling frequency after processing;

[0065]

[0066] (3) Filter and denoise the one-dimensional vibration signal and sound signal after the acquisition frequency is aligned (such as wavelet threshold denoising) to remove high-frequency noise; standardize the signal to enhance the consistency of the features;

[0067]

[0068] The sampling frequency of the sound signal and the vibration signal are both set to 10 kHz, and the duration of each sampling is 2 s.

[0069] Step S2 is specifically as follows:

[0070] (1) Performing continuous wavelet transform on the pre-processed vibration signal and sound signal;

[0071] Wavelet transform is used to decompose the signal into different frequency bands and obtain the time-frequency localization characteristics of the signal. The continuous wavelet transform (CWT) formula is:

[0072]

[0073] The result W(a,b) represents the components of the signal at different scales a and time b, forming a two-dimensional time-frequency diagram.

[0074] (2) Use the imshow function to draw a time-frequency graph. Draw the transformation coefficients as a two-dimensional image, with the x-axis representing time and the y-axis representing frequency. The drawing method is as follows:

[0075] Time-frequency plot = imshow(|W(a,b)|, extent = [t min ,t max ,f min ,f max ])

[0076] The extracted energy features, statistical features, spectral entropy, etc. are combined into a feature vector F = [E1, μ1, σ1, ..., H].

[0077] (3) Try different scale ranges and color mappings to ensure that frequency features are clearly visible.

[0078] (4) When the resolution requirement is high, increasing the number of scales can improve the frequency resolution, but the computational complexity will also increase.

[0079] (5) The unified time-frequency image size is (224, 224, 3)

[0080] Additional note: Morlet wavelet is selected as the mother wavelet, which has good centralization in the frequency domain.

[0081] Refer to the attached Figure 3 , step S3 is as follows:

[0082] (1) Construct ResNet-50 as a convolutional neural network to extract features from the time-frequency graphs of vibration signals and sound signals. Each neural network module can contain a convolution layer, a pooling layer, and a dropout layer.

[0083] Specific parameters include: each residual block of ResNet-50 includes 3 convolutional layers, using 1x1, 3x3, and 1x1 convolution kernels respectively; the number of convolution kernels in the network is 64, 128, 256, 512, and 2048, respectively; the convolution kernel size in each stage is 1x1 and 3x3, the activation function is ReLU, the pooling layer size is unified to 3x3, and the stride is 2;

[0084] (2) Convolutional neural networks are used to process vibration and sound signals, extracting complex features such as local, frequency band, and nonlinear characteristics through layer-by-layer convolution in the signal's time-frequency domain. Local features in the time-frequency graph are extracted through the convolution layer, and the pooling layer performs dimensionality reduction. Finally, the classification or regression results are output through the fully connected layer. When training the network, the backpropagation algorithm optimizes the convolution kernel and network parameters, enabling the network to effectively learn the characteristics of the time-frequency graph.

[0085] Table 1 Time-frequency features extracted by neural network

[0086]

[0087]

[0088]

[0089] (3) Fusion of vibration and sound signal features. A MultiHeadAttention layer is introduced into the fusion layer, and a Dense layer is used to process the input features. This enables the model to assign different weights to each modal feature, improving the representation capability of the fused features.

[0090] Specific parameters include: the input of each modality is processed through the Dense layer. Here, Dense(64) projects the input features into a 64-dimensional space (this dimension can be appropriately adjusted according to experimental results, such as using Dense(128) or Dense(32) to find the optimal representation space). layers.concatenate connects the feature vectors of the two modalities according to the last feature dimension. The MultiHeadAttention layer is used to assign different weights to the fused features. Here, 4 heads are used, and the dimension of each head is 64. Residual connections are added through layers.Add() to make the model easier to train. The fused features are further processed through a Dense layer (128 neurons), and then Dropout is used to prevent overfitting (Dropout is 0.3).

[0091] (4) After the fusion layer, the fused features are further processed by the Dense layer, and the final features are processed using a higher-dimensional layer Dense (128). This step can help extract more abstract features and further improve the representation ability of the model.

[0092] Step S4 is specifically as follows:

[0093] (1) Random partitioning is selected and the Knuth shuffle algorithm is used to shuffle the data. This algorithm randomly selects elements from the data and swaps their positions. Its time complexity is O(n), which ensures the fairness of the shuffle.

[0094] (2) Based on the total size of the dataset, the data is divided in the ratio of 7:2:1.

[0095] (3) Stratified sampling is used to ensure that the labels of each subset are evenly distributed.

[0096] Supplementary note: Use the random library in Python to randomly divide the data set and disrupt the data order to ensure uniform data distribution. This division method is based on the premise that there is sufficient data. If the data volume is insufficient, it is recommended to use cross-validation to evaluate the model.

[0097] Refer to the attached Figure 5 , step S5 is as follows:

[0098] (1) Construct the basic architecture of the ConvNext model, including four convolutional layers. ConvNeXt does not directly use explicit pooling layers, but instead implements downsampling through Patchify and Strided Convolution (convolution operations with a stride of 2). At the beginning of each stage, the spatial resolution is halved through a 2D convolution with a stride of 2. In the standard implementation of ConvNeXt, there is no explicit Dropout layer. However, to prevent overfitting, DropPath (random path dropout) is set to control model regularization. The number of filters in the convolution layer is increased layer by layer to enhance feature extraction capabilities.

[0099] Specific parameters include: the number of convolution kernels in the four convolutional layers is 96, 192, 384, and 768, respectively; the convolution kernel size is 7x7; the activation function is ReLU; the size of the pooling layer is uniformly 2x2; and the dropout rate of the Dropout layer is set to 0.2;

[0100] (2) Before the model output layer, a flattening layer is used to convert the feature map into a one-dimensional vector so that it can be input into the fully connected layer;

[0101] (3) Before feeding into the classifier, add a fully connected layer to enhance the representation ability of the model and introduce nonlinear transformation. Add a dropout layer after the fully connected layer to reduce the risk of overfitting. The activation function uses relu;

[0102] (4) Select categorical_crossentropy as the loss function, use the Adma optimizer to accelerate convergence, and specify accuracy as the evaluation metric;

[0103] (5) EarlyStopping can be used to dynamically monitor the loss of the validation set when training the model to avoid overfitting.

[0104] (6) Using the softmax activation function, we can accurately classify the four types of tool wear faults: normal, light wear, moderate wear, and heavy wear. The softmax function converts the numerical value output by the model into the probability of belonging to each category. By comparing the probabilities, we can determine the current wear state of the tool.

[0105] Although blade faults such as cracks and chipping are not classified within the four wear state categories, they will still cause unique changes in the vibration and sound signals during the signal processing and feature extraction stages. After continuous wavelet transform and feature extraction, these unique signal characteristics are included in the fusion feature. During model training, samples with faults such as cracks and chipping are also included in the training set, allowing the model to learn the characteristic patterns corresponding to these faults. When the model makes a prediction, if the input signal characteristics match the characteristic patterns of these faults in the training set, it can be determined that the tool has the corresponding fault, thus achieving comprehensive diagnosis of various tool faults.

[0106] The above content is only for explaining the technical idea of the present invention and cannot be used to limit the protection scope of the present invention. Any changes made on the basis of the technical solution in accordance with the technical idea proposed by the present invention shall fall within the protection scope of the claims of the present invention.

Claims

1. A gear milling machine tool fault diagnosis method based on continuous wavelet transform and deep learning of multimodal signal fusion, characterized in that: The steps include: S1: Obtain the vibration signal and sound signal of the CNC gear milling tool through the acceleration sensor and noise sensor and pre-process the signals; S2: Use the continuous wavelet transform algorithm to convert the one-dimensional vibration signal and sound signal into a two-dimensional time-frequency image. The wavelet image of the vibration signal and the wavelet image of the sound signal are cropped to the same size and input into the convolutional neural network (CNN) model. S3: Feature extraction and feature fusion: Using a deep learning model, the convolutional neural network (CNN), to extract the time-frequency features of vibration and sound signals, the extracted features are then fed into a multimodal network, and the features are fused in the fusion layer using the attention mechanism. S4: Divide the dataset into training, test, and validation sets in a ratio of 7:2:

1. S5: Use the ConvNext model combined with the softmax classifier to complete the fault classification of CNC gear milling tools; S6: Input the test set into the trained model to verify whether the model is feasible.

2. The fault diagnosis method based on continuous wavelet transform and deep learning of multimodal signal fusion according to claim 1, characterized in that: In S2, the one-dimensional vibration signal and the sound signal are converted into a two-dimensional time-frequency image using a continuous wavelet transform algorithm, the wavelet graph of the vibration signal and the wavelet graph of the sound signal are cropped to the same size, and input into a convolutional neural network (CNN) model, which mainly includes the following steps: S2.1: Perform a continuous wavelet transform on the preprocessed vibration and sound signals. The continuous wavelet transform (CWT) is a technique used to decompose a continuous time function into several wavelets. The CWT can construct a time-frequency signal with good localization in both the time and frequency domains. The wavelet transform is used to decompose the signal into different frequency bands and obtain the signal's time-frequency localization characteristics. The CWT formula is: Where: x(t) is the vibration signal, is the complex conjugate form of the wavelet function, defined as: a is the scale parameter that controls the frequency resolution; b is the translation parameter that controls the time resolution; ψ(t) is the mother wavelet function; the result W(a,b) represents the components of the signal at different scales a and time b, forming a two-dimensional time-frequency diagram; S2.2: Use the imshow function to plot the time-frequency graph. Plot the transformation coefficients as a two-dimensional image, with the x-axis representing time and the y-axis representing frequency. The plotting method is as follows: Time-frequency plot = imshow(|W(a,b)|, extent = [t min ,t max ,f min ,f max ]); The extracted energy features, statistical features, spectral entropy, etc. are combined into a feature vector F = [E1, μ1, σ1, ..., H]; S2.3: Experiment with different scale ranges and color mappings to ensure that frequency features are clearly visible. S2.4: When higher resolution is required, increasing the number of scales can improve frequency resolution, but the computational effort will also increase.

3. The fault diagnosis method based on continuous wavelet transform and deep learning of multimodal signal fusion according to claim 1, characterized in that: In S3, the time-frequency image converted by S2 is input into a convolutional neural network, and the time-frequency features of the vibration signal and the sound signal are extracted using a deep learning model convolutional neural network CNN. The extracted features are then input into a multimodal network, and the features are fused in the fusion layer through the Attention mechanism. The main steps include: S3.1: Construct a CNN convolutional neural network. The CNN convolutional neural network module includes a convolution layer, a pooling layer, and a dropout layer. S3.2: Use convolutional neural networks to process vibration and sound signals, extracting complex features such as local, frequency band, and nonlinear characteristics through layer-by-layer convolution in the time-frequency domain of the signal; S3.3: Concatenate the feature vectors of the vibration and sound signals. Introducing the Attention layer in the fusion layer enables the model to assign different weights to each modal feature, improving the representation capability of the fused feature.

4. The method for fault diagnosis of gear milling tools based on continuous wavelet transform and deep learning of multimodal signal fusion according to claim 1, characterized in that: In S4, the data set is divided based on the result of S3, and the fused time-frequency graph data set is divided into a training set, a test set, and a validation set in a ratio of 7:2:

1. The main steps include: S4.1: Choose random partitioning and use the Knuth shuffle algorithm to shuffle the data. This algorithm shuffles the data by randomly selecting elements from the data and swapping their positions. Its time complexity is O(n), which ensures fairness. S4.2: Divide the data into a ratio of 7:2:1 based on the total size of the dataset; S4.3: Stratified sampling is used to ensure that the labels of each subset are evenly distributed.

5. The fault diagnosis method based on continuous wavelet transform and deep learning of multimodal signal fusion according to claim 1, characterized in that: In S5, the ConvNext model is used in combination with the softmax classifier to classify the faults of the CNC gear milling tool on the data set divided in S4; S5.1: Build the basic architecture of the ConvNext model by adding two convolutional layers, each followed by a pooling layer and a dropout layer. Gradually increase the number of filters in the convolutional layers to enhance feature extraction, while keeping other parameters such as kernel size and stride unchanged. S5.2: After convolution and pooling, use a flatten layer to convert the feature map into a one-dimensional vector for input to the fully connected layer; S5.3: After the flattening layer, add a fully connected layer to enhance the representation capability of the model and introduce nonlinear transformations. Add a dropout layer after the fully connected layer to reduce the risk of overfitting. S5.4: Select categorical_crossentropy as the loss function, use the Adma optimizer to accelerate convergence, and specify accuracy as the evaluation metric. S5.5: Use EarlyStopping to dynamically monitor the validation set loss when training the model to avoid overfitting. S5.6: Use the softmax activation function to accurately classify the four types of tool wear faults.

6. A CNC gear milling tool fault diagnosis system based on continuous wavelet transform and deep learning of multimodal signal fusion, characterized in that: include: Data acquisition system, used to collect vibration and sound data; A data preprocessing system is used to preprocess the collected data and convert the one-dimensional data into a time-frequency image; Feature extraction and fusion system, used to extract features from the transformed time-frequency graph and fuse multimodal features; State recognition system: Identify the state of tool wear by establishing a time-frequency graph model.

Images