Machine tool feeding shaft running state monitoring method and edge deployment method and system based on multi-modal fusion

Abstract

The application belongs to the technical field of numerical control machine tool state monitoring and fault diagnosis, and discloses a machine tool feeding shaft running state monitoring method and edge deployment method and system based on multi-modal fusion. The power, vibration and temperature signals of the feeding shaft are synchronously collected by multi-modal sensors, and a sample set is constructed after preprocessing; a multi-modal deep neural network model composed of a feature extraction sub-network, a bidirectional gate recurrent unit and a single-layer long short-term memory network is used, and multi-head attention mechanism is combined to realize dynamic weighted fusion of cross-modal features; the multi-modal deep neural network model is deployed on an edge device after pruning and quantization optimization, real-time inference is realized, and online learning update is supported; the application solves the problems of single sensor mode, artificial feature extraction and poor model real-time in the prior art, significantly improves the abnormal recognition accuracy, robustness and industrial field adaptability, and can effectively identify early weak abnormalities and complex faults of the feeding shaft.

Description

Technical Field

[0001] This invention belongs to the field of CNC machine tool condition monitoring and fault diagnosis technology, and relates to a machine tool feed axis running status monitoring method and edge deployment method and system based on multimodal fusion. Background Technology

[0002] As a core moving component in precision machining, the feed axis of a CNC machine tool directly affects machining accuracy, production efficiency, and equipment safety. Currently, methods for monitoring the operating status of feed axes are mainly divided into three categories:

[0003] Traditional monitoring methods rely on rules or fixed thresholds. These methods preset fixed thresholds such as servo motor current, vibration signal amplitude, or temperature rise limits, and determine a fault when the monitored signal exceeds the threshold range. However, due to the complex and variable operating conditions of machine tools, fixed thresholds are difficult to adapt to the dynamically changing operating environment, resulting in insufficient ability to identify early and subtle anomalies, and are prone to false alarms and missed alarms under complex operating conditions.

[0004] Traditional machine learning methods, such as Support Vector Machines and Random Forests, have been attempted for feed axis status monitoring, but they heavily rely on manual feature extraction. Technicians need to extract static indicators such as peak value, mean, and kurtosis from signals based on experience, making it difficult to capture the potential relationships between multi-source signals. For example, feed axis jamming may simultaneously cause a sudden increase in current, a shift in the vibration spectrum, and a rise in temperature. Manually designed single features cannot fully characterize such abnormal patterns of multi-physics correlation, resulting in poor model generalization ability and difficulty in adapting to diverse processing scenarios.

[0005] Deep learning methods have emerged in recent years. While deep learning technology has been introduced into this field, it has significant limitations: on the one hand, most solutions rely on a single signal source, neglecting the value of multimodal signals collaboratively reflecting equipment status; on the other hand, network structure design is relatively basic, failing to effectively model the interaction patterns between multimodal signals and the long-term dependencies of time series, thus failing to fully exploit the dynamic evolution characteristics inherent in the signals. Furthermore, existing deep learning models mostly adopt offline training and deployment modes, resulting in high computational costs and poor real-time performance, making it difficult to meet the needs of real-time early warning and rapid edge response for machine tool feed axis operation status in industrial settings.

[0006] More importantly, existing monitoring methods are mostly limited to binary state judgments of "normal" and "abnormal," failing to quantify the degree of equipment performance degradation or effectively identify and warn of early stages such as "preliminary degradation." This results in a lack of tiered maintenance decisions, making it impossible to support advanced equipment management strategies centered on "predictive maintenance."

[0007] In summary, existing technologies still have significant shortcomings in the detection of the operating status of CNC machine tool feed axes. This invention proposes a machine tool feed axis operating status monitoring method and edge deployment system based on multimodal fusion to solve the above problems. Summary of the Invention

[0008] To overcome the aforementioned deficiencies of the prior art, this invention provides a machine tool feed axis operation status monitoring method and edge deployment system based on multimodal fusion, in order to solve the problems existing in the background art.

[0009] The technical solution of this invention:

[0010] A method for monitoring the operating status of machine tool feed axes and an edge deployment method based on multimodal fusion, comprising the following steps:

[0011] S1. Multimodal Operation Status Signal Acquisition: Power, vibration, and temperature signals of the feed axis are simultaneously acquired via a power sensor, a triaxial accelerometer, and a temperature sensor. The synchronous acquisition module synchronizes and acquires these signals, storing them as multimodal operation status signals in N×T tensor format. N is the number of sensor channels, and T is the number of sampling points within each sampling time. The three-dimensional acceleration sensor and temperature sensor are respectively arranged on the bearing housing near the motor end and the bearing housing far from the motor end of the feed axis, and the power sensor is connected to the servo driver of the feed axis.

[0012] S2. Signal Preprocessing and Sample Construction: Processing the acquired multimodal operating state signals Channel-by-channel preprocessing is performed, and low-pass filters are used to filter both the power and vibration signals. For the vibration signal, a signal kurtosis-based preprocessing method is applied. An improved wavelet denoising algorithm with dynamic adjustment is used for noise reduction processing, where... Represents the mathematical expectation. Represents a vibration signal sequence. The mean of the vibration signal sequence is denoted as . The standard deviation of the vibration signal sequence; for multimodal operating state signals Perform Z-score standardization , to obtain the matrix Samples are generated using a sliding time window, where the length of the sliding time window is given. The sliding step size is S Then the first k The sample is ,in k For sample index, ; Indicates from the first kS The sampling point starts with a cut-off length of... The multimodal operating status signal sequence; based on historical maintenance records or machine tool self-diagnostic system outputs labeled with categories such as normal state, slight degradation state, moderate degradation state, and severe degradation state, a sample set is constructed. ,in For the first i The category labels for each sample are divided according to the physical state of the feed axis and the machining accuracy requirements: normal state where the feed axis components are in good working order and the machining accuracy meets the set standards; slight degradation state where the feed axis components have early wear but the machining accuracy has not yet exceeded the tolerance; moderate degradation state where the performance of the feed axis components has significantly decreased and there is a risk of machining accuracy exceeding the tolerance; and severe degradation state where the feed axis components have failed or cannot guarantee safe operation.

[0013] S3. Construction of Multimodal Deep Neural Network Model: A multimodal deep neural network model is used to model and fuse the multimodal operating state signals separately. First, the signal obtained in step S2 is... k Sample Divided into power signal sequences Vibration signal sequence and temperature signal sequence ; power signal sequence The input consists of a feature extraction subnetwork with two one-dimensional convolutional layers, and the input is processed by convolution operations. To extract power fluctuation features, where the input of layer 0 is defined. , l Representation layer index, For the first l Feature mapping of layers, and The first l The convolutional kernel weights and bias parameters of the layer, ReLU is a linear rectified activation function, and the final pooling is the feature vector. ; Vibration signal sequence Input to a bidirectional gated recurrent unit (Bi-GRU) and through a local attention mechanism Weighting is used to extract the characteristics of periodic impacts and trajectory flutter, where, and These are the time intervals of the bidirectional gated recurrent unit (Bi-GRU). t Based on vibration signal input The calculated forward hidden state and backward hidden state, a t This represents the attention weight at time t. This is the transpose of the weight vector for the local attention mechanism. For normalized exponential functions, The function is a hyperbolic tangent activation function, W is a learnable parameter of the local attention mechanism, and the weighted output is a feature vector. ; temperature signal sequence A single-layer Long Short-Term Memory (LSTM) network is input to analyze the heat accumulation process and overload trend. After processing by a global average pooling layer, the output feature vector is determined. Concatenate the three feature vectors into Then, the input is subjected to adaptive weighted fusion using a multi-head attention mechanism to obtain the fused feature vector. Finally, the feature vectors are fused. The probability distribution of the output feed axis belonging to the normal state, slightly degraded state, moderately degraded state, and severely degraded state is obtained by using a fully connected layer and a softmax activation function. ,in p c Indicates that the sample belongs to the category c The predicted probability, and satisfying ;

[0014] S4. Training and Evaluation of Multimodal Deep Neural Network Model: The constructed sample set is divided into a training set, a validation set, and a test set. The multimodal deep neural network model is iteratively trained using the training set, and the cross-entropy loss function between the predicted probability distribution and the true label is calculated. ,in, L This represents the total number of samples in the training batch. For the sample k True label one-hot encoding, For multimodal deep neural network models, samples k Category c The predicted probability is obtained; the optimizer updates the parameters of the multimodal deep neural network model based on the gradient of the cross-entropy loss function, and the EarlyStopping mechanism monitors the validation set loss to prevent overfitting.

[0015] S5. Deployment and Online Application of Multimodal Deep Neural Network Model: The trained multimodal deep neural network model is compressed and converted in format, and then deployed to an edge computing device. The edge computing device is connected to the machine tool controller through a communication interface to achieve synchronous status feedback. Multimodal operating status signals are collected in real time. The multimodal operating status signals are preprocessed according to step S2, and the processed data is input into the multimodal deep neural network model in step S3 for inference calculation. When the probability of moderate or severe degradation in the output prediction probability distribution exceeds the preset threshold, the audible and visual alarm module and the human-machine interface are triggered, and the data is uploaded to the remote management system.

[0016] It supports online learning and regularly updates the multimodal deep neural network model using newly labeled data and a parameter-regularized incremental learning algorithm to adapt to machine tool performance drift.

[0017] Furthermore, the specific parameter configuration of the multimodal deep neural network model described in step S3 is as follows: In the feature extraction sub-network, the first one-dimensional convolutional layer has 32 convolutional kernels, a kernel size of 16, and a stride of 2; the second one-dimensional convolutional layer has 64 convolutional kernels, a kernel size of 8, and a stride of 1, and the output feature vector... The dimension is 64; in the Bi-GRU bidirectional gated recurrent unit, the number of hidden units in the Bi-GRU layer is 64, the time window length of the local attention mechanism is 200ms, and the output feature vector is... The dimension is 128; in the Long Short-Term Memory (LSTM) network, the number of hidden units in the LSTM layer is 32, and the output feature vector is... The dimension is 32; the fused feature vector is obtained by concatenating the three feature vectors. The dimension is 224; the multi-head attention mechanism contains three parallel attention heads, which focus on a time scale of 0.1s for capturing short-term transient features, a 1s for capturing medium-term trend features, and a 2s for capturing long-term correlation features.

[0018] Furthermore, the sample set mentioned in step S4 is randomly divided into a training set, a test set, and a validation set in a 6:2:2 ratio; the optimizer used is the Adam optimizer, and the initial learning rate is set to... The maximum number of training rounds is set to 60.

[0019] Furthermore, the model compression and format conversion in step S5 specifically involves: pruning and quantizing the multimodal deep neural network model, and converting it to TensorRT format; the communication interface is an industrial Ethernet interface, and the MODBUS communication protocol is used; the parameter regularization incremental learning algorithm specifically adopts the elastic weight consolidation algorithm, and its loss function is... ,in, The cross-entropy loss function is calculated based on the new batch of sample data in the current update cycle. These are the parameters of the multimodal deep neural network model that need to be updated. The optimal parameter values ​​retained after the multimodal deep neural network model converges on historical sample data represent the multimodal deep neural network model's memory of historical machine tool operating state characteristics; For parameters The diagonal elements of the Fisher information matrix are used to characterize the importance of this parameter in maintaining the accuracy of historical state recognition. These are hyperparameters used to weigh the importance of maintaining the performance of a multimodal deep neural network model on historical data against fitting new data.

[0020] An edge deployment system for monitoring the operating status of machine tool feed axes based on multimodal fusion is configured to execute the aforementioned method for monitoring the operating status of machine tool feed axes based on multimodal fusion and the edge deployment method. The edge deployment system for monitoring the operating status of machine tool feed axes based on multimodal fusion includes:

[0021] The data acquisition module executes step S1 to synchronously acquire the power signal, vibration signal, and temperature signal of the feed axis;

[0022] The preprocessing and sample construction module executes step S2, which performs low-pass filtering, noise reduction, standardization, and sample partitioning on the collected multimodal operating status signals, and constructs a labeled sample set.

[0023] In the model building and inference module, step S3 is executed, which uses a multimodal deep neural network model to extract features from the input signal sequence and fuse multi-head attention, and calculates the probability distribution of the running state of the feed axis.

[0024] The model training module executes step S4, which uses the training set data to iteratively train and optimize the parameters of the multimodal deep neural network model.

[0025] The online monitoring and update module executes step S5, which deploys the trained multimodal deep neural network model to the edge device for real-time inference and performs incremental learning and updates based on new data.

[0026] The application interaction module executes step S5, triggering an alarm and human-machine interface prompts when an anomaly is detected, and performing remote data archiving.

[0027] Furthermore, the hardware architecture of the machine tool feed axis operation status monitoring edge deployment system based on multimodal fusion includes:

[0028] The multimodal sensing layer, corresponding to the data acquisition module, includes a power sensor connected to the servo driver, a triaxial acceleration sensor installed in the bearing housing near the motor end and the bearing housing far from the motor end, a temperature sensor installed in the bearing housing near the motor end and the bearing housing far from the motor end, and a high-precision synchronous acquisition module supporting CAN communication.

[0029] The edge computing layer, corresponding to the preprocessing and sample construction module and the model construction and inference module, includes industrial edge devices, industrial Ethernet and model inference engine. The model inference engine supports TensorRT real-time inference and dynamic loading of multimodal deep neural network models. Moreover, the preprocessing and sample construction module is integrated inside the industrial edge device to perform signal filtering, denoising, standardization processing and sample partitioning and sample set construction.

[0030] The intelligent analysis layer, corresponding to the model training module and the online monitoring and update module, includes a data buffer pool, an incremental training unit, and a model evaluation unit. The data buffer pool is used to filter and store online collected samples; the incremental training unit is used to perform online incremental training and parameter optimization based on the elastic weight consolidation algorithm to adapt to machine tool performance drift; the model evaluation unit is used to evaluate the performance of new parameters before updating the multimodal deep neural network model, and only performs online updates of model weights when the performance index improves.

[0031] The application interaction layer, corresponding to the application interaction module, consists of a human-machine interface, an audible and visual alarm module, and a remote management system interface. It supports visualization of abnormal states, alarm prompts, and remote management archiving.

[0032] Furthermore, the data acquisition module of the multimodal sensing layer adopts a timestamp calibration mechanism to ensure that the time synchronization error of power signal, vibration signal, and temperature signal is ≤1ms.

[0033] Furthermore, the edge computing layer also includes a fault self-diagnosis unit, which monitors the sensor communication status and the inference performance of the multimodal deep neural network model in real time, and automatically switches to the backup sensor group when an anomaly is detected.

[0034] The beneficial effects of this invention are:

[0035] This invention breaks through the limitations of traditional methods that rely solely on a single modal signal. By fusing multi-source signals of power, vibration, and temperature, and employing a multi-head attention mechanism to dynamically weight different signal features, it can more comprehensively capture the multi-physics coupling characteristics under abnormal feed axis conditions. This mechanism significantly improves the identification accuracy of complex faults and minor anomalies, and has higher robustness compared to single-modal methods. It can effectively cope with the complex and ever-changing machining conditions of machine tools and reduce false alarms and missed alarms.

[0036] This invention employs a multi-layered network structure consisting of a deep convolutional network, a bidirectional gated recurrent unit, and a long short-term memory neural network. This structure can specifically mine the time dependence and nonlinear features of different modal signals. Compared with traditional classification methods based on static features, this structure can more accurately capture the dynamic evolution trend in the signal, achieve a sensitive response to the gradually changing state during the feed axis machining process, and improve the ability to identify early and subtle anomalies.

[0037] This invention significantly reduces the computational burden of multimodal deep neural network models by pruning and quantizing them, enabling low-latency real-time inference on industrial edge devices. Compared with solutions that rely solely on cloud computing, this deployment method is more adaptable to the high real-time requirements of industrial sites, with faster monitoring response and higher reliability. It can achieve real-time monitoring and rapid feedback of abnormal feed axis states, ensuring the stability and safety of the machining process. Attached Figure Description

[0038] Figure 1 This is a flowchart of the machine tool feed axis running status monitoring method based on multimodal fusion according to the present invention;

[0039] Figure 2 This is a simplified block diagram of the edge deployment system for monitoring the operating status of machine tool feed axes based on multimodal fusion, as described in this invention.

[0040] Figure 3 This is a simplified architecture diagram of the multimodal deep neural network model of this invention. Detailed Implementation

[0041] The specific embodiments of the present invention will be further described below with reference to the accompanying drawings and technical solutions.

[0042] Please see Figure 1 As shown, the machine tool feed axis running status monitoring method and edge deployment method based on multimodal fusion include the following steps:

[0043] S1. Multimodal Operation Status Signal Acquisition: Power, vibration, and temperature signals of the feed axis are simultaneously acquired via a power sensor, a triaxial accelerometer, and a temperature sensor. The synchronous acquisition module synchronizes and acquires these signals, storing them as multimodal operation status signals in N×T tensor format. N is the number of sensor channels, and T is the number of sampling points within each sampling time. The three-dimensional acceleration sensor and temperature sensor are respectively arranged on the bearing housing near the motor end and the bearing housing far from the motor end of the feed axis, and the power sensor is connected to the servo driver of the feed axis.

[0044] In practical implementation, the Z-axis feed system of a vertical machining center is taken as the monitoring object. The feed axis is driven by a ball screw by a servo motor to realize the vertical movement of the spindle box.

[0045] Power signal: Acquired in real time by a power sensor, including current and voltage signals, with a sampling frequency of 2000Hz;

[0046] Vibration signal: Install one triaxial acceleration sensor in the bearing housing near the motor end and one in the bearing housing far from the motor end to collect vibration acceleration in the X, Y, and Z directions at a sampling frequency of 2000Hz;

[0047] Temperature signal: Temperature sensors are installed on the motor housing and the bearing housing near and far from the motor end to collect the operating temperature of the motor and bearing housing at a sampling frequency of 2000Hz.

[0048] Power signals, vibration signals, and temperature signals are input to the synchronous acquisition module, and time synchronization is achieved through hardware triggering. Finally, they are stored as an N×T tensor, where N=10 channels, including 2 power channels, 6 vibration channels, and 2 temperature channels; T=4000 points, corresponding to a 2-second sampling duration.

[0049] S2. Signal Preprocessing and Sample Construction: Processing the acquired multimodal operating state signals Channel-by-channel preprocessing is performed, and low-pass filters are used to filter both the power and vibration signals. For the vibration signal, a signal kurtosis-based preprocessing method is applied. An improved wavelet denoising algorithm with dynamic adjustment is used for noise reduction processing, where... Represents the mathematical expectation. Represents a vibration signal sequence. The mean of the vibration signal sequence is denoted as . The standard deviation of the vibration signal sequence; for multimodal operating state signals Perform Z-score standardization , to obtain the matrix Samples are generated using a sliding time window, where the length of the sliding time window is given. The sliding step size is S Then the first k The sample is ,in k For sample index, ; Indicates from the first kS The sampling point starts with a cut-off length of... The multimodal operating status signal sequence; based on historical maintenance records or machine tool self-diagnostic system outputs labeled with categories such as normal state, slight degradation state, moderate degradation state, and severe degradation state, a sample set is constructed. ,in For the first i The category labels for each sample are divided according to the physical state of the feed axis and the machining accuracy requirements: normal state where the feed axis components are in good working order and the machining accuracy meets the set standards; slight degradation state where the feed axis components have early wear but the machining accuracy has not yet exceeded the tolerance; moderate degradation state where the performance of the feed axis components has significantly decreased and there is a risk of machining accuracy exceeding the tolerance; and severe degradation state where the feed axis components have failed or cannot guarantee safe operation.

[0050] In practical implementation, noise reduction is performed as follows: low-pass filters are used to filter power signals and vibration signals, while temperature signals are not filtered and are only normalized using Z-score.

[0051] The formula is:

[0052] (1)

[0053] Sample generation: The signal is segmented using a sliding time window with a window length of 2 seconds and a step size of 0.5 seconds. Each data segment is treated as a sample.

[0054] Labeling: Combining fault codes from the machine tool's self-diagnostic system, such as overload alarms, position deviation exceeding limits, and historical maintenance records, samples are labeled as normal, slightly degraded, moderately degraded, and severely degraded states, constructing a total sample set of 10,000, including 6,000 samples in the normal state, 2,000 samples in the slightly degraded state, 1,200 samples in the moderately degraded state, and 800 samples in the severely degraded state. The specific sample distribution can be adjusted according to the actual working conditions.

[0055] S3. Construction of Multimodal Deep Neural Network Model: A multimodal deep neural network model is used to model and fuse the multimodal operating state signals separately. First, the signal obtained in step S2 is... k Sample Divided into power signal sequences Vibration signal sequence and temperature signal sequence ; power signal sequence The input consists of a feature extraction subnetwork with two one-dimensional convolutional layers, and the input is processed by convolution operations. To extract power fluctuation features, where the input of layer 0 is defined. , l Representation layer index, For the first l Feature mapping of layers, and The first l The convolutional kernel weights and bias parameters of the layer, ReLU is a linear rectified activation function, and the final pooling is the feature vector. ; Vibration signal sequence Input to a bidirectional gated recurrent unit (Bi-GRU) and through a local attention mechanism Weighting is used to extract the characteristics of periodic impacts and trajectory flutter, where, and These are the time intervals of the bidirectional gated recurrent unit (Bi-GRU). t Based on vibration signal input The calculated forward hidden state and backward hidden state, a t This represents the attention weight at time t. This is the transpose of the weight vector for the local attention mechanism. For normalized exponential functions, The function is a hyperbolic tangent activation function, W is a learnable parameter of the local attention mechanism, and the weighted output is a feature vector. ; temperature signal sequence A single-layer Long Short-Term Memory (LSTM) network is input to analyze the heat accumulation process and overload trend. After processing by a global average pooling layer, the output feature vector is determined. Concatenate the three feature vectors into Then, the input is subjected to adaptive weighted fusion using a multi-head attention mechanism to obtain the fused feature vector. Finally, the feature vectors are fused. The probability distribution of the output feed axis belonging to the normal state, slightly degraded state, moderately degraded state, and severely degraded state is obtained by using a fully connected layer and a softmax activation function. ,in p c Indicates that the sample belongs to the category c The predicted probability, and satisfying ;

[0056] In practical implementation, the feature extraction subnetwork: The input two-dimensional power signal is processed through two layers of one-dimensional convolution to extract short-period power fluctuation features. The first one-dimensional convolutional layer has 32 kernels, a kernel size of 16, and a stride of 2. The second one-dimensional convolutional layer has 64 kernels, a kernel size of 8, and a stride of 1, outputting a feature vector. The dimension is 64.

[0057] Bi-Gated Cyclic Unit (Bi-GRU): Input a six-dimensional vibration signal, connect it to a bi-GRU network, hide 64 units, and use 2 layers to extract periodic impact and trajectory flutter features;

[0058] Long Short-Term Memory (LSTM) Network: Input a two-dimensional temperature signal and connect it to a Long Short-Term Memory (LSTM) network with 32 hidden units to analyze heat accumulation trends, such as temperature rise caused by continuous overload.

[0059] The splicing and fusion process includes 64-dimensional power, 128-dimensional vibration, and 32-dimensional temperature, which are spliced ​​into a 224-dimensional vector. Weights are dynamically allocated through a multi-head attention mechanism.

[0060] S4. Training and Evaluation of Multimodal Deep Neural Network Model: The constructed sample set is divided into a training set, a validation set, and a test set. The multimodal deep neural network model is iteratively trained using the training set, and the cross-entropy loss function between the predicted probability distribution and the true label is calculated. ,in, L This represents the total number of samples in the training batch. For the sample k True label one-hot encoding, For multimodal deep neural network models, samples k Category cThe predicted probability is obtained; the optimizer updates the parameters of the multimodal deep neural network model based on the gradient of the cross-entropy loss function, and the EarlyStopping mechanism monitors the validation set loss to prevent overfitting.

[0061] In practice, the training configuration is as follows: The TensorFlow framework is used, with the training, test, and validation sets divided in a 6:2:2 ratio. The loss function is cross-entropy, the optimizer is Adam, and the initial learning rate is... The batch size is 128, and the number of training rounds is 60.

[0062] Overfitting prevention: If the validation set loss does not decrease for 5 consecutive rounds, the Early Stopping mechanism is triggered to terminate training.

[0063] S5. Deployment and Online Application of Multimodal Deep Neural Network Model: The trained multimodal deep neural network model is compressed and converted in format, and then deployed to an edge computing device. The edge computing device is connected to the machine tool controller through a communication interface to achieve synchronous status feedback. Multimodal operating status signals are collected in real time. The multimodal operating status signals are preprocessed according to step S2, and the processed data is input into the multimodal deep neural network model in step S3 for inference calculation. When the probability of moderate or severe degradation in the output prediction probability distribution exceeds the preset threshold, the audible and visual alarm module and the human-machine interface are triggered, and the data is uploaded to the remote management system.

[0064] It supports online learning and regularly updates the multimodal deep neural network model using newly labeled data and a parameter-regularized incremental learning algorithm to adapt to machine tool performance drift.

[0065] In practice, model optimization involves pruning the trained multimodal deep neural network model, removing 10% of redundant parameters, performing INT8 quantization, and compressing the size of the multimodal deep neural network model.

[0066] Edge deployment: Converted to TensorRT format and deployed on industrial edge devices, it communicates with machine tool controllers via industrial Ethernet to achieve real-time inference;

[0067] Abnormal response: When the probability of moderate degradation and severe degradation exceeds the preset threshold, the audible and visual alarm module on the machine tool operation panel is triggered, and the abnormal data is uploaded to the remote management system for archiving.

[0068] As a further aspect of the present invention: in step S2, an improved wavelet threshold denoising algorithm is used on the vibration signal. By introducing the signal kurtosis value to dynamically adjust the threshold function, the impact vibration component is preserved.

[0069] The specific implementation is as follows:

[0070] Wavelet thresholding was used to denoise the vibration signal, with the db5 wavelet basis selected and a decomposition layer of 6 layers.

[0071] The formula for calculating the kurtosis value of a vibration signal is:

[0072] (2)

[0073] in: For the expectation, x This represents a preprocessed vibration signal sequence. The mean of the vibration signal sequence is denoted as . denoted as the standard deviation of the vibration signal sequence.

[0074] When kurtosis K When the threshold is ≤3 (close to a normal distribution, with no significant shock), a soft threshold function is used:

[0075] (3)

[0076] in: These are the wavelet decomposition coefficients. These are the coefficients after thresholding. For adaptive threshold, 1 represents the noise standard deviation estimated based on wavelet coefficients, and N represents the number of sampling points.

[0077] When kurtosis K When the threshold value is greater than 3, a hard threshold function is used to preserve the impact characteristics.

[0078] (4)

[0079] in: For indicator functions, when the condition The value is 1 when it is true, and 0 otherwise.

[0080] As a further aspect of the present invention: In step S3, the bidirectional gated cyclic unit Bi-GRU adds a local timing attention mechanism at the output end of the hidden layer to focus on capturing the periodic impact characteristics in the vibration signal.

[0081] The specific implementation is as follows:

[0082] A local timing attention layer is added at the hidden layer output of the bidirectional gated recurrent unit Bi-GRU, and the attention window size is set to 200ms.

[0083] For each moment within the window t Calculate its attention weights a t The formula is:

[0084]

[0085] in: For learnable parameters, d a For the attention dimension, d h Hidden state dimension for Bi-GRU and Bi-GRU, a bidirectional gated recirculation unit, is located in... t The forward and backward hidden states are obtained based on the vibration signal input at any time. This indicates a vector concatenation operation.

[0086] As a further aspect of the present invention: In step S3, the multi-head attention mechanism includes three parallel attention heads, which focus on short-term transient features, medium-term trend features and long-term correlation features respectively, and the features are spliced ​​after the weights are normalized by the Softmax function.

[0087] The specific implementation is as follows:

[0088] Configuration of three attention heads:

[0089] First 1 (Short Transient): Attention time window of 0.1s (200 sampling points), focusing on transient signal changes, such as sudden power jumps that stop abruptly;

[0090] First 2 (intermediate trend): Attention time window 1s (2000 sampling points), focusing on the trend of signal change, such as the gradual increase of vibration amplitude;

[0091] First 3 (long-term correlation): Attention time window of 2s (4000 sampling points), focusing on the coordinated changes of multimodal signals, such as the correlation between power increase and temperature increase;

[0092] For each attention head, given the input feature set { f 1, f 2, …}, First, calculate the attention score for each feature fi. ,in u1 , W1 , b1 These are learnable parameters specific to this attention focus. Then, the scores are normalized using the Softmax function to obtain the corresponding weights:

[0093] (6)

[0094] Finally, each feature f i According to their weight The weighted sums are then combined, and the weighted results of different attention points are concatenated to form the final fusion feature.

[0095] As a further aspect of the present invention: In step S5, after the multimodal deep neural network model is deployed, it supports online learning and adopts the elastic weight consolidation (EWC) algorithm to update the model parameters based on the new labeled data to retain historical knowledge and integrate new features. The model update cycle is set to 24 hours.

[0096] The specific implementation is as follows:

[0097] Data collection: Edge devices cache newly added samples daily, and incremental learning is triggered when the number of samples is ≥50;

[0098] EWC Algorithm Implementation:

[0099] When training with new samples, a regularization term is added to the loss function to protect key parameters. The formula is as follows:

[0100] (7)

[0101] Please see Figure 2 As shown, the edge deployment system for monitoring the operating status of machine tool feed axes based on multimodal fusion includes:

[0102] Multimodal sensing layer: Composed of a power sensor connected to the servo driver, a triaxial acceleration sensor installed in the bearing housing near the motor end and the bearing housing at the far motor end, a temperature sensor installed in the bearing housing near the motor end and the bearing housing at the far motor end, and a synchronous acquisition module supporting CAN communication, used to synchronously acquire multimodal operating status signals; among them, the high-precision synchronous acquisition module supports 8 channels of analog input, a sampling rate of 2000Hz, and is calibrated by GPS timestamp to ensure that the synchronization error of multiple signals is ≤1ms.

[0103] The edge computing layer, corresponding to the preprocessing and sample construction module and the model construction and inference module, includes industrial edge devices, industrial Ethernet and model inference engine. The model inference engine supports TensorRT real-time inference and dynamic loading of multimodal deep neural network models. Moreover, the preprocessing and sample construction module is integrated inside the industrial edge device to perform signal filtering, denoising, standardization processing and sample partitioning and sample set construction.

[0104] Industrial edge devices: Select industrial control computers equipped with GPUs;

[0105] Industrial Ethernet interface: Uses MODBUS TCP protocol to communicate with machine tool PLC;

[0106] Model inference engine: Integrates TensorRT SDK, supports dynamic model loading, and can switch between dedicated models with different feed axes.

[0107] Intelligent Analysis Layer: Includes a data buffer pool, incremental training unit, and model evaluation unit, used for training, optimizing, and updating multimodal deep neural network models online;

[0108] Multimodal deep neural network model: GPU-accelerated unit deployed on edge devices, inference latency ≤20ms;

[0109] The application interaction layer, corresponding to the application interaction module, consists of a human-machine interface, an audible and visual alarm module, and a remote management system interface. It supports visualization of abnormal states, alarm prompts, and remote management archiving.

[0110] Human-machine interface: 10.1-inch touch screen, displaying real-time signal waveforms of power, vibration, and temperature, as well as the operating status of the machine tool feed axes;

[0111] Audible and visual alarm module: connected to the machine tool column warning light or installed independently, alarm volume ≥85dB, light flashing frequency 2Hz;

[0112] Remote management system interface: Data is uploaded via OPC UA protocol, supporting integration with ERP system.

[0113] As a further aspect of the present invention: the synchronous acquisition module of the multimodal sensing layer adopts a timestamp calibration mechanism to ensure that the time synchronization error of power, vibration, and temperature signals is ≤1ms;

[0114] The specific implementation is as follows:

[0115] The synchronous acquisition module has a built-in GPS timing module that generates a global timestamp every 1ms, and each sensor signal is appended with this timestamp when it is received. After receiving the data, the edge device corrects the transmission delay through a timestamp alignment algorithm to ensure the consistency of power, vibration and temperature signals on the time axis.

[0116] As a further aspect of the present invention, the edge computing layer also includes a fault self-diagnosis unit, which can monitor the sensor communication status and model inference performance in real time, and automatically switch to the backup sensor group when an anomaly is detected.

[0117] The specific implementation is as follows:

[0118] The fault self-diagnosis unit monitors in real time:

[0119] Sensor communication status: Detected via CAN bus heartbeat packets; a timeout of ≥500ms is considered a fault.

[0120] Model inference performance: An inference delay greater than 50ms is considered abnormal for 10 consecutive times.

[0121] Backup sensor group: Same model as the main sensor, installed in a redundant position, and normally in low-power standby mode;

[0122] Switching mechanism: When a main sensor failure is detected, the system will automatically switch to the backup sensor within 50ms, and the HMI will display a message saying "Sensor switching, check main sensor".

[0123] In this invention, the sampling frequency of multimodal signal acquisition can be adjusted to 1000Hz~20000Hz according to the machine tool running speed; the window length and step size of the sliding time window can be shortened to 1s and the step size adjusted to 0.2s for high-speed machining scenarios; the order of the Butterworth filter in the preprocessing process can be adjusted in the range of 3 to 8; the wavelet basis selection and decomposition layer number for wavelet denoising; and structural parameters such as the convolution kernel size of the multimodal deep neural network, the number of hidden units of Bi-GRU / LSTM, and the number of attention heads, as well as hyperparameters such as the learning rate and number of training rounds of the model training, and the optimization parameters of the multimodal deep neural network model deployed at the edge are all adjustable parameterized settings.

Claims

1. A method for monitoring the operating status of machine tool feed axes and an edge deployment method based on multimodal fusion, characterized in that, Includes the following steps: S1. Multimodal Operation Status Signal Acquisition: Power, vibration, and temperature signals of the feed axis are simultaneously acquired via a power sensor, a triaxial accelerometer, and a temperature sensor. The synchronous acquisition module synchronizes and acquires these signals, storing them as multimodal operation status signals in N×T tensor format. N is the number of sensor channels, and T is the number of sampling points within each sampling duration. The triaxial acceleration sensor and temperature sensor are respectively arranged on the near motor end bearing housing and the far motor end bearing housing of the feed axis, and the power sensor is connected to the servo driver of the feed axis. S2. Signal Preprocessing and Sample Construction: Processing the acquired multimodal operating state signals Channel-by-channel preprocessing is performed, and low-pass filters are used to filter both the power and vibration signals. For the vibration signal, a signal kurtosis-based preprocessing method is applied. An improved wavelet denoising algorithm with dynamic adjustment is used for noise reduction processing, where... Represents the mathematical expectation. Represents a vibration signal sequence. The mean of the vibration signal sequence is denoted as . The standard deviation of the vibration signal sequence; Multimodal operating status signals Perform Z-score standardization , to obtain the matrix Samples are generated using a sliding time window, where the length of the sliding time window is given. The sliding step size is S Then the first k The sample is ,in k For sample index, ; Indicates from the first kS The sampling point starts with a cut-off length of... The multimodal operating status signal sequence; based on historical maintenance records or machine tool self-diagnostic system outputs labeled with categories such as normal state, slight degradation state, moderate degradation state, and severe degradation state, a sample set is constructed. ,in For the first i The category labels for each sample are divided according to the physical state of the feed axis and the machining accuracy requirements: normal state where the feed axis components are in good working order and the machining accuracy meets the set standards; slight degradation state where the feed axis components have early wear but the machining accuracy has not yet exceeded the tolerance; moderate degradation state where the performance of the feed axis components has significantly decreased and there is a risk of machining accuracy exceeding the tolerance; and severe degradation state where the feed axis components have failed or cannot guarantee safe operation. S3. Construction of Multimodal Deep Neural Network Model: A multimodal deep neural network model is used to model and fuse the multimodal operating state signals separately. First, the signal obtained in step S2 is... k Sample Divided into power signal sequences Vibration signal sequence and temperature signal sequence ; power signal sequence The input consists of a feature extraction subnetwork with two one-dimensional convolutional layers, and the input is processed by convolution operations. To extract power fluctuation features, where the input of layer 0 is defined. , l Representation layer index, For the first l Feature mapping of layers, and The first l The convolutional kernel weights and bias parameters of the layer, ReLU is a linear rectified activation function, and the final pooling is the feature vector. ; Vibration signal sequence Input to a bidirectional gated recurrent unit (Bi-GRU) and through a local attention mechanism Weighting is used to extract the characteristics of periodic impacts and trajectory flutter, where, and These are the time intervals of the bidirectional gated recurrent unit (Bi-GRU). t Based on vibration signal input The calculated forward hidden state and backward hidden state, a t This represents the attention weight at time t. This is the transpose of the weight vector for the local attention mechanism. For normalized exponential functions, The function is a hyperbolic tangent activation function, W is a learnable parameter of the local attention mechanism, and the weighted output is a feature vector. ; temperature signal sequence A single-layer Long Short-Term Memory (LSTM) network is input to analyze the heat accumulation process and overload trend. After processing by a global average pooling layer, the output feature vector is determined. Concatenate the three feature vectors into Then, the input is subjected to adaptive weighted fusion using a multi-head attention mechanism to obtain the fused feature vector. Finally, the feature vectors are fused. The probability distribution of the output feed axis belonging to the normal state, slightly degraded state, moderately degraded state, and severely degraded state is obtained by using a fully connected layer and a softmax activation function. ,in p c Indicates that the sample belongs to the category c The predicted probability, and satisfying ; S4. Training and Evaluation of Multimodal Deep Neural Network Model: The constructed sample set is divided into a training set, a validation set, and a test set. The multimodal deep neural network model is iteratively trained using the training set, and the cross-entropy loss function between the predicted probability distribution and the true label is calculated. ,in, L This represents the total number of samples in the training batch. For the sample k True label one-hot encoding, For multimodal deep neural network models, samples k Category c The predicted probability is obtained; the optimizer updates the parameters of the multimodal deep neural network model based on the gradient of the cross-entropy loss function, and the EarlyStopping mechanism monitors the validation set loss to prevent overfitting. S5. Deployment and Online Application of Multimodal Deep Neural Network Model: The trained multimodal deep neural network model is compressed and converted in format, and then deployed to an edge computing device. The edge computing device is connected to the machine tool controller through a communication interface to achieve synchronous status feedback. Multimodal operating status signals are collected in real time. The multimodal operating status signals are preprocessed according to step S2, and the processed data is input into the multimodal deep neural network model in step S3 for inference calculation. When the probability of moderate or severe degradation in the output prediction probability distribution exceeds the preset threshold, the audible and visual alarm module and the human-machine interface are triggered, and the data is uploaded to the remote management system. It supports online learning and regularly updates the multimodal deep neural network model using newly labeled data and a parameter-regularized incremental learning algorithm to adapt to machine tool performance drift.

2. The machine tool feed axis running status monitoring method and edge deployment method based on multimodal fusion according to claim 1, characterized in that, The specific parameter configuration of the multimodal deep neural network model described in step S3 is as follows: In the feature extraction sub-network, the first one-dimensional convolutional layer has 32 convolutional kernels, a kernel size of 16, and a stride of 2; the second one-dimensional convolutional layer has 64 convolutional kernels, a kernel size of 8, and a stride of 1, and the output feature vector... The dimension is 64; in the Bi-GRU bidirectional gated recurrent unit, the number of hidden units in the Bi-GRU layer is 64, the time window length of the local attention mechanism is 200ms, and the output feature vector is... The dimension is 128; in the Long Short-Term Memory (LSTM) network, the number of hidden units in the LSTM layer is 32, and the output feature vector is... The dimension is 32; the fused feature vector is obtained by concatenating the three feature vectors. The dimension is 224; the multi-head attention mechanism contains three parallel attention heads, which focus on a time scale of 0.1s for capturing short-term transient features, a 1s for capturing medium-term trend features, and a 2s for capturing long-term correlation features.

3. The machine tool feed axis running status monitoring method and edge deployment method based on multimodal fusion according to claim 2, characterized in that, In step S4, the sample set is randomly divided into a training set, a test set, and a validation set in a 6:2:2 ratio; the optimizer used is the Adam optimizer, and the initial learning rate is set to... ; The maximum number of training rounds is set to 60.

4. The machine tool feed axis running status monitoring method and edge deployment method based on multimodal fusion according to claim 3, characterized in that, The model compression and format conversion in step S5 specifically involves: pruning and quantizing the multimodal deep neural network model, and converting it to TensorRT format; the communication interface is an industrial Ethernet interface, using the MODBUS communication protocol; the parameter regularization incremental learning algorithm specifically employs the elastic weight consolidation algorithm, with a loss function of... ,in, The cross-entropy loss function is calculated based on the new batch of sample data in the current update cycle. These are the parameters of the multimodal deep neural network model that need to be updated. The optimal parameter values ​​retained after the multimodal deep neural network model converges on historical sample data represent the multimodal deep neural network model's memory of historical machine tool operating state characteristics; For parameters The diagonal elements of the Fisher information matrix are used to characterize the importance of this parameter in maintaining the accuracy of historical state recognition. These are hyperparameters used to weigh the importance of maintaining the performance of a multimodal deep neural network model on historical data against fitting new data.

5. An edge deployment system for monitoring the operating status of machine tool feed axes based on multimodal fusion, characterized in that, The multimodal fusion-based machine tool feed axis running status monitoring edge deployment system is configured to execute the multimodal fusion-based machine tool feed axis running status monitoring method and edge deployment method as described in any one of claims 1-4; The edge deployment system for monitoring the operating status of machine tool feed axes based on multimodal fusion includes: The data acquisition module executes step S1 to synchronously acquire the power signal, vibration signal, and temperature signal of the feed axis; The preprocessing and sample construction module executes step S2, which performs low-pass filtering, noise reduction, standardization, and sample partitioning on the collected multimodal operating status signals, and constructs a labeled sample set. In the model building and inference module, step S3 is executed, which uses a multimodal deep neural network model to extract features from the input signal sequence and fuse multi-head attention, and calculates the probability distribution of the running state of the feed axis. The model training module executes step S4, which uses the training set data to iteratively train and optimize the parameters of the multimodal deep neural network model. The online monitoring and update module executes step S5, which deploys the trained multimodal deep neural network model to the edge device for real-time inference and performs incremental learning and updates based on new data. The application interaction module executes step S5, triggering an alarm and human-machine interface prompts when an anomaly is detected, and performing remote data archiving.

6. The edge deployment system for monitoring the operating status of machine tool feed axes based on multimodal fusion according to claim 5, characterized in that, The hardware architecture of the machine tool feed axis running status monitoring edge deployment system based on multimodal fusion includes: The multimodal sensing layer, corresponding to the data acquisition module, includes a power sensor connected to the servo driver, a triaxial acceleration sensor installed on the near motor end bearing housing and the far motor end bearing housing, a temperature sensor installed on the near motor end bearing housing and the far motor end bearing housing, and a high-precision synchronous acquisition module supporting CAN communication. The edge computing layer, corresponding to the preprocessing and sample construction module and the model construction and inference module, includes industrial edge devices, industrial Ethernet and model inference engine. The model inference engine supports TensorRT real-time inference and dynamic loading of multimodal deep neural network models. Moreover, the preprocessing and sample construction module is integrated inside the industrial edge device to perform signal filtering, denoising, standardization processing and sample partitioning and sample set construction. The intelligent analysis layer, corresponding to the model training module and the online monitoring and update module, includes a data buffer pool, an incremental training unit, and a model evaluation unit. The data buffer pool is used to filter and store online collected samples; the incremental training unit is used to perform online incremental training and parameter optimization based on the elastic weight consolidation algorithm to adapt to machine tool performance drift; the model evaluation unit is used to evaluate the performance of new parameters before updating the multimodal deep neural network model, and only performs online updates of model weights when the performance index improves. The application interaction layer, corresponding to the application interaction module, consists of a human-machine interface, an audible and visual alarm module, and a remote management system interface. It supports visualization of abnormal states, alarm prompts, and remote management archiving.

7. The edge deployment system for monitoring the operating status of machine tool feed axes based on multimodal fusion according to claim 6, characterized in that, The data acquisition module of the multimodal sensing layer adopts a timestamp calibration mechanism to ensure that the time synchronization error of power signal, vibration signal, and temperature signal is ≤1ms.

8. The edge deployment system for monitoring the operating status of machine tool feed axes based on multimodal fusion according to claim 6, characterized in that, The edge computing layer also includes a fault self-diagnosis unit, which monitors the sensor communication status and the inference performance of the multimodal deep neural network model in real time, and automatically switches to the backup sensor group when an anomaly is detected.

Images