Multi-source uncertainty-guided multi-modal tool wear identification and prediction network model

Abstract

The invention belongs to the field of tool wear identification, and relates to a multi-source uncertainty guided multi-modal tool wear identification and prediction network model. The multi-modal tool wear identification and prediction network model comprises an identification branch and a prediction branch, and the identification branch identifies a tool wear area from a tool wear image and further calculates a quantized tool wear value; the prediction branch is used for further predicting future tool wear prediction based on a historical tool wear sequence obtained by a plurality of images through the identification branch; on the basis, weighted segmentation loss and prediction loss are calculated respectively, and high-precision identification and prediction are ensured to be realized in a few-sample scene in combination with multi-source uncertainty guidance.

Description

Technical Field

[0001] This invention belongs to the field of tool wear identification, and more specifically, relates to a multi-modal tool wear identification and prediction network model guided by multi-source uncertainty. Background Technology

[0002] Tool wear is a core issue in industrial automation and manufacturing, directly impacting equipment operating efficiency, production costs, and product quality stability. In modern machining processes, milling tools are widely used due to their versatility and high efficiency; however, tool wear leads to increased cutting forces, decreased machining accuracy, and even machine tool vibration, severely affecting process stability and production efficiency. Therefore, developing efficient and reliable tool condition monitoring (TCM) technology is crucial for optimizing equipment performance, reducing production costs, and improving the production efficiency of intelligent manufacturing systems. Existing research indicates that tool wear monitoring is not only key to improving product quality but also a vital foundation for proactive maintenance and real-time decision-making in intelligent manufacturing.

[0003] Existing TCM (Tool Condition Monitoring) technologies mainly include sensor-based and vision-based technologies. Sensor-based technologies assess tool wear by collecting signals such as cutting force, vibration, or acoustic emission; however, they are highly sensitive to changes in operating conditions (such as material properties, cutting parameters, or environmental noise), resulting in insufficient generalization ability and reliability across different machining scenarios. In contrast, vision-based tool condition monitoring technologies have received widespread attention in recent years due to their advantages of being non-contact, intuitive, and unaffected by signal interference. For example, existing technologies use machine vision methods to extract tool contours and quantify wear areas, or employ optimized image segmentation algorithms to identify specific tool wear patterns. These methods, combined with advanced image processing techniques and machine learning algorithms, can accurately detect and quantify tool wear characteristics, demonstrating the application potential of vision technology in tool condition monitoring.

[0004] However, existing vision-based TCM (Tracking Management Machine) technologies still have significant limitations. First, most technologies are limited to the identification and measurement of tool wear, lacking the ability to predict wear progression, making it difficult to meet the needs of proactive maintenance and real-time decision-making in intelligent manufacturing systems. Second, acquiring high-quality tool wear images is costly, requiring specialized imaging equipment and frequent processing interruptions, resulting in small datasets and a typical few-shot learning problem. Existing few-shot learning methods, such as data augmentation, transfer learning, and meta-learning, have limited effectiveness in tool wear monitoring, struggling to effectively capture complex wear textures or generalize to diverse tool types. Furthermore, the application of proactive learning-based methods in tool condition monitoring has not been fully explored, and their potential to improve model performance by selecting high-uncertainty samples has not been fully utilized. Summary of the Invention

[0005] In view of the above-mentioned defects or improvement needs of the existing technology, the present invention provides a multi-modal tool wear identification and prediction network model guided by multi-source uncertainty. Its purpose is to solve the technical problem that the tool wear monitoring effect is difficult to improve due to the small number of samples and high uncertainty in the existing technology.

[0006] To achieve the above objectives, according to one aspect of the present invention, a multi-source uncertainty-guided multimodal tool wear identification and prediction network model is provided, including a multimodal tool wear identification and prediction network (MRP), wherein the MRP includes an identification branch (RB) and a prediction branch (PB). The recognition branch RB uses the U-Net semantic segmentation network as its backbone and embeds a prototype feature enhancement module (PFE) to process the input tool wear image and generate a binary segmentation mask for the wear region. Specifically, the encoder of the U-Net semantic segmentation network extracts multi-scale features from the input tool wear image to capture the semantic details of the wear pattern; the prototype feature enhancement module (PFE) estimates the model uncertainty through Monte Carlo dropout, thereby reducing the dimensionality of the semantic details of the wear pattern to generate enhanced features; the decoder of the U-Net semantic segmentation network combines skip connections and enhanced features to reconstruct the segmentation mask. Model uncertainty is calculated through the prediction variance of multiple rounds of dropout forward propagation, reflecting the model confidence; data uncertainty is extracted from the prediction variance, emphasizing complex image regions; model uncertainty and data uncertainty are jointly used to guide the segmentation loss, optimize the learning process, focus on reliable predictions and complex regions, thereby improving segmentation robustness and accuracy in few-shot learning FSL scenarios; The prediction branch PB is used to process historical wear time series data and predict future wear trends based on CNN network, thus realizing multimodal fusion from image recognition to time series prediction.

[0007] Furthermore, the decoder reconstructs the segmentation mask through upsampling and feature refinement. The reconstructed segmentation mask utilizes the enhanced features of the PFE module and the skip connections of the encoder.

[0008] Furthermore, the decoder comprises four upsampling stages, employing bilinear interpolation and convolutional layers to progressively restore spatial resolution. Each stage fuses encoder features through skip connections, preserving spatial details to achieve accurate boundary segmentation. Subsequently, convolutional blocks refine features using batch normalization and ReLU activation functions, and finally, pixel-level segmentation probabilities are generated through convolutional layers with sigmoid activation. The segmentation loss is optimized by comparing the predicted mask with the real mask to ensure segmentation accuracy.

[0009] Furthermore, if the Prototype Feature Enhancement (PFE) module is not enabled, the encoder features are output directly; otherwise, the real mask is adjusted to the encoder feature size through interpolation.

[0010] Furthermore, when uncertainty calculation is enabled, Monte Carlo sampling is used to generate feature mean and variance. After adjusting the variance size, uncertainty weights are calculated and normalized to avoid noise interference. Subsequently, features and masks are extracted for each batch of samples. Foreground and background masks are generated through binarization. The areas of the foreground and background regions are calculated. Based on the uncertainty weights, the features are weighted and averaged to generate foreground and background prototypes, which are then spliced ​​into a prototype set. If attention enhancement is enabled, attention weights are calculated using a multilayer perceptron and softmax, the prototype is refined and expanded to the feature map size, and the original features and convolutional prototype features are fused by parameters to generate fused features. Subsequently, the features are further enhanced by combining parameter-weighted attention terms, and finally the final enhanced features are generated by Gaussian smoothing and output for the decoder to use, thereby improving the segmentation robustness and accuracy in low-sample scenarios.

[0011] Furthermore, it also includes a pixel-level weighted strategy MSU guided by multi-source uncertainty, used to dynamically identify the segmentation loss in the branch. The MSU includes the calculation of model uncertainty weights and data uncertainty calculation. The calculation of model uncertainty weights is achieved by processing the input image and the ground truth mask through the U-Net network, including: first, disabling dropout to generate deterministic output, then enabling dropout in training mode to generate multiple random outputs and stacking them into a prediction set; quantifying model uncertainty by calculating the variance of the prediction set; and highlighting regions with high uncertainty in model predictions by adding 1 and then performing convolution smoothing and normalization based on the calculated variance.

[0012] Furthermore, the calculation of data uncertainty weights is based on the analysis of intra-batch prediction variance: first, the batch mean and standard deviation are calculated, and a noise threshold is determined using a threshold factor; a noise mask is generated by comparing the variance with the threshold to identify high-noise regions; the data uncertainty weights amplify the weights of noise regions through the noise factor, and then convolutional smoothing and normalization are performed.

[0013] Furthermore, it also includes a dynamic weighted segmentation loss, which involves: calculating the entropy of the model weights and data weights respectively, and dynamically adjusting the weight factors based on the total entropy; finally, the binary cross-entropy loss is used to generate a weighted segmentation loss based on weighting, averaging the batch and spatial dimensions.

[0014] Furthermore, the tool wear prediction branch also includes a sample-level weighted strategy (TDU) guided by temporal variance uncertainty. This strategy enhances the robustness of tool wear time-series prediction in few-shot learning scenarios by calculating temporal weights, prioritizing stable patterns in historical wear sequences. Specifically: first, the input sequence is expanded into subsequences using a sliding window, and the local mean of each subsequence is calculated. Then, a mean matching the length of the input sequence is generated by copying and padding. Subsequently, the absolute deviation between the input sequence and the padding mean is calculated and averaged along the feature dimension to obtain the batch-level deviation. A threshold is set based on the batch mean and standard deviation of the batch-level deviation. The basic weights are calculated using the reciprocal of the normalized deviation. The weights of time-series points with deviations exceeding the threshold are halved. The weights are then smoothed using a one-dimensional Gaussian kernel to ensure continuity. Finally, the time-series weights are normalized and used to optimize the prediction loss.

[0015] Furthermore, it also includes the CPI, a comprehensive evaluation indicator for forecasting tasks:

[0016] Where i and j represent the experimental numbers of the two groups used for comparison, and m represents the general evaluation index, specifically including RSME, MAE, MAPE, and R2. Indicates the weights of RSME, MAE, MAPE, and R2; This represents the normalized values ​​of RSME, MAE, MAPE, and R2. The calculation methods for RSME, MAE, and MAPE are as follows: The normalization method for the R2 index is as follows: min( ) represents the minimum value among all the original measurements obtained by the methods participating in the comparison on the evaluation index m in the j-th experiment, max( ) represents the maximum value among all methods in the raw measurements obtained on the evaluation index m in the j-th experiment.

[0017] In general, the above-mentioned technical solutions conceived in this invention can achieve the following beneficial effects compared with the prior art.

[0018] 1. The present invention proposes a multimodal tool wear recognition and prediction network, which includes a recognition branch and a prediction branch. The recognition branch identifies the tool wear area from the tool wear image and further calculates the quantified tool wear value. The prediction branch makes a prediction of future tool wear based on the historical tool wear sequence obtained by the recognition branch from multiple images. On this basis, the weighted segmentation loss and prediction loss are calculated respectively. Combined with multi-source uncertainty guidance, high-precision recognition and prediction are achieved in scenarios with few samples.

[0019] 2. For the problem of few images in the recognition branch, this invention proposes a multi-source uncertainty strategy that combines model and data to jointly guide the loss function of the recognition branch, so as to enable the model to make robust predictions while urging the model to pay attention to the more valuable parts of the data. 3. In order to address the possibility of ineffective feature patterns in the identification branch, this invention proposes a prototype feature enhancement module to make the model pay more attention to feature patterns that are strongly correlated with wear.

[0020] 4. To address the problem of limited samples in time series data in the prediction branch, this invention proposes a strategy guided by the uncertainty of time series variance, which encourages the model to uncover the dynamic characteristics and patterns in the time series data. 5. To address the problem of inconsistent results when comparing multiple methods for common evaluation metrics in prediction tasks, making it difficult to intuitively and uniformly judge overall performance, this invention proposes a comprehensive evaluation index (CPI) for unified measurement, thereby providing a more scientific standard for judging consistency. Attached Figure Description

[0021] Figure 1 This is a schematic diagram of the architecture of a multi-source uncertainty-guided multimodal tool wear identification and prediction network model according to a preferred embodiment of the present invention. Detailed Implementation

[0022] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. Furthermore, the technical features involved in the various embodiments of this invention described below can be combined with each other as long as they do not conflict with each other.

[0023] To achieve the identification and prediction of tool wear under limited sample conditions, this invention proposes a multimodal tool wear identification and prediction network (MRP), a weighted strategy guided by multi-source uncertainty (MSU), a weighted strategy guided by temporal variance uncertainty (TDU), and a prototype feature enhancement module (PFE).

[0024] Multimodal Tool Wear Recognition and Prediction Network (MRP): Its technical solution first acquires images of the tool's side and bottom surfaces through an automated data acquisition platform. Then, it employs data augmentation techniques such as random flipping, rotation, and cropping to improve the model's robustness to different poses and occlusions. MRP consists of a Recognition Branch (RB) and a Prediction Branch (PB): RB, based on a U-Net network with embedded PFE modules, extracts multi-scale features from the images to generate segmentation masks for the wear region; PB, based on a CNN network, processes historical wear time-series data to predict future wear trends. Both branches are optimized through staged training, calculating weighted segmentation loss and prediction loss respectively. Combined with multi-source uncertainty guidance, this ensures high-precision recognition and prediction even in scenarios with few samples.

[0025] Multi-Source Uncertainty Guided Weighted Strategy (MSU): This strategy utilizes the U-Net model, generating deterministic and stochastic outputs by disabling and enabling dropout respectively, and calculating the prediction variance to quantify the model's uncertainty weights. Simultaneously, it generates data uncertainty weights through in-batch variance analysis, identifies noisy regions, and adjusts the weights accordingly. Both are then fused into a total weight through an entropy adjustment mechanism, smoothed and normalized, and applied to the binary cross-entropy loss to optimize the segmentation loss. This strategy enhances the model's robustness and segmentation accuracy in low-sample scenarios by emphasizing high-uncertainty regions and complex image regions.

[0026] Temporal variance uncertainty-guided weighted strategy (TDU): This technique expands the input sequence through a sliding window, calculates a local mean and compares it with the input sequence to generate a bias value, and then determines the batch mean and standard deviation to set a threshold. Basic weights are calculated based on the reciprocal of the bias, with weights halved in high-bias regions. These weights are then smoothed and normalized using a one-dimensional Gaussian kernel to generate temporal weights. These weights are applied to the mean squared error loss to optimize CNN network predictions, focusing on low-variability sequences and ensuring reliable prediction of future wear trends even in scenarios with few samples.

[0027] The Prototype Feature Enhancement (PFE) module extracts features from the encoder, calculates uncertainty using Monte Carlo sampling, and generates foreground and background prototypes. The prototypes are calculated using a weighted average of uncertainty weights, and attention weights are generated using a multilayer perceptron and softmax. After refining the prototypes, they are expanded to the feature map size. The final features are smoothly output by fusing the original and prototype features and adding weighted attention terms. This method improves the model's segmentation performance and generalization ability in low-sample scenarios by reducing noise impact and highlighting wear features.

[0028] The present invention will be described below with more specific embodiments: The present invention provides a multi-modal tool wear identification and prediction method guided by multi-source uncertainty, including a multi-modal tool wear identification and prediction network (MRP), a pixel-level weighting strategy (MSU) guided by multi-source uncertainty, a sample-level weighting strategy (TDU) guided by temporal variance uncertainty, and a prototype feature enhancement module (PFE), etc.

[0029] The aforementioned multimodal tool wear identification and prediction network (MRP) includes two modal branches: identification and prediction. The Multimodal Tool Wear Recognition and Prediction Network (MRP) achieves accurate identification of tool wear regions and prediction of future wear trends through a recognition branch (RB) and a prediction branch (PB). Its recognition branch uses a U-Net semantic segmentation network as its backbone, embedding a Prototype Feature Enhancement (PFE) module to process the input tool wear image and generate a binary segmentation mask for the wear region. Specifically, the encoder extracts multi-scale features to capture the semantic details of the wear pattern; the PFE module estimates model uncertainty through Monte Carlo (MC) dropout and generates enhanced features through dimensionality reduction; the decoder combines skip connections and enhanced features to reconstruct the segmentation mask. Model uncertainty is calculated through the prediction variance of multiple rounds of dropout forward propagation, reflecting model confidence; data uncertainty is extracted from the prediction variance, emphasizing complex image regions. Both branches jointly guide the segmentation loss, optimizing the learning process and focusing on reliable predictions and complex regions, thereby improving segmentation robustness and accuracy in few-shot learning (FSL) scenarios. The prediction branch (PB) processes historical wear time-series data and predicts future wear trends based on a CNN network, achieving a multimodal fusion from image recognition to time-series prediction.

[0030] The tool wear recognition branch network for MRP is constructed as follows: The Recognition Branch (RB) employs a U-Net network architecture with embedded Prototype Feature Enhancement (PFE) modules as its backbone to accurately identify wear regions from tool wear images. Its encoder extracts multi-scale contextual features from the input image through multi-stage convolutional operations. The encoder contains multiple convolutional blocks, each consisting of a 3×3 convolutional layer, batch normalization, and a ReLU activation function. Subsequently, max pooling is used to downsample the feature map, progressively reducing spatial dimensions and increasing the number of channels to form a hierarchical semantic feature representation. In few-shot learning (FSL) scenarios, the encoder utilizes pre-trained weights to initialize the feature extraction process, capturing robust and transferable feature representations. This enhances the model's generalization ability under data-scarce conditions, ensuring the effective extraction of key semantic features in segmentation tasks.

[0031] The Prototype Feature Enhancement (PFE) module optimizes the feature representation for few-shot segmentation. If not enabled, it directly outputs encoder features; otherwise, it adjusts the mask size through interpolation, generates uncertain weights through Monte Carlo sampling, extracts foreground and background prototypes, and concatenates them. A self-attention mechanism generates weights, refines and expands the prototypes, fuses the original and convolutional features, outputs enhanced features with Gaussian smoothing, and adjusts parameters to reduce noise and improve segmentation robustness.

[0032] The decoder reconstructs the segmentation mask through upsampling and feature refinement, utilizing enhanced features from the PFE module and skip connections from the encoder. The decoder comprises four upsampling stages, employing bilinear interpolation and convolutional layers to progressively restore spatial resolution. Each stage fuses encoder features through skip connections, preserving spatial details for accurate boundary segmentation. Subsequently, convolutional blocks refine features using batch normalization and ReLU activation functions, and finally, convolutional layers with sigmoid activation generate pixel-level segmentation probabilities. The segmentation loss is optimized by comparing the predicted mask with the ground truth mask, ensuring segmentation accuracy. This design, through multi-scale feature fusion and enhancement mechanisms, effectively improves the recognition accuracy of tool wear regions in low-sample scenarios.

[0033] Preferably, this embodiment also proposes a prototype feature enhancement module, which is embedded between the encoder and decoder of the identification branch network.

[0034] The Prototype Feature Enhancement (PFE) module optimizes the feature representation for few-shot segmentation through uncertainty-aware prototype learning and a self-attention mechanism. First, if the PFE module is not enabled, encoder features are directly output; otherwise, the ground truth mask is adjusted to the encoder feature size via interpolation. When uncertainty calculation is enabled, Monte Carlo sampling is used to generate the feature mean and variance. After adjusting the variance size, uncertainty weights are calculated and normalized to avoid noise interference. Subsequently, features and masks are extracted for each batch of samples. Foreground and background masks are generated through binarization, the areas of the foreground and background regions are calculated, and foreground and background prototypes are generated by weighted averaging based on uncertainty weights. These prototypes are then concatenated into a prototype set. If attention enhancement is enabled, attention weights are calculated using a multilayer perceptron and softmax, refining the prototypes to the feature map size. The original features and convolutional prototype features are then fused using parameter fusion to generate fused features. Subsequently, the features are further enhanced by parameter-weighted attention terms. Finally, Gaussian smoothing is used to generate the final enhanced features, which are output for the decoder, thereby improving the robustness and accuracy of segmentation in few-shot scenarios.

[0035] A multi-source uncertainty-guided pixel-level weighted strategy (MSU) is proposed for dynamically identifying segmentation loss in branches. The MSU includes model uncertainty calculation and data uncertainty calculation. The model uncertainty weights are calculated using the U-Net network to process the input image and the ground truth mask. First, dropout is disabled to generate deterministic outputs. Then, dropout is enabled in training mode, generating five random outputs, which are stacked into a prediction set. The model uncertainty is quantified by calculating the variance of these predictions. Based on the variance, the model uncertainty weights are generated by adding 1, then smoothed using a 3×3 convolutional kernel, and normalized to highlight regions of high uncertainty in the model predictions, thereby enhancing attention to regions of low model confidence.

[0036] The calculation of data uncertainty weights is based on the analysis of batch-specific prediction variance. First, the batch mean and standard deviation are calculated, and a noise threshold is determined using a threshold factor. A noise mask is generated by comparing the variance with the threshold to identify high-noise regions. In the basic weight calculation, the data uncertainty weights amplify the weights of noisy regions using a noise factor, and then a 3×3 convolution kernel is used for smoothing and normalization to ensure robust handling of noisy or blurred regions in the input image.

[0037] The dynamically weighted segmentation loss optimizes segmentation accuracy by combining the model and data uncertainty. The entropy of the model weights and data weights are calculated separately, and the weight factors are dynamically adjusted based on the total entropy, with an initial value of 1.0 for both to balance their contributions. The total weights are generated through weighted summation and normalized to a range of 0.5 to 2.0 to stabilize the loss. Finally, the binary cross-entropy loss is used to generate a weighted segmentation loss based on weighting, batch averaging, and spatial dimension, thereby improving the segmentation accuracy of tool wear in scenarios with few samples by emphasizing high uncertainty and complex regions.

[0038] Preferably, this embodiment constructs a tool wear prediction branch network; The tool wear prediction branch processes historical wear sequences using a convolutional neural network (CNN) to predict future tool wear trends, and can be trained using conventional CNN training methods. Preferably, in this embodiment, the network includes a feature extractor and a regressor architecture. The feature extractor consists of two one-dimensional convolutional layers, combining ReLU activation and dropout operations to extract temporal features from the input time-series data. Subsequently, the regressor generates the predicted future wear sequence through two fully connected layers. The prediction process constructs input-output sequence pairs from the time-series dataset and divides them into training and test sets. The network is optimized using a mean squared error loss function.

[0039] Preferably, this embodiment proposes a sample-level weighting strategy (TDU) guided by time series variance uncertainty.

[0040] TDU enhances the robustness of tool wear time-series prediction in few-shot learning scenarios by calculating temporal weights, prioritizing stable patterns in historical wear sequences. Its implementation first uses a sliding window to expand the input sequence into subsequences, calculates the local mean of each subsequence, and generates a mean matching the length of the input sequence through duplication and padding. Then, it calculates the absolute deviation between the input sequence and the padding mean, averaging along the feature dimension to obtain batch-level deviation. A threshold is set based on the batch mean and standard deviation of the deviation. The base weights are calculated using the reciprocal of the normalized deviation; time-series points with deviations exceeding the threshold have their weights halved. The weights are then smoothed using a one-dimensional Gaussian kernel to ensure continuity, and finally normalized to generate temporal weights used to optimize the prediction loss.

[0041] Preferably, a comprehensive evaluation index (CPI) is proposed for the prediction task. Its calculation method is as follows:

[0042] Where i and j represent the experimental numbers of the two groups used for comparison, m represents general evaluation indicators, specifically including RSME, MAE, MAPE, R2, etc. Wm represents the weights of RSME, MAE, MAPE, R2, etc. This represents the normalized value of indicators such as RSME, MAE, MAPE, and R2. The calculation methods for indicators such as RSME, MAE, and MAPE are as follows: For the R² index, its normalization calculation method is as follows: In a preferred embodiment, This represents the raw measurement value of the evaluation index m obtained using the i-th method in the j-th experiment. Here, i is the index of the various methods being compared, j is the index of different experimental scenarios or datasets, and m is any regression evaluation index selected from root mean square error (RMSE), mean absolute error (MAE), mean absolute percentage error (MAPE), and coefficient of determination (R²). Specifically, these are the index values ​​directly calculated in the corresponding experiments using the corresponding methods, used for subsequent normalization and comprehensive evaluation. min( ) represents the minimum value among all the original measurements obtained by the methods participating in the comparison on the evaluation index m in the j-th experiment, i.e., min( )=min i ( Correspondingly, the formula also involves max( Let represent the maximum value among all the original measurements obtained by all methods on the evaluation index m in the same j-th experiment. The minimum and maximum values ​​together constitute the range of values ​​for index m in this experiment, which is used to linearly normalize the original index values ​​of each method, thereby unifying the indices with different dimensions and directions of superiority or inferiority into dimensionless normalized scores, which facilitates subsequent weighted summation to form a comprehensive evaluation index.

[0043] Table 1 shows the recognition metrics on the test set after training with training sets of different sizes in the recognition branch under different strategies. Specifically, the entire image was first divided into training and test sets in an 8:2 ratio. Then, 50%, 30%, 20%, 10%, and 2% of the data from the training set were taken for training, respectively, representing 36%, 14.4%, 7.2%, 3.6%, and 1.4% of the entire dataset. In the table, Num represents the proportion of samples participating in training relative to the total, Unc represents the strategy used, Wd represents the proportion of data uncertainty, Wm represents the proportion of model uncertainty, ID represents the experiment number, and the remaining metrics represent common performance evaluation metrics in semantic segmentation tasks. For the parameter Unc, N indicates no uncertainty-guided strategy, D indicates only data uncertainty-guided strategy, M indicates only model uncertainty-guided strategy, and D+M indicates a combined model and data uncertainty-guided strategy.

[0044] Data shows that, under different scales of training data, each comparison consistently demonstrates that the performance of the uncertainty strategy jointly guided by the model and data is significantly better than the performance metrics of other strategies in the same group, and consistently shows that the performance reaches its best when the proportion of data uncertainty is twice the proportion of model uncertainty.

[0045] Table 1. Recognition metrics for identifying branches in cases with few samples.

[0046] Table 2 shows the prediction metrics in the small-sample case, with and without using the TDU strategy in the prediction branch. Specifically, firstly, the entire time series data is divided into training and test sets in an 8:2 ratio. Then, 50%, 30%, 20%, 10%, and 5% of the data from the training set are selected sequentially for training, representing 40%, 24%, 16%, 8%, and 4% of the entire dataset, respectively. In the table, Num represents the proportion of the samples participating in training relative to the total population, UT indicates whether TDU is used, T indicates TDU is used, and F indicates that TDU is not used. In addition, T represents the length of the historical sequence, T' represents the length of the predicted sequence, and the remaining metrics represent common performance evaluation metrics in prediction tasks. It should be noted that, given the issue that when comparing prediction metrics with and without TDU, some metrics increase while others decrease, the comprehensive evaluation metric CPI proposed in this invention can comprehensively consider factors and assign certain weights to metrics such as RMSE, MAE, MAPE, and R2 in the CPI evaluation calculation.

[0047] The effectiveness of TDU was verified based on the comprehensive evaluation index CPI. In adjacent experimental groups, all parameters were identical except for the parameter UT (ultrasound unit) indicating whether TDU was used. Therefore, the CPI of adjacent experimental groups was evaluated to verify whether the use of TDU improved the evaluation index. Data showed that in 20 comparative experiments with different sequence lengths, the CPI improved after using TDU, effectively verifying the effectiveness of TDU.

[0048] Table 2. Identification metrics for prediction branches in the case of few samples.

[0049] Those skilled in the art will readily understand that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A multi-modal tool wear identification and prediction network model guided by multi-source uncertainty, characterized in that, It includes a multimodal tool wear identification and prediction network (MRP), which comprises an identification branch (RB) and a prediction branch (PB). The recognition branch RB uses the U-Net semantic segmentation network as its backbone and embeds a prototype feature enhancement module PFE to process the input tool wear image and generate a binary segmentation mask for the wear region. Specifically, the encoder of the U-Net semantic segmentation network extracts multi-scale features from the input tool wear image to capture the semantic details of the wear pattern. The prototype feature enhancement module PFE estimates the model uncertainty through Monte Carlo dropout, thereby reducing the dimensionality of the semantic details of the wear pattern to generate enhanced features. The decoder of the U-Net semantic segmentation network combines skip connections and enhanced features to reconstruct the segmentation mask; Model uncertainty is calculated through the prediction variance of multiple rounds of dropout forward propagation, reflecting the model confidence; data uncertainty is extracted from the prediction variance, emphasizing complex image regions; model uncertainty and data uncertainty are jointly used to guide the segmentation loss, optimize the learning process, focus on reliable predictions and complex regions, thereby improving segmentation robustness and accuracy in few-shot learning FSL scenarios; The prediction branch PB is used to process historical wear time series data and predict future wear trends based on CNN network, thus realizing multimodal fusion from image recognition to time series prediction.

2. The multi-source uncertainty-guided multimodal tool wear identification and prediction network model as described in claim 1, characterized in that, The decoder reconstructs the segmentation mask through upsampling and feature refinement. The reconstructed segmentation mask utilizes the enhanced features of the PFE module and the skip connections of the encoder.

3. The multi-source uncertainty-guided multimodal tool wear identification and prediction network model as described in claim 2, characterized in that, The decoder comprises four upsampling stages, employing bilinear interpolation and convolutional layers to progressively restore spatial resolution. Each stage fuses encoder features through skip connections, preserving spatial details to achieve accurate boundary segmentation. Subsequently, convolutional blocks refine features using batch normalization and ReLU activation functions, and finally, pixel-level segmentation probabilities are generated through convolutional layers with sigmoid activation. The segmentation loss is optimized by comparing the predicted mask with the real mask to ensure segmentation accuracy.

4. A multi-source uncertainty-guided multimodal tool wear identification and prediction network model as described in any one of claims 1 to 3, characterized in that, If the Prototype Feature Enhancement (PFE) module is not enabled, the encoder features are output directly; otherwise, the real mask is adjusted to the encoder feature size by interpolation.

5. The multi-source uncertainty-guided multimodal tool wear identification and prediction network model as described in claim 4, characterized in that, When uncertainty calculation is enabled, Monte Carlo sampling is used to generate the feature mean and variance. After adjusting the variance size, the uncertainty weight is calculated and normalized to avoid noise interference. Subsequently, features and masks are extracted for each batch of samples. Foreground and background masks are generated through binarization. The areas of the foreground and background regions are calculated. Foreground and background prototypes are generated based on the uncertainty weighted average features and then spliced ​​together to form a prototype set. If attention enhancement is enabled, attention weights are calculated using a multilayer perceptron and softmax, the prototype is refined and expanded to the feature map size, and the original features and convolutional prototype features are fused through parameters to generate fused features; Subsequently, the features are further enhanced by combining parameter-weighted attention terms, and finally, the enhanced features are generated by Gaussian smoothing and output for the decoder to use, thereby improving the segmentation robustness and accuracy in low-sample scenarios.

6. The multi-source uncertainty-guided multimodal tool wear identification and prediction network model as described in claim 1, characterized in that, It also includes a pixel-level weighted strategy MSU guided by multi-source uncertainty, which is used to dynamically identify the segmentation loss in the branch. The MSU includes the calculation of model uncertainty weights and data uncertainty calculation. The calculation of model uncertainty weights is achieved by processing the input image and the ground truth mask through the U-Net network, including: first, disabling dropout to generate deterministic output, then enabling dropout in training mode to generate multiple random outputs and stacking them into a prediction set; quantifying model uncertainty by calculating the variance of the prediction set; and highlighting regions with high uncertainty in model predictions by adding 1 and then performing convolution smoothing and normalization based on the calculated variance.

7. The multi-source uncertainty-guided multimodal tool wear identification and prediction network model as described in claim 6, characterized in that, The calculation of data uncertainty weights is based on the analysis of intra-batch prediction variance: first, the batch mean and standard deviation are calculated, and a noise threshold is determined using a threshold factor; a noise mask is generated by comparing the variance with the threshold to identify high-noise regions. The data uncertainty weights are amplified by a noise factor to increase the weights of noisy regions, and then smoothed and normalized by convolution.

8. The multi-source uncertainty-guided multimodal tool wear identification and prediction network model as described in claim 6, characterized in that, It also includes a dynamic weighted segmentation loss, which involves calculating the entropy of the model weights and data weights separately, and dynamically adjusting the weight factors based on the total entropy; finally, the binary cross-entropy loss is used to generate a weighted segmentation loss based on weighting, averaging the batch size and spatial dimension.

9. The multi-source uncertainty-guided multimodal tool wear identification and prediction network model as described in claim 1, characterized in that, The tool wear prediction branch also includes a sample-level weighted strategy TDU guided by temporal variance uncertainty. By calculating temporal weights, the robustness of tool wear temporal prediction in the few-sample learning scenario is enhanced. It prioritizes stable patterns in historical wear sequences. Specifically, the input sequence is first expanded into subsequences using a sliding window, the local mean of each subsequence is calculated, and a mean matching the length of the input sequence is generated by copying and padding. Subsequently, the absolute deviation between the input sequence and the filled mean is calculated and averaged along the feature dimension to obtain the batch-level deviation. A threshold is set based on the batch mean and standard deviation of the batch-level deviation. The basic weights are calculated as the inverse of the normalized deviation. The weights of time series points with deviations exceeding the threshold are halved. The weights are then smoothed using a one-dimensional Gaussian kernel to ensure continuity. Finally, the time series weights are normalized and used to optimize the prediction loss.

10. The multi-source uncertainty-guided multimodal tool wear identification and prediction network model as described in claim 9, characterized in that, It also includes the CPI, a comprehensive evaluation indicator for forecasting tasks: Where i and j represent the experimental numbers of the two groups used for comparison, and m represents the general evaluation index, specifically including RSME, MAE, MAPE, and R2. Indicates the weights of RSME, MAE, MAPE, and R2; This represents the normalized values ​​of RSME, MAE, MAPE, and R2. The calculation methods for RSME, MAE, and MAPE are as follows: The normalization method for the R2 index is as follows: , Min( represents the original measured value of the evaluation index m obtained by using the i-th method in the j-th experiment.) ) represents the minimum value among all the original measurements obtained by the methods participating in the comparison on the evaluation index m in the j-th experiment, max( ) represents the maximum value among all methods in the raw measurements obtained on the evaluation index m in the j-th experiment.

Images