A tool wear prediction method and system based on machine vision

Abstract

The present invention belongs to the technical field related to numerical control machining and discloses a machine vision-based tool wear prediction method and system. The method includes: constructing a first sample set: the samples are tool wear images and wear area labels; constructing a wear area segmentation model for segmenting the wear area based on the tool wear images, and using the first sample set for training; constructing a second sample set: the samples are a sequence of tool wear values; the wear values ​​are calculated and obtained based on the wear areas segmented from the tool wear images by the wear area segmentation model; constructing a wear prediction model to predict the tool wear values, and using the second sample set for training; and using the trained wear prediction model to perform online prediction of the tool wear values. The present invention trains the wear area segmentation model to achieve online segmentation of the tool wear area, and further, online calculation of the wear value. This allows real-time prediction without disrupting production, thereby improving production efficiency.

Description

Technical Field

[0001] The present invention belongs to the technical field related to numerical control machining, and more specifically, relates to a tool wear prediction method and system based on machine vision. Background Art

[0002] CNC machine tools are essential processing equipment in the manufacturing industry, with cutting tools being their key actuators. However, during CNC machining, mechanical friction occurs between the tool, the chip, and the workpiece. Simultaneously, due to the effects of cutting forces and heat, the tool continuously wears. This wear gradually worsens over time, eventually leading to chipping, severely impacting machining efficiency, workpiece surface quality, and processing costs. Studies have shown that 20% of total machine tool downtime is due to tool failure. Accurately monitoring tool status can improve machining efficiency by 50% and reduce production costs by 40%. Therefore, accurate monitoring of tool wear is of great theoretical and practical significance.

[0003] Traditional tool wear monitoring methods typically rely on manual inspection and regular maintenance, which is time-consuming and prone to errors. With advances in computer technology, artificial intelligence, and other technologies, intelligent tool wear monitoring technology is attracting increasing research attention. Tool wear monitoring methods can be categorized as either indirect or direct, depending on the monitoring method.

[0004] The indirect method refers to indirectly reflecting the tool wear condition by analyzing signals that are strongly correlated with tool wear, such as cutting force, vibration, acoustic emission, and current. However, due to the characteristics of the indirect method, the indirect method has several inherent limitations. First, complex signal processing technology is required to filter out noise and ensure the accuracy of the signal. Second, the tool wear signal may be affected by many factors, such as cutting conditions, workpiece material, tool material, etc., which makes the relationship between tool wear complicated. Third, establishing an accurate mathematical model or machine learning algorithm requires a large amount of experimental data and computing resources. Fourth, the indirect monitoring method relies on the accuracy and stability of the sensor itself, and the sensor needs to be maintained and calibrated regularly, which adds uncertainty to the tool wear monitoring model.

[0005] Direct methods leverage machine vision technologies such as cameras and image processing to directly capture tool image information, enabling accurate detection of minute changes in the tool. However, compared to indirect methods, direct methods have been limited in use for a long time due to shortcomings such as the need for machine downtime, complex operation, limited measurement locations, and the inability to monitor in real time. Existing machine vision-based tool wear prediction research requires capturing tool images and performing offline wear value testing during training sample construction and prediction. This is cumbersome and significantly impacts machining efficiency. Summary of the Invention

[0006] In response to the above-mentioned defects or improvement needs of the prior art, the present invention provides a tool wear prediction method and system based on machine vision, which is used to solve the problem in the existing tool wear prediction research based on machine vision. The conventional machine vision tool wear prediction model needs to collect tool images and perform wear value testing offline during the training sample construction and prediction process, which is cumbersome to operate and seriously affects the processing efficiency.

[0007] To achieve the above objectives, according to one aspect of the present invention, a tool wear prediction method based on machine vision is provided, comprising:

[0008] Constructing a first sample set: the samples in the first sample set are tool wear images and corresponding wear area labels;

[0009] Constructing a wear area segmentation model: the wear area segmentation model is used to segment the wear area according to the tool wear image, and the wear area segmentation model is trained using the first sample set to obtain the trained wear area segmentation model;

[0010] Constructing a second sample set: the samples in the second sample set are a sequence of tool wear values; wherein the tool wear values ​​are calculated and obtained based on the wear area segmented from the tool wear image by the wear area segmentation model;

[0011] Constructing a wear prediction model: the wear prediction model is used to predict the wear value of the tool, and the wear prediction model is trained using the second sample set to obtain the trained wear prediction model;

[0012] Online prediction: During the machining process, the wear value of the tool is predicted online using the trained wear prediction model.

[0013] The tool wear prediction method based on machine vision provided by the present invention further includes:

[0014] Industrial cameras are respectively set up below and on the sides of the parts outside the processing area of ​​the machine tool to form image acquisition points;

[0015] During machining, the tool is controlled to move to the image acquisition point, and an image of the tool's bottom wear is acquired by an industrial camera below, and an image of the tool's side edge wear is acquired by an industrial camera on the side. The tool is then controlled to rotate at the image acquisition point to acquire wear images of different side edges of the tool; wherein the tool's bottom wear image and the tool's side edge wear image form a tool wear image;

[0016] The first sample set is constructed by marking the wear area of ​​the tool wear image.

[0017] According to the machine vision-based tool wear prediction method provided by the present invention, the wear area segmentation model is an improved YOLOv8 model, which includes a Conv module, an M module, an SPPF module, an Upsample module and a Concat module, wherein the M module is composed of a FasterNeXt module; the Conv module performs a convolution operation on the feature map to extract local features of the image; the FasterNeXt module generates an intermediate feature map after a convolution operation on the input feature map; the SPPF module realizes multi-scale extraction and fusion of features through multi-level spatial pyramid pooling; the Upsample module uses deconvolution to restore the resolution of the feature map; and the Concat module realizes feature fusion and jump connection.

[0018] According to the machine vision-based tool wear prediction method provided by the present invention, the wear area segmentation model is used to segment the tool wear image into the wear area, and the wear value of the tool is calculated and obtained, which specifically includes:

[0019] Obtain pixel coordinates of vertices in the wear area to form a vertex point set;

[0020] Traversing the vertex point set to obtain equations of line segments between any two vertices; and determining the original cutting edge line segment of the tool by screening the slope of the line segment and the number of vertices it passes through;

[0021] Calculate the vertical distances between the remaining points of the vertex point set and the original cutting edge line segment of the tool, and select the value with the largest vertical distance as the number of pixels of the maximum wear width;

[0022] According to the number of pixels of the maximum wear width and the size of the tool wear image, the actual value of the maximum wear width is obtained as the wear value.

[0023] According to the machine vision-based tool wear prediction method provided by the present invention, the wear prediction model is an MN structure Transformer-CNN model, which is composed of M stacked Transformer modules and N stacked CNN modules, and is used to predict future sequence data of tool wear values ​​based on historical sequence data of tool wear values;

[0024] The wear prediction model is trained based on the mean square error (MSE) shown in the following formula:

[0025]

[0026] Among them, y i is the actual value of the i-th data point, is the predicted value of the i-th data point, and n is the total number of predicted data points.

[0027] According to the tool wear prediction method based on machine vision provided by the present invention, building a wear prediction model further includes:

[0028] Dividing the second sample set into a training set and a test set;

[0029] The Grey Wolf algorithm is used to train the wear prediction model independently multiple times and minimize the average RMSE on the test set as the objective function. The optimal combination of hyperparameters M and N is optimized with M and N being integers in the range of [1,20] as the constraint conditions.

[0030] According to the tool wear prediction method based on machine vision provided by the present invention, the online prediction specifically includes:

[0031] The wear prediction model inputs a sequence of true values ​​of tool wear values ​​during the first prediction, and outputs a sequence of predicted values ​​of tool wear values;

[0032] Afterwards, the wear prediction model inputs a combination of the true value of the tool wear value and the predicted value output from the previous prediction, or a sequence of predicted values ​​output from the previous prediction, and outputs a sequence of predicted values ​​of the tool wear value, forming an advanced prediction mode.

[0033] According to the tool wear prediction method based on machine vision provided by the present invention, in the advance prediction mode, the online prediction further includes:

[0034] Timely correction strategy: comparing the predicted value sequence of tool wear values ​​output by the wear prediction model with the corresponding true value sequence of tool wear values ​​to obtain the error between the predicted value and the true value;

[0035] If the error is less than the preset threshold, online prediction will continue in the advance prediction mode; if the error is greater than or equal to the preset threshold, when the wear prediction model makes the next prediction, the input will be the true value sequence of the tool wear value to correct the model input; thereafter, online prediction will continue in the advance prediction mode.

[0036] According to the tool wear prediction method based on machine vision provided by the present invention, the true value of the tool wear value is calculated and obtained based on the wear area segmentation model of the tool wear image collected online, or is determined based on a wear value sample set established in advance through experiments.

[0037] According to another aspect of the present invention, a tool wear prediction system based on machine vision is provided, comprising a machine vision system, the machine vision system including a bottom camera and a side camera, the bottom camera being located below a portion outside a machining area of ​​a machine tool, the side camera being located to the side of the portion outside the machining area of ​​the machine tool, the machine vision system being configured to form an online image acquisition point on the machine tool;

[0038] The invention also includes a memory and a processor, wherein the memory stores a computer program, and the processor executes any one of the above-mentioned tool wear prediction methods based on machine vision when executing the computer program.

[0039] In general, compared with the prior art, the above technical solutions conceived by the present invention provide a tool wear prediction method and system based on machine vision:

[0040] 1. A direct method uses machine vision technology to acquire tool wear images. Training a wear region segmentation model based on the tool wear images enables online segmentation of the tool wear region, and then online calculation of the wear value, eliminating the need for offline testing to obtain the wear value. This significantly speeds up the acquisition of prediction model training sample sets and simplifies operations. Furthermore, when the wear prediction model acquired through wear value training is used for online prediction, it can also achieve online acquisition of the wear value, enabling real-time prediction without disrupting production. This prediction method can significantly improve production efficiency.

[0041] 2. The proposed automated online tool wear monitoring method can automatically obtain wear samples from the tool bottom and side surfaces during production, providing an innovative method for the automated collection of wear images.

[0042] 3. Although the number of layers of the optimized YOLOv8-FasterNeXt model, or the wear area segmentation model, has increased, its parameter count has been reduced to 86% of the original and its size has been reduced to 87% due to the structural characteristics of the replaced modules. On the RTX3080 graphics card, the inference speed per image has been reduced to 84%, and the computational complexity has been reduced to 89%. Furthermore, while the average accuracy of the model at IoU = 0.5 has not decreased, the average accuracy at IoU = 0.5-0.95 has improved. Overall, the improved YOLOv8 model has lightweight characteristics while also improving segmentation accuracy. It can identify wear areas with high accuracy, and the absolute measurement error on the test set is within 5μm, which can provide accurate input data for subsequent wear prediction models with small measurement errors.

[0043] 4. Under the traditional prediction model, the Transformer-CNN prediction model, also known as the wear prediction model, achieved accuracies of 99.98%, 99.98%, and 99.99% in three sets of experiments using public and experimental data, and 99.95%, 99.99%, and 99.99% in three sets of experiments using public and experimental data, respectively.

[0044] 5. The optimal parameters obtained by the Gray Wolf Algorithm outperformed other hyperparameter combinations on both public and experimental data, demonstrating the effectiveness of the Gray Wolf optimization algorithm.

[0045] 6. It was verified that the proposed GWO-Transformer-CNN tool wear prediction model can be unsupervisedly transferred to cross-tool, cross-working condition, cross-material, cross-equipment and other cross-domain tool wear prediction tasks with lower error and higher accuracy, proving the robustness and generalization of the proposed method. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 This is the overall framework diagram of the tool wear prediction method based on machine vision provided by the present invention;

[0047] Figure 2 This is the structure diagram of the YOLOv8-FasterNeXt tool wear area segmentation model;

[0048] Figure 3 This is the inference effect diagram of the tool wear area segmentation model, where (a) before wear; (b) early wear stage; (c) middle wear stage; (d) late wear stage;

[0049] Figure 4 It is a process of calculating the quantitative tool wear value, wherein: (a) the original cutting edge is identified; (b) the quantitative tool wear value is obtained; (c) the conversion between pixel value and actual length is performed;

[0050] Figure 5 It is the structure diagram of the wear prediction model;

[0051] Figure 6 It is the visualization result of the gray wolf algorithm solution process;

[0052] Figure 7 This is the principle diagram of the advance prediction model. DETAILED DESCRIPTION

[0053] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely for the purpose of explaining the present invention and are not intended to limit the present invention. In addition, the technical features involved in the various embodiments of the present invention described below may be combined with each other as long as they do not conflict with each other.

[0054] See also Figure 1 This embodiment provides a tool wear prediction method based on machine vision, which includes:

[0055] Constructing a first sample set: the samples in the first sample set are tool wear images and corresponding wear area labels;

[0056] Constructing a wear area segmentation model: the wear area segmentation model is used to segment the wear area according to the tool wear image, and the wear area segmentation model is trained using the first sample set to obtain the trained wear area segmentation model;

[0057] Constructing a second sample set: the samples in the second sample set are a sequence of tool wear values; wherein the tool wear values ​​are calculated and obtained based on the wear area segmented from the tool wear image by the wear area segmentation model;

[0058] Constructing a wear prediction model: the wear prediction model is used to predict the wear value of the tool, and the wear prediction model is trained using the second sample set to obtain the trained wear prediction model;

[0059] Online prediction: During the machining process, the wear value of the tool is predicted online using the trained wear prediction model.

[0060] This embodiment adopts a direct method with the help of machine vision technologies such as cameras and image processing. By training a wear area segmentation model based on the tool wear image, the online segmentation of the tool wear area can be achieved, and then the wear value can be calculated online. Based on the calculated wear value, a wear prediction model can be trained to obtain online prediction, which can achieve real-time prediction without interfering with production. This prediction method can greatly improve production efficiency. The unique advantages of this direct method in tool wear monitoring cannot be ignored. First, the direct method can provide very accurate wear measurement results, which can accurately reflect the actual wear status of the tool. Second, the wear value provided by the direct method is a direct and specific numerical value, which is easy to understand and analyze, and avoids the uncertainty brought by signal conversion in the indirect method. Third, the direct method is based on physical measurement, and its results have high reliability and repeatability, and are not easily affected by external factors. Therefore, the direct method has broad application potential and prospects in tool wear monitoring.

[0061] Furthermore, this embodiment considers that existing deep learning-based tool wear prediction models typically randomly divide the entire dataset into training and test sets, resulting in interpolation-based predictions. These models can only predict one set of outputs based on one set of inputs and cannot continuously predict tool wear for multiple steps into the future. Furthermore, existing research has limited research on multi-domain tool wear prediction across operating conditions, tools, materials, and equipment, which is crucial for practical production.

[0062] In view of the above shortcomings of the existing technology, the solutions proposed in this embodiment mainly include the following points:

[0063] Establish a fully automatic data acquisition platform to automatically acquire tool wear images;

[0064] Using tool wear images, a machine vision model is constructed to automatically identify tool wear areas and further calculate and obtain quantitative tool wear values;

[0065] Construct a tool wear prediction model to achieve the mapping relationship between historical and future tool wear sequences;

[0066] By improving the traditional prediction model and combining it with correction strategies, we can achieve predictions with ultra-long time steps while ensuring prediction accuracy.

[0067] Specifically, an automated data acquisition platform was built to automatically capture tool wear images. To capture tool wear images, a safe photo-taking point was set inside the machine tool. Two industrial cameras and lenses were installed below and to the side of the point to capture images of the tool's bottom and side surfaces, respectively. To facilitate the acquisition of images of each side edge of the tool, an M100CN instruction was developed to orient the machine tool's spindle to any angle, where N is the angle of rotation relative to the machine tool's encoder zero point. Furthermore, the camera's accompanying software provides a shortcut key for taking photos. The TCP / IP network protocol is used to establish communication between the machine tool and the PC. Combined with the FOCAS communication protocol, this enables the automatic capture of wear images of each tool's bottom and side edges.

[0068] After obtaining tool wear images, annotation software was used to annotate the wear areas and generate label files to construct a dataset. The dataset was then divided into training, validation, and test sets in a specific ratio. The FasterNeXt module was used to optimize the segmentation model structure, reducing model parameters and making the model lightweight while also improving inference accuracy. The training set was used to train the instance segmentation model, while the validation and test sets were used to evaluate the performance of the instance segmentation model. The test set was used to evaluate the accuracy of the wear measurement method based on the instance segmentation model. A deep learning instance segmentation model was constructed, using the wear images and their label files as input. The model was iteratively trained to determine the optimal parameters, enabling it to autonomously learn key features. Ultimately, it possessed the ability to segment tool wear regions, capable of pixel-level classification of input images and intelligently segmenting tool wear areas. After training, the model's performance was evaluated on the validation set. By refining the segmentation model structure and retraining the model, its performance on the validation set improved. To obtain quantitative tool wear values, the wear areas were segmented and then further calculated based on morphological analysis.

[0069] The tool wear prediction model uses a neural network to establish a nonlinear relationship between historical and future tool wear sequences, enabling the mapping of historical to future tool wear sequences. Given the powerful parallel computing capabilities of the Transformer model and the parameter sharing mechanism of convolutional networks, a MN-structured Transformer-CNN tool wear prediction model is developed to predict historical to future wear sequences. First, the input data is reshaped into a three-dimensional format (batch size, sequence length, features) so that it can be used as input for M stacked Transformer models. Second, a stack of M Transformer models is constructed, and their outputs are used as input for N stacked CNN models. Finally, the input is passed through each layer of the model to obtain the model output and calculate the loss with the labels. The model parameters are iteratively updated using a backpropagation algorithm to obtain the optimal parameter prediction model.

[0070] In addition, in order to ensure the robustness and stability of the model, the present invention uses the Gray Wolf Algorithm, takes the model independently trained 5 times and the average RMSE value on the test set as the objective function, and calculates the optimal hyperparameter combination M and N, which has a significant improvement in prediction performance compared with other parameter combinations.

[0071] Given that traditional interpolation prediction models lack the ability to perceive wear trends over very long time steps, an advance prediction mode is proposed. In this mode, the model fuses the prediction output of the previous time step with the input for each prediction, thus enabling prediction capabilities over infinite time steps. At the same time, the decline in prediction accuracy due to the accumulation of prediction errors is becoming increasingly serious. Therefore, this embodiment proposes a timely correction strategy. This timely correction strategy can compensate for the error accumulation problem in the advance prediction mode and promptly correct the model input. The combination of the advance prediction mode and the timely correction strategy enables the model to have very long time step prediction capabilities while also ensuring model prediction accuracy.

[0072] The prediction method provided in this embodiment was used to carry out cross-domain prediction, which effectively verified the high generalization ability of the prediction model. It can realize cross-domain prediction and use the model trained with source domain data to directly predict the data in the target domain, which reduces the data annotation cost and enhances the applicability and flexibility of the model, which is of great significance for engineering applications.

[0073] In another embodiment, a machine vision-based tool wear prediction system is provided, comprising a machine vision system, the machine vision system comprising a bottom camera and a side camera, the bottom camera being located below a portion outside a machining area of ​​a machine tool, the side camera being located to the side of the portion outside the machining area of ​​the machine tool, the machine vision system being configured to form an online image acquisition point on the machine tool;

[0074] The invention also includes a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the tool wear prediction method based on machine vision described in any one of the embodiments is executed.

[0075] This embodiment provides a cross-domain tool wear advance prediction method based on machine vision, including a fully automated data acquisition platform, a tool wear area segmentation model, a tool wear quantification method, a tool wear prediction model, a gray wolf algorithm optimization hyperparameter, a tool wear advance prediction mode, a timely correction strategy, and cross-domain prediction, such as Figure 1 The following is a clear and complete description of the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings.

[0076] Example 1

[0077] Build the fully automated data acquisition platform; that is, set industrial cameras below and on the side of the part outside the machine tool processing area to form image acquisition points; during the machine tool processing process, control the tool to move to the image acquisition point, obtain the bottom wear image of the tool through the industrial camera below, obtain the side edge wear image of the tool through the industrial camera on the side, and control the tool to rotate at the image acquisition point to obtain the wear images of different side edges of the tool; wherein, the bottom wear image of the tool and the side edge wear image of the tool form a tool wear image; by marking the wear area of ​​the tool wear image, construct a first sample set.

[0078] First, a safe photo-taking spot was set up inside the machine tool. Two industrial cameras and lenses were installed below and to the side of the spot to capture images of tool wear on the bottom and sides, respectively.

[0079] Secondly, to automatically measure the wear of each side edge of the tool, a custom M100CN command is used to orient the machine tool spindle to any angle, where N is the rotation angle relative to the machine tool encoder zero point. By controlling the spindle rotation angle, the side camera can capture wear images of different side edges.

[0080] Finally, a C# script was developed based on the FOCAS communication protocol to monitor the status of the machine tool. The tool wear image was automatically photographed and stored using shortcut keys. The specific implementation steps are shown in Table 1.

[0081] Table 1 Online measurement method for tool wear

[0082]

[0083]

[0084] Example 2

[0085] A tool wear region segmentation model was constructed. To identify tool wear regions, machine vision was used to segment the wear regions. Labeling software was used to manually mark the wear regions in the tool wear images in the first sample set. The resulting 2,176 samples were divided into training, validation, and test sets in a ratio of 1,546:546:84. The training set was used to train the YOLOv8 instance segmentation model, and the validation and test sets were used to evaluate the instance segmentation performance of the YOLOv8 model.

[0086] Use as Figure 2The improved YOLOv8 model architecture shown identifies the wear area of ​​the tool; that is, the wear area segmentation model is an improved YOLOv8 model, which is mainly composed of basic modules such as the Conv module, M module, SPPF module, Upsample module and Concat module; the Conv module performs convolution operations on the feature map to extract local features of the image; the SPPF module uses multi-level spatial pyramid pooling to achieve multi-scale feature extraction and fusion, thereby improving the generalization ability of the model; the Upsample module uses deconvolution to restore the resolution of the feature map, providing more accurate spatial information, thereby improving the positioning accuracy of the target; the Concat module implements feature fusion and skip connections, enhancing the model's feature expression capabilities;

[0087] In the native YOLOv8 architecture, M is composed of C2f modules. This module achieves efficient computation by splitting, processing, and fusing features, providing rich feature representations. In the proposed YOLOv8-FasterNeXt architecture, or the improved YOLOv8 model, FasterNeXt replaces the C2f module. In other words, M is composed of FasterNeXt modules. This FasterNeXt module convolves the input feature map to generate an intermediate feature map. One portion is processed by X FasterNetBlock submodules, while the other portion remains unchanged. The two portions are then concatenated and subjected to a layer of convolution to generate the final feature map. This optimized FasterNeXt module helps maintain computational efficiency while enhancing the network's expressive power, making it suitable for tasks that efficiently process feature maps.

[0088] These basic modules are based on Figure 2 The connections on the right form the instance segmentation model architecture, where different segment layers identify objects at large, medium, and small scales in the input image. This output layer, with anchor boxes of varying sizes, enables the model to extract and fuse features, ensuring robust detection of objects at multiple scales.

[0089] To verify the effectiveness of the improved YOLOv8-FasterNeXt, the comparison of inference indicators on the validation set before and after the improvement is shown in Table 2. It can be seen that although the number of layers of the optimized model has increased, due to the structural characteristics of the replacement module, the number of parameters is reduced to 86% of the original and the size is reduced to 87%. On the RTX3080 graphics card, the inference speed of each image is reduced to 84% and the computational complexity is reduced to 89%. More importantly, while the average accuracy of the model at IoU = 0.5 does not decrease, the average accuracy at IoU = 0.5-0.95 is improved. Overall, the improved YOLOv8-FasterNeXt model has lightweight characteristics while also improving segmentation accuracy.

[0090] like Figure 3 The results of model instance segmentation of randomly selected images from various stages of the tool's entire life cycle show that the wear area segmentation model can effectively identify the wear area at any stage of tool wear, providing a basis for subsequent tool wear measurement.

[0091] Table 2 Comparison of inference indicators before and after YOLOv8 model optimization

[0092]

[0093] Example 3

[0094] The tool wear value quantification method is to further calculate the tool wear value by segmenting the wear area; the International Organization for Standardization stipulates that the blunting standard of carbide tools during rough machining is: under normal wear of cutting tools, the average wear width of the tool back face VB average =0.3 or maximum wear width VB max =0.5 is the blunting standard; this embodiment adopts the maximum wear width VB max Conduct follow-up studies as a standard for blunting;

[0095] The trained YOLOv8-FasterNeXt can accurately identify irregular polygonal tool wear areas, and the vertex coordinates of the area can be used to further calculate the quantitative tool wear value VB max ; Each vertex of the wear area is a set of two-dimensional points (x i ,y j ),i=1,2,…,n;

[0096] The maximum wear width VB of the side edge is calculated max When the tool is cut, the original cutting edge position of the tool must be determined first; by observing Figure 3 Based on the tool wear characteristics, it is found that the original cutting edge of the tool can be approximated by the longest edge of the polygon; therefore, the point set on the wear area is traversed to obtain the equation of the line segment between each two points; the original cutting edge L of the tool can be easily determined by screening the line segments with fewer slopes and fewer points. ij ,like Figure 4 (a)

[0097] Specifically, for the irregular polygonal area obtained after segmentation, let the target set T = {t1, t2, ..., t n}, where each target t i It is a set of two-dimensional points on the polygonal area. Let point P = (x, y) be a point on the two-dimensional plane, and let a line segment represents the straight line segment connecting points p and q; line segment The length d(p,q) is obtained using the Euclidean distance and its slope is calculated; each point pair (p,q) in the target set is traversed, the distance and slope are calculated, and line segments that do not meet the conditions are filtered out. If the number of points passed by the line segment is greater than or equal to a threshold (the threshold set in the present invention is 20), the longest distance and line segment are updated. Finally, the longest line segment obtained after traversing all vertices of the entire polygon is the edge boundary of the tool when it is not worn, that is, the original cutting edge;

[0098] Then, given the point r = (x3, y3) and the line segment Calculate the distance from the point to the line segment; after obtaining the original cutting edge boundary of the tool, it is necessary to determine the point on the wear area farthest from the boundary, that is, calculate the distance between the remaining points on the polygon and the line segment L ij Vertical distance, the largest value is the maximum wear width VB max The number of pixels, such as Figure 4 (b)

[0099] On this basis, if you know the actual width or height of the image, you can convert VB max The actual value of the industrial camera software used provides basic distance and area measurement functions. After measurement, the actual width and height of the image are 9.7112 and 5.4663 mm respectively, and then the VB is converted max The actual value of Figure 4 (c) in the.

[0100] That is, the wear value of the tool is calculated based on the wear area segmentation model for the wear area segmented from the tool wear image, specifically including:

[0101] Obtain pixel coordinates of vertices in the wear area to form a vertex point set;

[0102] Traversing the vertex point set to obtain equations of line segments between any two vertices; and determining the original cutting edge line segment of the tool by screening the slope of the line segment and the number of vertices it passes through;

[0103] Calculate the vertical distances between the remaining points of the vertex point set and the original cutting edge line segment of the tool, and select the value with the largest vertical distance as the number of pixels of the maximum wear width;

[0104] According to the number of pixels of the maximum wear width and the size of the tool wear image, the actual value of the maximum wear width is obtained as the wear value.

[0105] Example 4

[0106] Tool wear prediction model; As shown in formula (1), the tool wear prediction model is used to mine the wear value history T1 time series data y calculated based on the wear area provided by the wear area segmentation model (t-T1)：t With the future T2 time series data y t.(t+T2)The relationship between them; this process is implemented using the Transformer model;

[0107] y t: (t+T2) =P(y (t -T1): t ) (1)

[0108] This embodiment uses the powerful parallel computing capability of the Transformer model and combines the parameter sharing mechanism of the convolutional network to propose a MN structure Transformer-CNN tool wear prediction model, such as Figure 5 That is, the wear prediction model is an MN structure Transformer-CNN model, which is composed of M stacked Transformer modules and N stacked CNN modules, and is used to predict the future sequence data of tool wear values ​​based on the historical sequence data of tool wear values;

[0109] The model input is a tensor in the form of a sequence. The Transformer consists of M stacked encoders and a decoder layer. Each encoder layer calculates multi-head attention on the input sequence to capture the internal relationships of the input sequence. The feed-forward layer performs feature extraction and nonlinear transformation. Normalization is performed after the multi-head attention mechanism layer and the feed-forward layer. The normalization layer helps reduce internal covariate shift during training and improves the stability of model training. Residual connections help accelerate the training process and prevent gradient vanishing and exploding.

[0110] The input tensor is input into N stacked convolutional networks through M stacked encoder and decoder layers, and finally outputs the predicted wear value through two layers of fully connected networks. During back propagation, the model propagates to each layer of the network with mean squared error (MSE), thereby updating the weights and biases of the network. That is, the wear prediction model is trained based on mean squared error (MSE), and the MSE loss calculation formula is shown in formula (2), y i is the actual value of the i-th data point, is the predicted value of the i-th data point, and n is the total number of predicted data points;

[0111]

[0112] The M stacked Transformer layers and the N stacked CNN layers serve as model hyperparameters and have a significant impact on the prediction results. Increasing the number of stacked layers can enhance the expressive power of the model, thereby enabling the model to better capture the complex relationships in the input sequence. However, too many stacked layers will increase the complexity and training cost of the model. Constructing the wear prediction model also includes: dividing the second sample set into a training set and a test set; using the gray wolf algorithm, with the wear prediction model independently trained multiple times and the minimum average RMSE on the test set as the objective function, and with M and N being integers in [1,20] as constraints, optimizing the optimal combination of hyperparameters M and N.

[0113] To verify the effectiveness of the GWO-Transformer-CNN tool wear prediction model—the MN-structured Transformer-CNN model after optimizing hyperparameters M and N using the Gray Wolf algorithm—we first validated its performance under traditional prediction mode. Tables 3 and 4 show its prediction metrics compared to other advanced models on test sets of public and experimental data, respectively. This demonstrates that the wear prediction model proposed in this paper exhibits superior prediction performance. C1, C4, and C6 are three publicly available datasets; Exp1, Exp2, and Exp3 are three datasets experimentally constructed based on the wear region segmentation model proposed in this paper.

[0114] Table 3 Prediction indicators of the prediction model on the public data test set

[0115]

[0116] Table 4 Prediction indicators of the prediction model on the experimental data test set

[0117]

[0118]

[0119] Example 5

[0120] The Gray Wolf Algorithm is used to optimize the hyperparameters of the tool wear prediction model. For regression tasks, RMSE, MSE, MAE and other indicators are generally used to evaluate the model performance. Due to the randomness of the initialization parameters of the neural network, the evaluation indicators of the model after training are also random. Therefore, in order to ensure the robustness and stability of the model, the average RMSE value of the model independently trained 5 times on the test set is used as the objective function of the model optimization when optimizing the M and N hyperparameters. The variables to be optimized M and N are integers in [1,20], n is the number of training times, and f is the training set. M，N (x) RMSE is the RMSE value under the combination of M and N values, so the hyperparameter optimization task of the prediction model can be defined as shown in formula (3);

[0121]

[0122] In order to make the model have a lower RMSE index, the gray wolf optimization algorithm was used to solve the optimal model hyperparameters M and N. The solution steps are shown in Table 5;

[0123] Table 5. Grey Wolf Algorithm Optimization Process of Prediction Model Hyperparameters

[0124]

[0125]

[0126] The visualization results of the gray wolf algorithm solution process are as follows Figure 6 As shown; it can be seen that the objective function has a minimum value when the optimal hyperparameters M and N are both in the range of [1,10];

[0127] To verify the effectiveness of GWO-Transformer-CNN, we divided M and N into three levels (low: 1, medium: 6, and high: 10), and conducted orthogonal experiments on the public and experimental data test sets. The results are shown in Tables 6 and 7, respectively. It can be seen that the optimal hyperparameters obtained by the Grey Wolf Algorithm have better prediction performance.

[0128] Table 6 Comparison of prediction indicators of different hyperparameters on the public data test set

[0129]

[0130] Table 7 Comparison of prediction indicators of different hyperparameters on the experimental data test set

[0131]

[0132] Example 6

[0133] The advance prediction mode; Figure 7 The difference between traditional tool wear prediction and the advanced prediction model is shown. The traditional tool wear prediction model relies on historical real data, that is, each prediction requires the provision of the actual value of the historical wear value. In this model, the wear value of the historical T1 time point needs to be calculated to serve as the input of the prediction model. Therefore, the prediction model needs to provide the historical wear data of T1*N time points for N predictions, which seriously affects the real-time performance of the prediction model.

[0134] In contrast, the advanced prediction mode feeds the predicted value output by the prediction model back to the input end to fuse it with the historical real value data, continuously updating the model input for the next prediction, thereby making continuous predictions; in this mode, the model can theoretically make an infinite number of recursive predictions based on a set of model inputs; however, as the number of predictions increases, the accumulation of errors cannot be ignored, so it is very necessary to use historical tool wear values ​​to correct the model input in a timely manner after a certain number of predictions.

[0135] That is, online prediction specifically includes:

[0136] The wear prediction model inputs a sequence of true values ​​of tool wear values ​​during the first prediction, and outputs a sequence of predicted values ​​of tool wear values;

[0137] Afterwards, the wear prediction model inputs a combination of the true value of the tool wear value and the predicted value output from the previous prediction, or a sequence of predicted values ​​output from the previous prediction, and outputs a sequence of predicted values ​​of the tool wear value, forming an advanced prediction mode.

[0138] Example 7

[0139] In the advance prediction mode, online prediction also includes:

[0140] Timely correction strategy: comparing the predicted value sequence of tool wear values ​​output by the wear prediction model with the corresponding true value sequence of tool wear values ​​to obtain the error between the predicted value and the true value;

[0141] If the error is less than a preset threshold, online prediction continues in advance prediction mode. If the error is greater than or equal to the preset threshold, the wear prediction model uses a sequence of true tool wear values ​​as input for the next prediction to correct the model input. Online prediction continues in advance prediction mode. During machining, timely corrections can be made at preset intervals or a preset number of tool changes.

[0142] Specifically, timely correction strategy; In the advanced prediction mode, the accumulation of prediction errors leads to an inevitable decrease in prediction accuracy; Therefore, a timely correction strategy is proposed, and its principle is shown in Table 8; S i T Indicates that the real historical sequence at the time point i is the model input, S i P Indicates that the output of the previous prediction is spliced ​​into the historical sequence as the model input when predicting the i-th time point; i ' represents the prediction result output at the i-th time point, P represents the prediction model, and △ represents the MAE value between the prediction result at the i-th time point and the actual wear sequence; when T1=3, T2=1, the first prediction occurs at the 4th time point. At this time, the model is based on the historical sequence S4T The prediction obtains Y4'; in the future prediction, the prediction can be made according to the advance prediction mode, and then the MAE between the previous prediction result and the true value is calculated at each preset time interval or preset number of tool intervals. If it is less than the error threshold δ, the input of the next prediction will use the previous prediction result fused with historical data as input; on the contrary, if the MAE of the previous prediction is greater than δ, the next prediction will use the real wear data as input, so as to avoid the accumulation of errors in time;

[0143] Specifically, the true value of the tool wear value is calculated based on the wear area segmented from the online tool wear image acquired by the wear area segmentation model, or determined based on a wear value sample set established in advance through experiments. During the online prediction process, predictions can be performed at regular intervals, and each prediction can be based on the number of cuts or processing time. Taking prediction based on the number of cuts as an example, the input of the wear prediction model for each prediction is a sequence of true wear values ​​corresponding to several cuts before the current moment, a fusion sequence of true wear values ​​and predicted wear values, or a sequence of predicted wear values. The output is a predicted wear value corresponding to at least one cut after the current moment.

[0144] The true value of the wear value can be obtained by moving the spindle to the image acquisition point after the corresponding tool processing to collect the tool wear image online, segmenting the wear area through the wear area segmentation model, and then obtaining the true value of the wear value corresponding to the corresponding tool processing through calculation; or, a processing experiment can be carried out in advance to obtain the wear value that changes with the number of processing tools through the processing experiment, and establish a wear value sample set related to the number of processing tools. In the online prediction process, the wear value corresponding to the corresponding tool processing can be directly selected from the wear value sample set as the true value of the wear value input to the model.

[0145] The true value of the wear value used for error judgment with the wear value prediction value in timely correction can be obtained in the same way, that is, the number of processing tools corresponding to the wear value prediction value can be obtained by online collection of tool mold damage images and then obtained through segmentation and calculation, or the wear value corresponding to the corresponding tool processing can be obtained by selecting the wear value sample set. There is no specific limitation.

[0146] To verify the effectiveness of the timely correction strategy in the look-ahead mode, predictions were performed on the public and experimental data test sets without using the correction strategy under different conditions of T1 and T2. The results are shown in Tables 9 and 10, respectively. Then, the prediction results after using the correction strategy on the public and experimental data are shown in Tables 11 and 12, respectively. It can be seen that after using the timely correction strategy, both error and accuracy are significantly improved at different sequence lengths.

[0147] Table 8 Principles of timely correction strategy

[0148]

[0149] Table 9 Prediction indicators of the prediction model without correction strategy in the advance mode on the public data test set

[0150]

[0151]

[0152] Table 10 Prediction indicators of the prediction model without correction strategy in the advance mode on the experimental data test set

[0153]

[0154]

[0155] Table 11 Prediction indicators of the prediction model using the correction strategy in the advance mode on the public data test set

[0156]

[0157] Table 12 Prediction indicators of the prediction model using the correction strategy in the advance mode on the experimental data test set

[0158]

[0159] Example 8

[0160] The cross-domain prediction method uses the trained model in the source domain to directly predict the data in the target domain, which has important practical significance for engineering applications.

[0161] The public data used in this paper is PHM2010, including data from three experiments, C1, C4, and C6. The experimental conditions and cutting parameters are shown in Table 13. It should be noted that the public data were conducted under the same working conditions using tools with different blade geometries and coatings.

[0162] Table 13 PHM2010 cutting parameters and experimental configuration

[0163]

[0164] When using experimental data in the present invention, a 12mm diameter 4-flute tungsten steel end mill was used to conduct cutting experiments on a rectangular mold steel. A side cutting method was adopted, and each time the material of the mold steel length was cut from the X direction, the tool was moved to the photographing point for photographing. A total of three independent cutting experiments, Exp1 to Exp3, were carried out, and the cutting parameters and experimental conditions are shown in Table 14. It should be noted that the same tool was used in the three experiments; Exp1 and Exp2 had the same working conditions but different workpieces; Exp2 and Exp3 had the same workpieces but different working conditions; Exp1 and Exp3 had different working conditions and workpieces.

[0165] Table 14 PHM2010 cutting parameters and experimental configuration

[0166]

[0167]

[0168] Since the three sets of data (C1, C4, and C6) in the public data were collected using different tools, in order to verify the generalization of the prediction model, the model trained on one set of data was used to unsupervisedly predict the data of another set of data, thereby verifying the model's cross-tool prediction performance. The results are shown in Table 15. It can be seen that the model has a prediction accuracy of at least 99.59% in cross-tool prediction, showing good generalization ability.

[0169] Table 15 Public data cross-tool prediction indicators

[0170]

[0171] Since the experimental data were collected under different working conditions and different workpiece materials, in order to verify the generalization ability of the model across working conditions and materials, the experimental data were used cross-wise, and the model trained with the source domain data was used to predict the target domain data. The results are shown in Table 16. It can be seen that the model has good generalization ability across working conditions and materials.

[0172] Table 16 Experimental data cross-condition and cross-material prediction indicators

[0173]

[0174] The publicly available PHM data was obtained by high-speed milling of nickel-chromium-iron alloy using a ball-end mill, while the Exp data was obtained by low-speed milling of mold steel using an end mill. The two data sets differ significantly in terms of tooling, workpiece materials, equipment, and operating conditions. To further explore the cross-domain prediction performance of the prediction model between the two types of data, the model trained on the Exp data was used to predict the PHM data. The results are shown in Table 17. As can be seen, the model trained on the Exp data achieves an accuracy of over 99.06% when used for unsupervised prediction of the PHM data, demonstrating excellent generalization capabilities.

[0175] Table 17: Public and experimental data cross-domain prediction indicators

[0176]

[0177] This paper verifies that the proposed GWO-Transformer-CNN tool wear prediction model can be unsupervisedly transferred to cross-tool, cross-material, cross-equipment and other cross-domain tool wear prediction tasks with lower error and higher accuracy, proving the robustness and generalization of the proposed method.

[0178] It will be easily understood by those skilled in the art that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A tool wear prediction method based on machine vision, characterized in that: include: Constructing a first sample set: the samples in the first sample set are tool wear images and corresponding wear area labels; Constructing a wear area segmentation model: the wear area segmentation model is used to segment the wear area according to the tool wear image, and the wear area segmentation model is trained using the first sample set to obtain the trained wear area segmentation model; Constructing a second sample set: the samples in the second sample set are a sequence of tool wear values; wherein the tool wear values ​​are calculated and obtained based on the wear area segmented from the tool wear image by the wear area segmentation model; Constructing a wear prediction model: the wear prediction model is used to predict the wear value of the tool, and the wear prediction model is trained using the second sample set to obtain the trained wear prediction model; Online prediction: During the machining process, the wear value of the tool is predicted online using the trained wear prediction model; The wear area segmentation model is an improved YOLOv8 model, which includes a Conv module, an M module, an SPPF module, an Upsample module, and a Concat module. The M module is composed of a FasterNeXt module. The Conv module performs a convolution operation on the feature map to extract local features of the image. The FasterNeXt module generates an intermediate feature map after a convolution operation on the input feature map. The SPPF module realizes multi-scale feature extraction and fusion through multi-level spatial pyramid pooling. The Upsample module uses deconvolution to restore the resolution of the feature map. The Concat module realizes feature fusion and jump connection. The wear prediction model is an MN structure Transformer-CNN model, which consists of M stacked Transformer modules and N stacked CNN modules, and is used to predict future sequence data of tool wear values ​​based on historical sequence data of tool wear values.

2. The tool wear prediction method based on machine vision according to claim 1, characterized in that: Also includes: Industrial cameras are respectively set up below and on the sides of the parts outside the processing area of ​​the machine tool to form image acquisition points; During machining, the tool is controlled to move to the image acquisition point, and an image of the tool's bottom wear is acquired by an industrial camera below, and an image of the tool's side edge wear is acquired by an industrial camera on the side. The tool is then controlled to rotate at the image acquisition point to acquire wear images of different side edges of the tool; wherein the tool's bottom wear image and the tool's side edge wear image form a tool wear image; The first sample set is constructed by marking the wear area of ​​the tool wear image.

3. The tool wear prediction method based on machine vision according to claim 1, characterized in that: Calculating the wear value of the tool based on the wear area segmentation model for the wear area segmented from the tool wear image specifically includes: Obtain pixel coordinates of vertices in the wear area to form a vertex point set; Traversing the vertex point set to obtain equations of line segments between any two vertices; and determining the original cutting edge line segment of the tool by screening the slope of the line segment and the number of vertices it passes through; Calculate the vertical distances between the remaining points of the vertex point set and the original cutting edge line segment of the tool, and select the value with the largest vertical distance as the number of pixels of the maximum wear width; According to the number of pixels of the maximum wear width and the size of the tool wear image, the actual value of the maximum wear width is obtained as the wear value.

4. The tool wear prediction method based on machine vision according to any one of claims 1 to 3, characterized in that: The wear prediction model is trained based on the mean square error (MSE) shown in the following formula: ； in, For the The actual value of the data point, For the The predicted value of the data point, is the total number of predicted data points.

5. The tool wear prediction method based on machine vision according to claim 4, characterized in that: Building a wear prediction model also includes: Dividing the second sample set into a training set and a test set; The Grey Wolf algorithm is used to train the wear prediction model independently multiple times and minimize the average RMSE on the test set as the objective function. The optimal combination of hyperparameters M and N is optimized with M and N being integers in the range of [1,20] as the constraint conditions.

6. The tool wear prediction method based on machine vision according to any one of claims 1 to 3, characterized in that: Online prediction specifically includes: The wear prediction model inputs a sequence of true values ​​of tool wear values ​​during the first prediction, and outputs a sequence of predicted values ​​of tool wear values; Afterwards, the wear prediction model inputs a combination of the true value of the tool wear value and the predicted value output from the previous prediction, or a sequence of predicted values ​​output from the previous prediction, and outputs a sequence of predicted values ​​of the tool wear value, forming an advanced prediction mode.

7. The tool wear prediction method based on machine vision according to claim 6, characterized in that: In the advance prediction mode, online prediction also includes: Timely correction strategy: comparing the predicted value sequence of tool wear values ​​output by the wear prediction model with the corresponding true value sequence of tool wear values ​​to obtain the error between the predicted value and the true value; If the error is less than the preset threshold, online prediction will continue in the advance prediction mode; if the error is greater than or equal to the preset threshold, when the wear prediction model makes the next prediction, the input will be the true value sequence of the tool wear value to correct the model input; thereafter, online prediction will continue in the advance prediction mode.

8. The tool wear prediction method based on machine vision according to claim 7, characterized in that: The true value of the tool wear value is calculated and obtained based on the wear area segmentation model for the wear area segmented from the online collected tool wear image, or is determined based on a wear value sample set established in advance through experiments.

9. A tool wear prediction system based on machine vision, characterized in that: The machine vision system includes a bottom camera and a side camera, wherein the bottom camera is located below the portion outside the processing area of ​​the machine tool, and the side camera is located to the side of the portion outside the processing area of ​​the machine tool. The machine vision system is used to form an online image acquisition point on the machine tool; The invention further comprises a memory and a processor, wherein the memory stores a computer program, and the processor executes the tool wear prediction method based on machine vision as described in any one of claims 1 to 8 when executing the computer program.

Images