Tool wear prediction method and system based on machine vision

Abstract

The invention belongs to the related technical field of numerical control machining, and discloses a tool wear prediction method and system based on machine vision, and the method comprises the steps: constructing a first sample set: a sample is a tool wear image and a wear region label; constructing a wear region segmentation model for segmenting a wear region according to the tool wear image, and training by using the first sample set; constructing a second sample set, wherein a sample is a wear value sequence of the cutter; the wear value is calculated and obtained based on a wear region segmented from the tool wear image by a wear region segmentation model; a wear prediction model is constructed to predict the wear value of the tool, and the second sample set is used for training; and performing online prediction on the wear value of the cutter by using the trained wear prediction model. According to the method, online segmentation of the tool wear area can be achieved by training the wear area segmentation model, then the wear value can be calculated online, real-time prediction can be achieved under the condition that production is not interfered, and the production efficiency is improved.

Description

Technical Field

[0001] The present invention belongs to the technical field related to numerical control machining, and more specifically, relates to a tool wear prediction method and system based on machine vision. Background Art

[0002] CNC machine tools are the main processing equipment in the manufacturing industry, and the tool is its key executive component. However, during the CNC machine tool processing, mechanical friction will occur between the tool and the chips and workpiece. At the same time, under the influence of cutting force and cutting heat, the tool will continue to wear, and its wear state will gradually increase over time and finally break, which seriously affects the processing efficiency, the surface quality of the workpiece and the processing cost. Studies have shown that 20% of the total downtime of machine tools is caused by fault shutdowns caused by tool failure; accurate monitoring of tool status can improve processing efficiency by 50% and reduce production costs by 40%. Therefore, accurate monitoring of tool wear status has important theoretical and practical significance.

[0003] Traditional tool wear monitoring methods usually rely on manual inspection and regular maintenance, which is time-consuming and prone to errors. With the advancement of computer technology, artificial intelligence and other technologies, intelligent tool wear monitoring technology has attracted more and more attention from researchers. According to different monitoring methods, tool wear monitoring methods can be divided into indirect method and direct method.

[0004] The indirect method refers to indirectly reflecting the wear of the tool by analyzing signals that are strongly correlated with tool wear, such as cutting force, vibration, acoustic emission and current. However, due to the characteristics of the indirect method, the indirect method has several inherent limitations. First, complex signal processing technology is required to filter out noise and ensure the accuracy of the signal. Second, the tool wear signal may be affected by many factors, such as cutting conditions, workpiece materials, tool materials, etc., which makes the relationship between tool wear complicated. Third, establishing an accurate mathematical model or machine learning algorithm requires a large amount of experimental data and computing resources. Fourth, the indirect monitoring method relies on the accuracy and stability of the sensor itself, and the sensor needs to be maintained and calibrated regularly, which adds uncertainty to the tool wear monitoring model.

[0005] The direct method uses machine vision technologies such as cameras and image processing to directly obtain the image information of the tool and accurately detect small changes on the tool. However, compared with the indirect method, the direct method has not been widely used for a long time in the past due to its shortcomings such as the need to stop the machine for detection, complex operation, measurement position restrictions, and inability to monitor in real time. In the existing tool wear prediction research based on machine vision, the usual machine vision tool wear prediction model needs to collect tool images and perform wear value tests offline during the construction of training samples and the prediction process. The operation is cumbersome and seriously affects the processing efficiency. Summary of the invention

[0006] In response to the above defects or improvement needs of the prior art, the present invention provides a tool wear prediction method and system based on machine vision, which is used to solve the problem that in the existing tool wear prediction research based on machine vision, the usual machine vision tool wear prediction model needs to collect tool images and perform wear value testing offline during the construction of training samples and the prediction process, which is cumbersome to operate and seriously affects the processing efficiency.

[0007] To achieve the above object, according to one aspect of the present invention, a tool wear prediction method based on machine vision is provided, comprising:

[0008] Constructing a first sample set: the samples in the first sample set are tool wear images and corresponding wear area labels;

[0009] Constructing a wear area segmentation model: the wear area segmentation model is used to segment the wear area according to the tool wear image, and the wear area segmentation model is trained using the first sample set to obtain the trained wear area segmentation model;

[0010] Constructing a second sample set: the samples in the second sample set are a wear value sequence of the tool; wherein the wear value of the tool is calculated and obtained based on the wear area segmented from the tool wear image by the wear area segmentation model;

[0011] Constructing a wear prediction model: the wear prediction model is used to predict the wear value of the tool, and the wear prediction model is trained using the second sample set to obtain the trained wear prediction model;

[0012] Online prediction: During the machining process of the machine tool, the wear value of the tool is predicted online using the trained wear prediction model.

[0013] The tool wear prediction method based on machine vision provided by the present invention also includes:

[0014] Industrial cameras are respectively arranged below and on the sides of the parts outside the processing area of ​​the machine tool to form image acquisition points;

[0015] During the machining process of the machine tool, the tool is controlled to move to the image acquisition point, a bottom wear image of the tool is acquired through the industrial camera below, a side edge wear image of the tool is acquired through the industrial camera on the side, and the tool is controlled to rotate at the image acquisition point to acquire wear images of different side edges of the tool; wherein the bottom wear image of the tool and the side edge wear image of the tool form a tool wear image;

[0016] The first sample set is constructed by marking the wear area of ​​the tool wear image.

[0017] According to the tool wear prediction method based on machine vision provided by the present invention, the wear area segmentation model is an improved YOLOv8 model, and the improved YOLOv8 model includes a Conv module, an M module, an SPPF module, an Upsample module and a Concat module, wherein the M module is composed of a FasterNeXt module; the Conv module performs a convolution operation on the feature map to extract local features of the image; the FasterNeXt module generates an intermediate feature map after a convolution operation on the input feature map; the SPPF module realizes multi-scale extraction and fusion of features through multi-level spatial pyramid pooling; the Upsample module uses deconvolution to restore the resolution of the feature map; and the Concat module realizes feature fusion and jump connection.

[0018] According to the machine vision-based tool wear prediction method provided by the present invention, the wear value of the tool is obtained by calculating the wear area segmented from the tool wear image based on the wear area segmentation model, specifically comprising:

[0019] Obtain pixel coordinates of vertices of the wear area to form a vertex point set;

[0020] Traversing the vertex point set, obtaining equations of line segments between two vertices; and determining the original cutting edge line segment of the tool by screening the slope of the line segment and the number of vertices passed;

[0021] Calculate the vertical distances between the remaining points of the vertex point set and the original cutting edge line segment of the tool, and select the value with the largest vertical distance as the number of pixels of the maximum wear width;

[0022] According to the number of pixels of the maximum wear width and the size of the tool wear image, the actual value of the maximum wear width is obtained as the wear value.

[0023] According to the machine vision-based tool wear prediction method provided by the present invention, the wear prediction model is an MN structure Transformer-CNN model, which is composed of M stacked Transformer modules and N stacked CNN modules, and is used to predict future sequence data of tool wear values ​​based on historical sequence data of tool wear values;

[0024] The wear prediction model is trained based on the mean square error MSE shown in the following formula:

[0025]

[0026] Among them, y i is the actual value of the ith data point, is the predicted value of the ith data point, and n is the total number of predicted data points.

[0027] According to the tool wear prediction method based on machine vision provided by the present invention, building a wear prediction model also includes:

[0028] Dividing the second sample set into a training set and a test set;

[0029] The Grey Wolf Algorithm is used to train the wear prediction model independently for multiple times and minimize the average RMSE on the test set as the objective function. M and N are integers in [1,20] as constraints to optimize the combination of the optimal hyperparameters M and N.

[0030] According to the tool wear prediction method based on machine vision provided by the present invention, the online prediction specifically includes:

[0031] The wear prediction model inputs a sequence of true values ​​of tool wear values ​​when predicting for the first time, and outputs a sequence of predicted values ​​of tool wear values;

[0032] Afterwards, the wear prediction model inputs a combination of the true value of the tool wear value and the predicted value output from the previous prediction, or a sequence of predicted values ​​output from the previous prediction, and outputs a sequence of predicted values ​​of the tool wear value, forming an advanced prediction mode.

[0033] According to the tool wear prediction method based on machine vision provided by the present invention, in the advance prediction mode, the online prediction further includes:

[0034] Timely correction strategy: comparing the predicted value sequence of tool wear values ​​output by the wear prediction model with the corresponding true value sequence of tool wear values ​​to obtain the error between the predicted value and the true value;

[0035] If the error is less than the preset threshold, online prediction will continue in the advance prediction mode; if the error is greater than or equal to the preset threshold, when the wear prediction model makes the next prediction, the input will be the true value sequence of the tool wear value to correct the model input; thereafter, online prediction will continue in the advance prediction mode.

[0036] According to the machine vision-based tool wear prediction method provided by the present invention, the true value of the tool wear value is calculated and obtained based on the wear area segmentation model of the tool wear image collected online, or is determined based on a wear value sample set established in advance through experiments.

[0037] According to another aspect of the present invention, there is provided a tool wear prediction system based on machine vision, comprising a machine vision system, the machine vision system comprising a lower camera and a side camera, the lower camera being arranged below a portion outside a machining area of ​​a machine tool, the side camera being arranged to the side of a portion outside a machining area of ​​a machine tool, the machine vision system being used to form an online image acquisition point on a machine tool;

[0038] It also includes a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, any of the above-mentioned tool wear prediction methods based on machine vision is executed.

[0039] In general, compared with the prior art, the above technical solutions conceived by the present invention provide a tool wear prediction method and system based on machine vision:

[0040] 1. The direct method is used to obtain tool wear images with the help of machine vision technology. The wear area segmentation model is trained based on the tool wear image to achieve online segmentation of the tool wear area, and then the wear value can be calculated online. There is no need to obtain the wear value through offline testing, which can significantly improve the acquisition speed of the prediction model training sample set and simplify the operation. When the wear prediction model obtained based on the wear value training is used for online prediction, the wear value can also be obtained online, and real-time prediction can be achieved without interfering with production. This prediction method can greatly improve production efficiency;

[0041] 2. The proposed automated online tool wear monitoring method can automatically obtain tool bottom and side wear samples during the production process, providing an innovative method for automated collection of wear images;

[0042] 3. Although the number of layers of the optimized YOLOv8-FasterNeXt model, i.e. the wear area segmentation model, has increased, its parameters have been reduced to 86% and its size has been reduced to 87% due to the structural characteristics of the replacement module. On the RTX3080 graphics card, the inference speed of each image has been reduced to 84% and the amount of calculation has been reduced to 89%. The average accuracy of the model at IoU=0.5 has not decreased, and the average accuracy at IoU=0.5-0.95 has been improved. In general, the improved YOLOv8 model has lightweight characteristics and improved segmentation accuracy. It can identify wear areas with high accuracy, and the absolute measurement error on the test set is within 5μm, which can provide accurate input data for the subsequent wear prediction model with a small measurement error.

[0043] 4. Under the traditional prediction mode, the Transformer-CNN prediction model, i.e. the wear prediction model, has an accuracy of 99.98%, 99.98%, 99.99% and 99.95%, 99.99%, 99.99% in three sets of experiments on public and experimental data respectively;

[0044] 5. The optimal parameters obtained by the Grey Wolf Algorithm outperformed other hyperparameter combinations on both public and experimental data, proving the effectiveness of the Grey Wolf Optimization Algorithm;

[0045] 6. It was verified that the proposed GWO-Transformer-CNN tool wear prediction model can be unsupervisedly transferred to cross-tool, cross-working condition, cross-material, cross-equipment and other cross-domain tool wear prediction tasks with lower error and higher accuracy, which proved the robustness and generalization of the proposed method. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 It is an overall framework diagram of the tool wear prediction method based on machine vision provided by the present invention;

[0047] Figure 2 This is the structure diagram of the YOLOv8-FasterNeXt tool wear area segmentation model;

[0048] Figure 3 This is the inference effect diagram of the tool wear area segmentation model, where (a) is not worn; (b) is the initial stage of wear; (c) is the middle stage of wear; (d) is the late stage of wear;

[0049] Figure 4 It is a process of calculating the quantified tool wear value, wherein: (a) the original cutting edge is identified; (b) the quantified tool wear value is obtained; (c) the conversion between pixel value and actual length is performed;

[0050] Figure 5 It is the structure diagram of the wear prediction model;

[0051] Figure 6 It is the visualization result of the solution process of the Grey Wolf Algorithm;

[0052] Figure 7 This is the principle diagram of the advance prediction mode. DETAILED DESCRIPTION

[0053] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention. In addition, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0054] See also Figure 1 This embodiment provides a tool wear prediction method based on machine vision, the method comprising:

[0055] Constructing a first sample set: the samples in the first sample set are tool wear images and corresponding wear area labels;

[0056] Constructing a wear area segmentation model: the wear area segmentation model is used to segment the wear area according to the tool wear image, and the wear area segmentation model is trained using the first sample set to obtain the trained wear area segmentation model;

[0057] Constructing a second sample set: the samples in the second sample set are a wear value sequence of the tool; wherein the wear value of the tool is calculated and obtained based on the wear area segmented from the tool wear image by the wear area segmentation model;

[0058] Constructing a wear prediction model: the wear prediction model is used to predict the wear value of the tool, and the wear prediction model is trained using the second sample set to obtain the trained wear prediction model;

[0059] Online prediction: During the machining process of the machine tool, the wear value of the tool is predicted online using the trained wear prediction model.

[0060] This embodiment adopts a direct method with the help of machine vision technologies such as cameras and image processing. By training the wear area segmentation model based on the tool wear image, the online segmentation of the tool wear area can be realized, and then the wear value can be calculated online. Based on the calculated wear value, the wear prediction model can be trained to obtain online prediction, which can achieve real-time prediction without interfering with production. This prediction method can greatly improve production efficiency. The unique advantages of this direct method in tool wear monitoring cannot be ignored. First, the direct method can provide very accurate wear measurement results, which can accurately reflect the actual wear state of the tool. Second, the wear value provided by the direct method is a direct and specific value, which is easy to understand and analyze, and avoids the uncertainty caused by signal conversion in the indirect method. Third, the direct method is based on physical measurement, and its results have high reliability and repeatability, and are not easily affected by external factors. Therefore, the direct method has broad application potential and prospects in tool wear monitoring.

[0061] Furthermore, this embodiment takes into account the existing tool wear prediction model based on deep learning, which generally divides the entire data set into training set and test set randomly in proportion, which belongs to interpolation prediction. The model can only predict a set of outputs based on a set of inputs, and cannot continuously predict tool wear for multiple steps in the future; in addition, existing research rarely conducts cross-domain tool wear prediction research across working conditions, tools, materials and equipment, and this research is of great significance in actual production.

[0062] In view of the above shortcomings of the prior art, the solutions proposed in this embodiment mainly include the following points:

[0063] Establish a fully automatic data acquisition platform to automatically acquire tool wear images;

[0064] Using tool wear images, a machine vision model is constructed to automatically identify tool wear areas and further calculate and obtain quantitative tool wear values;

[0065] Construct a tool wear prediction model to achieve the mapping relationship between historical and future tool wear sequences;

[0066] By improving the traditional prediction model and combining it with correction strategies, we can achieve predictions with ultra-long time steps while ensuring prediction accuracy.

[0067] Specifically, an automated data acquisition platform is built to automatically acquire tool wear images. When acquiring tool wear images, a safe photo-taking point is set inside the machine tool, and two industrial cameras and lenses are installed below and on the side to acquire images of the bottom and side surfaces of the tool, respectively. In order to obtain images of each side edge of the tool, an M100CN instruction is developed to orient the machine tool spindle to any angle, where N is the angle of rotation with the encoder zero point of the machine tool as the reference point. In addition, the software supporting the camera provides a shortcut key function for taking photos, uses the TCP / IP network protocol to establish communication between the machine tool and the PC, and combines the FOCAS communication protocol to automatically acquire images of the wear of each edge on the bottom and side surfaces of the tool.

[0068] After obtaining the tool wear image, the wear area is annotated using annotation software to generate a label file to construct a data set, and the data set is divided into a training set, a validation set, and a test set according to a certain ratio. With the help of the FasterNeXt module, the segmentation model structure is optimized, the model parameters are reduced, the model is lightweight, and the reasoning accuracy is also improved; the training set is used to train the instance segmentation model, the validation set and the test set are used to evaluate the performance of the instance segmentation model, and the test set is used to evaluate the accuracy of the wear measurement method based on the instance segmentation model. A deep learning instance segmentation model is built, and the wear image and its label file are used as the model input to iteratively train the model to determine the optimal parameters, so that it can learn key features autonomously, and finally has the ability to segment the tool wear area, can classify the input image at the pixel level, and intelligently segment the tool wear area. After the training, the performance of the model is evaluated on the validation set. By improving the segmentation model structure and retraining the model, its indicators on the validation set are improved. In order to obtain the quantitative tool wear value, the quantitative wear value is further calculated based on morphology on the basis of instance segmentation of the tool wear area.

[0069] The tool wear prediction model uses a neural network to establish a nonlinear relationship between historical and future tool wear sequences, and realizes the mapping of tool wear sequences from history to future. In view of the powerful parallel computing capability of the Transformer model and the parameter sharing mechanism of the convolutional network, the MN structure Transformer-CNN tool wear prediction model is established to complete the prediction of historical to future wear sequences. First, the input data is reshaped into a three-dimensional format [batch size, sequence length, features] so that it can be used as the input of M stacked Transformer models. Secondly, a Transformer model with M stacks is constructed, and its output is used as the input of N stacked CNN models. Finally, the input passes through each layer of the model to obtain the model output and calculate the loss with the label. The parameters of the model are iteratively updated through the back-propagation algorithm to obtain the optimal parameter prediction model.

[0070] In addition, in order to ensure the robustness and stability of the model, the present invention uses the Gray Wolf Algorithm, takes the model independently trained 5 times and the average RMSE value on the test set as the objective function, and calculates the optimal hyperparameter combination M and N, which has a significant improvement in prediction performance compared with other parameter combinations.

[0071] In view of the fact that the traditional interpolation prediction mode lacks the ability to perceive wear trends in ultra-long time steps, an advance prediction mode is proposed. In this mode, the model fuses the input with the prediction output of the previous time step each time it makes a prediction, so it has the prediction capability of infinite time steps. At the same time, the decrease in prediction accuracy caused by the accumulation of prediction errors is becoming more and more serious. Therefore, this embodiment proposes a timely correction strategy. The timely correction strategy can make up for the error accumulation problem in the advance prediction mode and correct the model input in time; the combination of the advance prediction mode and the timely correction strategy enables the model to have ultra-long time step prediction capabilities while ensuring the model prediction accuracy.

[0072] The prediction method provided in this embodiment has been used to carry out cross-domain prediction, which has effectively verified the high generalization ability of the prediction model. It can realize cross-domain prediction and use the model trained with source domain data to directly predict the data in the target domain, which reduces the data annotation cost and enhances the applicability and flexibility of the model, and is of great significance for engineering applications.

[0073] In another embodiment, a tool wear prediction system based on machine vision is further provided, comprising a machine vision system, wherein the machine vision system comprises a lower camera and a side camera, wherein the lower camera is arranged below a portion outside a machining area of ​​a machine tool, and the side camera is arranged to the side of a portion outside a machining area of ​​a machine tool, and wherein the machine vision system is used to form an online image acquisition point on a machine tool;

[0074] It also includes a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the tool wear prediction method based on machine vision described in any one of the embodiments is executed.

[0075] This embodiment provides a method for advanced prediction of tool wear across multiple domains based on machine vision, including a fully automated data acquisition platform, a tool wear region segmentation model, a tool wear quantification method, a tool wear prediction model, a gray wolf algorithm optimization hyperparameter, a tool wear advanced prediction mode, a timely correction strategy, and cross-domain prediction, such as Figure 1 The following is a clear and complete description of the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings.

[0076] Example 1

[0077] The fully automated data acquisition platform is constructed; that is, industrial cameras are respectively arranged below and on the sides of the parts outside the machining area of ​​the machine tool to form image acquisition points; during the machining process of the machine tool, the tool is controlled to move to the image acquisition point, the bottom wear image of the tool is acquired through the industrial camera below, the side edge wear image of the tool is acquired through the industrial camera on the side, and the tool is controlled to rotate at the image acquisition point to acquire wear images of different side edges of the tool; wherein the bottom wear image of the tool and the side edge wear image of the tool form a tool wear image; and the first sample set is constructed by annotating the wear area of ​​the tool wear image.

[0078] First, a safe photo-taking spot was set up inside the machine tool, and two industrial cameras and lenses were installed below and on the side to take images of the wear on the bottom and side surfaces of the tool respectively.

[0079] Secondly, in order to automatically measure the wear of each side edge of the tool, a custom M100CN command is used to orient the machine tool spindle to any angle, where N is the angle of rotation with the machine tool encoder zero point as the reference point; by controlling the rotation angle of the spindle, the side camera can collect wear images of different side edges;

[0080] Finally, a C# script was developed based on the FOCAS communication protocol to monitor the status of the machine tool, and shortcut keys were used to automatically take pictures and store tool wear images. The specific implementation steps are shown in Table 1.

[0081] Table 1 Online measurement method of tool wear

[0082]

[0083]

[0084] Example 2

[0085] A tool wear area segmentation model was constructed. In order to identify the tool wear area, machine vision was used to segment the tool wear area. The wear area in the tool wear image in the first sample set was manually marked using labeling software. The 2176 samples obtained were divided into training set, validation set and test set in the ratio of 1546:546:84. The training set was used to train the YOLOv8 instance segmentation model, and the validation set and test set were used to evaluate the instance segmentation performance of the YOLOv8 model.

[0086] Use as Figure 2The improved YOLOv8 model architecture shown identifies the wear area of ​​the tool; that is, the wear area segmentation model is an improved YOLOv8 model, and the improved YOLOv8 model is mainly composed of basic modules such as Conv module, M module, SPPF module, Upsample module and Concat module; the Conv module performs convolution operation on the feature map to extract local features of the image; the SPPF module realizes multi-scale extraction and fusion of features through multi-level spatial pyramid pooling, thereby improving the generalization ability of the model; the Upsample module uses deconvolution to restore the resolution of the feature map, providing more accurate spatial information, thereby improving the positioning accuracy of the target; the Concat module realizes feature fusion and jump connection, and enhances the model feature expression ability;

[0087] In the native YOLOv8 architecture, M is composed of C2f modules; this module achieves efficient calculation by splitting, processing and fusing features, and provides rich feature representation; in the proposed YOLOv8-FasterNeXt architecture, i.e., the improved YOLOv8 model, FasterNeXt is used to replace the C2f module, i.e., the M module is composed of FasterNeXt modules; the FasterNeXt module generates an intermediate feature map after convolution operation on the input feature map, one part of which is processed by X FasterNetBlock submodules, and the other part remains unchanged, and the two parts are concatenated and then a layer of convolution is performed to generate the final feature map; this optimized FasterNeXt module helps to maintain computational efficiency while enhancing the network's expressiveness, and is suitable for tasks that efficiently process feature maps;

[0088] These basic modules are based on Figure 2 The connection on the right constitutes the instance segmentation model architecture, where different Segment layers identify large, medium, and small scale objects in the input image. This output layer has anchor boxes of different sizes, which enables the model to extract and fuse features and ensure good detection of multi-scale objects.

[0089] In order to verify the effectiveness of the improved YOLOv8-FasterNeXt, the comparison of the inference indicators on the validation set before and after the improvement is shown in Table 2; it can be seen that although the number of layers of the optimized model has increased, due to the structural characteristics of the replacement module, the number of parameters is reduced to 86% of the original, and the size is reduced to 87%; on the RTX3080 graphics card, the inference speed of each picture is reduced to 84%, and the amount of calculation is reduced to 89%; more importantly, the average accuracy of the model at IoU = 0.5 has not decreased, and the average accuracy at IoU = 0.5-0.95 has improved; in general, the improved YOLOv8-FasterNeXt model has lightweight characteristics while improving segmentation accuracy;

[0090] like Figure 3 The results of model instance segmentation of randomly selected pictures from various stages of the entire life cycle of the tool. It can be seen that the wear area segmentation model can effectively identify the wear area at any stage of tool wear, providing a basis for subsequent tool wear measurement.

[0091] Table 2 Comparison of inference indicators before and after YOLOv8 model optimization

[0092]

[0093] Example 3

[0094] Tool wear value quantification method, the wear area is obtained by segmentation and then the tool wear value is further calculated; the International Organization for Standardization stipulates that the blunting standard of carbide tools during rough machining is: under normal wear of cutting tools, the average wear width of the tool back face VB average =0.3 or maximum wear width VB max =0.5 is the blunting standard; this embodiment adopts the maximum wear width VB max Conduct follow-up studies as a blunting standard;

[0095] The trained YOLOv8-FasterNeXt can accurately identify irregular polygonal tool wear areas, and the vertex coordinates of the area can be used to further calculate the quantitative tool wear value VB max ; Each vertex of the wear area is a set of two-dimensional points (x i ,y j ),i=1,2,…,n;

[0096] Find the maximum wear width VB of the side edge max When the tool is cut, the original cutting edge position of the tool must be determined first; by observing Figure 3 According to the tool wear characteristics, it is found that the original cutting edge of the tool can be approximated by the longest edge of the polygon; therefore, the point set on the wear area is traversed to obtain the equation of the line segment between two points; the original cutting edge L of the tool can be easily determined by screening the line segments with less slope and passing through the points ij ,like Figure 4 (a)

[0097] Specifically, for the irregular polygonal area obtained after segmentation, let the target set T = {t1, t2, ..., t n}, where each target t i is a set of two-dimensional points on the polygonal area. Let point P = (x, y) be a point on the two-dimensional plane, and let a line segment represents the straight line segment connecting points p and q; line segment The length d(p,q) is obtained using the Euclidean distance and its slope is calculated; each point pair (p,q) in the target set is traversed, the distance and slope are calculated, and the line segments that do not meet the conditions are filtered out. If the number of points passed by the line segment is greater than or equal to a threshold (the threshold set in the present invention is 20), the longest distance and line segment are updated. Finally, the longest line segment obtained after traversing all vertices of the entire polygon is the edge boundary of the tool when it is not worn, that is, the original cutting edge;

[0098] Then, given the point r = (x3, y3) and the line segment Calculate the distance from the point to the line segment; after obtaining the original cutting edge boundary of the tool, it is necessary to determine the point on the wear area farthest from the boundary, that is, calculate the distance between the remaining points on the polygon and the line segment L ij Vertical distance, the largest value is the maximum wear width VB max The number of pixels, such as Figure 4 (b)

[0099] On this basis, if we know the actual width or height of the image, we can calculate the VB max The industrial camera software used provides basic distance and area measurement functions. After measurement, the actual width and height of the image are 9.7112 and 5.4663 mm respectively, and then VB is calculated. max The actual value of Figure 4 (c) in .

[0100] That is, the wear value of the tool is calculated based on the wear area segmentation model for the wear area segmented from the tool wear image, specifically including:

[0101] Obtain pixel coordinates of vertices of the wear area to form a vertex point set;

[0102] Traversing the vertex point set, obtaining equations of line segments between two vertices; and determining the original cutting edge line segment of the tool by screening the slope of the line segment and the number of vertices passed;

[0103] Calculate the vertical distances between the remaining points of the vertex point set and the original cutting edge line segment of the tool, and select the value with the largest vertical distance as the number of pixels of the maximum wear width;

[0104] According to the number of pixels of the maximum wear width and the size of the tool wear image, the actual value of the maximum wear width is obtained as the wear value.

[0105] Example 4

[0106] Tool wear prediction model; as shown in formula (1), the tool wear prediction model is used to mine the wear value history T1 time series data y calculated based on the wear area provided by the wear area segmentation model (t-T1)：t With the future T2 time series data y t.(t+T2)The relationship between them; this process is implemented using the Transformer model;

[0107] y t: (t+T2) =P(y (t -T1): t ) (1)

[0108] This embodiment uses the powerful parallel computing capability of the Transformer model and combines the parameter sharing mechanism of the convolutional network to propose a MN structure Transformer-CNN tool wear prediction model. Figure 5 ; That is, the wear prediction model is an MN structure Transformer-CNN model, which is composed of M stacked Transformer modules and N stacked CNN modules, and is used to predict the future sequence data of tool wear values ​​based on the historical sequence data of tool wear values;

[0109] The model input is a tensor in the form of a sequence. The Transformer part consists of M stacked encoders and a layer of decoder. Each layer of encoder calculates multi-head attention on the input sequence to capture the internal relationship of the input sequence. The feed forward layer performs feature extraction and nonlinear transformation. Normalization is performed after the multi-head attention mechanism layer and the feed forward layer. The normalization layer helps to reduce the internal covariate shift in training and improve the stability of model training. The residual connection helps to accelerate the training process and prevent gradient disappearance and explosion.

[0110] The input tensor is input into N stacked convolutional networks through M stacked encoder and decoder layers, and finally the predicted wear value is output through two layers of fully connected networks. During back propagation, the model propagates to each layer of the network with mean square error (MSE), thereby updating the weights and biases of the network. That is, the wear prediction model is trained based on mean square error MSE, and the MSE loss calculation formula is shown in formula (2), y i is the actual value of the ith data point, is the predicted value of the i-th data point, and n is the total number of predicted data points;

[0111]

[0112] The number of M stacked Transformer layers and the number of N stacked CNN layers as hyperparameters of the model have an important impact on the prediction results; increasing the number of stacked layers can enhance the expressiveness of the model, so that the model can better capture the complex relationships in the input sequence; however, too many stacked layers will increase the complexity and training cost of the model; constructing the wear prediction model also includes: dividing the second sample set into a training set and a test set; using the gray wolf algorithm, with the wear prediction model independently trained multiple times and the minimum RMSE average value on the test set as the objective function, and with M and N being integers in [1,20] as constraints, optimizing the combination of the optimal hyperparameters M and N.

[0113] In order to verify the effectiveness of the GWO-Transformer-CNN tool wear prediction model, i.e., the MN structure Transformer-CNN model after optimizing the hyperparameters M and N using the Gray Wolf algorithm, its performance is first verified in the traditional prediction mode; its prediction indicators on the test sets of public and experimental data compared with other advanced models are shown in Tables 3 and 4 respectively; it can be seen that the wear prediction model of the present invention has better prediction performance. C1, C4, and C6 are three existing public data sets; Exp1, Exp2, and Exp3 are three data sets constructed through experiments based on the wear area segmentation model provided by the present invention.

[0114] Table 3 Prediction indicators of the prediction model on the public data test set

[0115]

[0116] Table 4 Prediction indicators of the prediction model on the experimental data test set

[0117]

[0118]

[0119] Example 5

[0120] The Gray Wolf Algorithm is used to optimize the hyperparameters of the tool wear prediction model. For regression tasks, RMSE, MSE, MAE and other indicators are generally used to evaluate the model performance. Due to the randomness of the initialization parameters of the neural network, the evaluation indicators of the model after training are also random. Therefore, in order to ensure the robustness and stability of the model, the average RMSE value of the model independently trained 5 times on the test set is used as the objective function of model optimization when optimizing the M and N hyperparameters. The variables to be optimized M and N are integers in [1,20], n is the number of training times, and f is the training set. M，N (x) RMSE is the RMSE value under the combination of M and N values, so the hyperparameter optimization task of the prediction model can be defined as shown in formula (3);

[0121]

[0122] In order to make the model have a lower RMSE index, the gray wolf optimization algorithm was used to solve the optimal model hyperparameters M and N. The solution steps are shown in Table 5;

[0123] Table 5 Grey Wolf Algorithm Optimization Prediction Model Hyperparameter Process

[0124]

[0125]

[0126] The visualization results of the gray wolf algorithm solution process are as follows Figure 6 As shown; it can be seen that the objective function has a minimum value when the optimal hyperparameters M and N are both in the range of [1,10];

[0127] In order to verify the effectiveness of GWO-Transformer-CNN, M and N are divided into 3 levels (low: 1, medium: 6, and high: 10), and orthogonal experiments are carried out on the public and experimental data test sets. The results are shown in Tables 6 and 7 respectively; it can be seen that the optimal hyperparameters obtained by the Grey Wolf Algorithm have better prediction performance.

[0128] Table 6 Comparison of prediction indicators of different hyperparameters on the public data test set

[0129]

[0130] Table 7 Comparison of prediction indicators of different hyperparameters on the experimental data test set

[0131]

[0132] Example 6

[0133] The advance prediction mode; Figure 7 The difference between the traditional tool wear prediction and the advanced prediction mode is shown; the traditional tool wear prediction mode relies on historical real data, that is, each prediction needs to provide the real value of the historical wear value; in this mode, the wear value of the historical T1 time point needs to be calculated to be used as the input of the prediction model; therefore, the N-time prediction of the prediction model requires the segmentation model to provide the historical wear data of T1*N time points, which seriously affects the real-time performance of the prediction model;

[0134] In contrast, the advanced prediction mode feeds back the predicted value output by the prediction model to the input end for integration with the historical real value data, continuously updating the model input for the next prediction, thereby making continuous predictions; in this mode, theoretically the model can make unlimited recursive predictions based on a set of model inputs; however, as the number of predictions increases, the accumulation of errors cannot be ignored, so it is very necessary to use historical tool wear values ​​to correct the model input in a timely manner after a certain number of predictions.

[0135] That is, online prediction specifically includes:

[0136] The wear prediction model inputs a sequence of true values ​​of tool wear values ​​when predicting for the first time, and outputs a sequence of predicted values ​​of tool wear values;

[0137] Afterwards, the wear prediction model inputs a combination of the true value of the tool wear value and the predicted value output from the previous prediction, or a sequence of predicted values ​​output from the previous prediction, and outputs a sequence of predicted values ​​of the tool wear value, forming an advanced prediction mode.

[0138] Example 7

[0139] In the look-ahead prediction mode, online prediction also includes:

[0140] Timely correction strategy: comparing the predicted value sequence of tool wear values ​​output by the wear prediction model with the corresponding true value sequence of tool wear values ​​to obtain the error between the predicted value and the true value;

[0141] If the error is less than the preset threshold, the online prediction is continued in the advance prediction mode; if the error is greater than or equal to the preset threshold, the real value sequence of the tool wear value is input when the wear prediction model makes the next prediction to correct the model input; then the online prediction is continued in the advance prediction mode. During the processing, timely correction can be made every preset time interval or preset number of cuts.

[0142] Specifically, timely correction strategy; In the advanced prediction mode, the accumulation of prediction errors leads to an inevitable decrease in prediction accuracy; Therefore, a timely correction strategy is proposed, and its principle is shown in Table 8; S i T Indicates that the real historical sequence at the i-th time point prediction is the model input, S i P Y represents the output concatenation of the previous prediction at the i-th time point as the model input; i ' represents the prediction result output at the i-th time point, P represents the prediction model, △ represents the MAE value between the prediction result at the i-th time point and the actual wear sequence; when T1=3, T2=1, the first prediction occurs at the 4th time point, at this time the model is based on the historical sequence S4T The prediction obtains Y4'; in the future prediction, the prediction can be carried out according to the advance prediction mode, and then the MAE between the previous prediction result and the true value is calculated at each preset time interval or preset number of knife intervals. If it is less than the error threshold δ, then the input of the next prediction will use the previous prediction result fused with historical data as input; on the contrary, if the MAE of the previous prediction is greater than δ, then the next prediction uses the real wear data as input, so as to avoid the accumulation of errors in time;

[0143] Specifically, the true value of the tool wear value is calculated and obtained based on the wear area segmented by the wear area segmentation model from the tool wear image collected online, or determined based on a wear value sample set established in advance through experiments. During the online prediction process, a prediction can be made at regular intervals, and each prediction can be made according to the number of cuts or processing time. Taking the prediction according to the number of cuts as an example: the input of the wear prediction model for each prediction is a sequence of true wear values ​​corresponding to several cutter processes before the current moment, a fusion sequence of true wear values ​​and predicted values, or a sequence of predicted wear values, and the output is a predicted wear value corresponding to at least one cutter process after the current moment.

[0144] The true value of the wear value can be obtained by moving the spindle to the image acquisition point after the corresponding tool processing to collect the tool wear image online, segmenting the wear area through the wear area segmentation model, and then calculating the true value of the wear value corresponding to the corresponding tool processing; or, a processing experiment can be carried out in advance to obtain the wear value that changes with the number of processing tools through the processing experiment, and establish a wear value sample set related to the number of processing tools. In the online prediction process, the wear value corresponding to the corresponding tool processing can be directly selected from the wear value sample set as the true value of the wear value input to the model.

[0145] The true value of the wear value used for error judgment with the predicted wear value in timely correction can be obtained in the same way, that is, the number of processing tools corresponding to the predicted wear value can be obtained by online collection of tool mold damage images and then segmentation and calculation, or the wear value corresponding to the corresponding tool processing can be obtained by selecting the wear value sample set from the wear value sample set, without specific limitation.

[0146] In order to verify the effectiveness of the timely correction strategy in the look-ahead mode, predictions were made on the public and experimental data test sets without using the correction strategy under different conditions of T1 and T2, and the results are shown in Tables 9 and 10, respectively. Then, the prediction results after using the correction strategy on the public and experimental data are shown in Tables 11 and 12, respectively. It can be seen that after using the timely correction strategy, both the error and the accuracy are significantly improved at different sequence lengths.

[0147] Table 8 Principles of timely correction strategy

[0148]

[0149] Table 9 Prediction indicators of the prediction model without correction strategy in the advance mode on the public data test set

[0150]

[0151]

[0152] Table 10 Prediction indicators of the prediction model in the advance mode without using the correction strategy on the experimental data test set

[0153]

[0154]

[0155] Table 11 Prediction indicators of the prediction model using the correction strategy in the advance mode on the public data test set

[0156]

[0157] Table 12 Prediction indicators of the prediction model using the correction strategy in the advance mode on the experimental data test set

[0158]

[0159] Example 8

[0160] The cross-domain prediction uses the trained model in the source domain to directly predict the data in the target domain, which has important practical significance for engineering applications;

[0161] The public data used in the present invention is PHM2010, including three experimental data of C1, C4 and C6, and the experimental conditions and cutting parameters are shown in Table 13; it should be noted that the public data is an experiment carried out under the same working conditions, using tools with different blade geometries and coatings;

[0162] Table 13 PHM2010 cutting parameters and experimental configuration

[0163]

[0164] When the present invention uses experimental data, a 12mm diameter 4-edge tungsten steel end mill is used to carry out cutting experiments on a rectangular mold steel; a side cutting method is adopted, and each time the material of the mold steel length is cut from the X direction, the tool is moved to the photo point for taking pictures; a total of three independent cutting experiments Exp1 to Exp3 are carried out, and the cutting parameters and experimental conditions are shown in Table 14; it should be noted that the same tool is used in the three experiments; Exp1 and Exp2 have the same working conditions, but different workpieces; Exp2 and Exp3 have the same workpieces, but different working conditions; Exp1 and Exp3 have different working conditions and workpieces;

[0165] Table 14 PHM2010 cutting parameters and experimental configuration

[0166]

[0167]

[0168] Since the three groups of data C1, C4, and C6 of the public data were collected using different tools, in order to verify the generalization of the prediction model, the model trained on one group of data was used to predict another group of data unsupervisedly, thereby verifying the cross-tool prediction performance of the model; the results are shown in Table 15, which shows that the prediction accuracy of the model in cross-tool prediction is at least 99.59%, which has good generalization ability;

[0169] Table 15 Public data cross-tool prediction indicators

[0170]

[0171] Since the experimental data are collected under different working conditions and different workpiece materials, in order to verify the generalization ability of the model across working conditions and materials, the experimental data are used cross-wise, and the model trained with the source domain data is used to predict the data in the target domain. The results are shown in Table 16. It can be seen that the model has good generalization ability across working conditions and materials.

[0172] Table 16 Experimental data cross-condition and cross-material prediction indicators

[0173]

[0174] The public data PHM is obtained by high-speed milling of nickel-chromium-iron alloy by a ball-end milling cutter, while the Exp data is obtained by low-speed milling of mold steel by an end milling cutter. The two are extremely different in terms of cutting tools, workpiece materials, equipment, and working conditions. In order to further explore the cross-domain prediction performance of the prediction model between the two types of data, the model trained with Exp data is used to predict PHM data. The results are shown in Table 17. It can be seen that when the model trained with Exp data predicts PHM data unsupervised, the prediction accuracy is above 99.06%, which has excellent generalization ability.

[0175] Table 17 Prediction indicators of public and experimental data across multiple domains

[0176]

[0177] The present invention verifies that the proposed GWO-Transformer-CNN tool wear prediction model can be unsupervisedly transferred to cross-tool, cross-material, cross-equipment and other cross-domain tool wear prediction tasks with lower error and higher accuracy, proving the robustness and generalization of the proposed method.

[0178] It will be easily understood by those skilled in the art that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A tool wear prediction method based on machine vision, characterized in that: include: Constructing a first sample set: the samples in the first sample set are tool wear images and corresponding wear area labels; Constructing a wear area segmentation model: the wear area segmentation model is used to segment the wear area according to the tool wear image, and the wear area segmentation model is trained using the first sample set to obtain the trained wear area segmentation model; Constructing a second sample set: the samples in the second sample set are a wear value sequence of the tool; wherein the wear value of the tool is calculated and obtained based on the wear area segmented from the tool wear image by the wear area segmentation model; Constructing a wear prediction model: the wear prediction model is used to predict the wear value of the tool, and the wear prediction model is trained using the second sample set to obtain the trained wear prediction model; Online prediction: During the machining process of the machine tool, the wear value of the tool is predicted online using the trained wear prediction model.

2. The tool wear prediction method based on machine vision according to claim 1, characterized in that: Also includes: Industrial cameras are respectively arranged below and on the sides of the parts outside the processing area of ​​the machine tool to form image acquisition points; During the machining process of the machine tool, the tool is controlled to move to the image acquisition point, a bottom wear image of the tool is acquired through the industrial camera below, a side edge wear image of the tool is acquired through the industrial camera on the side, and the tool is controlled to rotate at the image acquisition point to acquire wear images of different side edges of the tool; wherein the bottom wear image of the tool and the side edge wear image of the tool form a tool wear image; The first sample set is constructed by marking the wear area of ​​the tool wear image.

3. The tool wear prediction method based on machine vision according to claim 1, characterized in that: The wear area segmentation model is an improved YOLOv8 model, which includes a Conv module, an M module, an SPPF module, an Upsample module and a Concat module, wherein the M module is composed of a FasterNeXt module; the Conv module performs a convolution operation on the feature map to extract local features of the image; the FasterNeXt module generates an intermediate feature map after a convolution operation on the input feature map; the SPPF module realizes multi-scale extraction and fusion of features through multi-level spatial pyramid pooling; the Upsample module uses deconvolution to restore the resolution of the feature map; and the Concat module realizes feature fusion and jump connection.

4. The tool wear prediction method based on machine vision according to claim 1, characterized in that: The wear area segmented from the tool wear image is calculated based on the wear area segmentation model to obtain the wear value of the tool, specifically including: Obtain pixel coordinates of vertices of the wear area to form a vertex point set; Traversing the vertex point set, obtaining equations of line segments between two vertices; and determining the original cutting edge line segment of the tool by screening the slope of the line segment and the number of vertices passed; Calculate the vertical distances between the remaining points of the vertex point set and the original cutting edge line segment of the tool, and select the value with the largest vertical distance as the number of pixels of the maximum wear width; According to the number of pixels of the maximum wear width and the size of the tool wear image, the actual value of the maximum wear width is obtained as the wear value.

5. The tool wear prediction method based on machine vision according to any one of claims 1 to 4, characterized in that: The wear prediction model is an MN structure Transformer-CNN model, which is composed of M stacked Transformer modules and N stacked CNN modules, and is used to predict future sequence data of tool wear values ​​based on historical sequence data of tool wear values; The wear prediction model is trained based on the mean square error MSE shown in the following formula: Among them, y i is the actual value of the ith data point, is the predicted value of the ith data point, and n is the total number of predicted data points.

6. The tool wear prediction method based on machine vision according to claim 5, characterized in that: Building a wear prediction model also includes: Dividing the second sample set into a training set and a test set; The Grey Wolf Algorithm is used to train the wear prediction model independently for multiple times and minimize the average RMSE on the test set as the objective function. M and N are integers in [1,20] as constraints to optimize the combination of the optimal hyperparameters M and N.

7. The tool wear prediction method based on machine vision according to any one of claims 1 to 4, characterized in that: Online prediction specifically includes: The wear prediction model inputs a sequence of true values ​​of tool wear values ​​when predicting for the first time, and outputs a sequence of predicted values ​​of tool wear values; Afterwards, the wear prediction model inputs a combination of the true value of the tool wear value and the predicted value output from the previous prediction, or a sequence of predicted values ​​output from the previous prediction, and outputs a sequence of predicted values ​​of the tool wear value, forming an advanced prediction mode.

8. The tool wear prediction method based on machine vision according to claim 7, characterized in that: In the look-ahead prediction mode, online prediction also includes: Timely correction strategy: comparing the predicted value sequence of tool wear values ​​output by the wear prediction model with the corresponding true value sequence of tool wear values ​​to obtain the error between the predicted value and the true value; If the error is less than the preset threshold, online prediction will continue in the advance prediction mode; if the error is greater than or equal to the preset threshold, when the wear prediction model makes the next prediction, the input will be the true value sequence of the tool wear value to correct the model input; thereafter, online prediction will continue in the advance prediction mode.

9. The machine vision-based tool wear prediction method according to claim 8, characterized in that: The true value of the tool wear value is calculated and obtained based on the wear area segmented from the tool wear image collected online using the wear area segmentation model, or is determined based on a wear value sample set established in advance through experiments.

10. A tool wear prediction system based on machine vision, characterized in that: The machine vision system includes a bottom camera and a side camera, wherein the bottom camera is arranged below the part outside the processing area of ​​the machine tool, and the side camera is arranged to the side of the part outside the processing area of ​​the machine tool, and the machine vision system is used to form an online image acquisition point on the machine tool; It also includes a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the tool wear prediction method based on machine vision described in any one of claims 1 to 9 is executed.

Images