Tool wear detection method, device, storage medium and program product

Through multimodal data fusion and enhancement technology, the problem of low wear detection accuracy of CNC machine tools is solved, and higher detection accuracy and adaptability are achieved.

CN118617196BActive Publication Date: 2024-11-26ZKHM CNC SOFTWARE (SHENZHEN) CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411112269.9
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-08-14
Publication Date
2024-11-26
Estimated Expiration
2044-08-14

AI Technical Summary

Technical Problem

The existing CNC machine tool wear detection technology relies on a single type of sensor and is susceptible to noise interference. Traditional visual inspection is sensitive to ambient light and surface conditions, and the data processing is complex, resulting in low detection accuracy.

Method used

Multimodal data (vibration data, temperature data, sound data and visual data) are combined to generate enhanced data with similar distribution characteristics through the data augmentation model, and data features are extracted using the fusion model to determine the tool wear state.

Benefits of technology

It improves the accuracy of tool wear detection, enhances the ability to adapt to variable environments and complex data modes, and significantly improves the comprehensiveness and accuracy of detection.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118617196B_ABST
    Figure CN118617196B_ABST
Patent Text Reader

Abstract

The present application discloses a tool wear detection method, device, storage medium and program product, and relates to the technical field of digital data processing. The present application discloses a tool wear detection method, comprising: acquiring multimodal data collected by a sensor, wherein the multimodal data comprises at least two of vibration data, temperature data, sound data and visual data; inputting the multimodal data into a preset data enhancement model, acquiring enhanced data output by the data enhancement model, wherein the data enhancement model is used to generate the enhanced data having distribution characteristics similar to those of the multimodal data; combining the enhanced data with the multimodal data to form a data set; respectively extracting data features of different types of the data sets, and determining the wear state of the tool according to the different types of data features, so as to improve the accuracy of tool wear detection.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the technical field of digital data processing, and in particular to a tool wear detection method, device, storage medium and program product. Background Art

[0002] In modern manufacturing, CNC machine tools are key equipment for high-precision processing, and their performance directly affects product quality and production efficiency. Cutting tools are one of the core components of CNC machine tools, and their wear status is directly related to processing accuracy and product quality.

[0003] Currently, in the field of tool wear detection for CNC machine tools, a single type of sensor is relied upon to monitor the tool status, which lacks data diversity and is easily affected by noise. Although traditional visual inspection methods can monitor tool wear, they are sensitive to ambient lighting and tool surface conditions, and data processing is complex. They are difficult to adapt to changing processing environments and complex data patterns, and the accuracy of tool wear detection is low.

[0004] The above contents are only used to assist in understanding the technical solution of the present application and do not constitute an admission that the above contents are prior art. Summary of the invention

[0005] The main purpose of this application is to provide a tool wear detection method, device, storage medium and program product, aiming to solve the technical problem of low accuracy of tool wear detection.

[0006] To achieve the above objectives, the present application proposes a tool wear detection method, the method comprising:

[0007] Acquire multimodal data collected by a sensor, wherein the multimodal data includes at least two of vibration data, temperature data, sound data, and visual data;

[0008] Inputting the multimodal data into a preset data enhancement model to obtain enhanced data output by the data enhancement model, wherein the data enhancement model is used to generate the enhanced data having distribution characteristics similar to those of the multimodal data;

[0009] combining the enhanced data with the multimodal data to form a data set;

[0010] The data features of different types of data sets are extracted respectively, and the wear state of the tool is determined according to the different types of data features.

[0011] In one embodiment, after the step of combining the enhanced data with the multimodal data to form a data set, the method further includes:

[0012] The data set is input into a preset fusion model to obtain tool wear status information output by the fusion model; wherein the fusion model is used to extract data features of different types of data sets respectively, and fuse the different types of data features in the fully connected layer of the fusion model to obtain the wear status of the tool.

[0013] In one embodiment, the step of respectively extracting data features of different types of data sets and determining the wear state of the tool according to the different types of data features comprises:

[0014] determining time series data in the data set, the time series data comprising at least one of vibration time series data, temperature time series data, and sound time series data;

[0015] Inputting the time series data into a long short-term memory network model, and obtaining first feature information of the data set extracted by the long short-term memory network model, wherein the first feature information includes at least one of vibration feature information, temperature feature information, and sound feature information;

[0016] determining image data in the data set;

[0017] Inputting the image data into a convolutional neural network model, and obtaining second feature information of the data set extracted by the convolutional neural network model, wherein the second feature information includes image feature information;

[0018] The first feature information and the second feature information are input into the fully connected layer of the fusion model for fusion to obtain the tool wear state information.

[0019] In one embodiment, before the step of obtaining the enhanced data output by the data enhancement model, the method further includes:

[0020] Acquire training data, wherein the training data includes complete modality data;

[0021] A generative adversarial network or a variational autoencoder is trained based on the complete modal data to obtain the data enhancement model, so that the data enhancement model extracts the distribution characteristics of the training data and generates the enhanced data similar to the distribution characteristics of the multimodal data.

[0022] In one embodiment, the sensor includes a vibration sensor, a temperature sensor, a sound sensor and a visual sensor, and the sensor is arranged at a preset position of the machine tool. The step of acquiring multimodal data collected by the sensor includes:

[0023] Acquire detection data of vibration sensors, temperature sensors, sound sensors and visual sensors respectively;

[0024] Feature extraction is performed on the detection data respectively, wherein the features of the vibration data include spectrum amplitude and frequency components; the features of the temperature data include temperature change trends and abnormal points; the features of the sound data include spectrum features; the features of the visual data include texture features and shape features;

[0025] The multimodal data is determined according to the extracted data features.

[0026] In one embodiment, after the step of respectively extracting data features of different types of data sets and determining the wear state of the tool according to the different types of data features, the method further includes:

[0027] Determine a machine tool control strategy according to the tool wear state, wherein the machine tool control strategy includes machine tool operating parameters, and the machine tool operating parameters include at least one of a cutting speed, a feed speed, a cutting depth, and a cutting path;

[0028] The machine tool operation is controlled according to the machine tool control strategy.

[0029] In one embodiment, the step of determining the machine tool control strategy according to the tool wear state includes:

[0030] determining the degree of wear according to the wear state of the tool;

[0031] When the wear degree is less than a preset first degree value, reducing the cutting speed according to a preset first speed value;

[0032] When the wear degree is greater than a preset second degree value, the cutting speed is reduced or the cutting path is changed according to a preset second speed value, the second degree value is greater than the first degree value, and the second speed value is greater than the first speed value.

[0033] In addition, to achieve the above-mentioned purpose, the present application also proposes a tool wear detection device, which includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, and the computer program is configured to implement the steps of the tool wear detection method described above.

[0034] In addition, to achieve the above-mentioned purpose, the present application also proposes a storage medium, which is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by the processor, the steps of the tool wear detection method described above are implemented.

[0035] In addition, to achieve the above-mentioned purpose, the present application also provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, the steps of the tool wear detection method described above are implemented.

[0036] The present application provides a tool wear detection method, which obtains multimodal data collected by a sensor, wherein the multimodal data includes at least two of vibration data, temperature data, sound data and visual data; inputs the multimodal data into a preset data enhancement model to obtain enhanced data output by the data enhancement model, wherein the data enhancement model is used to generate enhanced data having distribution characteristics similar to those of the multimodal data; combines the enhanced data with the multimodal data to form a data set; extracts data features of different types of data sets respectively, and determines the wear state of the tool based on the data features of different types, thereby making full use of the advantages of multimodal data and improving the accuracy of tool wear detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0038] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0039] Figure 1 A schematic diagram of a process flow provided for the first embodiment of the tool wear detection method of the present application;

[0040] Figure 2 A schematic diagram of a method flow chart provided for the first embodiment of the tool wear detection method of the present application;

[0041] Figure 3 A schematic diagram of a process flow provided for the second embodiment of the tool wear detection method of the present application;

[0042] Figure 4 A schematic diagram of a process flow provided for the third embodiment of the tool wear detection method of the present application;

[0043] Figure 5 A schematic diagram of a process flow provided for the fourth embodiment of the tool wear detection method of the present application;

[0044] Figure 6 A schematic diagram of a process flow provided for the fifth embodiment of the tool wear detection method of the present application;

[0045] Figure 7 A schematic diagram of a method flow chart provided for the fifth embodiment of the tool wear detection method of the present application;

[0046] Figure 8 Schematic diagram of the device structure of the hardware operating environment involved in the tool wear detection method in the embodiment of the present application.

[0047] The purpose, features and advantages of this application will be further described in conjunction with the embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0048] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the present application and are not used to limit the present application.

[0049] In order to better understand the technical solution of the present application, a detailed description will be given below in conjunction with the accompanying drawings and specific implementation methods.

[0050] In modern manufacturing, CNC machine tools have become key equipment for improving production efficiency and ensuring product quality. However, current tool wear detection technology still faces many challenges, such as single sensor monitoring technology data and susceptibility to noise interference, traditional visual inspection is sensitive to the environment, data processing is complex and has poor adaptability, and machine learning technology requires big data and high generalization ability. These technical defects limit the accuracy of tool wear detection.

[0051] In view of the above problems, the present application proposes a tool wear detection method, which obtains multimodal data collected by a sensor, and the multimodal data includes at least two of vibration data, temperature data, sound data and visual data; inputs the multimodal data into a preset data enhancement model to obtain enhanced data output by the data enhancement model, and the data enhancement model is used to generate enhanced data with distribution characteristics similar to those of the multimodal data; combines the enhanced data with the multimodal data to form a data set; extracts data features of different types of data sets respectively, and determines the wear state of the tool based on the data features of different types, so as to make full use of the advantages of multimodal data and improve the accuracy of tool wear detection.

[0052] It should be noted that the execution subject of this embodiment can be a computing service device with data processing, network communication and program running functions, or an electronic device capable of realizing the above functions, etc. In order to more clearly explain this technical solution, this application is described with the machine tool control system as the execution subject.

[0053] Based on this, the first embodiment proposed in this application provides a tool wear detection method, referring to Figure 1 In this embodiment, the tool wear detection method includes steps S10 to S40:

[0054] Step S10, acquiring multimodal data collected by the sensor, wherein the multimodal data includes at least two of vibration data, temperature data, sound data and visual data.

[0055] It should be noted that there are missing modal data and unevenly distributed modal data in multimodal data. Missing modal data means that due to various reasons such as sensor failure, some modal data in the collected data are missing or incomplete data samples. Unevenly distributed modal data means that a certain modal data collected by the multimodal sensor is far more or less than the data samples of other modal data.

[0056] In a feasible implementation manner, the sensor includes a vibration sensor, a temperature sensor, a sound sensor and a visual sensor, and the sensor is arranged at a preset position of the machine tool. Figure 2 , step S10 includes steps S11 to S13:

[0057] Step S11, respectively acquiring detection data of the vibration sensor, the temperature sensor, the sound sensor and the visual sensor.

[0058] It should be noted that the vibration data detected by the vibration sensor reflects the vibration characteristics of the tool during operation, including the time domain waveform, amplitude, frequency, phase, and frequency domain characteristics of the vibration signal such as spectrum, spectrum density, etc.; the temperature data detected by the temperature sensor reflects the temperature change of the tool, including the real-time temperature, the temperature change trend over time, temperature distribution, and abnormal temperature points, etc.; the sound data detected by the sound sensor reflects the sound characteristics of the tool during operation, including the time domain waveform, frequency domain characteristics and volume of the sound signal, etc.; the visual data detected by the visual sensor such as the camera or other image acquisition equipment reflects the visual state of the tool or the processing area, including the high-definition image of the tool surface, the image segmentation result of the tool wear area, the texture and shape characteristics of the image, the defects of the tool edge and surface, etc.

[0059] Optionally, in addition to the above detection data, other detection data that can reflect the wear state of the tool can also be included. For example, laser scanning data can be obtained to reflect the shape of the tool or workpiece; optical data can be obtained to reflect the change in light intensity during the operation of the tool, etc.

[0060] Step S12, performing feature extraction on the detection data respectively, wherein the features of the vibration data include spectrum amplitude and frequency components; the features of the temperature data include temperature change trends and abnormal points; the features of the sound data include spectrum features; the features of the visual data include texture features and shape features.

[0061] Optionally, the vibration data is subjected to a Fourier transform (FFT) to convert the time domain data into frequency domain data. In the frequency domain, the amplitude of each frequency component represents the intensity of the frequency in the vibration signal. The amplitude of each frequency component is extracted as a feature.

[0062] Optionally, a time series analysis is performed on the temperature data, and the temperature change rate is calculated by sliding the window to obtain the temperature change trend. Statistical methods such as interquartile range (IQR) or standard score (Z-score) are used to identify data points that are significantly different from the normal temperature change pattern. For example, the interquartile range of the temperature data is calculated using IQR, and those data points that are out of range are identified as abnormal points.

[0063] Optionally, FFT is performed on the sound data to extract the amplitudes within a specific frequency range in its spectrum diagram, and these amplitudes represent the spectrum characteristics of the sound signal.

[0064] Optionally, texture information in visual data is extracted through image processing techniques, including Gray-Level Co-occurrence Matrix (GLCM), Local Binary Patterns (LBP), etc. to extract texture features such as contrast, energy, and entropy.

[0065] Step S13, determining the multimodal data according to the extracted data features.

[0066] In this embodiment, by respectively acquiring detection data from vibration, temperature, sound and visual sensors and performing feature extraction on these data to determine multimodal data, the accuracy and comprehensiveness of data analysis and tool status monitoring are effectively improved, the ability to identify abnormal situations under different environments and working conditions is enhanced, and the accuracy of tool wear detection is improved.

[0067] Step S20: input the multimodal data into a preset data enhancement model to obtain enhanced data output by the data enhancement model, wherein the data enhancement model is used to generate the enhanced data having distribution characteristics similar to those of the multimodal data.

[0068] It should be noted that distribution features refer to the statistical features of data in multimodal data, including mean, variance, probability density function, etc. These statistical features represent the distribution of data in various dimensions, including the central tendency of data, degree of dispersion, and shape of data distribution. Similarity of distribution features means that the generated enhanced data is statistically consistent with the multimodal data or differs within a preset range to make up for the defects of missing or uneven distribution in the collected multimodal data.

[0069] Optionally, statistical software such as Python's Pandas library or R language is used to calculate the mean, variance, standard deviation, skewness, kurtosis and other statistics of the data to analyze the central tendency and dispersion of multimodal data. Optionally, data visualization tools such as Matplotlib and Seaborn are used to draw histograms, box plots, scatter plots, etc., and analyze the distribution pattern of the data, such as symmetry, skewness, kurtosis, etc. Optionally, probability distribution functions such as normal distribution and Poisson distribution are used to fit the data to determine the distribution type of the data. The fitting process can estimate distribution parameters such as mean, variance, etc.

[0070] For example, it is assumed that a temperature sensor is installed on a machine tool to collect temperature data of the tool during operation. The temperature data includes the temperature readings of the tool at various time points during continuous operation. The collected temperature data is statistically analyzed to determine its distribution characteristics, such as the average temperature, the temperature change trend over time, and the temperature fluctuation range. Based on the distribution characteristics obtained from the analysis, a data enhancement model is constructed, and the temperature data is input into the data enhancement model. The model will generate new temperature data sequences that are statistically consistent with the input temperature data or differ within a preset range.

[0071] To better understand the solution provided in this example, this example is further explained in combination with specific application scenarios.

[0072] Assume that the input temperature data shows that the average temperature of the tool during operation is 50°C, the standard deviation is 5°C, and the temperature rises at the beginning of operation and stabilizes after reaching a certain time point. Then the enhanced data generated by the data enhancement model has an average temperature of 50±5°C, a standard deviation of 5°C, and the temperature value rises at the beginning and then stabilizes. The temperature value of a single data in the enhanced data may be different from the input temperature data, but its distribution characteristics are similar to the input temperature data.

[0073] Optionally, before inputting the multimodal data into a preset data enhancement model, the multimodal data is cleaned to remove noise and outliers, and the cleaned multimodal data is standardized or normalized to have a uniform scale.

[0074] Step S30: combining the enhanced data with the multimodal data to form a data set.

[0075] It should be noted that combining enhanced data with multimodal data can expand the size of the data set. If there is less data in some categories of multimodal data, data augmentation can generate more data to balance the data set and increase the diversity of the data set.

[0076] Optionally, missing parts in the multimodal data are filled according to the enhanced data to form a dataset.

[0077] For example, suppose there is a multimodal data including vibration data, temperature data, sound data and visual data, in which the sound data is missing. The data enhancement model is used to generate sound enhancement data with similar distribution characteristics to the multimodal data, and then the sound enhancement data is used to fill in the missing parts of the sound data, thereby forming a data set containing complete sound data.

[0078] Optionally, the enhanced data is used to replace the unevenly distributed part of the multimodal data to form a data set.

[0079] For example, suppose there is a multimodal data including vibration data, temperature data, sound data and visual data, where the collected vibration data is significantly less than other types of data. The data enhancement model is used to generate vibration enhanced data with similar distribution characteristics to the multimodal data, and then the generated vibration enhanced data replaces the original vibration data, thereby forming a data set containing complete vibration data.

[0080] Step S40 , extracting data features of different types of data sets respectively, and determining the wear state of the tool according to the different types of data features.

[0081] It should be noted that the data set after data enhancement has more complete modal data, which can improve the accuracy of tool wear state detection.

[0082] For example, it is assumed that abnormal spectral amplitudes and frequency components are extracted from vibration data, several abnormal high temperature points are extracted from temperature data, abnormal sound frequency components are extracted from sound data, and wear or crack features are extracted from visual data, and the tool wear state is determined based on the quantity or degree of the above features.

[0083] In this embodiment, enhanced data similar to the distribution characteristics of multimodal data is generated through the data enhancement model, and the multimodal data is combined to form a data set. Data features are further extracted and used to determine the wear state of the tool. This multimodal data fusion method fully utilizes the advantages of different sensor data and significantly improves the accuracy of tool wear detection.

[0084] Based on the first embodiment of the present application, in the second embodiment of the present application, the same or similar contents as those in the above-mentioned embodiment 1 can be referred to the above introduction, and will not be repeated in the following. Figure 3 , step S30 further includes step S50:

[0085] Step S50, inputting the data set into a preset fusion model to obtain tool wear status information output by the fusion model; wherein the fusion model is used to extract data features of different types of data sets respectively, and fuse the different types of data features in the fully connected layer of the fusion model to obtain the wear status of the tool.

[0086] It should be noted that the fusion model combines data features from different data sources to improve the accuracy of prediction and classification. The fully connected layer is a layer in the neural network that connects the input data features to all output nodes for integrating and processing information. Tool wear status information includes the specific wear conditions of the tool, such as the scratch length, number of scratches, wear points and wear area of ​​the tool.

[0087] Optionally, the tool wear status information can reflect the wear status of the tool. According to the importance and contribution of different data features, a weight is assigned to each data feature to perform weighted fusion of data features.

[0088] For example, assuming that the contribution of visual data features to the detection of tool wear status is 50%, the contribution of vibration data features to the detection of tool wear status is 30%, and the contribution of sound data features and temperature data features to the detection of tool wear status is 10% each, the data features are weightedly fused according to the above proportions to obtain the tool wear status.

[0089] To better understand the solution provided in this example, this example is further explained in combination with specific application scenarios.

[0090] The weight score is set according to the number of wear marks, the frequency components of abnormal vibration, the number of abnormal sound data, and the number of abnormal temperature points, where the higher the weight score, the more serious the damage to the tool. For example, when the number of wear marks is between 1-20, the weight score is 4, when the number of wear marks is between 20-40, the weight score is 6, and when the number of wear marks is between 40-80, the weight score is 8. The weight scores are weighted and fused according to the above contribution to obtain the wear status of the tool. Furthermore, assuming that the weight score in the visual data feature is 8, the weight score in the vibration data feature is 7, the weight score in the sound data feature is 6, and the weight score in the temperature data feature is 5, the weight score of the tool wear status can be obtained after weighted fusion to be 7.2.

[0091] In this embodiment, by comprehensively considering the data features provided by different sensors in the data set, the wear state of the tool can be evaluated more accurately.

[0092] Based on the above embodiment, in the third embodiment of the present application, the same or similar contents as those in the above embodiment can be referred to the above introduction, and will not be described in detail later. Figure 4 , step S40 may include S41 to S45:

[0093] Step S41, determining time series data in the data set, wherein the time series data includes at least one of vibration time series data, temperature time series data and sound time series data.

[0094] It should be noted that time series data is a series of data points arranged in chronological order, and is usually used to analyze phenomena or processes that change over time. Among them, vibration sensors usually record vibration signals when the equipment is running. These signals will change over time, forming vibration time series data, such as the vibration intensity per second when the tool is working; temperature sensors can continuously monitor and record the temperature changes of the tool. These recorded temperature readings are arranged in chronological order, forming temperature time series data; sound signals captured by sound sensors also change over time, and the recorded sound waveforms or sound features such as frequency and volume that change over time constitute sound time series data.

[0095] Step S42, inputting the time series data into the long short-term memory network model, and obtaining the first feature information of the data set extracted by the long short-term memory network model, wherein the first feature information includes at least one of vibration feature information, temperature feature information and sound feature information.

[0096] It should be noted that the Long Short-Term Memory Network (LSTM) is a special recurrent neural network (RNN) that can learn long-term dependency information for processing and predicting time series data. LSTM remembers information in long time series through its internal structures such as forget gate, input gate and output gate, and models the patterns in time series data. The first feature information is the feature representation learned by the LSTM model from the input time series data. The vibration feature information, temperature feature information and sound feature information are the features extracted from the vibration time series data, temperature time series data and sound time series data, respectively, including the frequency components and intensity changes of vibration, the trend and fluctuation pattern of temperature, the spectrum characteristics and intensity changes of sound, etc.

[0097] Step S43, determining image data in the data set.

[0098] It should be noted that image data contains rich visual information, such as color, shape, texture, spatial layout, etc.

[0099] Step S44: input the image data into a convolutional neural network model, and obtain second feature information of the data set extracted by the convolutional neural network model, where the second feature information includes image feature information.

[0100] It should be noted that the Convolutional Neural Network (CNN) is a deep learning model used to process and analyze image data or video data, and extract features from images or video frames through a series of convolutional layers, pooling layers, and fully connected layers. The second feature information is the feature representation learned by the CNN model from the input visual data, including color distribution, texture, shape, tool position and size, etc.

[0101] Step S45: input the first feature information and the second feature information into the fully connected layer of the fusion model for fusion to obtain the tool wear state information.

[0102] Optionally, a multilayer perceptron (MLP) is used to take the first feature information and the second feature information as input and fuse them through a fully connected layer.

[0103] It should be noted that MLP is a multi-layer feedforward network, which consists of three or more layers of neural networks, including input layer, hidden layer and output layer. The input layer is used to receive input data, the hidden layer contains multiple neurons for feature extraction and conversion, and the output layer is used to generate output results. Each layer is connected by a fully connected method.

[0104] Exemplarily, an MLP including several hidden layers is designed. The first feature information and the second feature information are taken as input. The MLP passes them to the hidden layer, and then passes them to the output layer through a full connection, and outputs a fused feature information. The tool wear status information is obtained based on the fused feature information.

[0105] In this embodiment, by determining the time series data and the image data, the long short-term memory network model is used to extract the first feature information and the convolutional neural network model is used to extract the second feature information, and then these two types of feature information are input into the fully connected layer of the fusion model for fusion, so as to obtain the tool wear status information, integrate the information of the multimodal data, and improve the accuracy of the tool wear status detection.

[0106] Based on the above embodiments, in the fourth embodiment of the present application, the same or similar contents as those in the above embodiments can be referred to the above introduction, and will not be described in detail later. Figure 5 , before step S20, steps S60 to S70 are also included:

[0107] Step S60, obtaining training data, wherein the training data includes complete modality data.

[0108] It should be noted that the training data is used to train the machine learning model so that the model can learn the data generation pattern. Complete modal data means that the modal data collected from multiple different sensors is evenly distributed and not missing, so as to ensure that the generated enhanced data is complete and comprehensive.

[0109] Optionally, training data is collected at a machine tool work site, and multiple sensors are used to collect multimodal data within a same time period. The collected training data may include data of the tool in an intact state and data in different wear states.

[0110] Optionally, historical working data of the tool is used as training data, or training data is generated by a machine learning model or simulation software.

[0111] Step S70, training a generative adversarial network or a variational autoencoder based on the complete modal data to obtain the data enhancement model, so that the data enhancement model extracts the distribution characteristics of the training data and generates the enhanced data similar to the distribution characteristics of the multimodal data.

[0112] It should be noted that Generative Adversarial Networks (GANs) consist of a generator and a discriminator. The generator is used to generate data with distribution characteristics similar to the input data, while the discriminator is used to distinguish between the input data and the generated data. Through repeated training, the generator can learn to generate enhanced data with distribution characteristics similar to the input multimodal data. Variational Auto-Encode (VAE) is a generative model that encodes the received input data into a distribution representation of latent variables and generates new data from the distribution representation of latent variables. VAE is trained by maximizing the likelihood estimate, that is, finding the model parameter values ​​that can make the input data and the generated data as similar as possible, so as to minimize the difference between the generated data and the real data, and achieve the effect of generating enhanced data with distribution characteristics similar to the input multimodal data.

[0113] For example, a set of multimodal training data is obtained in a machine tool work site test, the training data is divided into a training set and a validation set, a GANs or VAEs model is selected, and the training set is input into the model for training. During the training process, the weights and biases of the model are updated through the back propagation algorithm or the gradient descent algorithm, and multiple training cycles are performed until the accuracy of the model on the validation set reaches a preset threshold, forming a data enhancement model.

[0114] In this embodiment, by obtaining a training set containing complete modal data such as vibration, temperature, sound, and vision, a generative anti-network or a variational autoencoder is trained, and a data augmentation model is constructed to generate enhanced data with similar distribution characteristics to multimodal data, thereby enhancing the diversity and quality of the data.

[0115] Based on the above embodiments, in the fifth embodiment of the present application, the same or similar contents as those in the above embodiments can be referred to the above description, and will not be described in detail later. On this basis, the multimodal data includes missing modal data and unevenly distributed modal data, referring to Figure 6 , after step S40, steps S80 to S90 are also included:

[0116] Step S80, determining a machine tool control strategy according to the tool wear state, wherein the machine tool control strategy includes machine tool operating parameters, and the machine tool operating parameters include at least one of a cutting speed, a feed speed, a cutting depth, and a cutting path.

[0117] It should be noted that the machine tool control strategy is a plan to adjust the machine tool working parameters according to the tool wear state, with the purpose of optimizing the machine tool operation to reduce tool wear. For example, reducing the cutting speed and depth can reduce the friction and wear between the tool and the workpiece each time; reducing the feed rate can reduce the number of times the tool contacts the workpiece per unit time, thereby reducing tool wear and extending tool life; optimizing the cutting path can reduce the time the tool is in the non-machining area, thereby reducing tool wear.

[0118] Optionally, the machine tool control strategy may also include adjusting the concentration of the cutting fluid to reduce friction between the tool and the workpiece. For example, based on the processing conditions and tool wear, when the tool wear is more serious, the concentration of the cutting fluid is increased to provide better lubrication and cooling effects and reduce tool wear.

[0119] In one possible implementation, refer to Figure 7 , step S80 includes steps S81 to S83:

[0120] Step S81, determining the degree of wear according to the tool wear state.

[0121] Optionally, the tool wear state is quantitatively evaluated, for example, the tool wear state is quantified into continuous numerical values ​​to represent the degree of wear as described in the second embodiment.

[0122] Step S82, when the wear degree is less than a preset first degree value, reducing the cutting speed according to a preset first speed value.

[0123] Step S83, when the wear degree is greater than a preset second degree value, reduce the cutting speed or change the cutting path according to a preset second speed value, the second degree value is greater than the first degree value, and the second speed value is greater than the first speed value.

[0124] The wear degree of the tool is monitored in real time, and the monitored wear degree is compared with the preset first degree value. If the wear degree is less than the preset first degree value, it means that the tool is still in a relatively good working condition, and the cutting speed is reduced according to the preset first speed value. When the wear degree of the tool is greater than the preset second degree value, it means that the performance of the tool has been significantly reduced, and it is necessary to further reduce the cutting speed or take other measures to reduce further wear of the tool.

[0125] For example, when the tool wear is detected to be less than 50%, the system will automatically reduce the cutting speed to 80% of the original speed, which can reduce tool wear and maintain high processing quality. When the tool wear is detected to be greater than 60%, the system will automatically reduce the cutting speed to 60% of the original speed. If the wear continues to increase and exceeds 70%, in addition to reducing the cutting speed, the cutting path is also changed to extend the tool life.

[0126] In this embodiment, by monitoring the tool wear state, when the wear degree is low, the cutting speed is appropriately reduced to reduce wear; when the wear degree is high, the cutting speed is further reduced or the cutting path is changed, thereby significantly extending the tool life and maintaining the processing quality.

[0127] Step S90, controlling the operation of the machine tool according to the machine tool control strategy.

[0128] In this embodiment, by analyzing the wear state of the tool and determining a control strategy for adjusting the working parameters of the machine tool to control the operation of the machine tool, the wear of the tool can be reduced and the service life can be extended.

[0129] It should be noted that the above examples are only used to understand the present application and do not constitute a limitation on the tool wear detection method of the present application. More simple transformations based on this technical concept are all within the protection scope of the present application.

[0130] The present application provides a tool wear detection device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the tool wear detection method in the above-mentioned embodiment one.

[0131] Reference below Figure 8 , which shows a schematic diagram of the structure of a tool wear detection device suitable for implementing the embodiment of the present application. The tool wear detection device in the embodiment of the present application may include but is not limited to mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 8 The tool wear detection device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0132] like Figure 8As shown, the tool wear detection device may include a processing device 1001 (such as a central processing unit, a graphics processor, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM: Read Only Memory) 1002 or a program loaded from a storage device 1003 to a random access memory (RAM: Random Access Memory) 1004. In RAM1004, various programs and data required for the operation of the tool wear detection device are also stored. The processing device 1001, ROM1002 and RAM1004 are connected to each other through a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Generally, the following systems can be connected to the I / O interface 1006: an input device 1007 including, for example, a touch screen, a touch pad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; an output device 1008 including, for example, a liquid crystal display (LCD: Liquid Crystal Display), a speaker, a vibrator, etc.; a storage device 1003 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 can allow the tool wear detection device to communicate with other devices wirelessly or by wire to exchange data. Although the tool wear detection device with various systems is shown in the figure, it should be understood that it is not required to implement or have all the systems shown. More or fewer systems can be implemented or have alternatively.

[0133] In particular, according to the embodiments disclosed in the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are executed.

[0134] The tool wear detection device provided by the present application adopts the tool wear detection method in the above embodiment, which can solve the technical problem of low accuracy of tool wear detection. Compared with the prior art, the beneficial effects of the tool wear detection device provided by the present application are the same as the beneficial effects of the tool wear detection method provided by the above embodiment, and other technical features in the tool wear detection device are the same as the features disclosed in the method of the previous embodiment, which will not be repeated here.

[0135] It should be understood that the various parts disclosed in this application can be implemented by hardware, software, firmware or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0136] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0137] The present application provides a computer-readable storage medium having computer-readable program instructions (ie, computer programs) stored thereon, and the computer-readable program instructions are used to execute the tool wear detection method in the above-mentioned embodiment.

[0138] The computer-readable storage medium provided in the present application may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, systems or devices, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM: Random Access Memory), a read-only memory (ROM: Read Only Memory), an erasable programmable read-only memory (EPROM: Erasable Programmable Read Only Memory or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM: CD-Read Only Memory), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, system or device. The program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (Radio Frequency: Radio Frequency), etc., or any suitable combination of the above.

[0139] The computer-readable storage medium may be included in the tool wear detection device; or may exist independently without being assembled into the tool wear detection device.

[0140] The computer-readable storage medium carries one or more programs. When the one or more programs are executed by the tool wear detection device, the tool wear detection device can write computer program codes for performing the operations of the present application in one or more programming languages ​​or a combination thereof. The programming languages ​​include object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as an independent software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (for example, using an Internet service provider to connect through the Internet).

[0141] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0142] The modules involved in the embodiments described in this application may be implemented by software or hardware, wherein the name of the module does not constitute a limitation on the unit itself in some cases.

[0143] The readable storage medium provided in the present application is a computer-readable storage medium, which stores computer-readable program instructions (i.e., computer programs) for executing the above-mentioned tool wear detection method, and can solve the technical problem of low accuracy of tool wear detection. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in the present application are the same as the beneficial effects of the tool wear detection method provided in the above-mentioned embodiment, and will not be elaborated here.

[0144] The present application also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the steps of the tool wear detection method as described above are implemented.

[0145] The computer program product provided by the present application can solve the technical problem of low accuracy of tool wear detection. Compared with the prior art, the beneficial effects of the computer program product provided by the present application are the same as the beneficial effects of the tool wear detection method provided by the above embodiment, which will not be repeated here.

[0146] The above descriptions are only some embodiments of the present application, and are not intended to limit the patent scope of the present application. All equivalent structural changes made using the contents of the present application specification and drawings under the technical concept of the present application, or direct / indirect applications in other related technical fields are included in the patent protection scope of the present application.

Claims

1. A tool wear detection method, characterized in that: The method comprises: Acquire multimodal data collected by a sensor, wherein the multimodal data includes at least two of vibration data, temperature data, sound data, and visual data; Inputting the multimodal data into a preset data enhancement model to obtain enhanced data output by the data enhancement model, wherein the data enhancement model is used to generate the enhanced data having distribution characteristics similar to those of the multimodal data; combining the enhanced data with the multimodal data to form a data set; Inputting the data set into a preset fusion model, wherein the fusion model is used to extract data features of different types of the data set respectively, and fuse the different types of data features in a fully connected layer of the fusion model to obtain tool wear state information; determining time series data in the data set, the time series data comprising at least one of vibration time series data, temperature time series data, and sound time series data; Inputting the time series data into a long short-term memory network model, and obtaining first feature information of the data set extracted by the long short-term memory network model, wherein the first feature information includes at least one of vibration feature information, temperature feature information, and sound feature information; determining image data in the data set; Inputting the image data into a convolutional neural network model, and obtaining second feature information of the data set extracted by the convolutional neural network model, wherein the second feature information includes image feature information; The first feature information and the second feature information are input into the fully connected layer of the fusion model for fusion to obtain the tool wear state information.

2. The tool wear detection method according to claim 1, characterized in that: The step of inputting the multimodal data into a preset data enhancement model to obtain enhanced data output by the data enhancement model comprises: Acquire training data, wherein the training data includes complete modality data; A generative adversarial network or a variational autoencoder is trained based on the complete modal data to obtain the data augmentation model, so that the data augmentation model extracts the distribution characteristics of the training data and generates the augmented data similar to the distribution characteristics of the multimodal data.

3. The tool wear detection method according to claim 1, characterized in that: The sensor includes a vibration sensor, a temperature sensor, a sound sensor and a visual sensor. The sensor is arranged at a preset position of the machine tool. The step of acquiring multimodal data collected by the sensor includes: Acquire detection data of vibration sensors, temperature sensors, sound sensors and visual sensors respectively; Extracting features from the detection data respectively, wherein the features of the vibration data include spectrum amplitude and frequency components, the features of the temperature data include temperature change trends and abnormal points, the features of the sound data include spectrum features, and the features of the visual data include texture features and shape features; The multimodal data is determined according to the extracted data features.

4. The tool wear detection method according to claim 1, characterized in that: After the step of inputting the first feature information and the second feature information into the fully connected layer of the fusion model for fusion to obtain the tool wear state information, the method further includes: Determine a machine tool control strategy according to the tool wear state, wherein the machine tool control strategy includes machine tool operating parameters, and the machine tool operating parameters include at least one of a cutting speed, a feed speed, a cutting depth, and a cutting path; The machine tool operation is controlled according to the machine tool control strategy.

5. The tool wear detection method according to claim 4, characterized in that: The step of determining the machine tool control strategy according to the tool wear state comprises: determining the degree of wear according to the wear state of the tool; When the wear degree is less than a preset first degree value, reducing the cutting speed according to a preset first speed value; When the wear degree is greater than a preset second degree value, the cutting speed is reduced or the cutting path is changed according to a preset second speed value, the second degree value is greater than the first degree value, and the second speed value is greater than the first speed value.

6. A tool wear detection device, characterized in that: The device comprises: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the tool wear detection method according to any one of claims 1 to 5.

7. A storage medium, characterized in that: The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the tool wear detection method according to any one of claims 1 to 5 are implemented.

8. A computer program product, characterized in that The computer program product comprises a computer program, and when the computer program is executed by a processor, the steps of the tool wear detection method according to any one of claims 1 to 5 are implemented.