Cutter wear state monitoring method and device based on semi-supervised learning and medium

Abstract

The invention provides a tool wear state monitoring method and device based on semi-supervised learning and a medium. The method comprises the steps that a multi-mode tool wear data set and a semi-supervised teacher-student learning model are constructed; performing forward reasoning on the multi-modal data without the labels according to a teacher model to obtain a plurality of pseudo labels; co-training the student models; generating a confrontation network and a dynamic data enhancement strategy according to a preset condition, and processing the semi-supervised teacher-student learning model; according to an integrated learning strategy of the model snapshot, migrating a pre-trained model weight to an image branch of the semi-supervised teacher-student learning model so as to optimize the semi-supervised teacher-student learning model; and inputting the multi-modal data of the to-be-detected tool in the machining process into the semi-supervised teacher-student learning model to obtain a wear state prediction result corresponding to the to-be-detected tool so as to realize high-precision and strong-generalization real-time monitoring and prediction of the wear state of the tool by fusing the multi-modal sensing information and the visual information.

Description

Technical Field

[0001] This invention relates to the field of mechanical manufacturing process monitoring and intelligent manufacturing, and in particular to a method, equipment and medium for monitoring tool wear status based on semi-supervised learning. Background Technology

[0002] In intelligent manufacturing systems, cutting tools, as the execution units directly involved in cutting processes, directly determine the machining quality, production efficiency, and manufacturing costs of the workpiece. Statistics show that approximately 10%-40% of unplanned machine tool downtime is caused by abnormal wear or sudden breakage of cutting tools. Therefore, achieving real-time and accurate monitoring of cutting tool wear conditions, and making predictive maintenance and tool replacement decisions accordingly, is of paramount importance for improving the reliability, economy, and safety of intelligent manufacturing systems. Summary of the Invention

[0003] To address the aforementioned technical problems, the technical solution adopted by this invention is as follows: According to one aspect of this application, a method for monitoring tool wear state based on semi-supervised learning is provided, comprising: Step S100: Construct a multimodal tool wear dataset by collecting a number of labeled multimodal data and a number of unlabeled multimodal data; the multimodal data includes at least vibration signals during the cutting process, multi-channel force signals, and image signals of the tool flank wear area; Step S200: Construct a semi-supervised teacher-student learning model; the semi-supervised teacher-student learning model includes a student model and a teacher model with the same structure and shared parameters; Step S300: Based on the teacher model, perform forward inference on several unlabeled multimodal data in the multimodal tool wear dataset to obtain several pseudo-labeled multimodal data. Step S400: Train the student model together based on several pseudo-labeled multimodal data, several unlabeled multimodal data in the multimodal tool wear dataset, and several labeled multimodal data. Step S500: Generate an adversarial network and dynamic data augmentation strategy according to preset conditions, and process the semi-supervised teacher-student learning model; Step S600: According to the ensemble learning strategy of model snapshot, the pre-trained model weights are transferred to the image branch of the semi-supervised teacher-student learning model to optimize the semi-supervised teacher-student learning model. Step S700: Deploy the optimized semi-supervised teacher-student learning model in the online monitoring system of the tool under test; Step S800: Input the multimodal data of the tool under test during the machining process collected in real time into the semi-supervised teacher-student learning model to obtain the wear state prediction result of the tool under test.

[0004] According to another aspect of this application, a non-transitory computer-readable storage medium is provided, wherein at least one instruction or at least one program is stored in the storage medium, the at least one instruction or the at least one program being loaded and executed by a processor to implement the aforementioned tool wear condition monitoring method based on semi-supervised learning.

[0005] According to another aspect of this application, an electronic device is provided, including a processor and the aforementioned non-transitory computer-readable storage medium.

[0006] The present invention has at least the following beneficial effects: This invention discloses a semi-supervised learning-based tool wear state monitoring method. It constructs a multimodal tool wear dataset by collecting labeled and unlabeled multimodal data. It also constructs a semi-supervised teacher-student learning model containing a student model and a teacher model with identical structures and shared parameters. The teacher model performs forward inference on the unlabeled multimodal data in the multimodal tool wear dataset to obtain pseudo-labeled multimodal data. The student model is then jointly trained using the pseudo-labeled multimodal data, the unlabeled multimodal data in the multimodal tool wear dataset, and the labeled multimodal data. Finally, an adversarial network and dynamic data augmentation strategies are generated based on preset conditions to further enhance the semi-supervised teacher-student learning model. The system processes and integrates learning strategies based on model snapshots, transferring pre-trained model weights to the image branch of the semi-supervised teacher-student learning model to optimize it. In practical applications, the optimized semi-supervised teacher-student learning model is deployed in the online monitoring system of the tool under test. The multimodal data of the tool under test during the machining process is collected in real time and input into the semi-supervised teacher-student learning model to obtain the wear state prediction result of the tool under test. This overcomes the shortcomings of existing technologies, such as dependence on a large amount of labeled data and weak model generalization ability. It is suitable for real industrial scenarios where labeled data is scarce. By integrating multimodal sensing information and visual information, it can achieve high-precision, strong generalization, real-time monitoring and prediction of tool wear state. Attached Figure Description

[0007] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0008] Figure 1 A flowchart illustrating a tool wear state monitoring method based on semi-supervised learning provided in an embodiment of the present invention; Figure 2A detailed flowchart of the tool wear state monitoring method based on semi-supervised learning provided in this embodiment of the invention; Figure 3 This is a schematic diagram of the structure of the multimodal feature fusion network provided in an embodiment of the present invention; Figure 4 A flowchart illustrating the pseudo-label generation and filtering steps provided in an embodiment of the present invention; Figure 5 The loss function change curve and the schematic diagram of the growth of the number of pseudo-labels during the training process of the teacher-student model provided in this embodiment of the invention; Figure 6 This is a comparison curve of the predicted tool wear value and the actual measured value on the test set provided in an embodiment of the present invention. Detailed Implementation

[0009] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0010] Currently, existing tool condition monitoring methods can be mainly divided into two categories: physical model-based methods and data-driven methods.

[0011] I. Physical model-based methods rely on mechanical and thermal modeling of the cutting process. Although they have clear physical meaning, their models are complex, parameters are difficult to obtain, and they are difficult to adapt to changing machining conditions.

[0012] II. Data-driven methods, especially deep learning models, can automatically learn the complex mapping relationship between tool state and sensor signals from historical data, and have become the mainstream of current research. However, existing data-driven methods face several serious challenges in practical applications: The scarcity and high cost of labeled data: The "true value" of tool wear status, such as the width of the wear band on the back face VB, usually requires the tool to be removed during machining breaks and obtained through offline measurement equipment such as tool microscopes. This process is cumbersome, time-consuming, and will interrupt continuous production. Therefore, the number of data samples that can be accurately labeled is extremely limited, which constitutes the bottleneck of the fully supervised learning model. Insufficient model generalization ability: When a model trained under a specific working condition is applied to another working condition, its monitoring performance often drops significantly due to the difference in signal distribution. Therefore, a single model is difficult to cover complex multi-working-condition production environments. Class imbalance problem: During the entire life cycle of a tool, the state of severe wear or breakage accounts for only a very small part, resulting in a serious imbalance between the number of "normal" and "abnormal" samples in the dataset. Therefore, the model tends to predict the majority class, thus becoming insensitive to key abnormal states. Insufficient utilization of multimodal information: The cutting process contains a variety of information such as force, vibration, sound, temperature, power and tool image. Therefore, existing methods mostly rely on single or a few signals, failing to fully explore the complementarity and synergy between multi-source information, which limits the further improvement of monitoring accuracy. Real-time performance and computational cost: Although complex deep learning models have high accuracy, they have high computational costs and are difficult to deploy on resource-constrained industrial edge devices, making it difficult to meet the real-time requirements of online monitoring.

[0013] In summary, based on the aforementioned technical problems, semi-supervised learning (SSL) provides a feasible technical path. It aims to train models using both a small amount of labeled data and a large amount of readily available unlabeled data, thereby significantly reducing dependence on labeled data while maintaining high performance. However, effectively applying SSL to the field of tool monitoring requires solving specific technical challenges such as multimodal data fusion, pseudo-label quality, and model stability. Therefore, this application proposes a tool wear condition monitoring method based on semi-supervised learning.

[0014] like Figure 1 and Figure 2 As shown, the tool wear state monitoring method based on semi-supervised learning proposed in this application includes: Step S100: Construct a multimodal tool wear dataset by collecting a number of labeled multimodal data and a number of unlabeled multimodal data; Multimodal data includes at least vibration signals during the cutting process, multi-channel force signals, and image signals of the wear area on the tool's flank face.

[0015] First, a tool wear monitoring device is constructed, comprising a multi-channel force sensor, a triaxial vibration sensor, a micro-displacement sensor, a data acquisition card, an industrial camera, a dual telecentric microscope, and a computer terminal. The multi-channel force sensor is mounted on the spindle of the milling cutter (i.e., the tool under test) to collect the triaxial cutting force signals (i.e., multi-channel force signals) along the X, Y, and Z axes. The triaxial vibration sensor is fixed to the workpiece to collect vibration acceleration signals (i.e., vibration signals). The micro-displacement sensor is fixed to the spindle of the tool under test to monitor the micro-deformation state of the spindle system. The data acquisition card and industrial camera are connected via signal transmission, and the data acquisition card is connected to each sensor. The computer terminal includes a computer memory, processor, program, and acquisition module.

[0016] Furthermore, step S100 includes steps S110-S150: Step S110: Based on the multi-channel force sensor installed on the spindle of the tool under test and the vibration sensor fixed on the workpiece to be machined, collect the milling processing sensing signal corresponding to the wear state of the tool under test at the current moment. Milling machining sensing signals include vibration signals and multi-channel force signals.

[0017] Step S120: After each pass of the tool under test, acquire the back face image of the tool under test using an industrial camera and a dual telecentric microscope; The back face image focuses on the wear zone area.

[0018] Step S130: Obtain the process parameters of the tool under test during each cutting operation; Process parameters include cutting speed, feed per tooth, depth of cut, and effective cutting time.

[0019] Step S140: Use the milling machining sensor signals and process parameters as multimodal data, and use the corresponding back face wear band width VB value in the back face image as wear label to establish a labeled data subset and an unlabeled data subset. Among them, the labels corresponding to the multimodal data in the labeled data subset are wear labels, while the samples in the unlabeled data subset are multimodal data that were continuously collected during the production process but were not measured offline.

[0020] The width VB of the wear band on the back face in the back face image can be measured by a professional using image annotation tools.

[0021] On the other hand, the labels corresponding to multimodal data in the labeled data subset can also directly reference the wear labels in the public dataset.

[0022] Step S150: Perform signal preprocessing and image preprocessing on several multimodal data from the labeled and unlabeled data subsets to obtain a multimodal tool wear dataset.

[0023] Signal preprocessing is performed on several multimodal data from labeled and unlabeled data subsets to identify and remove invalid signals during the tool feed and retraction stages.

[0024] Step S150 includes steps S151-S157: Step S151: Define a sliding window of a preset length on the entire cutting signal curve of the cutting process of the tool under test. The sliding window slides point by point along the time sequence of the cutting signals in several multimodal data in the labeled data subset and the unlabeled data subset. Step S152: After traversing all multimodal data in the labeled and unlabeled data subsets by the sliding window, obtain several variances of all multimodal data corresponding to each sliding of the sliding window; Step S153: Based on several variances, determine the variance threshold using the preset quartile method; Step S154: Perform a forward traversal of the cutting signals in several multimodal data in the labeled and unlabeled data subsets according to the time sequence, and determine the first time point where the variance is greater than the variance threshold as the entry point. Step S155: Perform reverse traversal of the cutting signals in several multimodal data in the labeled data subset and the unlabeled data subset according to the time sequence, and determine the first time point where the variance is greater than the variance threshold as the tool retraction point. Step S156: Delete the multimodal data in the labeled and unlabeled data subsets whose corresponding time points are not between the tool entry and exit points. Step S157: Using a preset wavelet soft thresholding denoising algorithm and Butterworth low-pass filtering algorithm, noise reduction and smoothing are performed on several back face images corresponding to the labeled and unlabeled data subsets. The images are then preprocessed with grayscale conversion, contrast enhancement, and region cropping to obtain a multimodal tool wear dataset.

[0025] Step S200: Construct a semi-supervised teacher-student learning model; Using the Mean Teacher framework, a student model F is constructed that contains identical structures and shares parameters. S (θ S ) and a teacher model F T (θ T A semi-supervised teacher-student learning model; Among them, the parameter θ of the teacher model T The parameter θ of the student model S The result of the exponential moving average during the training process; θ T (t) =αθ T (t-1) +(1-α)θ S (t) ; α is the preset smoothing coefficient, and the value of α ranges from [0.99, 0.999]; t is any time point in the training process.

[0026] Furthermore, the structures of the teacher and student models include: The image feature extraction branch employs a lightweight CNN (Convolutional Neural Network) encoder. The input is a preprocessed back face image, which is used to perform convolution and pooling operations on the preprocessed back face image to extract high-level features related to wear (such as the texture, shape, contour, and area information of the wear band) from the pixels, resulting in a high-level visual feature vector representing the image content of the back face image. The image feature extraction branch can capture the texture, shape, and contour information of the wear area. The sensor signal feature extraction branch is used to extract time-domain, frequency-domain, and time-frequency-domain features from the preprocessed milling sensor signal to form an initial feature vector. Then, the initial feature vector is input into a bidirectional LSTM network (Long Short-Term Memory) to capture the long-term temporal dependencies and trends in the milling sensor signal and obtain a temporal feature vector. Feature fusion module, such as Figure 3 As shown, the high-level visual feature vector and the temporal feature vector are concatenated and then input into a multi-head self-attention mechanism. This mechanism automatically learns and weights the important information from different modalities by calculating the correlation between different multimodal data features in the high-level visual feature vector and the temporal feature vector, so as to obtain the fused feature representation vector. The output layer maps the fused feature representation vector to the prediction target. The fused feature representation vector passes through a fully connected layer and outputs the regression prediction or state classification probability of the tool wear value.

[0027] Step S300: Based on the teacher model, perform forward inference on several unlabeled multimodal data in the multimodal tool wear dataset to obtain several pseudo-labeled multimodal data. Furthermore, such as Figure 4 As shown, step S300 includes steps S310-S380: Step S310: Input each multimodal data in the unlabeled data subset into the teacher model and the student model respectively, and use the prediction generated by the teacher model as a pseudo-label; Step S320: The difference between the predictions generated by the student model and the pseudo-labels is determined as the unsupervised loss function L. unsup ; Step S330: Input each multimodal data point from the labeled data subset into the student model, and have the student model output the supervised loss function L. sup ; Step S340: Integrate the unsupervised loss function and the supervised loss function into a joint loss function L. total ; Wherein, the joint loss function L total =L sup +λ(t)·L unsup ; λ(t) is a weight function that changes over time (number of training rounds). It is usually a Gaussian heating function, which is small in the early stages of training and then gradually increases to stabilize the training process.

[0028] Step S350: Update the parameters of the student model through backpropagation, and update the parameters of the teacher model by using the exponential moving average of the parameters of the student model. Step S360: The output probability of the semi-supervised teacher-student learning model is calibrated using a temperature scaling strategy; Step S370: Retain the multimodal data whose maximum output probability after calibration is greater than the preset confidence threshold, and delete the remaining multimodal data from the multimodal tool wear dataset; Step S380: The category corresponding to the output probability whose maximum value is greater than the preset confidence threshold is determined as a pseudo label.

[0029] Step S400: Train the student model together based on several pseudo-labeled multimodal data, several unlabeled multimodal data in the multimodal tool wear dataset, and several labeled multimodal data. like Figure 5 As shown, this is a diagram illustrating the change curve of the loss function and the growth of the number of pseudo-labels during the teacher-student model training process.

[0030] Step S500: Generate an adversarial network and dynamic data augmentation strategy according to preset conditions, and process the semi-supervised teacher-student learning model; Furthermore, step S500 includes steps S510-S520: Step S510: Generate an adversarial network according to preset conditions. Using process parameters and wear status as conditions, generate corresponding multimodal sensing signal features and add them to the multimodal tool wear dataset. To address the problem of insufficient minority class samples, such as those with severe wear, a conditional generative adversarial network (cGAN) is trained to generate realistic multimodal sensor signal features based on process parameters and wear status, thereby expanding the minority class sample.

[0031] Step S520: During the training process of the semi-supervised teacher-student learning model, the image data of the input data is subjected to any of the following processing: rotation, translation, scaling, adjustment of brightness and contrast, addition of Gaussian noise, and the sensor signal of the input data is subjected to any of the following processing: time distortion, window distortion, and random masking.

[0032] The enhancement strategy in step S520 is dynamically generated in each round of training, which greatly increases the diversity of data.

[0033] Step S600: According to the ensemble learning strategy of model snapshot, the pre-trained model weights are transferred to the image branch of the semi-supervised teacher-student learning model to optimize the semi-supervised teacher-student learning model. Furthermore, step S600 includes steps S610-S620: Step S610: Use random forest as the meta-learner to perform weighted voting or averaging on the prediction results of the semi-supervised teacher-student learning model. In the inference phase, instead of using separate student or teacher models, model snapshots saved from multiple stages of training are integrated. Specifically, a random forest is used as a meta-learner to perform weighted voting or averaging of the prediction results from these deep models, thereby reducing variance and improving prediction accuracy and robustness.

[0034] Step S620: Initialize the image branch of the semi-supervised teacher-student learning model using model weights pre-trained on a large image dataset.

[0035] For the image branch, the model weights are initialized using models pre-trained on large image datasets. By fine-tuning the entire network or just the last few layers, the model can be quickly adapted to the specific domain of the tool image, accelerating convergence and improving performance.

[0036] Step S700: Deploy the optimized semi-supervised teacher-student learning model in the online monitoring system of the tool under test; The trained semi-supervised teacher-student learning model is deployed in the actual production environment and connected to data acquisition cards and sensor networks to form an online monitoring system.

[0037] Step S800: Input the multimodal data of the tool under test during the machining process collected in real time into the semi-supervised teacher-student learning model to obtain the wear state prediction result of the tool under test. Furthermore, step S800 includes steps S810-S820: Step S810: Input the multimodal data of the tool under test during the machining process collected in real time into the semi-supervised teacher-student learning model to obtain the wear prediction value output by the semi-supervised teacher-student learning model. Step S820: If the wear prediction value is greater than or equal to the preset wear threshold, then the tool wear is determined as the wear state prediction result corresponding to the tool under test; otherwise, the tool no wear is determined as the wear state prediction result corresponding to the tool under test.

[0038] like Figure 6As shown, during the machining process of the tool under test, force and vibration signals are collected in real time, and tool images are acquired at set intervals (e.g., after each pass). After undergoing the same preprocessing as in the training phase, these data are input into the deployed semi-supervised teacher-student learning model to obtain the real-time wear prediction value VB. pred Then set a wear threshold VB. threshold (According to national standard GB / T16460-2016, it can be set to 300μm), when VB pred ≥VB threshold When this happens, the system will automatically trigger an alarm signal, prompting the operator to check the cutting tool or perform a tool change, thereby preventing deterioration of machining quality or damage to the equipment.

[0039] Furthermore, this application also proposes an online tool wear condition monitoring system that applies the above-mentioned semi-supervised learning-based tool wear condition monitoring method, which includes: The multi-source information acquisition module includes a multi-channel force sensor, a triaxial vibration sensor, a micro-displacement sensor, and an industrial camera; The data preprocessing and feature extraction module is used to perform signal denoising, image segmentation, and multimodal feature alignment. The semi-supervised learning inference engine, deployed on a graphics processor, is used to load and run a trained semi-supervised teacher-student learning model and execute pseudo-label generation and model update logic. The human-machine interaction and early warning module is used to display the tool health status curve, remaining life percentage, and trigger audible and visual alarms in real time.

[0040] This invention is applicable to real-world industrial scenarios where labeled data is scarce. Its primary objective is to overcome the shortcomings of existing technologies, such as reliance on large amounts of labeled data and weak model generalization ability. By designing an efficient teacher-student learning framework and a pseudo-label generation mechanism, a semi-supervised learning framework is built. The model can learn richer and more fundamental data distribution features from massive amounts of unlabeled data, effectively overcoming the dependence on large amounts of labeled data, resulting in high monitoring accuracy and strong generalization ability. A multimodal feature extraction network capable of deeply integrating vibration, force signals, and time-series images is constructed. Advanced data balancing and model optimization strategies are introduced to improve the model's robustness and accuracy under class imbalance and multiple operating conditions. Ultimately, a high-precision, low-cost, and highly adaptive online tool wear monitoring system is achieved. Through multimodal feature fusion, the model can comprehensively utilize the complementary advantages of different information, exhibiting stronger fault tolerance to single sensor failures or signal quality degradation. Data balancing and dynamic enhancement techniques effectively alleviate the class imbalance problem, making the model equally sensitive to rare tool breakage events. Ensemble learning and transfer learning strategies further enhance the model's adaptability and stability under different machine tools, workpiece materials, and cutting parameters.

[0041] Meanwhile, this invention minimizes the amount of data that needs to be manually labeled, reducing the cost of acquiring labels, which was originally expensive and time-consuming, by about 70%. It is economical and has low implementation costs, making it possible to deploy and apply high-precision tool condition monitoring technology on a large scale in small and medium-sized enterprises.

[0042] Embodiments of the present invention also provide a computer program product including program code, which, when the program product is run on an electronic device, causes the electronic device to perform the steps of the methods described above in various exemplary embodiments of the present invention.

[0043] Furthermore, although the steps of the method in this disclosure are described in a specific order in the accompanying drawings, this does not require or imply that the steps must be performed in that specific order, or that all the steps shown must be performed to achieve the desired result. Additional or alternative steps may be omitted, multiple steps may be combined into one step, and / or a step may be broken down into multiple steps.

[0044] From the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of this disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, external hard drive, etc.) or on a network, including several instructions to cause a computing device (such as a personal computer, server, mobile terminal, or network device, etc.) to execute the methods according to the embodiments of this disclosure.

[0045] In an exemplary embodiment of this disclosure, an electronic device capable of implementing the above-described method is also provided.

[0046] Those skilled in the art will understand that various aspects of the present invention can be implemented as systems, methods, or program products. Therefore, various aspects of the present invention can be specifically implemented in the following forms: entirely in hardware, entirely in software (including firmware, microcode, etc.), or in a combination of hardware and software, collectively referred to herein as “circuit,” “module,” or “system.”

[0047] An electronic device according to this embodiment of the invention. The electronic device is merely an example and should not be construed as limiting the functionality or scope of the embodiments of the invention.

[0048] Electronic devices are manifested in the form of general-purpose computing devices. Components of an electronic device may include, but are not limited to: at least one processor, at least one memory, and buses connecting different system components (including memory and processor).

[0049] The storage device stores program code that can be executed by the processor to perform the steps described in the "Exemplary Methods" section above, according to various exemplary embodiments of the present invention.

[0050] The storage may include readable media in the form of volatile storage, such as random access memory (RAM) and / or cache memory, and may further include read-only memory (ROM).

[0051] The storage may also include programs / utilities having a set (at least one) of program modules, including but not limited to: an operating system, one or more applications, other program modules, and program data, each or some combination of these examples may include an implementation of a network environment.

[0052] A bus can represent one or more of several bus architectures, including a memory bus or memory controller, a peripheral bus, a graphics acceleration port, a processor, or a local bus that uses any of the various bus architectures.

[0053] Electronic devices can also communicate with one or more external devices (such as keyboards, pointing devices, Bluetooth devices, etc.), one or more devices that enable users to interact with the electronic device, and / or any device that enables the electronic device to communicate with one or more other computing devices (such as routers, modems, etc.). This communication can be performed through input / output (I / O) interfaces. Furthermore, electronic devices can also communicate with one or more networks (such as local area networks (LANs), wide area networks (WANs), and / or public networks, such as the Internet) via network adapters.

[0054] In exemplary embodiments of this disclosure, a computer-readable storage medium is also provided, on which a program product capable of implementing the methods described above is stored. In some possible embodiments, various aspects of the invention may also be implemented as a program product comprising program code that, when the program product is run on a terminal device, causes the terminal device to perform the steps of the various exemplary embodiments of the invention described in the "Exemplary Methods" section of this specification.

[0055] The program product may employ any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0056] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A readable signal medium may also be any readable medium other than a readable storage medium, capable of sending, propagating, or transmitting programs for use by or in conjunction with an instruction execution system, apparatus, or device.

[0057] The program code contained on the readable medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical fiber, RF, etc., or any suitable combination thereof.

[0058] Program code for performing the operations of this invention can be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java and C++, and conventional procedural programming languages ​​such as C or similar languages. The program code can execute entirely on the user's computing device, partially on the user's device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0059] Furthermore, the above figures are merely illustrative of the processes included in the method according to exemplary embodiments of the present invention, and are not intended to be limiting. It is readily understood that the processes shown in the above figures do not indicate or limit the temporal order of these processes. Additionally, it is readily understood that these processes may be executed synchronously or asynchronously, for example, in multiple modules.

[0060] It should be noted that although several modules or units for the device used to perform actions have been mentioned in the detailed description above, this division is not mandatory. In fact, according to embodiments of this disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided and embodied by multiple modules or units.

[0061] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method for monitoring tool wear state based on semi-supervised learning, characterized in that, include: Step S100: Construct a multimodal tool wear dataset by collecting a number of labeled multimodal data and a number of unlabeled multimodal data; the multimodal data includes at least vibration signals during the cutting process, multi-channel force signals, and image signals of the tool flank wear area; Step S200: Construct a semi-supervised teacher-student learning model; the semi-supervised teacher-student learning model includes a student model and a teacher model with the same structure and shared parameters; Step S300: Based on the teacher model, perform forward reasoning on several unlabeled multimodal data in the multimodal tool wear dataset to obtain several pseudo-labeled multimodal data; Step S400: Train the student model together based on several pseudo-labeled multimodal data, several unlabeled multimodal data and several labeled multimodal data in the multimodal tool wear dataset; Step S500: Generate an adversarial network and dynamic data augmentation strategy according to preset conditions, and process the semi-supervised teacher-student learning model; Step S600: According to the ensemble learning strategy of model snapshot, the pre-trained model weights are transferred to the image branch of the semi-supervised teacher-student learning model to optimize the semi-supervised teacher-student learning model. Step S700: Deploy the optimized semi-supervised teacher-student learning model in the online monitoring system of the tool under test; Step S800: Input the multimodal data of the tool under test during the machining process, which is collected in real time, into the semi-supervised teacher-student learning model to obtain the wear state prediction result corresponding to the tool under test.

2. The method according to claim 1, characterized in that, Step S100 includes: Step S110: Based on the multi-channel force sensor installed on the spindle of the tool under test and the vibration sensor fixed on the workpiece to be processed, the milling machining sensing signal corresponding to the wear state of the tool under test at the current moment is collected; the milling machining sensing signal includes vibration signal and multi-channel force signal; Step S120: After each pass of the tool under test, acquire the back face image of the tool under test using an industrial camera and a dual telecentric microscope; Step S130: Obtain the process parameters of the tool under test during each cutting operation; the process parameters include cutting speed, feed per tooth, depth of cut, and effective cutting time; Step S140: Using the milling processing sensor signal and the process parameters as multimodal data, and the corresponding flank wear band width VB value in the flank image as the wear label, a tagged data subset and an untagged data subset are established; wherein, the label corresponding to the multimodal data in the tagged data subset is the wear label; Step S150: Perform signal preprocessing and image preprocessing on several multimodal data in the labeled data subset and the unlabeled data subset to obtain a multimodal tool wear dataset.

3. The method according to claim 2, characterized in that, Step S150 includes: Step S151: Define a sliding window of a preset length on the entire cutting signal curve of the cutting process of the tool under test. The sliding window slides point by point along the time sequence of the cutting signals in several multimodal data in the labeled data subset and the unlabeled data subset. Step S152: After the sliding window has traversed all the multimodal data in the labeled data subset and the unlabeled data subset, obtain several variances of all the multimodal data corresponding to each sliding of the sliding window; Step S153: Based on the aforementioned variances, determine the variance threshold using a preset quartile method; Step S154: Perform a forward traversal of the cutting signals in several multimodal data in the labeled data subset and the unlabeled data subset according to the time sequence, and determine the first time point where the variance is greater than the variance threshold as the entry point. Step S155: Perform reverse traversal of the cutting signals in several multimodal data in the labeled data subset and the unlabeled data subset according to the time sequence, and determine the first time point where the variance is greater than the variance threshold as the tool retraction point. Step S156: Delete the multimodal data whose corresponding time point is not between the tool entry point and the tool retraction point from the multimodal data in the labeled data subset and the unlabeled data subset; Step S157: Using a preset wavelet soft thresholding denoising algorithm and Butterworth low-pass filtering algorithm, denoise and smooth the corresponding back face images in the labeled data subset and the unlabeled data subset, and perform grayscale conversion, contrast enhancement and region cropping preprocessing to obtain a multimodal tool wear dataset.

4. The method according to claim 3, characterized in that, Step S200 includes: Step S210: Using the Mean Teacher framework, construct a student model F. S (θ S ) and a teacher model F T (θ T A semi-supervised teacher-student learning model; Among them, the parameter θ of the teacher model T The parameter θ of the student model S The result of the exponential moving average during the training process; θ T (t) =αθ T (t-1) +(1-α)θ S (t) α is the preset smoothing coefficient, and the value of α ranges from [0.99, 0.999]; t is any time point in the training process; The structures of the teacher model and the student model include: The image feature extraction branch is used to perform convolution and pooling operations on the preprocessed back face image to extract high-level features related to wear from the pixels, and obtain a high-level visual feature vector representing the image content of the back face image. The sensor signal feature extraction branch is used to process the preprocessed milling sensor signal to capture the long-term temporal dependence and change trend in the milling sensor signal and obtain the temporal feature vector. The feature fusion module is used to automatically learn and weightedly fuse important information from different modalities by calculating the correlation between different multimodal data features in the high-level visual feature vector and the temporal feature vector to obtain a fused feature representation vector. The output layer is used to map the fused feature representation vector to the prediction target and output the regression prediction or state classification probability of the tool wear value.

5. The method according to claim 4, characterized in that, Step S300 includes: Step S310: Input each multimodal data in the unlabeled data subset into the teacher model and the student model respectively, and use the prediction generated by the teacher model as a pseudo-label; Step S320: Determine the difference between the predictions generated by the student model and the pseudo-labels as the unsupervised loss function; Step S330: Input each multimodal data in the labeled data subset into the student model, and have the student model output a supervised loss function; Step S340: Integrate the unsupervised loss function and the supervised loss function into a joint loss function; Step S350: Update the parameters of the student model through backpropagation, and update the parameters of the teacher model by using the exponential moving average of the parameters of the student model. Step S360: The output probability of the semi-supervised teacher-student learning model is calibrated using a temperature scaling strategy; Step S370: Retain the multimodal data whose maximum output probability after calibration is greater than the preset confidence threshold, and delete the remaining multimodal data from the multimodal tool wear dataset; Step S380: The category corresponding to the output probability whose maximum value is greater than the preset confidence threshold is determined as a pseudo label.

6. The method according to claim 5, characterized in that, Step S500 includes: Step S510: Generate an adversarial network according to preset conditions. Using process parameters and wear status as conditions, generate corresponding multimodal sensing signal features and add them to the multimodal tool wear dataset. Step S520: During the training process of the semi-supervised teacher-student learning model, the image data of the input data is subjected to any of the following processing: rotation, translation, scaling, adjustment of brightness and contrast, addition of Gaussian noise, and the sensor signal of the input data is subjected to any of the following processing: time distortion, window distortion, and random masking.

7. The method according to claim 6, characterized in that, Step S600 includes: Step S610: Use random forest as a meta-learner to perform weighted voting or averaging on the prediction results of the semi-supervised teacher-student learning model; Step S620: Initialize the image branch of the semi-supervised teacher-student learning model using model weights pre-trained on a large image dataset.

8. The method according to claim 7, characterized in that, Step S800 includes: Step S810: Input the multimodal data of the tool under test during the machining process collected in real time into the semi-supervised teacher-student learning model to obtain the wear prediction value output by the semi-supervised teacher-student learning model. Step S820: If the wear prediction value is greater than or equal to the preset wear threshold, then the tool wear is determined as the wear state prediction result corresponding to the tool under test; otherwise, the tool no wear is determined as the wear state prediction result corresponding to the tool under test.

9. A non-transitory computer-readable storage medium, characterized in that, The storage medium stores at least one instruction or at least one program segment, which is loaded and executed by a processor to implement the method as described in any one of claims 1-8.

10. An electronic device, characterized in that, Includes a processor and the non-transitory computer-readable storage medium as described in claim 9.

Images