Tool damage state monitoring method and equipment based on zero sample learning and medium

Abstract

The invention provides a tool breakage state monitoring method and device based on zero sample learning and a medium, and the method comprises the steps: generating a high-dimensional semantic vector according to a word vector corresponding to a tool-breakage structured text description and a graph embedding vector corresponding to a tool-breakage knowledge graph; generating a multi-modal sensing feature vector according to the data of the to-be-detected cutter in the cutting process; mapping the multi-modal sensing feature vector to a semantic space corresponding to the high-dimensional semantic vector; generating a virtual sample by taking the high-dimensional semantic vector corresponding to the sample of the invisible damaged category as a condition; training a zero sample classifier according to the virtual sample; the dynamic threshold value is determined according to the zero sample classifier, real-time early warning monitoring is conducted on the to-be-detected tool based on the dynamic threshold value, high-precision recognition of unknown damage categories can be achieved by means of priori knowledge, namely semantic attributes, of the tool and data of known damage modes, and therefore the adaptive capacity of a monitoring system in the variable machining environment is improved.

Description

Technical Field

[0001] This invention relates to the field of intelligent manufacturing and equipment condition monitoring technology, and in particular to a method, equipment and medium for monitoring tool breakage condition based on zero-shot learning. Background Technology

[0002] In precision CNC machining, the cutting tool, as a key component that directly performs cutting tasks, has its health condition directly affecting machining quality, efficiency, and cost. Tool breakage is one of the main failure modes, and if not detected in time, it may lead to workpiece scrap, machine tool damage, or even safety accidents.

[0003] Currently, mainstream tool breakage monitoring methods can be divided into two categories: physical model-based methods and data-driven methods. The latter, especially deep learning models such as Convolutional Neural Networks (CNNs), has attracted much attention due to its powerful feature learning capabilities. However, these methods have inherent limitations, as follows:

[0004] Heavy reliance on large amounts of labeled data: Supervised learning models need to provide hundreds or thousands of labeled samples for each damage mode. However, in real industrial scenarios, samples of some severe damage (such as broken blades) are readily available, but samples of micro-damage (such as micro-chipping and thermal cracks) are scarce, making it difficult to construct a balanced training set.

[0005] Poor generalization ability: When the trained model faces tool types or new damage patterns that have not appeared in the training set, its recognition performance drops sharply and it cannot adapt to the needs of frequent tool changes in flexible manufacturing systems.

[0006] Insufficient utilization of multimodal information: Most studies only use single-type sensor signals, such as vibration signals or force signals, and fail to fully integrate the complementary value of multi-source information such as vision, force, and vibration.

[0007] Zero-shot learning, by introducing "semantic attributes," transfers knowledge from known categories to unknown categories, providing a solution to the aforementioned problems. However, in the field of tool monitoring, which has a strong physical context, how to construct an effective semantic space, how to integrate multimodal sensing information, and how to ensure the real-time performance and robustness of the model in the industrial field remain key technical challenges to be solved. Summary of the Invention

[0008] To address the aforementioned technical problems, the technical solution adopted by this invention is as follows:

[0009] According to one aspect of this application, a method for monitoring tool breakage state based on zero-shot learning is provided, comprising:

[0010] Step S100: Generate a high-dimensional semantic vector based on the word vectors corresponding to the pre-defined structured text description of the tool-damage and the graph embedding vectors corresponding to the pre-defined tool-damage knowledge graph.

[0011] Step S200: Based on the collected triaxial vibration signal, triaxial force signal and tool back face image of the tool under test during the cutting process, perform data preprocessing and feature fusion to generate a multimodal sensing feature vector;

[0012] Step S300: Map the multimodal sensing feature vectors to the semantic space corresponding to the high-dimensional semantic vectors;

[0013] Step S400: Based on the preset improved generative adversarial network, generate corresponding virtual samples using the high-dimensional semantic vectors corresponding to samples of the preset invisible damage category as conditions.

[0014] Step S500: Construct a zero-shot classifier based on a semantic autoencoder and train the zero-shot classifier based on virtual samples.

[0015] Step S600: Determine the dynamic threshold based on the zero-shot classifier, and perform real-time early warning monitoring of the cutting tool to be detected based on the dynamic threshold.

[0016] According to another aspect of this application, a non-transitory computer-readable storage medium is provided, wherein at least one instruction or at least one program is stored in the storage medium, the at least one instruction or the at least one program being loaded and executed by a processor to implement the aforementioned zero-shot learning-based tool breakage state monitoring method.

[0017] According to another aspect of this application, an electronic device is provided, including a processor and the aforementioned non-transitory computer-readable storage medium.

[0018] The present invention has at least the following beneficial effects:

[0019] The zero-shot learning-based tool breakage state monitoring method of this invention first generates a high-dimensional semantic vector based on the word vectors corresponding to the structured text description of tool-breakage and the graph embedding vectors corresponding to the tool-breakage knowledge graph. Then, it performs data preprocessing and feature fusion on the collected triaxial vibration signals, triaxial force signals, and tool back face images of the tool during the cutting process to generate a multimodal sensing feature vector. This multimodal sensing feature vector is mapped to the semantic space corresponding to the high-dimensional semantic vector. Based on an improved generative adversarial network, virtual samples are generated using the high-dimensional semantic vectors corresponding to samples of invisible breakage categories as conditions. A zero-shot classifier based on a semantic autoencoder is then constructed. Virtual samples are used to train a zero-shot classifier. Finally, a dynamic threshold is determined based on the zero-shot classifier, and real-time early warning monitoring is performed on the tool to be inspected based on the dynamic threshold. By fusing vibration signals, force signals, visual signals, and text and graph knowledge, a richer and more robust semantic representation is constructed. Combined with an improved GAN loss function, the generated virtual samples are ensured to be both realistic and semantically consistent, effectively solving the data missing problem in zero-shot learning. It can utilize the prior knowledge of the tool, namely semantic attributes and known damage patterns, to achieve high-precision identification of unknown damage categories, significantly reducing the dependence on labeled data and improving the adaptability of the monitoring system in variable processing environments. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 A flowchart of a tool breakage state monitoring method based on zero-shot learning provided in an embodiment of the present invention;

[0022] Figure 2 A detailed overall flowchart of the tool breakage state monitoring method based on zero-shot learning provided in this embodiment of the invention;

[0023] Figure 3 This is a schematic diagram of the hardware setup of the multimodal data acquisition system provided in an embodiment of the present invention;

[0024] Figure 4 This is a schematic diagram of the structure of a semantic mapping network provided in an embodiment of the present invention;

[0025] Figure 5 A schematic diagram of the structure and loss function of the improved generative adversarial network provided in an embodiment of the present invention;

[0026] Figure 6 This is a schematic diagram of the structure of the semantic autoencoder and the zero-sample classification decision process provided in an embodiment of the present invention. Detailed Implementation

[0027] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0028] This application proposes a tool breakage state monitoring method based on zero-shot learning, such as... Figure 1 and Figure 2 As shown, it includes:

[0029] Step S100: Generate a high-dimensional semantic vector based on the word vectors corresponding to the pre-defined structured text description of the tool-damage and the graph embedding vectors corresponding to the pre-defined tool-damage knowledge graph.

[0030] Furthermore, step S100 includes steps S110-S150:

[0031] Step S110: Generate a structured text description for each tool-damage category according to the type of tool to be inspected;

[0032] Step S120: Extract word vectors of structured text descriptions corresponding to each tool-damage category using a pre-set deep bidirectional pre-trained language model;

[0033] The breakage modes in this application are divided into visible breakage and invisible breakage for zero-sample evaluation. Visible breakage includes three types: normal wear, chipping, and cracks. Invisible breakage includes two types: broken blade and edge edge.

[0034] The end mills to be tested can be selected from categories such as carbide end mills and ball end mills. A structured text description is generated for each "tool-damage" combination, such as "carbide end mill chipped". The 768-dimensional word vector a_text(y) is extracted using the BERT (Bidirectional Encoder Representations from Transformers) pre-trained model (i.e., a deep bidirectional pre-trained language model) to capture rich linguistic semantics, where y is the corresponding "tool-damage" category.

[0035] Step S130: Construct a tool-damage knowledge graph containing tool attributes, damage mode entities, and their interrelationships;

[0036] The nodes in the tool-breakage knowledge graph include entities such as tool type, material, coating, geometric parameters, and breakage mode, while the edges represent the physical, causal, or statistical relationships between them.

[0037] Step S140: Extract the graph embedding vectors of the damage mode nodes from the tool-damage knowledge graph through a preset graph convolutional network;

[0038] We use a two-layer graph convolutional network to learn the embedding representation of nodes in the tool-breakage knowledge graph, and specifically extract the 256-dimensional graph embedding vector a_kg(y) of the breakage pattern node to capture structured domain knowledge.

[0039] Step S150: Concatenate the word vectors and graph embedding vectors and perform principal component analysis to reduce dimensionality, thereby generating high-dimensional semantic vectors.

[0040] The 768-dimensional word vectors and the 256-dimensional graph embedding vectors are concatenated to form a 1024-dimensional fusion vector. Then, principal component analysis is used to reduce the dimensionality of the 1024-dimensional fusion vector to 512 dimensions, resulting in the final high-dimensional semantic vector a(y).

[0041] Dimensionality reduction aims to remove redundancy, reduce computational complexity, and enhance the compactness of the representation.

[0042] 512 dimensions are a dimension that has been verified through extensive experiments to achieve the best balance between information preservation and model efficiency.

[0043] Furthermore, step S150 includes steps S151-S155:

[0044] Step S151: Concatenate the word vector and graph embedding vector corresponding to each tool-damage category in the first dimension to obtain the fusion vector corresponding to each tool-damage category;

[0045] For each category y of "tool-damage", its word vector a_text(y) and graph embedding vector a_kg(y) are concatenated in the first dimension to obtain a 1024-dimensional fused vector a_fused(y).

[0046] ;

[0047] Step S152: Stack the fusion vectors corresponding to all tool-damage categories into a data matrix, and standardize each feature dimension of the data matrix to obtain a standardized data matrix;

[0048] Stack the 1024-dimensional fusion vectors a_fused(y) of all N categories of "tool-damage" (y∈[1,N]; including visible and invisible classes) into a data matrix X with shape [N,1024]. Standardize each column (i.e., each feature dimension) of the data matrix X so that its mean is 0 and its variance is 1.

[0049] Step S153: Calculate the covariance matrix of the standardized data matrix, and perform eigenvalue decomposition on the covariance matrix to obtain the corresponding eigenvalues ​​and eigenvectors.

[0050] Step S154: Sort several feature values ​​in descending order, and determine the feature values ​​that are among the first preset target number of the sorted feature values ​​as target feature values.

[0051] The variance contribution rate and cumulative variance contribution rate of each principal component are calculated. The goal is to retain more than 95% of the original variance information. After calculation, the top 512 principal components (i.e., the preset target number) are selected, which are the feature vectors corresponding to the maximum 512 feature values. This goal can be achieved or exceeded. Therefore, the output dimension of the high-dimensional semantic vector is set to 512 dimensions.

[0052] Step S155: Project the fusion vector corresponding to each tool-damage category after standardization onto a subspace spanned by feature vectors corresponding to several target feature values ​​to obtain a high-dimensional semantic vector corresponding to each tool-damage category.

[0053] The standardized original fused vector a_fused(y) is projected onto the subspace spanned by these first 512 feature vectors, and the relevant formula is as follows: Where mean is the mean vector of the training data; V_{512} is the projection matrix composed of the first 512 feature vectors, with shape [1024, 512].

[0054] Finally, a 512-dimensional high-dimensional semantic vector a(y) is obtained for each "tool-damage" category y; this vector will be used for the training and inference of all subsequent models.

[0055] Step S200: Based on the collected triaxial vibration signal, triaxial force signal and tool back face image of the tool under test during the cutting process, perform data preprocessing and feature fusion to generate a multimodal sensing feature vector;

[0056] Furthermore, step S200 includes steps S210-S260:

[0057] Step S210, as follows Figure 3As shown, the triaxial acceleration vibration sensor is fixed to the surface of the workpiece or worktable by a magnetic base, so that the three sensitive axes of the triaxial acceleration vibration sensor are aligned with the three axes of the coordinate system of the machine tool to be tested, so as to accurately capture the vibration response during the processing.

[0058] Step S220: Install the multi-channel force sensor in the spindle taper hole of the machine tool; the multi-channel force sensor integrates a triaxial force sensing unit.

[0059] Multi-channel force sensors can be used with intelligent force-measuring tool holders. The intelligent force-measuring tool holders have strain gauges embedded inside, which can directly measure the three-dimensional cutting force during the cutting process.

[0060] The machine tool is a vertical milling machining center, which uses different types of milling cutters to mill high-hardness workpieces.

[0061] Step S230: Fix the industrial camera to the observation window outside the machining cavity using a rigid bracket, so that the lens of the industrial camera is facing the tool to be inspected set on the spindle of the machining tool;

[0062] To eliminate interference from cutting fluid reflections and shadows, the industrial camera is also equipped with an LED coaxial light source, ensuring clear images of the back face can be obtained at any time when the machine is stopped.

[0063] The triaxial acceleration vibration sensor, multi-channel force sensor, and industrial camera are all connected to the multi-channel data acquisition card via data cables, and the sensor signal acquisition module on the computer terminal performs synchronous data acquisition and processing.

[0064] Step S240: During the cutting process of the tool under test, the triaxial vibration signal collected by the triaxial acceleration vibration sensor and the triaxial force signal collected by the multi-channel force sensor are decomposed by wavelet packet to obtain the sensing feature vector.

[0065] The triaxial vibration and force signals were processed by wavelet decomposition: the triaxial vibration and force signals during the stable cutting stage were extracted, and 5-level db4 wavelet packet decomposition was performed. The 8 sub-frequency bands with the most concentrated energy were selected, and their energy values ​​were calculated, generating a total of 48-dimensional time-frequency domain features for the 6 sensing channels.

[0066] Step S250: Global average pooling feature extraction is performed on the tool back face image of the tool to be inspected captured by the industrial camera during the cutting process to obtain the image feature vector.

[0067] The YOLOv5 model (an existing object detection model) is used to automatically locate the tool region in the tool flank image and crop out the Region of Interest (ROI). Then, contrast-limited adaptive histogram equalization and Gaussian filtering are performed to enhance the contrast and filter the noise of the ROI. The ROI is then input into the pre-trained ResNet-50 model, and the global average pooling features of its last convolutional layer are extracted to obtain a 2048-dimensional image feature vector.

[0068] Step S260: Normalize the sensing feature vector and the image feature vector by their maximum and minimum values ​​respectively, and then concatenate them to obtain the multimodal sensing feature vector.

[0069] The 48-dimensional sensing feature vector and the 2048-dimensional image feature vector are normalized and then concatenated to form a 2096-dimensional multimodal sensing feature vector x, which serves as the unified input for subsequent models.

[0070] Step S300: Map the multimodal sensing feature vector to the semantic space corresponding to the high-dimensional semantic vector to achieve semantic alignment between sensing features and semantic features;

[0071] Furthermore, step S300 includes steps S310-S340:

[0072] Step S310: Construct a three-layer semantic mapping network that uses a three-layer fully connected neural network as the mapping network;

[0073] The three-layer semantic mapping network uses a three-layer fully connected neural network as the mapping network φ. The input layer is 2096-dimensional, the two hidden layers have 1024 and 512 neurons respectively, and the ReLU activation function is used. The output layer is 512-dimensional, which is the same as the dimension of the semantic space corresponding to the high-dimensional semantic vector.

[0074] Specifically, the structure of the three-layer semantic mapping network is a three-layer fully connected neural network, such as... Figure 4 As shown, it includes:

[0075] An input layer, whose input is a 2096-dimensional multimodal sensing feature vector x;

[0076] The first hidden layer, containing 1024 neurons, performs a linear transformation on the input and then applies the ReLU activation function, calculated as follows: Where W1 represents the weight matrix with shape [1024, 2096]; b1 represents the bias vector with shape

[1024] ; and ReLU represents the modified linear unit activation function, defined as... It introduces nonlinearity, enabling the network to learn more complex mapping relationships;

[0077] The second hidden layer, containing 512 neurons, is obtained by linearly transforming the output of the first hidden layer and then applying the activation function again. , where W2 represents the weight matrix and b2 represents the bias vector;

[0078] An output layer that performs a linear transformation on the output of the second hidden layer. ;

[0079] Step S320: Minimize the Euclidean distance between the output vector of the three-layer semantic mapping network and the high-dimensional semantic vector using the mean squared error loss function;

[0080] To train this three-layer semantic mapping network, a loss function needs to be defined to quantify the difference between the mapped output φ(x) and the target semantic vector a(y), and to guide the optimization of the network parameters, thereby forcing the clustering of sensor features of the same category in the semantic space. Therefore, the mean squared error loss function L_map is defined as follows:

[0081] ;

[0082] in, For the network to input features x i The output of a(y); i ) represents the corresponding real high-dimensional semantic vector; λ is the L2 regularization coefficient, set to 0.001; θ is the network parameter.

[0083] Adding L2 regularization can prevent network overfitting.

[0084] Step S330: Train the three-layer semantic mapping network for several training cycles using samples of the visible damage category;

[0085] In the data preparation and initialization phase of the training stage of the three-layer semantic mapping network, only samples of the visible damage category are used for training, and each training sample is a paired data (x i ,a(y i ), where x i It is a 2096-dimensional multimodal sensing feature vector, a(y i ) is its corresponding 512-dimensional real high-dimensional semantic vector.

[0086] During the parameter initialization phase, the "Kaiming initialization" strategy is used to initialize all network weight matrices (W1, W2, W3). This is an optimized initialization method for the ReLU activation function, which can accelerate training convergence. All bias vectors (b1, b2, b3) are initialized to 0.

[0087] For the optimizer selection and configuration, the Adam optimizer was adopted, with a learning rate set to 1e-4. This is a relatively small learning rate to ensure a smooth training process and avoid oscillations. The default parameters of the Adam optimizer in the PyTorch framework were used: beta1=0.9, beta2=0.999, eps=1e-8.

[0088] Next, the three-layer semantic mapping network is trained iteratively for 100 training epochs. Within each epoch, the training dataset is randomly shuffled and divided into multiple batches, each containing N=32 samples. For each batch, the mapping output {φ(x_1),φ(x_2),...,φ(x_32)} is calculated sequentially through the input layer, hidden layer, and output layer. Based on the loss function formula L_map, the total loss for the current batch is calculated. Using the chain rule, the gradient of the loss function with respect to all network parameters (W1,b1,W2,b2,W3,b3) is calculated. The Adam optimizer updates all parameters in the network based on the calculated gradients.

[0089] Step S340: Based on the trained three-layer semantic mapping network, map the multimodal sensing feature vectors to the semantic space corresponding to the high-dimensional semantic vectors.

[0090] After the three-layer semantic mapping network is trained, the network parameters θ have converged to a state that allows the mapping function φ(·) to project the multimodal sensing feature vector x to the correct position in the semantic space and use it for subsequent steps.

[0091] Step S400: Based on the preset improved generative adversarial network, generate corresponding virtual samples using the high-dimensional semantic vectors corresponding to samples of the preset invisible damage category as conditions.

[0092] Furthermore, step S400 includes steps S410-S430:

[0093] Step S410: Construct an improved generative adversarial network;

[0094] Among them, such as Figure 5 As shown, the improved generative adversarial network includes a generator and a discriminator:

[0095] The generator consists of four fully connected layers (with dimensions of 1024, 1024, 1024, and 2096 respectively). The input is a concatenation of a 100-dimensional random noise vector (to ensure the diversity of generated samples) and a 512-dimensional high-dimensional semantic vector (to specify the damage category to be generated). The input is passed through the four fully connected layers, activated by LeakyReLU, and finally outputs a 2096-dimensional virtual sensing feature vector constrained to [-1, 1] by the Tanh activation function.

[0096] The discriminator consists of three fully connected layers (with dimensions of 512, 512, and 512 respectively). The input is a concatenation of a real or virtual 2096-dimensional multimodal sensing feature vector and a corresponding 512-dimensional high-dimensional semantic vector. The discriminator is activated by LeakyReLU through the three fully connected layers and finally outputs a scalar discriminant score.

[0097] Step S420: Train the generator and discriminator according to the preset improved loss function until the generative adversarial network reaches Nash equilibrium.

[0098] The improved loss function is: ;

[0099] Where L_adv is the adversarial loss of Wasserstein GAN with Gradient Penalty; This is a loss due to circular consistency. The semantic consistency loss is α=10, β=5.

[0100] The generator and discriminator are trained using an improved loss function until a Nash equilibrium is reached. At this point, the discriminator struggles to distinguish between real and virtual features, while the generator produces high-quality virtual samples. A total loss is defined for the generator, consisting of three parts:

[0101] Adversarial loss: Encouraging the generation of virtual features Achieve a high score on the discriminator;

[0102] Cyclic consistency loss: the generated virtual features The input is fed into a pre-trained semantic mapping network φ with fixed parameters. This network is able to... Map back to the vicinity of the conditional semantic vector a(y) used when it was generated;

[0103] Semantic consistency loss: It directly constrains the generated virtual features It requires that its corresponding semantic attributes be highly consistent with the conditional semantic vector a(y);

[0104] The total loss of the generator is the weighted sum of these three parts.

[0105] Step S430: Input the high-dimensional semantic vectors corresponding to the samples of the invisible damage category in the sample dataset as conditions into the trained generative adversarial network to generate the corresponding virtual samples.

[0106] From the entire sample dataset, extract samples of all "invisible" damage categories, such as "tumor". For each "invisible" class sample, obtain its corresponding 512-dimensional high-dimensional semantic vector a(y) from step S100. This vector will serve as conditional information to guide the generator in generating virtual features for a specific category.

[0107] For each category of invisible damage, a large amount of random noise and its corresponding semantic vector are input to generate many virtual features. By calculating their similarity in distribution to a small number of real, invisible class samples, such as the maximum mean difference (MMD), high-quality virtual samples that are closest to the distribution of real data are retained. Finally, these selected high-quality virtual features, together with their class labels, constitute a virtual training set and are incorporated into the final training of the zero-shot classifier.

[0108] Step S500: Construct a zero-shot classifier based on a semantic autoencoder and train the zero-shot classifier based on virtual samples.

[0109] Furthermore, step S500 includes steps S510-S540:

[0110] Step S510: Train a semantic autoencoder on the high-dimensional semantic vectors corresponding to the visible damage category samples in the sample dataset;

[0111] like Figure 6 As shown, a semantic autoencoder is trained using the high-dimensional semantic vector a(y) corresponding to the samples of the visible damage category, and its loss function is: Where E is the encoder and D is the decoder.

[0112] The encoder E of the semantic autoencoder compresses the 512-dimensional semantic vector to a 128-dimensional bottleneck layer, and the decoder D of the semantic autoencoder reconstructs it back to 512 dimensions. Minimizing the reconstruction error L_SAE enables the autoencoder to learn the core manifold of the semantic space.

[0113] Step S520: Input the virtual sample into the three-layer semantic mapping network to obtain the corresponding mapping features;

[0114] Step S530: Encode and decode the mapping features using a semantic autoencoder to obtain a reconstructed semantic vector;

[0115] Step S540: Calculate the cosine similarity between the reconstructed semantic vector and the high-dimensional semantic vectors corresponding to all damaged categories. Use the damaged category with the highest cosine similarity as the prediction result to train the zero-shot classifier of the semantic autoencoder.

[0116] For the test sample, φ(x_test) is first obtained through the mapping network φ, and then the reconstructed semantic vector a_pred is obtained through the encoder E and decoder D. Finally, the cosine similarity between a_pred and the semantic vectors of all categories is calculated, and the most similar category is taken as the prediction result. This process transforms the classification problem into a nearest neighbor search in the semantic space.

[0117] Step S600: Determine the dynamic threshold based on the zero-shot classifier, and perform real-time early warning monitoring of the tool to be detected based on the dynamic threshold;

[0118] Furthermore, step S600 includes steps S610-S620:

[0119] Step S610: Input the multimodal sensing feature vector corresponding to the cutting process of the tool to be detected into the zero-shot classifier to obtain the corresponding prediction confidence of the zero-shot classifier output.

[0120] Step S620: If the prediction confidence is lower than the preset dynamic threshold, it is determined that the tool to be tested is abnormal or has suffered unknown damage.

[0121] In the initial stage of operation, the collected data are analyzed, and the mean μ and standard deviation σ of the prediction confidence level are calculated. A dynamic threshold is then set. Where μ is the mean of the highest predicted similarity between historical normal samples and known damaged samples; σ is its standard deviation; and k is an adjustable parameter, usually set to 2.

[0122] During online monitoring, if the prediction confidence level of real-time data falls below this threshold, it is determined to be a tool malfunction or unknown damage, and an alarm is immediately triggered to prompt the operator to intervene.

[0123] This invention is applicable to tool breakage detection under multiple working conditions and tool types. Through a complete technical solution from semantic construction, multimodal fusion, generative data augmentation to monitoring and early warning, it provides a feasible path to achieve intelligent, adaptive, and low-maintenance-cost tool condition monitoring. The core lies in linking prior knowledge with sensor data through semantic space and compensating for the lack of data in unknown categories through generative models. It identifies and warns of newly emerging tool breakage patterns that lack labeled samples. By utilizing prior knowledge of tools, namely semantic attributes, and data on known breakage patterns, the model can accurately identify new tool types or new breakage patterns that have never seen real samples during the training phase without relying on a large amount of labeled data. It has high robustness, strong adaptability, and powerful zero-shot recognition capability, which can accurately identify new tool types or new breakage patterns that have never seen real samples during the training phase, greatly expanding the application scope of the monitoring system.

[0124] The zero-shot learning-based tool breakage state monitoring method of this invention first generates a high-dimensional semantic vector based on the word vectors corresponding to the structured text description of tool-breakage and the graph embedding vectors corresponding to the tool-breakage knowledge graph. Then, it performs data preprocessing and feature fusion on the collected triaxial vibration signals, triaxial force signals, and tool back face images during the cutting process to generate a multimodal sensing feature vector. This multimodal sensing feature vector is mapped to the semantic space corresponding to the high-dimensional semantic vector. Based on an improved generative adversarial network, virtual samples are generated using the high-dimensional semantic vectors corresponding to samples of invisible breakage categories as conditions. A zero-shot classifier based on a semantic autoencoder is constructed. The zero-shot classifier is trained using the virtual samples. Finally, the zero-shot classifier determines the breakage state. A dynamic threshold is defined, and real-time early warning monitoring of the tool under inspection is performed based on the dynamic threshold. By integrating vibration signals, force signals, visual signals, and text and graph knowledge, a richer and more robust semantic representation is constructed. Combined with an improved GAN loss function, the generated virtual samples are ensured to be both realistic and semantically consistent, effectively solving the data missing problem in zero-shot learning. Finally, the tool breakage mode is monitored and early warning is issued in real time based on the dynamic threshold. A complete hardware construction scheme ensures that the method can run stably and in real time in actual industrial environments. It can utilize the prior knowledge of the tool, namely semantic attributes and known breakage modes, to achieve high-precision identification of unknown breakage categories, significantly reducing the dependence on labeled data and improving the adaptability of the monitoring system in variable processing environments.

[0125] Embodiments of the present invention also provide a computer program product including program code, which, when the program product is run on an electronic device, causes the electronic device to perform the steps of the methods described above in various exemplary embodiments of the present invention.

[0126] Furthermore, although the steps of the method in this disclosure are described in a specific order in the accompanying drawings, this does not require or imply that the steps must be performed in that specific order, or that all the steps shown must be performed to achieve the desired result. Additional or alternative steps may be omitted, multiple steps may be combined into one step, and / or a step may be broken down into multiple steps.

[0127] From the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of this disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, external hard drive, etc.) or on a network, including several instructions to cause a computing device (such as a personal computer, server, mobile terminal, or network device, etc.) to execute the methods according to the embodiments of this disclosure.

[0128] In an exemplary embodiment of this disclosure, an electronic device capable of implementing the above-described method is also provided.

[0129] Those skilled in the art will understand that various aspects of the present invention can be implemented as systems, methods, or program products. Therefore, various aspects of the present invention can be specifically implemented in the following forms: entirely in hardware, entirely in software (including firmware, microcode, etc.), or in a combination of hardware and software, collectively referred to herein as “circuit,” “module,” or “system.”

[0130] An electronic device according to this embodiment of the invention. The electronic device is merely an example and should not be construed as limiting the functionality or scope of the embodiments of the invention.

[0131] Electronic devices are manifested in the form of general-purpose computing devices. Components of an electronic device may include, but are not limited to: at least one processor, at least one memory, and buses connecting different system components (including memory and processor).

[0132] The storage device stores program code that can be executed by the processor to perform the steps described in the "Exemplary Methods" section above, according to various exemplary embodiments of the present invention.

[0133] The storage may include readable media in the form of volatile storage, such as random access memory (RAM) and / or cache memory, and may further include read-only memory (ROM).

[0134] The storage may also include programs / utilities having a set (at least one) of program modules, including but not limited to: an operating system, one or more applications, other program modules, and program data, each or some combination of these examples may include an implementation of a network environment.

[0135] A bus can represent one or more of several bus architectures, including a memory bus or memory controller, a peripheral bus, a graphics acceleration port, a processor, or a local bus that uses any of the various bus architectures.

[0136] Electronic devices can also communicate with one or more external devices (such as keyboards, pointing devices, Bluetooth devices, etc.), one or more devices that enable users to interact with the electronic device, and / or any device that enables the electronic device to communicate with one or more other computing devices (such as routers, modems, etc.). This communication can be performed through input / output (I / O) interfaces. Furthermore, electronic devices can also communicate with one or more networks (such as local area networks (LANs), wide area networks (WANs), and / or public networks, such as the Internet) via network adapters.

[0137] In exemplary embodiments of this disclosure, a computer-readable storage medium is also provided, on which a program product capable of implementing the methods described above is stored. In some possible embodiments, various aspects of the invention may also be implemented as a program product comprising program code that, when the program product is run on a terminal device, causes the terminal device to perform the steps of the various exemplary embodiments of the invention described in the "Exemplary Methods" section of this specification.

[0138] The program product may employ any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0139] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A readable signal medium may also be any readable medium other than a readable storage medium, capable of sending, propagating, or transmitting programs for use by or in conjunction with an instruction execution system, apparatus, or device.

[0140] The program code contained on the readable medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical fiber, RF, etc., or any suitable combination thereof.

[0141] Program code for performing the operations of this invention can be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java and C++, and conventional procedural programming languages ​​such as C or similar languages. The program code can execute entirely on the user's computing device, partially on the user's device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0142] Furthermore, the above figures are merely illustrative of the processes included in the method according to exemplary embodiments of the present invention, and are not intended to be limiting. It is readily understood that the processes shown in the above figures do not indicate or limit the temporal order of these processes. Additionally, it is readily understood that these processes may be executed synchronously or asynchronously, for example, in multiple modules.

[0143] It should be noted that although several modules or units for the device used to perform actions have been mentioned in the detailed description above, this division is not mandatory. In fact, according to embodiments of this disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided and embodied by multiple modules or units.

[0144] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method for monitoring tool breakage state based on zero-shot learning, characterized in that, include: Step S100: Generate a high-dimensional semantic vector based on the word vectors corresponding to the pre-defined structured text description of the tool-damage and the graph embedding vectors corresponding to the pre-defined tool-damage knowledge graph. Step S200: Based on the collected triaxial vibration signal, triaxial force signal and tool back face image of the tool under test during the cutting process, perform data preprocessing and feature fusion to generate a multimodal sensing feature vector; Step S300: Map the multimodal sensing feature vector to the semantic space corresponding to the high-dimensional semantic vector; Step S400: Based on the preset improved generative adversarial network, generate corresponding virtual samples using the high-dimensional semantic vectors corresponding to samples of the preset invisible damage category as conditions. Step S500: Construct a zero-shot classifier based on a semantic autoencoder, and train the zero-shot classifier based on the virtual samples; Step S600: Determine the dynamic threshold based on the zero-sample classifier, and perform real-time early warning monitoring of the cutting tool to be detected based on the dynamic threshold.

2. The method according to claim 1, characterized in that, Step S100 includes: Step S110: Generate a structured text description for each tool-damage category according to the type of tool to be inspected; Step S120: Extract word vectors of structured text descriptions corresponding to each tool-damage category using a pre-set deep bidirectional pre-trained language model; Step S130: Construct a tool-damage knowledge graph containing tool attributes, damage mode entities, and their interrelationships; Step S140: Extract the graph embedding vector of the damage mode node from the tool-damage knowledge graph through a preset graph convolutional network; Step S150: Concatenate the word vectors and the graph embedding vectors and perform principal component analysis for dimensionality reduction to generate high-dimensional semantic vectors.

3. The method according to claim 2, characterized in that, Step S150 includes: Step S151: Concatenate the word vector and the graph embedding vector corresponding to each tool-damage category in the first dimension to obtain the fusion vector corresponding to each tool-damage category; Step S152: Stack the fusion vectors corresponding to all tool-damage categories into a data matrix, and standardize each feature dimension of the data matrix to obtain a standardized data matrix; Step S153: Calculate the covariance matrix of the standardized data matrix, and perform eigenvalue decomposition on the covariance matrix to obtain the corresponding eigenvalues ​​and eigenvectors; Step S154: Sort the several feature values ​​in descending order, and determine the feature values ​​that are among the first preset target number of the sorted feature values ​​as target feature values; Step S155: Project the fused vector corresponding to each tool-damage category after standardization onto a subspace spanned by feature vectors corresponding to several target feature values ​​to obtain a high-dimensional semantic vector corresponding to each tool-damage category.

4. The method according to claim 3, characterized in that, Step S200 includes: Step S210: Fix the triaxial acceleration vibration sensor to the surface of the workpiece or worktable using a magnetic base, so that the three sensitive axes of the triaxial acceleration vibration sensor are aligned with the three axes of the coordinate system of the machine tool for processing the tool to be tested. Step S220: Install the multi-channel force sensor in the spindle taper hole of the machine tool; the multi-channel force sensor integrates a triaxial force sensing unit. Step S230: Fix the industrial camera to the observation window outside the processing cavity using a rigid bracket, so that the lens of the industrial camera is facing the tool to be inspected set on the spindle of the machine tool; Step S240: During the cutting process of the tool to be tested, the triaxial vibration signal collected by the triaxial acceleration vibration sensor and the triaxial force signal collected by the multi-channel force sensor are subjected to wavelet packet decomposition to obtain the sensing feature vector. Step S250: During the cutting process of the tool to be inspected, the image of the tool back face of the tool to be inspected acquired by the industrial camera is subjected to global average pooling feature extraction to obtain the image feature vector. Step S260: Normalize the sensing feature vector and the image feature vector by their maximum and minimum values ​​respectively, and then concatenate them to obtain a multimodal sensing feature vector.

5. The method according to claim 4, characterized in that, Step S300 includes: Step S310: Construct a three-layer semantic mapping network that uses a three-layer fully connected neural network as the mapping network; the dimension of the output layer of the three-layer semantic mapping network is the same as the dimension of the semantic space corresponding to the high-dimensional semantic vector; Step S320: Minimize the Euclidean distance between the output vector of the three-layer semantic mapping network and the high-dimensional semantic vector using the mean squared error loss function; Step S330: Train the three-layer semantic mapping network for several training cycles using samples of the visible damage category; Step S340: Based on the trained three-layer semantic mapping network, map the multimodal sensing feature vector to the semantic space corresponding to the high-dimensional semantic vector.

6. The method according to claim 5, characterized in that, Step S400 includes: Step S410: Construct a generative adversarial network; the generative adversarial network includes a generator and a discriminator; the generator includes four fully connected layers, and the discriminator includes three fully connected layers; Step S420: Train the generator and the discriminator according to the preset improved loss function until the generative adversarial network reaches Nash equilibrium. Step S430: Input the high-dimensional semantic vectors corresponding to the samples of the invisible damage category in the sample dataset as conditions into the trained generative adversarial network to generate corresponding virtual samples.

7. The method according to claim 6, characterized in that, Step S500 includes: Step S510: Train a semantic autoencoder on the high-dimensional semantic vectors corresponding to the visible damage category samples in the sample dataset; Step S520: Input the virtual sample into the three-layer semantic mapping network to obtain the corresponding mapping features; Step S530: Encode and decode the mapping features using the semantic autoencoder to obtain a reconstructed semantic vector; Step S540: Calculate the cosine similarity between the reconstructed semantic vector and the high-dimensional semantic vectors corresponding to all damaged categories, and take the damaged category with the largest cosine similarity as the prediction result to train the zero-shot classifier of the semantic autoencoder.

8. The method according to claim 7, characterized in that, Step S600 includes: Step S610: Input the multimodal sensing feature vector corresponding to the cutting tool during the cutting process into the zero-shot classifier to obtain the corresponding prediction confidence output by the zero-shot classifier; Step S620: If the prediction confidence level is lower than the preset dynamic threshold, it is determined that the tool to be detected is abnormal or has suffered unknown damage.

9. A non-transitory computer-readable storage medium, characterized in that, The storage medium stores at least one instruction or at least one program segment, which is loaded and executed by a processor to implement the method as described in any one of claims 1-8.

10. An electronic device, characterized in that, Includes a processor and the non-transitory computer-readable storage medium as described in claim 9.

Images