Turning machine tool processing monitoring system

Abstract

The present application relates to the field of machine tool processing monitoring, and specifically discloses a turning machine tool processing monitoring system, which uses artificial intelligence technology based on the field of deep learning to monitor the working video of the tool, the sound signal of the tool working, and the vibration signal of the tool working to determine whether the tool of the turning machine tool is excessively worn. In this way, by real-time monitoring of the working condition of the tool of the turning machine tool, the quality of the processed workpiece is guaranteed, the processing efficiency is improved, and accidents are prevented.

Description

Technical Field

[0001] The present application relates to the field of machine tool processing monitoring, and more specifically to a turning machine tool processing monitoring system. Background Art

[0002] Turning machine tools are a common metal processing equipment used to machine workpieces into the desired shape and size by rotating the workpiece and using cutting tools. Turning machine tools are mainly used to process cylindrical, conical, spherical and threaded workpieces. The basic working principle of turning machine tools is to fix the workpiece on the spindle, drive the workpiece to rotate through the rotation of the spindle, and use cutting tools (tools) to cut the workpiece. Cutting tools are usually made of carbide materials, and different types of tools can be selected as needed to achieve different turning operations. For the manufacturing industry, the quality of the workpiece is very important. With the development of science and technology, more and more industries require high-precision equipment to work, such as the medical field, which cannot be separated from the support of high-precision workpieces. When the turning machine tool is working, tool wear is one of the important factors affecting the quality of the workpiece. When the tool is worn excessively, the cutting force will increase, resulting in problems such as workpiece size deviation, surface quality degradation and even scratches.

[0003] Therefore, an optimized turning machine tool processing monitoring system is expected. Summary of the invention

[0004] In order to solve the above technical problems, this application is proposed. The embodiment of the application provides a turning machine tool processing monitoring system, which uses artificial intelligence technology based on the field of deep learning to monitor the video of the tool working, the sound signal of the tool working, and the vibration signal of the tool working to determine whether the tool of the turning machine tool is excessively worn. In this way, by real-time monitoring of the working condition of the tool of the turning machine tool, the quality of the processed workpiece is guaranteed, the processing efficiency is improved, and accidents are prevented.

[0005] According to one aspect of the present application, a turning machine tool processing monitoring system is provided, which includes: a tool working data acquisition module, which is used to acquire monitoring videos of the tool working in a predetermined time period, vibration signals of the tool working, and sound signals of the tool working;

[0006] a tool work data analysis module, used for processing the monitoring video of the tool working in the predetermined time period, the sound signal of the tool working and the vibration signal of the tool working, to obtain a tool work monitoring semantic association feature matrix, a tool sound feature matrix and a tool vibration feature matrix;

[0007] The tool wear condition analysis module is used to analyze the tool work monitoring semantic association feature matrix, the tool sound feature matrix and the tool vibration feature matrix to obtain the result of whether the tool of the turning machine tool is excessively worn.

[0008] In summary, the turning machine tool processing monitoring system provided by the present application uses artificial intelligence technology based on the field of deep learning to monitor the video of the tool working, the sound signal of the tool working, and the vibration signal of the tool working to determine whether the tool of the turning machine tool is excessively worn. In this way, by real-time monitoring of the working condition of the tool of the turning machine tool, the quality of the processed workpiece is guaranteed, the processing efficiency is improved, and accidents are prevented. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings required for use in the embodiments or the prior art description are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. In the drawings, the same reference numerals generally represent the same components or steps.

[0010] Figure 1 4 is a block diagram of a turning machine tool processing monitoring system according to an embodiment of the present application.

[0011] Figure 2 It is a block diagram of a tool working data analysis module in a turning machine tool processing monitoring system according to an embodiment of the present application.

[0012] Figure 3 It is a block diagram of a tool operation monitoring feature extraction unit in a turning machine tool processing monitoring system according to an embodiment of the present application.

[0013] Figure 4 It is a block diagram of a tool monitoring semantic association feature extraction subunit in a turning machine tool processing monitoring system according to an embodiment of the present application. DETAILED DESCRIPTION

[0014] Below, in order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0015] Figure 1 FIG. 1 is a block diagram of a lathe processing monitoring system according to an embodiment of the present application. Figure 1As shown, according to the turning machine tool processing monitoring system 100 of the embodiment of the present application, it includes: a tool working data acquisition module 110, which is used to obtain the monitoring video of the tool working in a predetermined time period, the vibration signal of the tool working, and the sound signal of the tool working; a tool working data analysis module 120, which is used to process the monitoring video of the tool working in the predetermined time period, the sound signal of the tool working, and the vibration signal of the tool working, and obtain the tool working monitoring semantic association feature matrix, the tool sound feature matrix, and the tool vibration feature matrix; and a tool wear condition analysis module 130, which is used to analyze the tool working monitoring semantic association feature matrix, the tool sound feature matrix, and the tool vibration feature matrix to obtain the result of whether the tool of the turning machine tool is excessively worn.

[0016] In the above-mentioned turning machine tool processing monitoring system 100, the tool working data acquisition module 110 is used to obtain the monitoring video of the tool working in a predetermined time period, the vibration signal of the tool working, and the sound signal of the tool working. As mentioned in the above-mentioned background technology, when the turning machine tool is working, tool wear is one of the important factors affecting the quality of the workpiece. When the tool wears excessively, the cutting force will increase, resulting in problems such as workpiece dimensional deviation, reduced surface quality, and even scratches. Therefore, an optimized turning machine tool processing monitoring system is expected.

[0017] In response to the above technical problems, a turning machine tool processing monitoring system is proposed, which uses artificial intelligence technology based on the field of deep learning to monitor the video of the tool working, the sound signal of the tool working, and the vibration signal of the tool working to determine whether the tool of the turning machine tool is excessively worn. In this way, by real-time monitoring of the working condition of the tool of the turning machine tool, the quality of the processed workpiece is guaranteed, the processing efficiency is improved, and accidents are prevented.

[0018] At present, deep learning and neural networks have been widely used in computer vision, natural language processing, speech signal processing and other fields. In addition, deep learning and neural networks have also shown a level close to or even beyond that of humans in image classification, object detection, semantic segmentation, text translation and other fields.

[0019] In recent years, the development of deep learning and neural networks has provided new solutions and solutions for turning machine tool processing monitoring systems.

[0020] Specifically, first, the monitoring video of the tool working in a predetermined time period, the vibration signal of the tool working, and the sound signal of the tool working are obtained. The monitoring video can provide intuitive visual information, including the working state of the tool, changes and abnormalities in the cutting process, etc. By analyzing the monitoring video, the wear of the tool, the working quality and the processing effect can be observed, and whether there are abnormal conditions, such as tool breakage, vibration during the cutting process, etc., can be detected. At the same time, the tool will vibrate when working, and the vibration signal can reflect the working state of the tool and the changes in the cutting process. By monitoring and analyzing the vibration signal of the tool, the degree of wear of the tool, the cutting quality, and whether there is abnormal vibration, such as excessive vibration amplitude, abnormal frequency, etc. can be judged. The tool also generates sound when working, and the sound signal can provide information about the working state of the tool and the cutting process. By analyzing the sound signal of the tool, the degree of wear of the tool, the working quality, and whether there is abnormal sound, such as abnormal noise, abnormal frequency, etc. can be judged. The comprehensive use of monitoring video, vibration signal and sound signal can obtain multi-dimensional information and have a more comprehensive understanding of the working condition of the tool. By analyzing and processing these signals, the degree of tool wear can be monitored in real time, and timely warnings and corresponding maintenance measures can be taken to ensure processing quality, improve efficiency, and prevent accidents. In the above-mentioned turning machine tool processing monitoring system 100, the tool work data analysis module 120 is used to process the monitoring video of the tool working in the predetermined time period, the sound signal of the tool working, and the vibration signal of the tool working, and obtain the tool work monitoring semantic association feature matrix, the tool sound feature matrix, and the tool vibration feature matrix. The monitoring video, sound signal, and vibration signal contain a large amount of raw data. By processing and analyzing these data, features related to the tool working state and wear can be extracted. These features can be image features in the video, spectral features of the sound signal, frequency domain or time domain features of the vibration signal, etc. By extracting features, complex monitoring data can be converted into vector representations in a high-dimensional feature space, which is convenient for subsequent analysis and processing.

[0021] Figure 2 FIG. 1 is a block diagram of a tool work data analysis module in a lathe machining monitoring system according to an embodiment of the present application. Figure 2 As shown, the tool work data analysis module 120 includes: a tool work monitoring feature extraction unit 121, which is used to analyze the monitoring video of the tool when it is working, and obtain a tool work monitoring semantic association feature matrix; a tool sound feature extraction unit 122, which is used to analyze the sound signal of the tool when it is working, and obtain a tool sound feature matrix; a tool vibration feature extraction unit 123, which is used to analyze the vibration signal of the tool when it is working, and obtain a tool vibration feature matrix.

[0022] Figure 3FIG. 1 is a block diagram of a tool operation monitoring feature extraction unit in a lathe machining monitoring system according to an embodiment of the present application. Figure 3 As shown, the tool work monitoring feature extraction unit 121 includes: a tool work monitoring key frame extraction subunit 1211, which is used to extract multiple tool work monitoring key frames from the monitoring video of the tool working at a predetermined sampling frequency; a tool work monitoring feature extraction subunit 1212, which is used to pass the multiple tool work monitoring key frames through a tool work monitoring feature extractor based on a convolutional neural network model to obtain multiple tool work monitoring feature vectors; and a tool monitoring semantic association feature extraction subunit 1213, which is used to pass the multiple tool work monitoring feature vectors through a tool monitoring feature extractor based on a converter to obtain a tool work monitoring semantic association feature matrix.

[0023] More specifically, in order to reduce the amount of data and extract key information, multiple tool operation monitoring key frames are extracted from the monitoring video of the tool operation. Considering that the monitoring video is usually a continuous image sequence, it contains a lot of redundant information. By extracting key frames, important information in the video sequence can be extracted, thereby reducing the amount of data and facilitating subsequent processing and analysis.

[0024] Further, in order to extract feature information from key frames, multiple tool work monitoring key frames are passed through a tool work monitoring feature extractor based on a convolutional neural network model to obtain multiple tool work monitoring feature vectors. The tool work monitoring key frames contain rich visual information, such as tool shape, changes in the cutting process, tool wear, etc. Through the feature extractor based on the convolutional neural network model, these key information can be automatically learned and extracted, and converted into a representative feature vector. At the same time, by extracting feature vectors, the dimension of the original image data can be reduced to a lower dimensional space. This can reduce the overhead of data storage and calculation, and help improve the efficiency and accuracy of subsequent algorithms. Specifically, in an embodiment of the present application, the tool work monitoring feature extraction subunit 1212 is used to: use each layer of the tool work monitoring feature extractor to perform convolution processing on the input data in the forward pass of the layer, mean pooling processing of the local feature matrix along the channel dimension, and nonlinear activation processing to output the multiple tool work monitoring feature vectors by the last layer of the tool work monitoring feature extractor, wherein the input of the tool work monitoring feature extractor is the multiple tool work monitoring key frames.

[0025] More specifically, in order to capture the semantic association and contextual information between tool work monitoring data, multiple tool work monitoring feature vectors are passed through a converter-based tool monitoring feature extractor to obtain a tool work monitoring semantic association feature matrix. The tool monitoring feature extractor is a converter-based context encoder model. Considering that the tool work monitoring feature vector contains information about the tool working state, and there may be certain semantic associations between different feature vectors. Through the converter-based feature extractor, the correlation between feature vectors can be learned, thereby capturing the semantic information of the tool work monitoring data. At the same time, tool work monitoring data is usually time series data, and there may be certain contextual relationships between data at adjacent time points. By extracting feature vectors and constructing a feature matrix, the features of adjacent time points can be integrated to obtain more comprehensive contextual information.

[0026] Figure 4 FIG. 1 is a block diagram of a tool monitoring semantic association feature extraction subunit in a lathe machining monitoring system according to an embodiment of the present application. Figure 4 As shown, the tool monitoring semantic association feature extraction subunit 1213 includes: a query vector construction secondary subunit 11, which is used to arrange the multiple tool work monitoring feature vectors into an input vector; a vector conversion secondary subunit 12, which is used to convert the input vector into a query vector and a key vector respectively through a learnable embedding matrix; a self-attention secondary subunit 13, which is used to calculate the product between the query vector and the transposed vector of the key vector to obtain a self-attention association matrix; a normalization secondary subunit 14, which is used to normalize the self-attention association matrix to obtain a standardized self-attention association matrix; an attention calculation secondary subunit 15, which is used to input the standardized self-attention association matrix into a Softmax activation function for activation to obtain a self-attention feature matrix; an attention application secondary subunit 16, which is used to multiply the self-attention feature matrix with each tool work monitoring feature vector in the multiple tool work monitoring feature vectors to obtain the multiple tool work monitoring semantic association feature vectors; and a matrixization secondary subunit 17, which is used to arrange the multiple tool work monitoring semantic association feature vectors in two dimensions to obtain a tool work monitoring semantic association feature matrix.

[0027] Specifically, in the embodiment of the present application, the tool sound feature extraction unit 122 includes: a tool sound preprocessing subunit, which is used to preprocess the sound signal of the tool when it is working to obtain the sound signal of the tool working after preprocessing and the vibration signal of the tool working after preprocessing; a tool sound frequency domain feature extraction subunit, which is used to perform Fourier transform on the sound signal of the tool working after preprocessing to obtain a plurality of tool sound frequency domain statistics; and a tool sound feature extraction subunit, which is used to pass the waveform diagram of the plurality of tool sound frequency domain statistics and the sound signal of the tool when it is working through a tool sound feature extractor based on the Clip model to obtain a tool sound feature matrix.

[0028] More specifically, in order to extract useful features and reduce the impact of noise, the vibration signal and sound signal of the tool are preprocessed to obtain the preprocessed tool working sound signal and vibration signal. The vibration signal and sound signal of the tool contain rich information, such as frequency, amplitude, energy, etc. Through preprocessing, useful features such as spectrum features, time domain features, etc. can be extracted for subsequent analysis and processing. The vibration signal and sound signal of the tool may be interfered by environmental noise, such as the sound of machine operation and the sound of the surrounding environment. Preprocessing can help remove or reduce the impact of these noises and improve the accuracy and reliability of the signal. At the same time, preprocessing can filter the signal, adjust the gain, and other operations to enhance the characteristics and information of the signal. This helps to improve the performance of subsequent algorithms and the accuracy of the results.

[0029] More specifically, the preprocessed tool working sound signal and vibration signal are Fourier transformed to obtain multiple tool sound frequency domain statistics and multiple tool vibration frequency domain statistics, which can analyze the characteristics and properties of the signal from the frequency domain perspective. Fourier transform can convert time domain signals into frequency domain representations and decompose signals into components of different frequencies. By Fourier transforming the tool working sound signal and vibration signal, frequency domain features such as spectrum shape and frequency distribution can be extracted to more comprehensively describe the frequency domain characteristics of the signal. Multiple statistical values ​​can be extracted from the Fourier transformed frequency domain signal to obtain a richer feature representation. These features can be used in applications such as tool wear monitoring, fault diagnosis and prediction. In addition, the most representative features can be selected or dimension reduction can be performed according to task requirements to reduce the dimension and computational complexity of the data.

[0030] More specifically, in order to convert the sound signal into a feature representation that can be used for machine learning and deep learning algorithms, multiple frequency domain statistics of tool sounds and waveforms of sound signals when the tool is working are passed through a tool sound feature extractor based on the Clip model to obtain a tool sound feature matrix. At the same time, the frequency domain statistics and waveforms of the sound signal usually have a high dimension. Through the tool sound feature extractor based on the Clip model, the high-dimensional sound signal can be converted into a low-dimensional feature vector or feature matrix, thereby reducing the dimension of the data and facilitating subsequent processing and analysis. The tool sound feature matrix can provide important feature information of the sound signal when the tool is working. These features may include spectral features, time domain features, energy features, etc. of the sound, which can reflect important information such as the status of the tool, working quality, and degree of wear.

[0031] Specifically, in the embodiment of the present application, specifically, in the embodiment of the present application, the tool sound feature extraction subunit includes: a frequency domain statistical feature encoding secondary subunit, which is used to pass the multiple tool sound frequency domain statistical values ​​through the sequence encoder of the tool sound feature extractor to obtain a tool sound frequency domain statistical feature vector; a tool sound signal waveform encoding secondary subunit, which is used to pass the waveform diagram of the sound signal of the tool when working through the image encoder of the tool sound feature extractor to obtain a tool sound waveform feature vector; and a tool sound feature optimization encoding secondary subunit, which is used to use the optimization encoder of the tool sound feature extractor to optimize the tool sound waveform feature vector based on the tool sound frequency domain statistical feature vector to obtain a tool sound feature matrix.

[0032] Specifically, in an embodiment of the present application, the frequency domain statistical feature encoding secondary subunit includes: a first-scale tool sound feature extraction tertiary subunit, which is used to pass the tool sound frequency domain feature statistics through the first convolution layer of the sequence encoder of the trained tool sound feature extractor to obtain a first neighborhood scale tool sound frequency domain statistical feature vector, wherein the first convolution layer has a first one-dimensional convolution kernel of a first length; a second-scale current feature extraction tertiary subunit, which is used to pass the tool sound frequency domain feature statistics through the second convolution layer of the sequence encoder of the trained tool sound feature extractor to obtain a second neighborhood scale tool sound frequency domain statistical feature vector, wherein the second convolution layer has a second one-dimensional convolution kernel of a second length, and the first length is different from the second length; and a multi-scale cascade secondary subunit, which is used to cascade the first neighborhood scale tool sound frequency domain statistical feature vector and the second neighborhood scale tool sound frequency domain statistical feature vector to obtain the tool sound frequency domain statistical feature vector.

[0033] Specifically, in the embodiment of the present application, the first-scale tool sound feature extraction three-level subunit is used to: use the first convolution layer of the sequence encoder of the trained tool sound feature extractor to perform one-dimensional convolution encoding on the tool sound audio frequency feature statistics using the following first convolution encoding formula to obtain a first neighborhood scale tool sound audio frequency statistical feature vector; wherein the first convolution encoding formula is:

[0034]

[0035] Among them, a is the width of the first convolution kernel in the x direction, F(a) is the first convolution kernel parameter vector, G(xa) is the local vector matrix operated with the convolution kernel function, w is the size of the first convolution kernel, and X represents the audio frequency feature statistical value of the tool sound.

[0036] Specifically, in the embodiment of the present application, the second-scale tool sound feature extraction three-level subunit is used to: use the second convolution layer of the sequence encoder of the trained tool sound feature extractor to perform one-dimensional convolution encoding on the tool sound audio frequency feature statistics using the following second convolution encoding formula to obtain the second neighborhood scale tool sound audio frequency statistical feature vector; wherein the second convolution encoding formula is:

[0037]

[0038] Wherein, b is the width of the second convolution kernel in the x direction, F(b) is the second convolution kernel parameter vector, G(xb) is the local vector matrix operated with the convolution kernel function, m is the size of the second convolution kernel, and X represents the audio frequency feature statistical value of the tool sound.

[0039] Specifically, in an embodiment of the present application, the secondary subunit for extracting the tool sound waveform feature is further used to: use each layer of the image encoder of the tool sound feature extractor to respectively perform the following on the input data in the forward transfer of the layer: convolution processing on the input data to obtain a convolution feature map; pooling based on a feature matrix on the convolution feature map to obtain a pooled feature map; and nonlinear activation on the pooled feature map to obtain an activation feature map; wherein the output of the last layer of the image encoder of the tool sound feature extractor is the tool sound waveform feature vector, and the input of the first layer of the image encoder of the tool sound feature extractor is the waveform of the sound signal when the tool is working. Specifically, in an embodiment of the present application, the secondary subunit for joint current encoding is further used to: use the joint encoder of the tool sound feature extractor to optimize the feature encoding of the tool sound waveform feature vector based on the tool sound frequency domain statistical feature vector using the following joint encoding formula to obtain a tool sound feature matrix; wherein the joint encoding formula is:

[0040]

[0041] Among them, V s represents the frequency domain statistical feature vector of the tool sound, represents the transposed vector of the tool sound frequency domain statistical feature vector, V b represents the tool sound waveform feature vector, M b represents the tool sound feature matrix, Represents vector multiplication.

[0042] More specifically, in order to convert the vibration signal into a feature representation that can be used for machine learning and deep learning algorithms, multiple tool vibration frequency domain statistics and waveforms of the vibration signal when the tool is working are passed through a tool vibration feature extractor based on the Clip model to obtain a tool vibration feature matrix. Specifically, the processing method of the multiple tool vibration frequency domain statistics and waveforms of the vibration signal when the tool is working can refer to the processing steps of the multiple tool sound frequency domain statistics and waveforms of the sound signal when the tool is working.

[0043] Specifically, in the embodiment of the present application, the tool vibration feature extraction unit 123 includes: a tool vibration preprocessing subunit, which is used to preprocess the vibration signal of the tool when it is working to obtain the vibration signal of the tool after preprocessing; a tool vibration frequency domain feature extraction subunit, which is used to perform Fourier transform on the vibration signal of the tool after preprocessing to obtain a plurality of tool vibration frequency domain statistics; and a tool vibration feature extraction subunit, which is used to pass the waveform diagram of the plurality of tool vibration frequency domain statistics and the vibration signal of the tool when it is working through a tool vibration feature extractor based on the Clip model to obtain a tool vibration feature matrix.

[0044] In the above-mentioned turning machine tool processing monitoring system 100, the tool wear condition analysis module 130 is used to analyze the tool work monitoring semantic association feature matrix, the tool sound feature matrix and the tool vibration feature matrix to obtain the result of whether the tool of the turning machine tool is excessively worn. By analyzing the feature matrix, features related to tool wear can be extracted. These features may include position offset when the tool is working, changes in vibration spectrum, changes in sound features, etc. By comparing these features with the feature differences under normal working conditions, it can be determined whether the tool is excessively worn.

[0045] Specifically, in the embodiment of the present application, the tool wear condition analysis module 130 includes: a tool feature fusion unit, which is used to fuse the tool work monitoring semantic association feature matrix, the tool sound feature matrix and the tool vibration feature matrix to obtain a tool feature fusion matrix; an optimization unit, which is used to perform probability density domain related migration hyperconvex projection measurement on the tool feature fusion matrix to obtain an optimized tool feature fusion matrix; and a tool wear degree analysis unit, which is used to pass the optimized tool feature fusion matrix through a classifier to obtain a classification result, and the classification result is used to indicate whether the tool of the turning machine tool is excessively worn.

[0046] More specifically, the tool work monitoring semantic association feature matrix, the tool sound feature matrix and the tool vibration feature matrix are fused to obtain a tool feature fusion matrix. The working state and performance of a tool are usually determined by multiple aspects of information. The tool work monitoring semantic association feature matrix can provide semantic information related to the tool state, the tool sound feature matrix can provide a feature representation of the sound signal, and the tool vibration feature matrix can provide a feature representation of the vibration signal. By fusing these different types of feature matrices, multiple aspects of information can be comprehensively considered to provide a more comprehensive and accurate tool feature description.

[0047] Specifically, in the embodiment of the present application, the tool feature fusion unit is used to fuse the tool work monitoring semantic association feature matrix, the tool sound feature matrix and the tool vibration feature matrix using the following three fusion formulas to obtain the tool feature fusion matrix; wherein the three fusion formulas are:

[0048]

[0049] Among them, M1 represents the tool work monitoring semantic association feature matrix, M2 represents the tool sound feature matrix, M3 represents the tool vibration feature matrix, M represents the tool feature fusion matrix, α1, α2 and α3 represent the weighting parameters of the tool work monitoring semantic association feature matrix, the tool sound feature matrix and the tool vibration feature matrix respectively. Indicates sum by position.

[0050] In particular, in the technical solution of the present application, considering that the tool work monitoring semantic association feature matrix is ​​extracted from the monitoring video of the tool work, a plurality of tool work monitoring feature vectors are obtained by a tool work monitoring feature extractor based on a convolutional neural network model, and a tool work monitoring semantic association feature matrix is ​​obtained by a tool monitoring feature extractor based on a converter. This feature matrix may be based on the time series data of the video frame, each feature vector corresponds to the feature representation of a video frame, and the dimension may be consistent with the feature representation dimension of the video frame. The tool sound feature matrix and the tool vibration feature matrix are extracted from the sound signal and vibration signal when the tool is working. These signals may be continuous time series data collected by sensors, and multiple sound frequency domain statistics and vibration frequency domain statistics are obtained after preprocessing and Fourier transformation. These statistics may have different dimensions because the sound signal and the vibration signal may have different feature representation methods. When the tool work monitoring semantic association feature matrix, the tool sound feature matrix and the tool vibration feature matrix are fused, they will have dimensional misalignment. If the correct dimension alignment operation is not performed, the fused tool feature fusion matrix may have out-of-distribution eigenvalues, that is, the fusion result contains mismatched features from different feature matrices, which may affect the final classification result. In order to solve this problem, the tool feature fusion matrix is ​​measured by probability density domain related migration hyperconvex projection.

[0051] Specifically, in an embodiment of the present application, the optimization unit is used to: perform probability density domain related migration hyperconvex projection measurement on the tool feature fusion matrix according to the following optimization formula to obtain an optimized tool feature fusion matrix; wherein the optimization formula is:

[0052]

[0053] Wherein, M represents the tool feature fusion matrix, m i，j represents the eigenvalue of the position (i, j) in the tool feature fusion matrix, softmax represents the normalized exponential function, log represents the logarithmic function value with base 2, β represents the hyperparameter, and m i，j ' represents the eigenvalue of the (i, j) position in the tool feature fusion matrix.

[0054] That is, in the process of fusing the tool work monitoring semantic association feature matrix, the tool sound feature matrix and the tool vibration feature matrix, there is a dimensional misalignment between the tool work monitoring semantic association feature matrix, the tool sound feature matrix and the tool vibration feature matrix, resulting in the existence of out-of-distribution eigenvalues ​​in the fused tool feature fusion matrix.

[0055] In view of the above technical problems, in the technical solution of the present application, the tool feature fusion matrix is ​​measured by probability density domain related migration hyperconvex projection, which maps the tool feature fusion matrix from the original space to a hyperconvex space through the probability density domain related migration hyperconvex projection measurement function, so that the feature vectors of different data densities have the same dimension and distribution in the hyperconvex space. The hyperconvex space mapping can eliminate the dimensional misalignment and distribution inconsistency between the global feature vector of the tourist attraction and the feature vector associated with the point cloud coordinates, so that the fused tool feature fusion matrix does not have out-of-distribution eigenvalues, but has a more compact and robust feature representation. That is, the tool feature fusion matrix is ​​synthesized with boundary constraints using the probability density domain related migration hyperconvex projection measurement function, which can avoid excessive fragmentation of the decision area in the classification target domain due to the out-of-distribution samples of the feature value set, thereby improving the classification performance of the tool feature fusion matrix.

[0056] More specifically, in order to determine whether the tool of a turning machine tool is excessively worn, the optimized tool feature fusion matrix is ​​passed through a classifier to obtain a classification result. Inputting the optimized tool feature fusion matrix into the classifier can achieve automated judgment and early warning. Traditional tool wear monitoring usually relies on manual experience and observation, while automated classifiers can monitor and judge tools in real time or offline. This can save human resources and improve the efficiency and accuracy of tool wear monitoring.

[0057] Specifically, in the embodiment of the present application, the tool wear degree analysis unit is used to: use the classifier to process the optimized tool feature fusion matrix with the following classification formula to obtain the classification result; wherein the classification formula is:

[0058] =softmax{(W c , B c )|Project(F)}

[0059] Among them, Project(F) represents the projection of the optimized tool feature fusion matrix into a vector, W e is the weight matrix, B c represents a bias vector, softmax represents a normalized exponential function, and o represents the classification result.

[0060] In summary, the lathe machining monitoring system according to the embodiment of the present application has been explained, which uses artificial intelligence technology based on the field of deep learning to monitor the video of the tool working, the sound signal of the tool working, and the vibration signal of the tool working to determine whether the tool of the lathe is excessively worn. In this way, by real-time monitoring of the working condition of the tool of the lathe, the quality of the machined workpiece is guaranteed, the machining efficiency is improved, and accidents are prevented.

[0061] As described above, the lathe machining monitoring system 100 according to the embodiment of the present application can be implemented in various terminal devices, such as a lathe machining monitoring server, etc. In one example, the lathe machining monitoring system 100 according to the embodiment of the present application can be integrated into the terminal device as a software module and / or a hardware module. For example, the lathe machining monitoring system 100 can be a software module in the operating system of the terminal device, or can be an application developed for the terminal device; of course, the lathe machining monitoring system 100 can also be one of the many hardware modules of the terminal device.

[0062] Alternatively, in another example, the turning machine tool processing monitoring system 100 and the terminal device may also be separate devices, and the turning machine tool processing monitoring system 100 may be connected to the terminal device via a wired and / or wireless network and transmit interactive information in accordance with an agreed data format.

[0063] The "first", "second", "third", "fourth", etc. (if any) referred to in this application are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods or devices.

[0064] It should be noted that the descriptions involving "first", "second", etc. in this application are only for descriptive purposes and cannot be understood as indicating or implying their relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In addition, the technical solutions between the various embodiments can be combined with each other, but they must be based on the ability of ordinary technicians in this field to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such combination of technical solutions does not exist and is not within the scope of protection required by this application.

[0065] Specific examples are used herein to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea. At the same time, for those skilled in the art, according to the idea of ​​the present application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

[0066] The block diagrams of the devices, apparatuses, equipment, and systems involved in this application are only illustrative examples and are not intended to require or imply that they must be connected, arranged, and configured in the manner shown in the block diagram. As will be appreciated by those skilled in the art, these devices, apparatuses, equipment, and systems can be connected, arranged, and configured in any manner. Words such as "including", "comprising", "having", etc. are open words, referring to "including but not limited to", and can be used interchangeably with them. The words "or" and "and" used here refer to the words "and / or" and can be used interchangeably with them, unless the context clearly indicates otherwise. The words "such as" used here refer to the phrase "such as but not limited to", and can be used interchangeably with them.

[0067] It should also be noted that in the apparatus, device and method of the present application, each component or each step can be decomposed and / or recombined. Such decomposition and / or recombination should be regarded as equivalent solutions of the present application.

[0068] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use the present application. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects without departing from the scope of the present application. Therefore, the present application is not intended to be limited to the aspects shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

[0069] The above description has been given for the purpose of illustration and description. In addition, this description is not intended to limit the embodiments of the present application to the forms disclosed herein. Although multiple example aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, changes, additions and sub-combinations thereof.

Claims

1. A turning machine tool processing monitoring system, characterized in that: include: A tool working data acquisition module is used to acquire monitoring video of the tool working in a predetermined time period, a vibration signal of the tool working, and a sound signal of the tool working; a tool work data analysis module, used for processing the monitoring video of the tool working in the predetermined time period, the sound signal of the tool working and the vibration signal of the tool working, to obtain a tool work monitoring semantic association feature matrix, a tool sound feature matrix and a tool vibration feature matrix; A tool wear condition analysis module, used for analyzing the tool work monitoring semantic association feature matrix, the tool sound feature matrix and the tool vibration feature matrix, to obtain a result of whether the tool of the turning machine tool is excessively worn; Wherein, the tool wear analysis module includes: A tool feature fusion unit, used for fusing the tool work monitoring semantic association feature matrix, the tool sound feature matrix and the tool vibration feature matrix to obtain a tool feature fusion matrix; An optimization unit, used for performing probability density domain related migration hyperconvex projection measurement on the tool feature fusion matrix to obtain an optimized tool feature fusion matrix; a tool wear degree analysis unit, used for passing the optimized tool feature fusion matrix through a classifier to obtain a classification result, wherein the classification result is used to indicate whether the tool of the turning machine tool is excessively worn; Wherein, the optimization unit is used for: The tool feature fusion matrix is ​​measured by probability density domain related migration hyperconvex projection using the following optimization formula to obtain an optimized tool feature fusion matrix; Wherein, the optimization formula is: Wherein, M represents the tool feature fusion matrix, m i,j represents the eigenvalue of the position (i, j) in the tool feature fusion matrix, softmax represents the normalized exponential function, log represents the logarithmic function value with base 2, β represents the hyperparameter, and m i,j ′ represents the eigenvalue of the (i, j) position in the tool feature fusion matrix.

2. The turning machine tool processing monitoring system according to claim 1, characterized in that: The tool working data analysis module comprises: A tool operation monitoring feature extraction unit is used to analyze the monitoring video of the tool when it is working to obtain a tool operation monitoring semantic association feature matrix; A tool sound feature extraction unit, used for analyzing the sound signal of the tool when it is working, to obtain a tool sound feature matrix; The tool vibration feature extraction unit is used to analyze the vibration signal of the tool when it is working to obtain the tool vibration feature matrix.

3. The turning machine tool processing monitoring system according to claim 2, characterized in that: The tool operation monitoring feature extraction unit comprises: A tool operation monitoring key frame extraction subunit is used to extract a plurality of tool operation monitoring key frames from the monitoring video of the tool operation at a predetermined sampling frequency; A tool work monitoring feature extraction subunit, used for passing the plurality of tool work monitoring key frames through a tool work monitoring feature extractor based on a convolutional neural network model to obtain a plurality of tool work monitoring feature vectors; The tool monitoring semantic association feature extraction subunit is used to obtain a tool working monitoring semantic association feature matrix by passing the plurality of tool working monitoring feature vectors through a converter-based tool monitoring feature extractor.

4. The turning machine tool processing monitoring system according to claim 3, characterized in that: The tool monitoring semantic association feature extraction subunit includes: A query vector construction secondary subunit is used to arrange the plurality of tool operation monitoring feature vectors into an input vector; A vector conversion secondary subunit, used to convert the input vector into a query vector and a key vector respectively through a learnable embedding matrix; A self-attention secondary subunit, configured to calculate the product between the query vector and the transposed vector of the key vector to obtain a self-attention association matrix; A normalization secondary subunit, used for normalizing the self-attention association matrix to obtain a normalized self-attention association matrix; The attention calculation secondary subunit is used to input the standardized self-attention association matrix into the Softmax activation function for activation to obtain a self-attention feature matrix; an attention applying secondary subunit, configured to respectively multiply the self-attention feature matrix by each of the plurality of tool work monitoring feature vectors to obtain the plurality of tool work monitoring semantic association feature vectors; The matrixed secondary sub-unit is used to arrange the plurality of tool operation monitoring semantic association feature vectors in two dimensions to obtain a tool operation monitoring semantic association feature matrix.

5. The turning machine tool processing monitoring system according to claim 4, characterized in that: The tool sound feature extraction unit comprises: A tool sound preprocessing subunit, used for preprocessing the sound signal of the tool when it is working to obtain a sound signal of the tool working after preprocessing and a vibration signal of the tool working after preprocessing; A tool sound frequency domain feature extraction subunit is used to perform Fourier transform on the sound signal of the tool operation after the preprocessing to obtain a plurality of tool sound frequency domain statistical values; The tool sound feature extraction subunit is used to obtain a tool sound feature matrix by passing the plurality of tool sound frequency domain statistical values ​​and the waveform diagram of the sound signal when the tool is working through a tool sound feature extractor based on the Clip model.

6. The turning machine tool processing monitoring system according to claim 5, characterized in that: The tool vibration feature extraction unit comprises: A tool vibration preprocessing subunit, used for preprocessing the vibration signal of the tool when it is working to obtain a preprocessed tool working vibration signal; A tool vibration frequency domain feature extraction subunit is used to perform Fourier transform on the pre-processed tool working vibration signal to obtain a plurality of tool vibration frequency domain statistical values; The tool vibration feature extraction subunit is used to obtain a tool vibration feature matrix by passing the plurality of tool vibration frequency domain statistical values ​​and the waveform diagram of the vibration signal of the tool when it is working through a tool vibration feature extractor based on the Clip model.

7. The turning machine tool processing monitoring system according to claim 6, characterized in that: The tool feature fusion unit is used for: The tool work monitoring semantic association feature matrix, the tool sound feature matrix and the tool vibration feature matrix are fused using the following three fusion formulas to obtain the tool feature fusion matrix; Among them, the three fusion formulas are: Among them, M1 represents the tool work monitoring semantic association feature matrix, M2 represents the tool sound feature matrix, M3 represents the tool vibration feature matrix, M represents the tool feature fusion matrix, a1, a2 and a3 represent the weighting parameters of the tool work monitoring semantic association feature matrix, the tool sound feature matrix and the tool vibration feature matrix respectively, Indicates sum by position.

8. The turning machine tool processing monitoring system according to claim 7, characterized in that: The tool wear degree analysis unit is used for: Using the classifier to process the optimized tool feature fusion matrix with the following classification formula to obtain the classification result; Among them, the classification formula is: O=softmax{(W c ,B c )|Project(F)} Among them, Project(F) represents the projection of the optimized tool feature fusion matrix into a vector, W c is the weight matrix, B c represents a bias vector, softmax represents a normalized exponential function, and O represents the classification result.

Images