Method for determining the state of a numerically controlled device

Abstract

The application provides a method for determining the machining state of a numerical control equipment, which can be applied to the technical field of intelligent manufacturing. The method comprises the following steps: acquiring target multi-modal data collected in real time in the process of machining a metal material by using the numerical control equipment; generating a visual feature vector based on morphological data, motion trajectory data and temperature data, and generating a signal feature vector based on vibration data and concentration data; fusing the visual feature vector and the signal feature vector to obtain a fused feature, wherein the fusion weights of the visual feature vector and the signal feature vector are determined based on the importance of multi-modal features to the machining state; and determining the machining state of the numerical control equipment based on the fused feature. The application also provides a device for determining the machining state of a numerical control equipment, an equipment, a storage medium and a program product.

Description

Technical Field

[0001] This application relates to the field of intelligent manufacturing technology, and more specifically to a method for determining the processing status of CNC equipment. Background Technology

[0002] Related CNC machining status monitoring technologies are mainly achieved through visual inspection or indirect signal analysis. For example, by using infrared thermal imaging and improved target detection algorithms, extreme anomalies such as metal chip entanglement can be identified, or the tool status can be indirectly inferred through spindle vibration signals and current data.

[0003] However, the above methods have at least the following problems: they can only achieve single abnormal alarms or local state monitoring; their functional coverage is limited, focusing mainly on static detection of extreme abnormalities such as metal chip entanglement, making it difficult to cover the entire life cycle of machining states such as normal cutting, slight wear, and chatter, and they only focus on abnormal results rather than the cutting process; the monitoring method is indirect, mainly relying on non-core signals such as vibration and current, which are easily affected by environmental noise, machine tool idling, and other factors, resulting in a high false alarm rate and a lack of physical interpretability; the early warning timing is delayed, with alarms triggered mostly after the abnormality occurs, making it difficult to predict and intervene in advance through early minute changes in the dynamic characteristics of metal chips, thus making it difficult to achieve real-time loss prevention. Summary of the Invention

[0004] In view of the above problems, this application provides a method, apparatus, equipment, storage medium and program product for determining the machining state of CNC equipment.

[0005] According to the first aspect of this application, a method for determining the machining state of a CNC machine is provided, comprising: acquiring real-time target multimodal data during the CNC machining of metal materials using the CNC machine, the target multimodal data including: morphological data and motion trajectory data corresponding to metal chips generated during the CNC machining process, temperature data of the cutting area, vibration data of the CNC machine, and concentration data of metal chips in the coolant; generating a visual feature vector based on the morphological data, motion trajectory data, and temperature data, and generating a signal feature vector based on the vibration data and concentration data; fusing the visual feature vector and the signal feature vector to obtain a fused feature, wherein the fusion weights of the visual feature vector and the signal feature vector are determined based on the importance of the multimodal features to the machining state; and determining the machining state of the CNC machine based on the fused feature.

[0006] According to embodiments of this application, acquiring real-time target multimodal data includes: acquiring raw multimodal data; performing image preprocessing and signal preprocessing on the raw multimodal data to obtain initial multimodal data; determining the metal chip profile based on the initial multimodal data, and determining the total perimeter, total area, aspect ratio, curling helix degree, and fracture rate of the metal chip based on the metal chip profile to obtain morphological data; and determining the average ejection velocity and trajectory divergence of the metal chip based on the initial multimodal data to obtain trajectory data.

[0007] According to an embodiment of this application, image preprocessing includes: performing noise reduction processing on the original image in the original multimodal data using a nonlocal mean filtering algorithm; performing brightness correction and temperature calibration on the infrared image in the original multimodal data to determine the temperature data corresponding to the image grayscale value; matching the image template for the predetermined area of ​​the tool with the original image to determine the real-time coordinates of the predetermined area of ​​the tool; and determining the region of interest and background region in the original image based on the real-time coordinates.

[0008] According to an embodiment of this application, signal preprocessing includes: filtering the original vibration data in the original multimodal data to obtain filtered vibration data; eliminating the trend term bias of the filtered vibration data using the least squares method and performing amplitude standardization to obtain target vibration data; smoothing the original concentration data in the original multimodal data to obtain processed concentration data; and calculating the metal chip concentration change rate and contamination level growth rate based on the processed concentration data, wherein the contamination level growth rate characterizes the rate of increase of the metal impurity contamination level in the coolant.

[0009] According to an embodiment of this application, the raw multimodal data is obtained by: acquiring multiple sets of sensor data collected by various sensors; and performing timestamp alignment processing on the multiple sets of sensor data to obtain the raw multimodal data.

[0010] According to an embodiment of this application, the method further includes: determining a target warning level corresponding to the machining state of the CNC equipment; and determining a target warning strategy corresponding to the target warning level based on a predetermined correspondence between multiple warning levels and multiple warning strategies.

[0011] The second aspect of this application provides a device for determining the machining state of a CNC machine, comprising: an acquisition module for acquiring real-time target multimodal data during the CNC machining of metal materials using the CNC machine, the target multimodal data including: morphological data and motion trajectory data corresponding to metal chips generated during the CNC machining process, temperature data of the cutting area, vibration data of the CNC machine, and concentration data of the metal chips in the coolant; a generation module for generating a visual feature vector based on the morphological data, motion trajectory data, and temperature data, and generating a signal feature vector based on the vibration data and concentration data; a fusion module for fusing the visual feature vector and the signal feature vector to obtain fused features, wherein the fusion weights of the visual feature vector and the signal feature vector are determined based on the importance of the multimodal features to the machining state; and a determination module for determining the machining state of the CNC machine based on the fused features.

[0012] A third aspect of this application provides an electronic device comprising: one or more processors; and a memory for storing one or more computer programs, wherein the one or more processors execute the one or more computer programs to implement the steps of the method described above.

[0013] A fourth aspect of this application also provides a computer-readable storage medium having a computer program or instructions stored thereon, which, when executed by a processor, implement the steps of the above-described method.

[0014] The fifth aspect of this application also provides a computer program product, including a computer program or instructions that, when executed by a processor, implement the steps of the above-described method. Attached Figure Description

[0015] The above-mentioned contents, other objects, features and advantages of this application will become clearer from the following description of embodiments with reference to the accompanying drawings, in which:

[0016] Figure 1 A flowchart illustrating a method for determining the machining state of a CNC machine according to an embodiment of this application is shown schematically.

[0017] Figure 2 The illustration shows a schematic diagram of a method for determining the machining state of a CNC machine according to an embodiment of this application;

[0018] Figure 3 This schematic diagram illustrates the structural block diagram of a CNC equipment machining state determination device according to an embodiment of this application;

[0019] Figure 4 A block diagram schematically illustrates an electronic device suitable for implementing a method for determining the machining state of a CNC machine according to an embodiment of this application. Detailed Implementation

[0020] The embodiments of this application will now be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of this application. In the following detailed description, numerous specific details are set forth to provide a thorough understanding of the embodiments of this application for ease of explanation. However, it will be apparent that one or more embodiments may be implemented without these specific details. Furthermore, descriptions of well-known structures and technologies are omitted in the following description to avoid unnecessarily obscuring the concepts of this application.

[0021] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of this application. The terms “comprising,” “including,” etc., as used herein indicate the presence of features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0022] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein are to be interpreted in a manner consistent with the context of this specification, and not in an idealized or overly rigid way.

[0023] When using expressions such as "at least one of A, B and C", they should generally be interpreted in accordance with the meaning that is commonly understood by those skilled in the art (e.g., "a system having at least one of A, B and C" should include, but is not limited to, a system having A alone, a system having B alone, a system having C alone, a system having A and B, a system having A and C, a system having B and C, and / or a system having A, B and C, etc.).

[0024] Relevant CNC machining status monitoring methods include, for example, the following:

[0025] Infrared thermal imaging technology is used to identify whether metal chips are entangled in the machining area. However, this method only extracts the morphology, temperature characteristics, and area of ​​the easily entangled region of the metal chips to achieve binary detection of the single anomaly of metal chip entanglement. It can only determine whether the metal chips are "entangled" and cannot identify various CNC machining states such as normal cutting, tool wear, chatter, and pseudo-cutting. Moreover, it can only issue an alarm after the metal chips have accumulated and formed entanglement, which has obvious monitoring lag. It can only solve the alarm problem of the single machining anomaly of metal chip entanglement.

[0026] This paper improves the target detection accuracy of metal chip entanglement by refining deep learning algorithms and designs a telescopic vision inspection mechanism to solve the interference and contamination problems in image acquisition inside machine tools. However, this method only recognizes and detects metal chip entanglement as a static target, extracting only the static visual features of the metal chips to determine whether entanglement exists, ignoring the acquisition and analysis of dynamic physical features such as velocity vectors and morphological evolution sequences during the metal chip cutting process. Furthermore, this scheme only detects the extreme machining anomaly of metal chip entanglement and cannot determine the entire lifecycle state of CNC machining; its improved algorithms mostly belong to complex deep learning models, requiring high computing power from edge computing devices, and can only detect the result of metal chip entanglement, unable to explain the specific physical causes behind the abnormal metal chip morphology, such as tool wear and material hardness changes.

[0027] This method indirectly infers tool wear and breakage status by using spindle vibration signals, identifying monitoring indicators that are sensitive to tool condition and less affected by operating parameters. However, this method relies solely on the indirect signal of spindle vibration and does not directly collect the relevant characteristics of the core physical product of the cutting process, metal chips. This results in insufficient monitoring sensitivity under light cutting or micro-feed conditions. Furthermore, the vibration signal is easily interfered with by non-cutting factors such as machine tool idling, guideway friction, and material hardness fluctuations, making it difficult to accurately distinguish between scenarios such as "effective cutting" and "idling vibration," creating a blind spot in status determination. In addition, this method relies on mathematical processing methods such as power spectrum analysis to extract abstract features, lacking physical intuition. It can only determine whether the tool is worn or broken, but cannot further distinguish the causes of different types of cutting anomalies such as tool wear and fixture loosening, and can only make indirect inferences.

[0028] In view of this, embodiments of this application provide a method for determining the machining state of CNC equipment. Figure 1 A flowchart illustrating a method for determining the machining state of a CNC machine according to an embodiment of this application is shown. Figure 2 The diagram illustrates a method for determining the machining state of a CNC machine according to an embodiment of this application.

[0029] like Figure 1 As shown, the method for determining the machining state of CNC equipment in this embodiment includes operations S110 to S140.

[0030] During the operation of S110, when CNC machining of metal materials is performed using CNC equipment, real-time target multimodal data is acquired. The target multimodal data includes: morphological data and motion trajectory data corresponding to the metal chips generated during the CNC machining process, temperature data of the cutting area, vibration data of the CNC equipment, and concentration data of the metal chips in the coolant.

[0031] In operation S120, visual feature vectors are generated based on morphological data, motion trajectory data, and temperature data, and signal feature vectors are generated based on vibration data and concentration data.

[0032] In operation S130, the visual feature vector and the signal feature vector are fused to obtain fused features. The fusion weights of the visual feature vector and the signal feature vector are determined based on the importance of the multimodal features to the processing state.

[0033] In operation S140, the machining status of the CNC equipment is determined based on the fusion features.

[0034] For example, the CNC machining process may include a machining flow in which a CNC machine tool, according to a preset program, cuts a metal blank workpiece at high speed by rotating a cutting tool. For instance, the spindle drives the cutting tool to rotate, and the cutting tool squeezes and scrapes the surface of the metal workpiece to remove excess metal and produce a part of a specified size. Metal chips may include fragments of metal that are cut off by the cutting tool, such as iron filings. Metal chips can reflect the machining state. For example, when the machining state is a normal and stable cutting state, the metal is cut smoothly, producing continuous, curled, long, and complete metal chips. The metal chips are uniform in size, with concentrated splashing and a convergent trajectory. The cutting temperature is stable, and the metal chips accumulate slowly in the coolant with small vibration amplitude. When the machining state is abnormal due to tool or equipment failure, the metal chips may become finer and more fragmented, with changes in curl, resulting in fragmentation. The metal chips may also splash and scatter, the cutting temperature may rise, and the concentration of metal chips in the coolant may increase.

[0035] like Figure 2 As shown in the data acquisition layer, various types of sensors can be pre-deployed, such as high-speed industrial cameras, infrared thermal imagers, triaxial vibration sensors, and online iron wear sensors.

[0036] Multimodal data can be acquired through various types of sensors. For example, explosion-proof high-speed industrial cameras and infrared thermal imagers can be used as vision units; high-frequency triaxial vibration sensors and online iron wear sensors can be used as signal units; an industrial control computer equipped with a high-performance GPU can be configured as an edge computing gateway; the vision unit can be aimed at the cutting area of ​​the tool tip at a 45° downward angle to ensure coverage of the entire iron chip trajectory; the vibration sensor is magnetically fixed to the end face of the spindle bearing seat; the iron wear sensor is connected in series in the coolant return pipeline; the vision data can be transmitted via industrial Ethernet, and the vibration and concentration signals are accessed via industrial bus to ensure no data loss and low latency.

[0037] For example, morphological data may include geometric parameters such as the shape and size of the iron chip, curling, and degree of breakage; motion trajectory data may include the speed and dispersion of the iron chip splash; temperature data may include the cutting area of ​​the tool tip and the real-time temperature of the iron chip; vibration data may include the vibration value of the machine tool spindle and during cutting; and concentration data may include the amount of iron chip accumulated in the coolant and the concentration change index.

[0038] Morphological data, motion trajectory data, and temperature data can be concatenated in a fixed order. For example, morphological data can be placed first, motion trajectory data second, and temperature data last, forming a one-dimensional numerical sequence to obtain a visual feature vector. Vibration data and concentration data can also be concatenated in a fixed order, such as vibration data first and concentration data last, to obtain a signal feature vector.

[0039] For example, an improved dual-branch fusion neural network architecture can be designed to construct a multimodal fusion decision model. The visual branch in the aforementioned neural network architecture can be improved based on a classic convolutional neural network by introducing deformable convolutional layers to enhance the feature extraction capability for irregular chip shapes and output high-dimensional visual features. The signal branch in the aforementioned neural network architecture can adopt a one-dimensional convolutional neural network architecture, specifically designed to extract temporal features from vibration and chip concentration signals. The fusion layer in the aforementioned neural network architecture can employ an attention mechanism to weightedly fuse the features output from the dual branches, automatically learning the importance of different features for machining state determination. The output layer includes a classification head and a regression head; the classification head is used to identify the machining state type, and the regression head is used to predict the remaining tool life.

[0040] Data that is more helpful in judging tool wear and cutting abnormalities can be assigned higher weights, while data with less impact can be assigned lower weights. Weights are automatically allocated based on the contribution of various indicators to the judgment of machining status. For example... Figure 2 As shown, for a real-time inference engine, the processing status can be determined by the fused features that integrate all monitoring information.

[0041] like Figure 2As shown, at the application decision level, the machining status can be used as a basis for judgment, automatically identifying the current overall machining condition of the CNC machine tool, distinguishing different machining states such as normal cutting, slight wear, heavy wear, chatter, and tool breakage, and predicting tool life based on the machining status. For example, when the chip shape is regular, the spatter is concentrated, the vibration is slightly increased, and the cutting temperature rises slowly, the tool may only have slight wear; when the chips are fine and broken, the trajectory divergence is increased, the vibration fluctuation is intensified, the temperature continues to rise, and the pollution level growth rate increases significantly, the tool may have reached moderate wear; when there are a large number of powdery chips, the spatter is extremely scattered, the vibration is violently excessive, the cutting temperature is high, and the coolant metal concentration increases sharply, it indicates that the tool is severely worn.

[0042] According to the embodiments of this application, by using the morphological data and motion trajectory data corresponding to metal chips as the core judgment basis, and combining multimodal data such as temperature data, vibration data, and concentration data to determine the processing state, the problem of misjudgment such as "pseudo-cutting" can be eliminated, the absolute reliability of the processing state judgment result can be improved, and early intervention can be made based on the early minute changes in the dynamic characteristics of metal chips. This achieves full coverage of the processing state from idling and normal cutting to multi-stage tool wear, chatter, and tool breakage, completing the upgrade from qualitative monitoring of "abnormality" to quantitative monitoring of "health measurement and remaining life prediction".

[0043] According to embodiments of this application, when training and optimizing the improved dual-branch fusion neural network, a labeled dataset covering various machining materials, tool types, and machining states can be constructed. The labeled dataset can include scenarios such as normal cutting, pseudo-cutting, different degrees of tool wear, chatter, tool breakage, and chip entanglement, and is divided into training, validation, and test sets according to a predetermined ratio (e.g., 7:21). A transfer learning strategy can be used for model training. The visual branch network parameters are pre-trained using the collected dataset, and then the entire network is fine-tuned based on the constructed labeled dataset. Appropriate optimizers, loss functions, and training hyperparameters are set, and an early stopping strategy is used to prevent overfitting. After training, model quantization and inference acceleration techniques are used to optimize the model, improving inference speed and meeting the latency requirements of real-time monitoring.

[0044] According to embodiments of this application, training can be performed using a high-performance GPU server, for example, the following operations can be performed:

[0045] Data augmentation is achieved through the development of automated scripts, with all operations executed according to fixed rules to ensure reproducibility. Visual data augmentation includes batch execution of random horizontal flipping (50% probability), random rotation within ±15° (bilinear interpolation filling), brightness adjustment of 0.8-1.2 times, and contrast adjustment of 0.9-1.1 times for chip images. This preserves the core features of the chips while expanding the diversity of the dataset and improving the model's adaptability to changes in lighting and angle. Signal data augmentation includes adding Gaussian noise with a signal-to-noise ratio of 20dB to simulate environmental interference to the vibration and chip concentration time-series data, performing time stretching within the range of 0.8-1.2 times to simulate processing speed fluctuations, and adding slight baseline drift to simulate sensor zero-point offset, allowing the model to adapt to signal fluctuations in actual working conditions in advance.

[0046] During model training, the preprocessed dataset can be randomly divided into training, validation, and test sets according to a predetermined ratio (e.g., 7:2:1). Data is loaded in batches using a data loading tool, with the training set loaded at 32 samples per batch and the validation and test sets at 16 samples per batch. Simultaneously, feature data is normalized to complete data preparation. In parameter configuration, the initial learning rate can be set to 1e-4, the optimizer's weight decay coefficient to 1e-5, and the loss function to be a weighted sum of classification loss (cross-entropy) and regression loss (MAE) (weight ratio 7:3). The visual branch is initialized with pre-trained weights, while the signal branch is randomly initialized. All parameters are loaded onto the GPU for computation.

[0047] After training begins, the program iterates for 100 rounds. After each round, the performance is evaluated using a validation set. The training loss and accuracy are monitored in real time using visualization tools. An early stopping strategy is adopted: if the validation set loss does not decrease for 10 consecutive rounds, the learning rate is halved. A model checkpoint is saved every 20 rounds to avoid training interruption and loss of results.

[0048] After training, the recognition accuracy, false alarm rate and false negative rate of various processing states can be statistically analyzed on the test set. The confusion matrix can be used to locate scenarios that are prone to misjudgment (such as the initial wear of the tool and normal cutting scenarios).

[0049] If the accuracy of a certain state recognition is low (e.g., below 90%), a predetermined amount of labeled data for that scenario can be added, the weights of the model's base layer can be frozen, only the fully connected layer and fusion layer can be fine-tuned, and the model can be retrained (e.g., retrained for 30 rounds). If the inference speed is not up to standard (e.g., single frame latency exceeds 50ms), model quantization or simplification of the number of attention heads can be used to optimize the model, ensuring that the accuracy decreases by no more than 2% while the speed is improved. If the error in the prediction of the remaining tool life is large, the regression loss weights can be adjusted, accurate life labeling data can be added, and the model can be retrained to achieve a balance between accuracy and speed.

[0050] After optimization, batch operation monitoring can be performed. For example, a workshop-level centralized monitoring platform can be built to summarize the system operation data of all equipment in real time, including the accuracy of processing status recognition, alarm response time, false alarm / missed alarm rate, hardware operation status, etc., and performance warning thresholds can be set (such as automatically triggering an alarm when the recognition accuracy is lower than 90%).

[0051] like Figure 2 As shown, continuous optimization and iteration are also possible. For example, regular system performance reviews across the entire workshop can be conducted to analyze common and unique issues. For common issues (such as false alarms due to strong light interference), the unified image preprocessing algorithm can be upgraded, for example, by adding an adaptive strong light suppression module. For unique issues (such as unclear chip features due to intermittent cutting), feature extraction parameters can be adjusted accordingly, such as by adding a statistical dimension for chip generation frequency. Actual on-site processing data can be regularly integrated (such as adding 10,000 labeled samples) to fine-tune the model, continuously improving the system's adaptability to different processing scenarios. Ultimately, this will achieve stable coverage of all CNC equipment in the workshop, with an overall recognition accuracy of ≥95% and a false alarm rate of ≤2%, effectively supporting real-time monitoring and intelligent judgment of the workshop's processing status.

[0052] According to embodiments of this application, acquiring real-time target multimodal data includes: acquiring raw multimodal data; performing image preprocessing and signal preprocessing on the raw multimodal data to obtain initial multimodal data; determining the metal chip profile based on the initial multimodal data, and determining the total perimeter, total area, aspect ratio, curling helix degree, and fracture rate of the metal chip based on the metal chip profile to obtain morphological data; and determining the average ejection velocity and trajectory divergence of the metal chip based on the initial multimodal data to obtain trajectory data.

[0053] For example, the raw multimodal data includes time-stamped data acquired by sensors; the raw multimodal data may contain interference information, such as... Figure 2As shown in the data computing layer (for data processing and inference), the data preprocessing module can perform image preprocessing and signal preprocessing to obtain initial multimodal data.

[0054] The contour of metal chips can be extracted using edge detection algorithms, and morphological data such as total perimeter, total area, aspect ratio, curling helix degree, and fracture rate of metal chips can be calculated based on the contour of metal chips. Morphological data can characterize the geometric shape and generation amount of metal chips.

[0055] For example, the total perimeter can be the total length of the edge lines of a single metal chip; the total area can be the pixel area occupied by the metal chip in the image; the aspect ratio can be the ratio of the longest width to the shortest width of the metal chip, which can be used to distinguish between long, curled chips and fine particles; the curling helix can characterize the degree of curling and entanglement of the metal chip; and the breakage rate can be the proportion of small, broken metal chips in the original image to all metal chips.

[0056] In the chip dynamic feature extraction module, the motion vector of metal chips can be calculated based on optical flow, and the motion characteristics such as the average ejection speed and trajectory divergence of metal chips generated during the cutting process can be statistically analyzed. For example, the position changes of the same group of metal chips in consecutive frames can be tracked by comparing the original images before and after, and the average flight speed of the metal chips flying outward from the tool tip can be calculated based on the position changes. The range of metal chip splashing and scattering in the original image can be statistically analyzed, and the motion trajectory divergence can be calculated. For example, the smaller the value of the trajectory divergence, the more concentrated the metal chips and the smoother the cutting; the larger the value, the more scattered the metal chips, the more the machine tool vibrates and the abnormal cutting state occurs.

[0057] For example, time-domain features such as root mean square value, kurtosis, and peak factor can be extracted from the preprocessed vibration signal, and a signal feature vector can be constructed by combining it with the rate of change of metal chip concentration. The visual feature vector and the signal feature vector can be aligned according to the timestamp, and the aligned visual feature vector and the signal feature vector can be spliced ​​together to form a multi-dimensional feature vector. The multi-dimensional feature vector can comprehensively reflect the multi-dimensional feature vector of the processing state, providing comprehensive input data for model inference.

[0058] According to an embodiment of this application, image preprocessing includes: performing noise reduction processing on the original image in the original multimodal data using a nonlocal mean filtering algorithm; performing brightness correction and temperature calibration on the infrared image in the original multimodal data to determine the temperature data corresponding to the image grayscale value; matching the image template for the predetermined area of ​​the tool with the original image to determine the real-time coordinates of the predetermined area of ​​the tool; and determining the region of interest and background region in the original image based on the real-time coordinates.

[0059] For example, the original image captured by the high-speed camera may contain interfering information such as coolant splashes, water droplets, and metal dust in the air. By using nonlocal mean filtering for noise reduction, the above-mentioned interfering information can be effectively removed, while retaining the detailed features such as the edge and shape of the metal chips.

[0060] Hardware defects in infrared cameras can lead to uneven brightness in infrared images, thus requiring brightness correction (such as non-uniformity correction). Since infrared images only contain grayscale values ​​and cannot be directly used as temperature readings, a conversion relationship between grayscale values ​​and actual degrees Celsius can be established, allowing the calculation of the corresponding actual temperature for each grayscale value in the infrared image. An adaptive binarization algorithm can be used to segment the temperature region of metal chips to highlight the metal chip target.

[0061] The image template for the predetermined area of ​​the cutting tool can include a standard appearance image of the cutting tool tip. For example, a standard appearance image of the cutting tool tip can be obtained in advance as a template, and a template matching algorithm can be used to automatically search for the position of the cutting tool tip in each frame of the original image, and output the horizontal and vertical coordinates of the cutting tool tip in the original image in real time to achieve automatic tracking of the cutting tool tip.

[0062] The real-time coordinates of the tracked tool tip can be used as the center to define the area containing the chip generation region as the region of interest, and the other areas in the original image other than the region of interest can be used as the background region. For example, the area where the machine tool, water tank and other targets are located can be divided into the background region. The background can be ignored directly in subsequent calculations, reducing the amount of computation and improving the real-time processing speed.

[0063] According to an embodiment of this application, signal preprocessing includes: filtering the original vibration data in the original multimodal data to obtain filtered vibration data; eliminating the trend term bias of the filtered vibration data using the least squares method and performing amplitude standardization to obtain target vibration data; smoothing the original concentration data in the original multimodal data to obtain processed concentration data; and calculating the metal chip concentration change rate and contamination level growth rate based on the processed concentration data, wherein the contamination level growth rate characterizes the rate of increase of the metal impurity contamination level in the coolant.

[0064] For example, the raw vibration data directly collected by the vibration sensor may contain interference information such as low-frequency vibration of the machine tool frame and high-frequency electromagnetic noise from the circuit. By filtering, the interference information can be removed and the effective vibration signal generated by the cutting tool can be retained. For example, the Butterworth bandpass filter algorithm can be used to filter out the low-frequency frame resonance and high-frequency electromagnetic noise in the raw vibration data.

[0065] When a sensor operates for a long time, the baseline may drift slowly (such as the overall value gradually becoming higher or lower, i.e., trend term deviation). The baseline can be corrected by the least squares algorithm, and amplitude standardization can be performed after the correction. Amplitude standardization can include mapping vibration values ​​with large differences in magnitude to a fixed numerical range to facilitate feature extraction and model training.

[0066] Iron powder concentration sensors are susceptible to the effects of water flow and the random passage of fine metal chips, which may cause the sensor readings to fluctuate frequently and irregularly. By performing data smoothing processing, random fluctuations can be smoothed out, noise can be suppressed, and the output concentration data with a stable trend that closely matches the actual changes in metal chip content can be produced. For example, data smoothing processing can be performed using a moving average filtering algorithm.

[0067] The rate of change in metal chip concentration can include the speed at which the metal chip content in the coolant increases or decreases per unit time; it can also pre-classify metal contamination levels, such as light, moderate, and heavy contamination, and the rate of increase in contamination level can measure the speed at which the severity of contamination increases. By calculating the rate of change in metal chip concentration and the rate of increase in contamination level based on the processed concentration data, effective data support can be provided for feature fusion.

[0068] By using image preprocessing and signal preprocessing algorithms such as dynamic background subtraction and ROI region dynamic locking, we can effectively overcome on-site interference such as cutting fluid splashing, dust obstruction, and changes in lighting. Through the complementary architecture of multimodal data, each modal data can verify each other and make collaborative judgments, thus maintaining high judgment accuracy even when a single sensor temporarily fails, and adapting to CNC machining scenarios with different materials and processes.

[0069] According to an embodiment of this application, the raw multimodal data is obtained by: acquiring multiple sets of sensor data collected by various sensors; and performing timestamp alignment processing on the multiple sets of sensor data to obtain the raw multimodal data.

[0070] For example, independent hardware such as high-speed cameras, vibration sensors, and metal chip concentration sensors collect data independently, which may result in millisecond-level time differences, causing the data collected by these sensors to be misaligned. Therefore, timestamp alignment processing can be performed on multiple sets of sensor data. For example, the hardware timestamp of each sensor can be obtained, and a hardware timestamp can be generated for each set of sensor data collected; sensor data generated at the same time can be matched to align multiple sets of sensor data.

[0071] like Figure 2As shown, in the multi-source data synchronization unit, time deviations between multiple devices can be eliminated by aligning the timestamps of multiple sets of sensor data, thus avoiding multimodal data asynchrony and improving the accuracy of the processing status determination results.

[0072] According to the embodiments of this application, a unified sensor data acquisition interface can be developed based on the industrial general communication protocol to realize standardized data access of high-speed cameras, infrared thermal imagers, vibration sensors, and iron wear sensors. The acquisition frequency of each device can be flexibly configured according to actual monitoring needs, and is not limited here.

[0073] According to embodiments of this application, multiple sets of sensor data can be checked for anomalies in real time. If at least one set of sensor data is determined to be abnormal, a local cache and device reconnection mechanism can be automatically triggered. For example, sensor data anomalies may include transmission interruptions, such as loose wiring, communication lag, or disconnection of cameras, vibration sensors, or concentration sensors, resulting in the inability to receive real-time data; or amplitude exceeding limits, such as sensor malfunction or signal distortion causing sensor data values ​​to be extremely high or low. Local caching may include storing the valid sensor data collected in real time into local storage when data is abnormal. Device reconnection may include the system automatically initiating communication retry, repeatedly attempting to re-establish the data transmission channel with the faulty sensor.

[0074] According to an embodiment of this application, the above method further includes: determining a target early warning strategy corresponding to a target early warning level based on a predetermined correspondence between multiple early warning levels and multiple early warning strategies.

[0075] For example, an industrial communication client can be developed to communicate with the machine tool's programmable logic controller and issue control commands to execute the target warning strategy based on the target warning level output by the model.

[0076] The target warning strategy may include: adjusting the machining parameters when the target warning level is lower than the preset level (i.e., when the machining status is slightly abnormal); and triggering the machine tool to stop suddenly when the target warning level is equal to or higher than the preset level (i.e., when the machining status is severely abnormal) to avoid the accident from escalating.

[0077] A multi-channel early warning module can be developed to support audible and visual alarms, SMS and internal enterprise communication tools to promptly push abnormal information to relevant personnel, including abnormal equipment, time of occurrence, type of abnormality and handling suggestions.

[0078] According to embodiments of this application, the machining status of CNC equipment can also be visualized.

[0079] like Figure 2As shown, at the application decision level (for human-machine interaction and early warning), a human-machine interface can be designed based on a graphical interface development framework. The human-machine interface can be equipped with an HMI status display panel to display CNC equipment processing monitoring data and early warning information in real time.

[0080] For example, the human-machine interface can include functional areas such as machining status dashboard, feature visualization, anomaly diagnosis, and historical data query. The human-machine interface can display information such as machining status, metal chip morphology, and remaining tool life in real time, and provide anomaly cause analysis and handling suggestions, and support the query and export of historical data.

[0081] By visualizing the processing status and providing tiered early warnings, an intelligent monitoring closed-loop system can be built, significantly improving equipment production efficiency and on-site control capabilities. It can also achieve real-time early warnings and adaptive parameter adjustments at the second level, making the processing status judgment results highly physically interpretable, reducing the professional threshold for on-site operation, and facilitating rapid decision-making and problem handling.

[0082] like Figure 2 As shown, in the execution control layer, the OPC UA (Industrial Equipment Common Standardized Communication Protocol) communication unit can serve as a data exchange bridge between the monitoring software and the CNC machine tool. For example, it can receive information such as machining anomalies and warning levels, as well as corresponding control commands, and translate the control commands issued by the software into the machine tool PLC. The programmable logic controller (PLC) can recognize standard signals and adaptively adjust cutting parameters. For example, when a non-emergency fault such as mild or moderate tool wear is detected, it can automatically optimize cutting process parameters instead of stopping the machine directly. This includes automatically reducing the spindle speed, adjusting the depth of cut, and reducing the cutting load on the tool, thereby extending the tool life while ensuring workpiece quality. Different types of actions (such as emergency stop, speed reduction, and chip removal reminders) can be set according to different warning levels. For example, in cases of moderate abnormality, the spindle and feed speeds can be significantly reduced to decrease cutting impact. In cases of severe risk (such as tool breakage, severe chatter, or cutting overheating), the cutting feed can be directly cut off and the spindle stopped to prevent workpiece scrap and machine damage through emergency stop. Chip removal reminders can be issued when there is a large accumulation of chips and the concentration of metal chips in the coolant exceeds the standard.

[0083] like Figure 2As shown, a processing status database and a chip feature knowledge base can be constructed. For example, the processing status database can store target multimodal data corresponding to each processing status, and the chip feature knowledge base can store full-volume time-series working condition data generated in real time during the operation of CNC equipment. Thus, the data assets of the entire processing process can be realized, providing real and traceable data support for production process optimization and processing parameter tuning. Furthermore, the model can be iterated and fine-tuned online based on the multimodal database to continuously improve the model's adaptability to different machine types and working conditions, and realize the system's self-optimization and upgrading.

[0084] The method for determining the machining status of CNC equipment in this application embodiment can be adapted to various workshop scenarios and machine tools, and its installation and deployment methods are flexible and varied. Therefore, it has high compatibility and flexible deployment, which can reduce the cost of enterprise implementation and promotion. The camera, vibration sensor, and concentration sensor are independent external accessories that can be disassembled and replaced separately. Through the above-mentioned external modular design, there is no need to modify the machine tool body structure, and it is compatible with mainstream CNC systems. Through the layered architecture design of data acquisition, application decision, and layered decision, it can support batch deployment of multiple machine models, greatly reducing the system implementation and maintenance costs, and therefore has good value for large-scale promotion.

[0085] Based on the above-described method for determining the machining state of CNC equipment, this application also provides a device for determining the machining state of CNC equipment. The following will be combined with... Figure 3 The device is described in detail.

[0086] Figure 3 The diagram illustrates the structure of a device for determining the machining state of a CNC machine according to an embodiment of this application.

[0087] like Figure 3 As shown, the CNC equipment machining state determination device 300 of this embodiment includes: an acquisition module 310, used to acquire real-time target multimodal data during the CNC machining of metal materials using CNC equipment. The target multimodal data includes: morphological data and motion trajectory data corresponding to metal chips generated during the CNC machining process, temperature data of the cutting area, vibration data of the CNC equipment, and concentration data of metal chips in the coolant.

[0088] The generation module 320 is used to generate visual feature vectors based on morphological data, motion trajectory data and temperature data, and to generate signal feature vectors based on vibration data and concentration data;

[0089] The fusion module 330 is used to fuse visual feature vectors and signal feature vectors to obtain fused features. The fusion weights of the visual feature vectors and signal feature vectors are determined based on the importance of the multimodal features to the processing state.

[0090] The determination module 340 is used to determine the machining status of the CNC equipment based on the fusion features.

[0091] According to an embodiment of this application, the acquisition module includes: an acquisition submodule for acquiring raw multimodal data; a preprocessing submodule for performing image preprocessing and signal preprocessing on the raw multimodal data to obtain initial multimodal data; a first determination submodule for determining the metal chip profile based on the initial multimodal data, and determining the total perimeter, total area, aspect ratio, curling helix degree, and fracture rate of the metal chip based on the metal chip profile to obtain morphological data; and a second determination submodule for determining the average ejection velocity and trajectory divergence of the metal chip based on the initial multimodal data to obtain trajectory data.

[0092] The preprocessing submodule includes: an image preprocessing unit, used to perform noise reduction processing on the original image in the original multimodal data using a nonlocal mean filtering algorithm; to perform brightness correction and temperature calibration on the infrared image in the original multimodal data, and to determine the temperature data corresponding to the image grayscale value; to match the image template of the predetermined area of ​​the tool with the original image to determine the real-time coordinates of the predetermined area of ​​the tool; and to determine the region of interest and background area in the original image based on the real-time coordinates.

[0093] The preprocessing submodule also includes: a signal preprocessing unit, used to filter the original vibration data in the original multimodal data to obtain filtered vibration data; to eliminate the trend term bias of the filtered vibration data using the least squares method and to perform amplitude standardization to obtain target vibration data; to perform data smoothing on the original concentration data in the original multimodal data to obtain processed concentration data; and to calculate the metal chip concentration change rate and the contamination level growth rate based on the processed concentration data, wherein the contamination level growth rate characterizes the rate of increase of the metal impurity contamination level in the coolant.

[0094] The acquisition submodule includes: an acquisition unit for acquiring multiple sets of sensor data from various sensors; and a timestamp alignment unit for performing timestamp alignment processing on the multiple sets of sensor data to obtain the original multimodal data.

[0095] The device for determining the machining status of CNC equipment also includes: a warning level determination module, used to determine the target warning level corresponding to the machining status of the CNC equipment; and a warning strategy determination module, used to determine the target warning strategy corresponding to the target warning level based on the pre-determined correspondence between multiple warning levels and multiple warning strategies.

[0096] According to embodiments of this application, any multiple modules among the acquisition module 310, generation module 320, fusion module 330, and determination module 340 can be merged into one module, or any one of these modules can be split into multiple modules. Alternatively, at least some of the functions of one or more of these modules can be combined with at least some of the functions of other modules and implemented in one module. According to embodiments of this application, at least one of the acquisition module 310, generation module 320, fusion module 330, and determination module 340 can be at least partially implemented as hardware circuitry, such as a field-programmable gate array (FPGA), a programmable logic array (PLA), a system-on-a-chip, a system-on-a-substrate, a system-on-package, an application-specific integrated circuit (ASIC), or implemented in hardware or firmware by any other reasonable means of integrating or packaging circuitry, or implemented in any one of software, hardware, and firmware methods, or in a suitable combination of any of these. Alternatively, at least one of the acquisition module 310, generation module 320, fusion module 330, and determination module 340 can be at least partially implemented as a computer program module, which can perform corresponding functions when the computer program module is run.

[0097] Figure 4 A block diagram schematically illustrates an electronic device suitable for implementing a method for determining the machining state of a CNC machine according to an embodiment of this application.

[0098] like Figure 4 As shown, an electronic device 400 according to an embodiment of this application includes a processor 401, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 402 or a program loaded from a storage portion 408 into a random access memory (RAM) 403. The processor 401 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or an associated chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 401 may also include onboard memory for caching purposes. The processor 401 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of this application.

[0099] RAM 403 stores various programs and data required for the operation of electronic device 400. Processor 401, ROM 402, and RAM 403 are interconnected via bus 404. Processor 401 executes various operations of the method flow according to embodiments of this application by executing programs in ROM 402 and / or RAM 403. It should be noted that programs may also be stored in one or more memories other than ROM 402 and RAM 403. Processor 401 may also execute various operations of the method flow according to embodiments of this application by executing programs stored in one or more memories.

[0100] According to embodiments of this application, the electronic device 400 may further include an input / output (I / O) interface 405, which is also connected to a bus 404. The electronic device 400 may also include one or more of the following components connected to the input / output (I / O) interface 405: an input section 406 including a keyboard, mouse, etc.; an output section 407 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 408 including a hard disk, etc.; and a communication section 409 including a network interface card such as a LAN card, modem, etc. The communication section 409 performs communication processing via a network such as the Internet. A drive 410 is also connected to the input / output (I / O) interface 405 as needed. A removable medium 411, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on the drive 410 as needed so that computer programs read from it can be installed into the storage section 408 as needed.

[0101] This application also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or it may exist independently and not assembled into the device / apparatus / system. The computer-readable storage medium carries one or more programs, which, when executed, implement the method according to the embodiments of this application.

[0102] According to embodiments of this application, the computer-readable storage medium can be a non-volatile computer-readable storage medium, such as including but not limited to: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this application, the computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. For example, according to embodiments of this application, the computer-readable storage medium may include ROM 402 and / or RAM 403 and / or one or more memories other than ROM 402 and RAM 403 described above.

[0103] Embodiments of this application also include a computer program product comprising a computer program containing program code for performing the methods shown in the flowchart. When the computer program product is run on a computer system, the program code is used to cause the computer system to implement the methods provided in the embodiments of this application.

[0104] When the computer program is executed by the processor 401, it performs the functions defined in the system / apparatus of this application embodiment. According to the embodiments of this application, the systems, apparatuses, modules, units, etc., described above can be implemented by computer program modules.

[0105] In one embodiment, the computer program may rely on a tangible storage medium such as an optical storage device or a magnetic storage device. In another embodiment, the computer program may also be transmitted and distributed in the form of signals over a network medium, and downloaded and installed via communication section 409, and / or installed from removable medium 411. The program code contained in the computer program can be transmitted using any suitable network medium, including but not limited to: wireless, wired, etc., or any suitable combination thereof.

[0106] In such an embodiment, the computer program can be downloaded and installed from a network via communication section 409, and / or installed from removable medium 411. When the computer program is executed by processor 401, it performs the functions defined in the system of this application embodiment. According to embodiments of this application, the systems, devices, apparatuses, modules, units, etc., described above can be implemented by computer program modules.

[0107] According to embodiments of this application, program code for executing the computer programs provided in the embodiments of this application can be written in any combination of one or more programming languages. Specifically, these computational programs can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, languages ​​such as Java, C++, Python, "C", or similar programming languages. The program code can be executed entirely on the user's computing device, partially on the user's device, partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0108] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0109] Those skilled in the art will understand that the features described in the various embodiments of this application can be combined and / or combined in various ways, even if such combinations or combinations are not explicitly described in this application. In particular, the features described in the various embodiments of this application can be combined and / or combined in various ways without departing from the spirit and teachings of this application. All such combinations and / or combinations fall within the scope of this application.

[0110] The embodiments of this application have been described above. However, these embodiments are merely illustrative and not intended to limit the scope of this application. Although various embodiments have been described above, this does not mean that the measures in the various embodiments cannot be used advantageously in combination. Without departing from the scope of this application, those skilled in the art can make various substitutions and modifications, all of which should fall within the scope of this application.

Claims

1. A method for determining the machining state of a CNC machine, characterized in that, The method includes: During the CNC machining of metal materials using CNC equipment, real-time target multimodal data is acquired. The target multimodal data includes: morphological data and motion trajectory data corresponding to the metal chips generated during the CNC machining process, temperature data of the cutting area, vibration data of the CNC equipment, and concentration data of the metal chips in the coolant. A visual feature vector is generated based on the morphological data, the motion trajectory data, and the temperature data, and a signal feature vector is generated based on the vibration data and the concentration data. The visual feature vector and the signal feature vector are fused to obtain fused features, wherein the fusion weights of the visual feature vector and the signal feature vector are determined based on the importance of the multimodal features to the processing state; Based on the fusion features, the machining status of the CNC equipment is determined.

2. The method according to claim 1, characterized in that, The acquisition of real-time target multimodal data includes: Acquire raw multimodal data; The original multimodal data is subjected to image preprocessing and signal preprocessing to obtain initial multimodal data; The metal chip profile is determined based on the initial multimodal data, and the total perimeter, total area, aspect ratio, curling helix degree, and fracture rate of the metal chip are determined based on the metal chip profile to obtain the morphological data. Based on the initial multimodal data, the average ejection velocity and trajectory divergence of the metal chips are determined, and the trajectory data is obtained.

3. The method according to claim 2, characterized in that, The image preprocessing includes: The original images in the original multimodal data are denoised using a nonlocal mean filtering algorithm. Brightness correction and temperature calibration are performed on the infrared image in the original multimodal data to determine the temperature data corresponding to the image grayscale value; The image template for the predetermined area of ​​the tool is matched with the original image to determine the real-time coordinates of the predetermined area of ​​the tool. The region of interest and the background region in the original image are determined based on the real-time coordinates.

4. The method according to claim 2, characterized in that, The signal preprocessing includes: The original vibration data in the original multimodal data is filtered to obtain filtered vibration data; The trend term bias of the filtered vibration data is eliminated by using the least squares method, and the amplitude is standardized to obtain the target vibration data. The original concentration data in the original multimodal data is smoothed to obtain the processed concentration data; Based on the processed concentration data, the change rate of metal chip concentration and the growth rate of contamination level are calculated, wherein the growth rate of contamination level represents the rate at which the contamination level of metal impurities in the coolant increases.

5. The method according to claim 2, characterized in that, The process of obtaining raw multimodal data: Acquire multiple sets of sensor data collected by various sensors; The multiple sets of sensor data are time-stamp aligned to obtain the original multimodal data.

6. The method according to claim 1, characterized in that, The method further includes: Determine the target early warning level corresponding to the machining status of the CNC equipment; Based on the pre-determined correspondence between multiple warning levels and multiple warning strategies, a target warning strategy corresponding to the target warning level is determined.

7. A device for determining the machining state of CNC equipment, characterized in that, The device includes: The acquisition module is used to acquire real-time target multimodal data during the CNC machining of metal materials using CNC equipment. The target multimodal data includes: morphological data and motion trajectory data corresponding to the metal chips generated during the CNC machining process, temperature data of the cutting area, vibration data of the CNC equipment, and concentration data of the metal chips in the coolant. The generation module is used to generate a visual feature vector based on the morphological data, the motion trajectory data, and the temperature data, and to generate a signal feature vector based on the vibration data and the concentration data; A fusion module is used to fuse the visual feature vector and the signal feature vector to obtain fused features, wherein the fusion weights of the visual feature vector and the signal feature vector are determined based on the importance of the multimodal features to the processing state; The determination module is used to determine the machining status of the CNC equipment based on the fusion features.

8. An electronic device, comprising: One or more processors; Memory, used to store one or more computer programs. The characteristic feature is that the one or more processors execute the one or more computer programs to implement the steps of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program or instructions stored thereon, characterized in that, When the computer program or instructions are executed by a processor, they implement the steps of the method according to any one of claims 1 to 6.

10. A computer program product comprising a computer program that, when executed by a processor, implements the method according to any one of claims 1 to 6.

Images