A method for detecting broken needles in micro drill bits based on multimodal information fusion

Abstract

This invention relates to the field of micro-drilling equipment inspection technology, specifically to a method for detecting broken drill bits based on multimodal information fusion. The method first constructs a multimodal time-series dataset, collecting data through non-invasive sensors and labeling it with multiple operating conditions and states. Multi-scale resampling technology is used to preprocess and enhance the original signals, generating multi-scale sequences and fusing them into a six-channel fused time-series matrix. A condition-time-series dual-stream fusion model based on cross-attention is constructed, utilizing a condition-time-series cross-attention fusion processor to actively query and weight time-series features based on condition information, overcoming class imbalance while performing end-to-end supervised training. In the online detection deployment phase, Monte Carlo Dropout is used to assess prediction uncertainty, and a dual-threshold decision based on the average broken bit probability and prediction uncertainty indicator is combined to achieve robust detection and alarm for broken bits and early degradation.

Description

Technical Field

[0001] This invention relates to the field of micro-drilling equipment testing technology, specifically to a method for detecting broken micro-drill bits based on multimodal information fusion. Background Technology

[0002] With the rapid development of high-tech industries such as electronics and aerospace, the requirements for the processing precision and efficiency of micro-components are becoming increasingly stringent. Micro-drilling, as an important micro-machining method, is widely used in fields such as printed circuit board drilling and micro-hole array manufacturing. However, micro-drills have a small diameter and a fragile structure, making them extremely prone to brittle fracture during high-speed drilling. Micro-drill bit breakage is a sudden failure that can not only lead to workpiece scrap and a severe decline in the surface quality of the machined parts, but may also damage the machine tool spindle, causing serious economic losses and production stoppages.

[0003] Currently, the main methods for detecting broken micro drill bits include the following:

[0004] (1) Signal monitoring method based on traditional sensors: This method mainly uses traditional sensors such as force, vibration or current to monitor physical signals during the drilling process. For example, the sudden change in spindle power or current signal can be used to infer whether the drill bit has broken; or the vibration signal of the machine tool or workpiece can be monitored by an accelerometer. However, the signal energy of the micro-drilling process itself is weak and is easily interfered with by background noise from the spindle motor, feed system, etc. The transient impact signal of the broken needle is often submerged in the complex working condition changes and environmental noise, resulting in insufficient detection sensitivity and anti-interference ability. In addition, single modal information is difficult to fully characterize the complex physical phenomena at the moment of needle breakage.

[0005] (2) Offline / online inspection methods based on machine vision: These methods use industrial cameras to image the drill bit before or during machining, and check the integrity of the drill bit's cutting edge through image processing and analysis. However, visual inspection requires interrupting or slowing down the machining process, making it impossible to achieve true real-time online inspection; and for high-speed drilling processes at high speeds, it is difficult to accurately capture and judge the moment of needle breakage.

[0006] (3) Single-signal intelligent detection methods based on traditional machine learning or deep learning: With the development of artificial intelligence, some studies have attempted to use deep learning models for feature extraction and fault classification of single acquired signals. However, most of these methods rely only on the temporal information of a single mode and fail to fully utilize the complementary advantages between signals from different physical fields. Furthermore, they are typically trained only for specific operating conditions, facing the challenge of a sharp decline in generalization due to changes in operating conditions in real production. In addition, existing methods generally lack effective solutions for extreme class imbalanced needle breakage samples and for evaluating the robustness of models to uncertain states.

[0007] In summary, existing micro-drill bit broken needle detection methods still have limitations in the following key aspects: insufficient multimodal information fusion, i.e., most methods rely on a single sensor, resulting in insufficient feature representation; poor generalization under working conditions: most existing intelligent models ignore the significant impact of working condition parameters, including spindle speed and feed rate, on signal features, making it difficult to maintain high accuracy and robustness in variable real-world machining environments; the scarcity of broken needle samples has not been effectively addressed: during training, the extreme class imbalance problem of scarce broken needle samples causes the model to easily bias towards normal states, making it difficult to guarantee the recall and accuracy of broken needle faults; lack of robustness and early degradation warning capabilities: existing methods usually only provide definite classification results, lacking quantitative assessment of prediction uncertainty, making it difficult to distinguish between high-confidence broken needle faults and early degradation or abnormal states that the model has not seen before, limiting the safety and early warning capabilities of their online applications.

[0008] Therefore, developing an intelligent detection method that can achieve deep fusion of multimodal signals, actively incorporate working condition information, effectively address data imbalance, and possess the ability to predict uncertainty assessment, in order to solve the challenges of accuracy, robustness, and generalization in the detection of broken needles in micro-drills in industrial applications, is of great practical significance. Summary of the Invention

[0009] The purpose of this invention is to provide a micro-drill bit broken needle detection method based on multimodal information fusion, so as to solve the existing technical problems in the background art.

[0010] To solve the above-mentioned technical problems, the present invention provides the following technical solution: a micro-drill bit broken needle detection method based on multimodal information fusion, comprising the following steps:

[0011] S1: Collect full-band acoustic timing signals and real-time spindle power timing data, label them with multiple operating conditions and multiple states, and construct a multimodal timing dataset;

[0012] S2: Perform signal preprocessing and feature enhancement on the multimodal time series dataset, and output a multi-channel fused sequence dataset;

[0013] S3: Based on the cross-attention mechanism, a dual-stream fusion model for detecting broken needles, combining working conditions and time series, is constructed. This model is trained and optimized using a multi-channel fusion sequence dataset to enable active querying and weighting of time series features based on working condition information, accurately identifying the broken needle state, and outputting the final optimized online state classification model.

[0014] S4: Online detection deployment and prediction uncertainty assessment; Deploy the online state classification model output from step S3 to the actual processing, and achieve robust detection of drill bit breakage and early degradation by assessing the uncertainty of the model prediction.

[0015] Based on the above technical solution, the construction of the multimodal time series dataset in step S1 includes the following steps:

[0016] S11 Multimodal Sensor Deployment and Data Acquisition: A high-fidelity industrial microphone is installed outside the spindle box to non-contactly acquire full-band acoustic timing signals during the drilling process; real-time spindle power timing data driving the spindle servo motor is synchronously acquired through the built-in data interface of the machine tool CNC system and an external power analysis module.

[0017] S12 Multi-condition Experiment Matrix Design: The micro drill bit is subjected to drilling experiments under various working conditions such as different spindle speeds, different feed rates, and different workpiece materials. Multi-modal sensor data of the micro drill bit under different health states, such as normal drilling state and drill bit breakage state, are collected and labeled with working condition labels and status labels.

[0018] S13 Data Integration: Integrate the data obtained in steps S11 and S12 to finally output a structured multimodal time series dataset with operating conditions and status labels.

[0019] Based on the above technical solution, the construction of the multi-channel fusion sequence dataset in step S2 includes the following steps:

[0020] S21 Time Series Data Segmentation and Sample Construction: The two original signals in the multi-modal time series dataset of step S1, namely the full-band acoustic time series signal and the real-time spindle power time series data, are segmented, and the resulting signal segments are combined to form original multi-modal samples; at the same time, the original multi-modal samples are assigned the operating condition label and status label corresponding to their acquisition time.

[0021] S22 Temporal feature enhancement based on multi-scale resampling: The one-dimensional signal segment in step S21 is preserved as the original sequence; the original sequence is filtered by a low-pass filter and a fixed downsampling factor is used to extract the filtered sequence to obtain a short-term trend sequence; then, the short-term trend sequence is upsampled and restored to a fixed length T of the original one-dimensional signal segment using linear interpolation to obtain a low-frequency trend sequence.

[0022] A bandpass filter is used to filter the one-dimensional signal segment to extract the target frequency band signal sensitive to the broken needle event. The Tiger-Kaiser energy operator is then performed on the target frequency band signal to obtain the instantaneous energy sequence. Boundary filling processing is then performed to obtain the high-frequency energy sequence.

[0023] S23 Multimodal and multiscale sequence fusion: The acoustic raw sequence, acoustic low-frequency trend sequence, acoustic high-frequency energy sequence, power raw sequence, power low-frequency trend sequence, and power high-frequency energy sequence obtained in step S22 are spliced ​​together to form a six-channel fusion time matrix.

[0024] S24 Multi-channel fusion sequence dataset output: The six-channel fusion time series matrix generated in step S23 and the set of working condition labels and status labels corresponding to the matrix in step S21 are combined to obtain a structured preprocessed sequence dataset, namely the multi-channel fusion sequence dataset.

[0025] Based on the above technical solution, the construction of the working condition-timing dual-stream fusion needle breakage detection model in step S3 includes the following steps:

[0026] S31 Dual-Stream Cross-Attention Model Construction: The dual-stream cross-attention model includes a temporal feature encoder, a working condition context encoder, a working condition-temporal cross-attention fusion unit, and a state classifier. The input of the temporal feature encoder is the six-channel fused temporal matrix in step S24, used to capture the temporal dependencies within the temporal matrix and output a high-dimensional temporal feature sequence. The input of the working condition context encoder is the working condition label in step S13, mapping the discrete working condition label to a dense working condition context vector. The input of the working condition-temporal cross-attention fusion unit is the high-dimensional temporal feature sequence and the working condition context vector, outputting a weighted fusion vector that integrates working condition information. The input of the state classifier is the weighted fusion vector of the working condition, outputting the probability distribution of the corresponding state label.

[0027] S32 End-to-end supervised training strategy and optimization: The dual-stream cross-attention model constructed in step S31 is trained end-to-end using the multi-channel fused sequence dataset from step S24. The focus loss function is used and a modulation factor is introduced to concentrate computational resources and optimization gradients on the difficult-to-classify and scarce broken needle samples. Through backpropagation, all parameters of the temporal feature encoder, working condition context encoder, cross-attention fusion unit and state classifier are optimized simultaneously, so that the model can accurately locate broken needle features from complex temporal signals by using working condition information while overcoming data imbalance.

[0028] S33 Consolidation and Output of Online State Classification Model: After the supervised training in step S32 converges, all parameter weights of the optimized complete dual-stream cross-attention model in step S31 are consolidated as the final online state classification model.

[0029] Based on the above technical solution, the temporal feature encoder includes a one-dimensional convolutional layer and a Transformer encoder layer consisting of N stacked layers.

[0030] Each Transformer encoder layer consists of two core sub-layers, and the output of each sub-layer is appended with a residual connection and subjected to layer normalization. The first core sub-layer is a multi-head self-attention module, which captures global temporal dependencies by calculating the correlation weights between all time steps within the sequence. The output of this sub-layer is then residually connected to the input and subjected to layer normalization. The second core sub-layer is a position feedforward network, consisting of two linear transformation layers and a ReLU activation function. This network is applied independently at each time step of the sequence. The output of this sub-layer is also residually connected to the input and subjected to layer normalization.

[0031] Based on the above technical solution, the working condition context encoder includes a working condition embedding layer, and the specific mapping process is as follows:

[0032] First, the input working condition labels are decomposed into categorical parameters and numerical parameters. Second, the categorical parameters are processed through independent embedding layers to obtain categorical embedding vectors. The numerical parameters are normalized and then mapped to numerical embedding vectors through fully connected layers to ensure that they are in the same feature space as the categorical embeddings. Finally, all independent categorical embedding vectors and numerical embedding vectors are concatenated and fused to form the final dense working condition context vector.

[0033] Based on the above technical solution, the working condition-time series cross-attention fusion processor includes a cross-attention layer, and the construction process of the cross-attention layer includes the following steps:

[0034] First, the input working condition context vector and time series feature sequence are projected through independent linear transformation matrices to generate query matrix Q, key matrix K, and value matrix V. Among them, query matrix Q is obtained by projecting the working condition context vector through linear transformation, while key matrix K and value matrix V are both obtained by projecting the time series feature sequence through their respective independent linear transformations.

[0035] Secondly, the attention weight matrix is ​​obtained by calculating the dot product of the query matrix Q and the key matrix K, scaling the matrix, and then applying the Softmax function.

[0036] Finally, the attention weight matrix and the value matrix V are weighted and summed to obtain the output of the attention layer. This output is then processed by residual connection and layer normalization to form the final working condition weighted fusion vector.

[0037] Based on the above technical solution, the state classifier includes a fully connected layer and an output layer arranged sequentially. The fully connected layer includes 128 neurons, a ReLU activation function, and a Dropout layer. The output layer includes two neurons, corresponding to the normal drilling state and the drill bit breakage state, respectively, and uses the Softmax activation function to output the probability distribution of the corresponding state labels.

[0038] Based on the above technical solution, step S4, online detection deployment and prediction uncertainty assessment, includes the following steps:

[0039] S41 Monte Carlo Dropout Model Enablement: Configure the online state classification model output from step S33 to keep it active during online prediction;

[0040] S42 Online Diagnosis and Random Forward Propagation: The real-time data stream of the full-band acoustic time-series signal and real-time spindle power time-series data acquired in step S11 is subjected to a complete preprocessing and feature enhancement process according to step S2, continuously generating a real-time multi-channel fusion sequence dataset.

[0041] S43 Bayesian approximation and dual-threshold decision-making: Parallel uncertainty assessment and decision-making are performed on the real-time data in step S42. Monte Carlo sampling and probability distribution calculation are performed to output multiple different state label probability distributions. Statistical indicators are calculated based on the state label probability distributions to calculate uncertainty indicators. The average needle breakage probability and prediction uncertainty indicator are output in real time. Then, a dual-pre-support decision-making and alarm fusion mechanism is executed to determine the current state of the drill bit and output the corresponding early warning signal.

[0042] Based on the above technical solution, the Monte Carlo sampling and probability distribution calculation process is as follows: a real-time six-channel fusion time series matrix and the corresponding real-time operating condition label are input into the online state classification model configured by S41, and V random forward propagation are performed continuously. The Dropout layer activated by S41 is used to output multiple different state label probability distributions.

[0043] The uncertainty index calculation process is as follows: collect the probability distributions of multiple generated state labels, calculate their statistical indicators, and output two core metrics, namely the average needle breakage probability and the prediction uncertainty indicator; the average needle breakage probability is obtained by calculating the mean of multiple probability distributions, representing the model's average judgment on the occurrence of needle breakage; the prediction uncertainty indicator is obtained by calculating the variance of multiple probability distributions, quantifying the model's confidence in the current judgment.

[0044] Based on the above technical solution, the dual-threshold decision and alarm fusion mechanism is as follows:

[0045] When the average needle breakage probability calculated in real time is greater than a preset high-confidence needle breakage threshold, the system determines that a needle breakage has occurred regardless of the prediction uncertainty and immediately triggers an emergency stop signal for the machine tool.

[0046] When the average needle breakage probability is within a preset medium-level threshold range, but its prediction uncertainty indicator is significantly higher than the uncertainty baseline threshold, the system determines that the drill bit is in an unstable early degradation state that the model has not seen, and outputs a tool change warning signal.

[0047] Based on the above technical solution, the uncertainty baseline threshold is obtained through uncertainty baseline calibration, including the following process:

[0048] From the multi-channel fusion sequence dataset in step S24, all samples of normal drilling status are selected. For each normal sample, the probability distribution calculation and uncertainty index calculation process in step S43 above are performed to obtain its predicted uncertainty indicator value. After calculating the uncertainty index of all normal samples, its overall distribution is statistically analyzed, and a high quantile is set as the uncertainty baseline threshold.

[0049] The beneficial effects of the technical solution provided by this invention are as follows:

[0050] Compared with existing technologies, the beneficial effects of this application are mainly reflected in the following aspects:

[0051] This invention constructs a multimodal, multi-condition time-series dataset and enhances its multi-scale features. It collects and fuses two types of multimodal time-series information: full-band acoustic signals and real-time spindle power data, constructing a multi-condition, multi-state dataset of broken needles in micro-drilling. Simultaneously, through multi-scale resampling technology, the one-dimensional signal is converted into a six-channel fused time-series matrix integrating the original signal, low-frequency trend data, and high-frequency energy data, enhancing the characterization ability of transient impacts and slow-changing trends in broken needles.

[0052] Based on the cross-attention working condition-time series dual-stream fusion model, this invention constructs a working condition-time series dual-stream fusion architecture. The core is the design of a dedicated working condition-time series cross-attention fusion device, which realizes the active query and weighting of high-dimensional time series feature sequences by the working condition context vector, integrates drilling environment parameters into the feature extraction process, and improves the generalization of the model under different processing conditions and the accuracy of broken needle identification.

[0053] By combining Monte Carlo Dropout's prediction uncertainty assessment with dual-threshold decision-making, this invention configures Monte Carlo Dropout during the online detection deployment phase to estimate the model's prediction uncertainty. By calculating the average needle breakage probability and prediction uncertainty indicator, and applying dual-threshold decision fusion rules, it can not only achieve emergency shutdown for high-confidence needle breakage but also provide early degradation warnings based on high uncertainty, greatly improving the robustness and practicality of detection. Attached Figure Description

[0054] Figure 1 This is a schematic diagram illustrating the construction process of the multimodal time-series dataset in this invention;

[0055] Figure 2 This is a schematic diagram of the process of signal preprocessing and feature enhancement for multimodal time series datasets in this invention;

[0056] Figure 3 This is a schematic diagram of the architecture of the working condition-time sequence dual-stream fusion needle breakage detection model in this invention;

[0057] Figure 4 This is a schematic diagram of the online detection deployment and prediction uncertainty assessment process in this invention;

[0058] Figure 5 This is a comparison diagram of the time-series waveforms after time-series feature enhancement by multi-scale resampling in this invention;

[0059] Figure 6 This is a performance comparison chart of the dual-stream fusion model and the single-stream model under different operating conditions in this invention; Detailed Implementation

[0060] The present invention will be further described below with reference to the accompanying drawings and embodiments:

[0061] In this invention, unless otherwise explicitly specified and limited, the terms "installation," "connection," "linking," and "fixing," etc., should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral part; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal communication of two components or the interaction between two components. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0062] In the description of this invention, it should be understood that the terms "left", "right", "front", "rear", "top", "bottom", etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing this invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this invention.

[0063] like Figures 1 to 6 As shown, a method for detecting broken micro-drill bits based on multimodal information fusion includes the following steps:

[0064] S1: Collect full-band acoustic timing signals and real-time spindle power timing data, label them with multiple operating conditions and multiple states, and construct a multimodal timing dataset;

[0065] S2: Perform signal preprocessing and feature enhancement on the multimodal time series dataset, and output a multi-channel fused sequence dataset;

[0066] S3: Based on the cross-attention mechanism, a dual-stream fusion model for detecting broken needles, combining working conditions and time series, is constructed. This model is trained and optimized using a multi-channel fusion sequence dataset to enable active querying and weighting of time series features based on working condition information, accurately identifying the broken needle state, and outputting the final optimized online state classification model.

[0067] S4: Online detection deployment and prediction uncertainty assessment; Deploy the online state classification model output from step S3 to the actual processing, and achieve robust detection of drill bit breakage and early degradation by assessing the uncertainty of the model prediction.

[0068] This invention proposes a micro-drill bit broken needle detection method based on multimodal information fusion, which can achieve deep fusion of multimodal signals, actively incorporate working condition information, effectively address data imbalance, and has the ability to assess predictive uncertainty. Specifically, a multimodal time-series dataset is first constructed by acquiring full-band acoustic time-series signals and real-time spindle power time-series data through non-invasive sensors, and labeling them with multiple working conditions and states. Then, multi-scale resampling technology is used to preprocess and enhance the features of the original signals, generating six sequences including the original sequence, low-frequency trend sequence, and high-frequency energy sequence, which are then fused into a six-channel fused time-series matrix through channel splicing. More importantly, this application constructs a working condition-time-series dual-stream fusion model based on cross-attention. This model includes independent time-series feature encoders and working condition context encoders. The core is to use a working condition-time-series cross-attention fusion processor to actively query and weight time-series features based on working condition information. This model uses a focus loss function to perform end-to-end supervised training while overcoming class imbalance. Finally, during the online detection deployment phase, Monte Carlo Dropout is used to assess predictive uncertainty, and a dual-threshold decision based on the average needle breakage probability and predictive uncertainty indicator is combined to achieve robust detection and alarm for needle breakage and early degradation.

[0069] Based on the above technical solution, the construction of the multimodal time series dataset in step S1 includes the following steps:

[0070] S11 Multimodal Sensor Deployment and Data Acquisition: A high-fidelity industrial microphone is installed outside the spindle box to non-contactly acquire full-band acoustic timing signals during the drilling process; real-time spindle power timing data driving the spindle servo motor is synchronously acquired through the built-in data interface of the machine tool CNC system and an external power analysis module.

[0071] S12 Multi-condition Experiment Matrix Design: The micro drill bit is subjected to drilling experiments under various working conditions such as different spindle speeds, different feed rates, and different workpiece materials. Multi-modal sensor data of the micro drill bit under different health states, such as normal drilling state and drill bit breakage state, are collected and labeled with working condition labels and status labels.

[0072] Preferably, the multimodal sensor data under different health states includes: normal drilling state: baseline data of stable drilling holes by a brand new or well-maintained micro drill bit under all working conditions; drill bit breakage state: brittle fracture of the drill bit induced by fatigue drilling or under controlled conditions, and high-speed acquisition of multimodal time-series data before and after the fracture instant.

[0073] S13 Data Integration: Integrate the data obtained in steps S11 and S12 to finally output a structured multimodal time series dataset with operating conditions and status labels.

[0074] This step aims to construct a multimodal time-series dataset to provide sufficient data for subsequent deep learning time-series model training, enabling it to accurately distinguish between normal drilling and momentary needle breakage. The specific process is as follows: Figure 1As shown, the process mainly includes three steps: First, multimodal sensor deployment and data acquisition: On a precision micro-drilling machining center test bench, non-invasive multiphysics sensors are deployed and information is acquired for key signal sources in the drilling process. Second, multi-condition experimental matrix design: To ensure the model can adapt to the varied working conditions of micro-drilling in real machining, this step simulates typical parameter combinations on the test bench. Through the CNC system, the micro-drill bit is controlled to conduct drilling experiments under various spindle speeds, various feed rates, and different workpiece materials. Under each of the above working condition combinations, multimodal sensor data of the micro-drill bit in different health states are collected, and each data point is accurately labeled with its corresponding working condition label and status label. Third, data integration: This step integrates the data from S11 and S12, ultimately outputting a structured multimodal time-series dataset with operating condition and status labels. Specifically, it includes two types of data: multimodal time-series data and label data. The multimodal time-series data includes full-band acoustic timing signals and real-time spindle power timing data. The label data includes operating condition labels and status labels. The operating condition labels indicate the specific spindle speed, feed rate, and workpiece material corresponding to each data acquisition. The status labels indicate the drill bit health status corresponding to each data, including normal drilling status and drill bit breakage status.

[0075] Based on the above technical solution, the construction of the multi-channel fusion sequence dataset in step S2 includes the following steps:

[0076] S21 Time Series Data Segmentation and Sample Construction: The two original signals in the multi-modal time series dataset of step S1, namely the full-band acoustic time series signal and the real-time spindle power time series data, are segmented, and the resulting signal segments are combined to form original multi-modal samples; at the same time, the original multi-modal samples are assigned the operating condition label and status label corresponding to their acquisition time.

[0077] Preferably, the segmentation operation in step S21 includes the following process:

[0078] A fixed-length overlapping sliding window method is adopted, with a sampling window of fixed length T and an overlap rate of M, to synchronously slide and cut multimodal time series data. Each slide extracts two synchronous signal segments from two signal sources, including an acoustic signal segment and a power signal segment.

[0079] S22 Temporal feature enhancement based on multi-scale resampling: The one-dimensional signal segment in step S21 is preserved as the original sequence; the original sequence is filtered by a low-pass filter and a fixed downsampling factor is used to extract the filtered sequence to obtain a short-term trend sequence; then, the short-term trend sequence is upsampled and restored to a fixed length T of the original one-dimensional signal segment using linear interpolation to obtain a low-frequency trend sequence.

[0080] A bandpass filter is used to filter the one-dimensional signal segment to extract the target frequency band signal sensitive to the broken needle event. The Tiger-Kaiser energy operator is then performed on the target frequency band signal to obtain the instantaneous energy sequence. Boundary filling processing is then performed to obtain the high-frequency energy sequence.

[0081] Considering that the broken needle signal exhibits characteristics at different frequency scales, this step performs multi-scale sequence generation on the two one-dimensional signal segments within each original multimodal sample in S21; specifically:

[0082] First, for each signal segment, its original sequence is preserved, namely the acoustic original sequence and the power original sequence.

[0083] Secondly, a Butterworth low-pass filter of a specific order is used, and a low cutoff frequency is set to filter the one-dimensional signal segment. A fixed downsampling factor N is used to extract the filtered sequence to obtain a short time-series trend sequence. The short time-series trend sequence is then upsampled to restore the original one-dimensional signal segment to a fixed length T using linear interpolation, thereby obtaining the low-frequency trend sequence, namely the acoustic low-frequency trend sequence and the power low-frequency trend sequence.

[0084] Finally, to isolate the high-frequency impact signal, a bandpass filter of a specific order and cutoff frequency range is used to filter the one-dimensional signal segment to extract the target frequency band signal sensitive to the broken needle event. The Teager-Kaiser Energy Operator (TKEO) operation is then performed on this target frequency band signal. This operator is a nonlinear operator specifically designed to estimate the instantaneous energy of a signal and is highly sensitive to transient impacts caused by broken needles. This operation converts the target frequency band signal into an instantaneous energy sequence. This sequence is close to zero when the signal is stable, but generates an energy peak at the moment of the broken needle impact. Boundary padding is performed on the processed instantaneous energy sequence to obtain the final high-frequency energy sequence, ensuring its length strictly matches the original one-dimensional signal segment. This yields the acoustic high-frequency energy sequence and the power high-frequency energy sequence. Therefore, this step ultimately yields the original acoustic sequence, the acoustic low-frequency trend sequence, the acoustic high-frequency energy sequence, the original power sequence, the power low-frequency trend sequence, and the power high-frequency energy sequence.

[0085] S23 Multimodal and multiscale sequence fusion: The acoustic raw sequence, acoustic low-frequency trend sequence, acoustic high-frequency energy sequence, power raw sequence, power low-frequency trend sequence, and power high-frequency energy sequence obtained in step S22 are spliced ​​together to form a six-channel fusion time matrix.

[0086] This step integrates the separated sequences from multiple sources and scales in S22 into a unified feature representation. Specifically, channel splicing is used to splice the six sequences corresponding to the same original multimodal sample in S21 along the channel dimension to form a six-channel fusion time series matrix.

[0087] S24 Multi-channel fusion sequence dataset output: The six-channel fusion time series matrix generated in step S23 and the set of working condition labels and status labels corresponding to the matrix in step S21 are combined to obtain a structured preprocessed sequence dataset, namely the multi-channel fusion sequence dataset.

[0088] This step ultimately outputs a structured preprocessed sequence dataset, which will serve as the final input to the S3 deep learning model. This dataset consists of two parts: six-channel fused sequence data and label data. The six-channel fused sequence data includes a large number of six-channel fused time-series matrices generated in step S23, while the label data includes the operating condition labels and status labels retained in step S21, corresponding one-to-one with each six-channel fused time-series matrix.

[0089] This step involves signal preprocessing and multi-scale sequence construction. It aims to preprocess and feature-engineer the multimodal time-series dataset output by S13, transforming the original one-dimensional time-series signal into a fused sequence that highlights multi-scale dynamic characteristics and is suitable for use as input to time-series models. The specific workflow of this step is as follows: Figure 2 As shown.

[0090] Based on the above technical solution, the construction of the working condition-timing dual-stream fusion needle breakage detection model in step S3 includes the following steps:

[0091] S31 Dual-Stream Cross-Attention Model Construction: The dual-stream cross-attention model includes a temporal feature encoder, a working condition context encoder, a working condition-temporal cross-attention fusion unit, and a state classifier. The input of the temporal feature encoder is the six-channel fused temporal matrix in step S24, used to capture the temporal dependencies within the temporal matrix and output a high-dimensional temporal feature sequence. The input of the working condition context encoder is the working condition label in step S13, mapping the discrete working condition label to a dense working condition context vector. The input of the working condition-temporal cross-attention fusion unit is the high-dimensional temporal feature sequence and the working condition context vector, outputting a weighted fusion vector that integrates working condition information. The input of the state classifier is the weighted fusion vector of the working condition, outputting the probability distribution of the corresponding state label.

[0092] S32 End-to-end supervised training strategy and optimization: The dual-stream cross-attention model constructed in step S31 is trained end-to-end using the multi-channel fused sequence dataset from step S24. The focus loss function is used and a modulation factor is introduced to concentrate computational resources and optimization gradients on the difficult-to-classify and scarce broken needle samples. Through backpropagation, all parameters of the temporal feature encoder, working condition context encoder, cross-attention fusion unit and state classifier are optimized simultaneously, so that the model can accurately locate broken needle features from complex temporal signals by using working condition information while overcoming data imbalance.

[0093] This step utilizes the preprocessed sequence dataset from S24 to perform end-to-end supervised training on the model defined in S31. The training process faces an extreme class imbalance challenge: the number of broken needle samples is extremely small compared to normal drilling samples. To address this, this step employs a focus loss function instead of the standard classification cross-entropy loss function. This function introduces a modulation factor, causing the model to automatically reduce its focus on the massive number of easily classifiable normal drilling samples during training, concentrating computational resources and gradient optimization on the difficult-to-classify and scarce broken needle samples. The training process optimizes this focus loss function, which calculates the difference between the state classifier's prediction and the true state label. Through backpropagation, all parameters of the temporal feature encoder, the working condition context encoder, the cross-attention fusion unit, and the state classifier are simultaneously optimized, forcing the model to learn how to accurately locate broken needle features from complex temporal signals using working condition information, while overcoming data imbalance.

[0094] S33 Consolidation and Output of Online State Classification Model: After the supervised training in step S32 converges, all parameter weights of the optimized complete dual-stream cross-attention model in step S31 are consolidated as the final online state classification model.

[0095] The model is encapsulated and saved as the final online state classification model of this invention, and delivered to step S4 for online detection and deployment.

[0096] This step is the core step of the micro-drill bit broken needle detection method of this invention. It aims to train a deep fusion model using the preprocessed sequence dataset output by S24. The core objective of this model is to construct two independent encoding streams to process time-series information and working condition information respectively, and to achieve active querying and weighting of time-series features based on working conditions through a dedicated cross-attention module, thereby realizing broken needle identification. The key model architecture and process of this step are as follows: Figure 3 As shown.

[0097] In step S31, a dual-stream fusion model architecture consisting of the above four core components is constructed. The specific structure of the core components is described in detail below.

[0098] Based on the above technical solution, the temporal feature encoder includes a one-dimensional convolutional layer and a Transformer encoder layer consisting of N stacked layers.

[0099] Each Transformer encoder layer consists of two core sub-layers, and the output of each sub-layer is appended with a residual connection and subjected to layer normalization. The first core sub-layer is a multi-head self-attention module, which captures global temporal dependencies by calculating the correlation weights between all time steps within the sequence. The output of this sub-layer is then residually connected to the input and subjected to layer normalization. The second core sub-layer is a position feedforward network, consisting of two linear transformation layers and a ReLU activation function. This network is applied independently at each time step of the sequence. The output of this sub-layer is also residually connected to the input and subjected to layer normalization.

[0100] Based on the above technical solution, the working condition context encoder includes a working condition embedding layer, and the specific mapping process is as follows:

[0101] First, the input working condition labels are decomposed into categorical parameters and numerical parameters. Second, the categorical parameters are processed through independent embedding layers to obtain categorical embedding vectors. The numerical parameters are normalized and then mapped to numerical embedding vectors through fully connected layers to ensure that they are in the same feature space as the categorical embeddings. Finally, all independent categorical embedding vectors and numerical embedding vectors are concatenated and fused to form the final dense working condition context vector.

[0102] The working condition context vector is a fixed-length output and fully encodes all the key parameters of the current drilling environment.

[0103] Based on the above technical solution, the working condition-time series cross-attention fusion processor includes a cross-attention layer, and the construction process of the cross-attention layer includes the following steps:

[0104] First, the input working condition context vector and time series feature sequence are projected through independent linear transformation matrices to generate query matrix Q, key matrix K, and value matrix V. Among them, query matrix Q is obtained by projecting the working condition context vector through linear transformation, while key matrix K and value matrix V are both obtained by projecting the time series feature sequence through their respective independent linear transformations.

[0105] Secondly, the attention weight matrix is ​​obtained by calculating the dot product of the query matrix Q and the key matrix K, scaling the matrix, and then applying the Softmax function.

[0106] Finally, the attention weight matrix and the value matrix V are weighted and summed to obtain the output of the attention layer. This output is then processed by residual connection and layer normalization to form the final working condition weighted fusion vector.

[0107] The working condition-time series cross-attention fusion fusion builder constructs a dedicated cross-attention layer, using the working condition context vector as the query and the time series feature sequence as both key and value. This allows the model to actively query the time series features most relevant to the current working condition and perform weighted aggregation, ultimately outputting a working condition weighted fusion vector that has fully integrated the working condition information.

[0108] Based on the above technical solution, the state classifier includes a fully connected layer and an output layer arranged sequentially. The fully connected layer includes 128 neurons, a ReLU activation function, and a Dropout layer. The output layer includes two neurons, corresponding to the normal drilling state and the drill bit breakage state, respectively, and uses the Softmax activation function to output the probability distribution of the corresponding state labels.

[0109] Based on the above technical solution, step S4, online detection deployment and prediction uncertainty assessment, includes the following steps:

[0110] S41 Monte Carlo Dropout Model Enablement: Configure the online state classification model output from step S33 to keep it active during online prediction;

[0111] This step configures the online state classification model, which was solidified in step S33 above, for use in the prediction phase. Specifically, it modifies the model's inference behavior so that it remains active during online prediction of the Dropout layer defined in the state classifier in S3.1, instead of being turned off as usual.

[0112] S42 Online Diagnosis and Random Forward Propagation: The real-time data stream of the full-band acoustic time-series signal and real-time spindle power time-series data acquired in step S11 is subjected to a complete preprocessing and feature enhancement process according to step S2, continuously generating a real-time multi-channel fusion sequence dataset.

[0113] This step is the real-time operation phase of the online state classification model. First, a real-time data processing flow is executed, using the industrial microphones and power analysis module deployed in S11 to acquire full-band acoustic timing signals and real-time spindle power timing data in real time. Second, the acquired real-time data stream is continuously processed according to the complete preprocessing flow defined in S21, S22, and S23 to generate a real-time six-channel fusion timing matrix.

[0114] S43 Bayesian approximation and dual-threshold decision-making: Parallel uncertainty assessment and decision-making are performed on the real-time data in step S42. Monte Carlo sampling and probability distribution calculation are performed to output multiple different state label probability distributions. Statistical indicators are calculated based on the state label probability distributions to calculate uncertainty indicators. The average needle breakage probability and prediction uncertainty indicator are output in real time. Then, a dual-pre-support decision-making and alarm fusion mechanism is executed to determine the current state of the drill bit and output the corresponding early warning signal.

[0115] This step performs parallel uncertainty assessment and decision-making on the real-time data generated by S42. This step first defines the core computational process: firstly, Monte Carlo sampling and probability distribution generation; secondly, uncertainty index calculation. The specific process is described below. During online execution, for each piece of real-time data generated by S42, the above-defined stochastic inference and probability distribution generation and uncertainty index calculation processes are executed in real time to obtain its average needle breakage probability and prediction uncertainty indicator.

[0116] This step aims to deploy the online condition classification model output by S33 into actual machining, and to achieve robust detection of drill bit breakage and early degradation by evaluating the uncertainty of the model's predictions. The specific flowchart is as follows: Figure 4 As shown.

[0117] Based on the above technical solution, the Monte Carlo sampling and probability distribution calculation process is as follows: a real-time six-channel fusion time series matrix and the corresponding real-time operating condition label are input into the online state classification model configured by S41, and V random forward propagation are performed continuously. The Dropout layer activated by S41 is used to output multiple different state label probability distributions.

[0118] The uncertainty index calculation process is as follows: collect the probability distributions of multiple generated state labels, calculate their statistical indicators, and output two core metrics, namely the average needle breakage probability and the prediction uncertainty indicator; the average needle breakage probability is obtained by calculating the mean of multiple probability distributions, representing the model's average judgment on the occurrence of needle breakage; the prediction uncertainty indicator is obtained by calculating the variance of multiple probability distributions, quantifying the model's confidence in the current judgment.

[0119] Based on the above technical solution, the dual-threshold decision and alarm fusion mechanism is as follows:

[0120] When the average needle breakage probability calculated in real time is greater than a preset high-confidence needle breakage threshold, the system determines that a needle breakage has occurred regardless of the prediction uncertainty and immediately triggers an emergency stop signal for the machine tool.

[0121] When the average needle breakage probability is within a preset medium-level threshold range, but its prediction uncertainty indicator is significantly higher than the uncertainty baseline threshold, the system determines that the drill bit is in an unstable early degradation state that has not been seen in the model. At this time, a tool change warning signal is output to prompt the operator to change the tool preventively.

[0122] Based on the above technical solution, the uncertainty baseline threshold is obtained through uncertainty baseline calibration, including the following process:

[0123] From the multi-channel fusion sequence dataset in step S24, all samples of normal drilling status are selected. For each normal sample, the probability distribution calculation and uncertainty index calculation process in step S43 above are performed to obtain its predicted uncertainty indicator value. After calculating the uncertainty index of all normal samples, its overall distribution is statistically analyzed, and a high quantile is set as the uncertainty baseline threshold.

[0124] The uncertainty baseline calibration operation is set to be performed before the online classification model is formally deployed to actual processing.

[0125] Experimental Results and Analysis

[0126] To verify the effectiveness of the proposed multimodal information fusion-based micro-drill bit broken needle detection method, this experiment focuses on micro-drilling, encompassing multimodal time-series data including full-band acoustic signals, real-time spindle power, multiple operating parameters including spindle speed, feed rate, workpiece material, and drill bit status (normal, broken). The experiment emphasizes the effectiveness of multi-scale feature enhancement, the performance of the operating condition-time-series dual-stream fusion model, and the robustness of prediction uncertainty assessment. The aim is to comprehensively evaluate the accuracy and practical performance of the proposed method for broken needle detection under complex and variable operating conditions.

[0127] 1. Validation of the effectiveness of multi-scale sequence construction and feature enhancement

[0128] The experiment first verified the effectiveness of the multi-scale sequence construction and feature enhancement strategy proposed in step S2. By converting the original signal into a six-channel fused time-series matrix that integrates the original sequence, low-frequency trend sequence, and high-frequency energy sequence, the aim was to enhance the characterization of transient impacts and slowly changing trends of the broken needle. To visually demonstrate the highlighting effect of different scale sequences on the broken needle characteristics, this embodiment selects a typical sample containing a broken needle event and visualizes its acoustic signal time-series waveforms at three scales, as shown below. Figure 5 As shown in the figure. The horizontal axis represents the time step, and the vertical axis represents the signal amplitude. The different sub-graphs from top to bottom show the original acoustic sequence, the acoustic low-frequency trend sequence, and the acoustic high-frequency energy sequence, respectively. The vertical dashed line in the figure marks the approximate time point when the needle breakage event occurred.

[0129] Experimental results show that the original sequence contains the transient impact of the broken needle, but is also affected by background noise and low-frequency motion trends, resulting in a low signal-to-noise ratio. The low-frequency trend sequence effectively filters out high-frequency noise and highlights the slow energy change trend before and after the needle breakage, which helps to identify early degradation. The high-frequency energy sequence generates a significant and sharp energy peak at the moment of needle breakage, and has extremely high sensitivity and instantaneous localization capability for the broken needle impact signal. This verifies the effective enhancement effect of the Tiger-Kaiser energy operator combined with bandpass filtering on the transient characteristics of the broken needle.

[0130] 2. Performance Analysis and Cross-Attention Contribution of the Operating Condition-Time Sequence Dual-Stream Fusion Model

[0131] To verify the generalization and recognition accuracy of the condition-time dual-stream fusion model based on cross-attention in step S3 under multiple conditions, this experiment evaluated its performance on a test set covering different spindle speeds, feed rates, and workpiece materials. Simultaneously, by removing the condition context encoder and cross-attention fusion module, an ablation model based solely on the time-series feature encoder, i.e., a single-stream model, was constructed to compare and analyze the effect of fused condition information.

[0132] The experiment used the F1 score of needle breakage detection under different operating conditions as the performance index, and used stacked bar charts to compare the performance of the dual-flow fusion model and the single-flow model in three typical operating condition combinations: Condition 1: high speed / high feed / material A; Condition 2: low speed / low feed / material B; Condition 3: medium speed / variable feed / material C. Figure 6 As shown, the horizontal axis represents different test condition combinations; the vertical axis represents the F1 score of the broken needle detection; and the bars of different colors represent the single-stream model and the dual-stream fusion model, respectively.

[0133] Experimental results show that the single-stream model has a certain detection capability under conditions such as working condition 1, but its F1 score drops significantly under working conditions 2 and 3, which differ greatly from the training set distribution, indicating insufficient generalization. The dual-stream fusion model achieves significantly higher F1 scores than the single-stream model in all three different working condition combinations. Particularly in working conditions 2 and 3, the dual-stream model shows the greatest performance improvement. The results verify that the working condition-temporal cross-attention fusion processor effectively realizes the active querying and weighting of temporal features based on working condition context information, thereby greatly improving the model's adaptability to complex and changing working conditions and the accuracy of broken needle identification, demonstrating the superiority and beneficial effects of the dual-stream fusion architecture of this invention.

[0134] The foregoing has shown and described the basic principles and main features of the present invention. It will be apparent to those skilled in the art that the present invention is not limited to the details of the above exemplary embodiments. Therefore, the embodiments should be considered as exemplary and not restrictive. The scope of the present invention is defined by the appended claims rather than the foregoing description. Therefore, it is intended that all variations falling within the meaning and scope of the equivalents of the claims be included within the present invention.

[0135] Furthermore, it should be understood that although this specification describes embodiments, not every embodiment contains only one independent technical solution. This narrative style is merely for clarity. Those skilled in the art should consider the specification as a whole, and the technical solutions in each embodiment can also be appropriately combined to form other embodiments that can be understood by those skilled in the art.

Claims

1. A method for detecting broken needles in micro drill bits based on multimodal information fusion, characterized in that, Includes the following steps: S1: Collect full-band acoustic timing signals and real-time spindle power timing data, label them with multiple operating conditions and multiple states, and construct a multimodal timing dataset; S2: Perform signal preprocessing and feature enhancement on the multimodal time series dataset, and output a multi-channel fused sequence dataset; S3: Based on the cross-attention mechanism, a dual-stream fusion model for detecting broken needles, combining working conditions and time series, is constructed. This model is trained and optimized using a multi-channel fusion sequence dataset to enable active querying and weighting of time series features based on working condition information, accurately identifying the broken needle state, and outputting the final optimized online state classification model. S4: Online detection deployment and prediction uncertainty assessment; Deploy the online state classification model output from step S3 to the actual processing, and achieve robust detection of drill bit breakage and early degradation by assessing the uncertainty of the model prediction.

2. The method for detecting broken micro-drill bits based on multimodal information fusion according to claim 1, characterized in that, The construction of the multimodal time series dataset in step S1 includes the following steps: S11 Multimodal Sensor Deployment and Data Acquisition: A high-fidelity industrial microphone is installed outside the spindle box to non-contactly acquire full-band acoustic timing signals during the drilling process; real-time spindle power timing data driving the spindle servo motor is synchronously acquired through the built-in data interface of the machine tool CNC system and an external power analysis module. S12 Multi-condition Experiment Matrix Design: The micro drill bit is subjected to drilling experiments under various working conditions such as different spindle speeds, different feed rates, and different workpiece materials. Multi-modal sensor data of the micro drill bit under different health states, such as normal drilling state and drill bit breakage state, are collected and labeled with working condition labels and status labels. S13 Data Integration: Integrate the data obtained in steps S11 and S12 to finally output a structured multimodal time series dataset with operating conditions and status labels.

3. The method for detecting broken micro-drill bits based on multimodal information fusion according to claim 1, characterized in that, The construction of the multi-channel fusion sequence dataset in step S2 includes the following steps: S21 Time Series Data Segmentation and Sample Construction: The two original signals in the multi-modal time series dataset of step S1, namely the full-band acoustic time series signal and the real-time spindle power time series data, are segmented, and the resulting signal segments are combined to form original multi-modal samples; at the same time, the original multi-modal samples are assigned the operating condition label and status label corresponding to their acquisition time. S22 Temporal feature enhancement based on multi-scale resampling: The one-dimensional signal segment in step S21 is preserved as the original sequence; the original sequence is filtered by a low-pass filter and a fixed downsampling factor is used to extract the filtered sequence to obtain a short-term trend sequence; then, the short-term trend sequence is upsampled and restored to a fixed length T of the original one-dimensional signal segment using linear interpolation to obtain a low-frequency trend sequence. A bandpass filter is used to filter the one-dimensional signal segment to extract the target frequency band signal sensitive to the broken needle event. The Tiger-Kaiser energy operator is then performed on the target frequency band signal to obtain the instantaneous energy sequence. Boundary filling processing is then performed to obtain the high-frequency energy sequence. S23 Multimodal and multiscale sequence fusion: The acoustic raw sequence, acoustic low-frequency trend sequence, acoustic high-frequency energy sequence, power raw sequence, power low-frequency trend sequence, and power high-frequency energy sequence obtained in step S22 are spliced ​​together to form a six-channel fusion time matrix. S24 Multi-channel fusion sequence dataset output: The six-channel fusion time series matrix generated in step S23 and the set of working condition labels and status labels corresponding to the matrix in step S21 are combined to obtain a structured preprocessed sequence dataset, namely the multi-channel fusion sequence dataset.

4. The method for detecting broken micro-drill bits based on multimodal information fusion according to claim 1, characterized in that, The construction of the working condition-time sequence dual-stream fusion needle breakage detection model in step S3 includes the following steps: S31 Dual-Stream Cross-Attention Model Construction: The dual-stream cross-attention model includes a temporal feature encoder, a working condition context encoder, a working condition-temporal cross-attention fusion unit, and a state classifier. The input of the temporal feature encoder is the six-channel fused temporal matrix in step S24, used to capture the temporal dependencies within the temporal matrix and output a high-dimensional temporal feature sequence. The input of the working condition context encoder is the working condition label in step S13, mapping the discrete working condition label to a dense working condition context vector. The input of the working condition-temporal cross-attention fusion unit is the high-dimensional temporal feature sequence and the working condition context vector, outputting a weighted fusion vector that integrates working condition information. The input of the state classifier is the weighted fusion vector of the working condition, outputting the probability distribution of the corresponding state label. S32 End-to-end supervised training strategy and optimization: The dual-stream cross-attention model constructed in step S31 is trained end-to-end using the multi-channel fused sequence dataset from step S24. The focus loss function is used and a modulation factor is introduced to concentrate computational resources and optimization gradients on the difficult-to-classify and scarce broken needle samples. Through backpropagation, all parameters of the temporal feature encoder, working condition context encoder, cross-attention fusion unit and state classifier are optimized simultaneously, so that the model can accurately locate broken needle features from complex temporal signals by using working condition information while overcoming data imbalance. S33 Consolidation and Output of Online State Classification Model: After the supervised training in step S32 converges, all parameter weights of the optimized complete dual-stream cross-attention model in step S31 are consolidated as the final online state classification model.

5. The micro-drill bit broken needle detection method based on multimodal information fusion according to claim 4, characterized in that, The temporal feature encoder includes a one-dimensional convolutional layer and a Transformer encoder layer consisting of N stacked layers; Each Transformer encoder layer consists of two core sub-layers, and the output of each sub-layer is appended with a residual connection and subjected to layer normalization. The first core sub-layer is a multi-head self-attention module, which captures global temporal dependencies by calculating the correlation weights between all time steps within the sequence. The output of this sub-layer is then residually connected to the input and subjected to layer normalization. The second core sublayer is a position feedforward network consisting of two linear transformation layers and a ReLU activation function. This network is applied independently at each time step of the sequence. The output of this sublayer is also residually connected to the input and then processed by layer normalization.

6. The micro-drill bit broken needle detection method based on multimodal information fusion according to claim 4, characterized in that, The operating condition context encoder includes an operating condition embedding layer, and the specific mapping process is as follows: First, the input working condition labels are decomposed into categorical parameters and numerical parameters. Second, the categorical parameters are processed through independent embedding layers to obtain categorical embedding vectors. The numerical parameters are normalized and then mapped to numerical embedding vectors through fully connected layers to ensure that they are in the same feature space as the categorical embeddings. Finally, all independent categorical embedding vectors and numerical embedding vectors are concatenated and fused to form the final dense working condition context vector.

7. The micro-drill bit broken needle detection method based on multimodal information fusion according to claim 4, characterized in that, The working condition-time series cross-attention fusion processor includes a cross-attention layer, and the construction process of the cross-attention layer includes the following steps: First, the input working condition context vector and time series feature sequence are projected through independent linear transformation matrices to generate query matrix Q, key matrix K, and value matrix V. Among them, query matrix Q is obtained by projecting the working condition context vector through linear transformation, while key matrix K and value matrix V are both obtained by projecting the time series feature sequence through their respective independent linear transformations. Secondly, the attention weight matrix is ​​obtained by calculating the dot product of the query matrix Q and the key matrix K, scaling the matrix, and then applying the Softmax function. Finally, the attention weight matrix and the value matrix V are weighted and summed to obtain the output of the attention layer. This output is then processed by residual connection and layer normalization to form the final working condition weighted fusion vector.

8. The micro-drill bit broken needle detection method based on multimodal information fusion according to claim 4, characterized in that, The state classifier includes a fully connected layer and an output layer arranged sequentially. The fully connected layer includes 128 neurons, a ReLU activation function, and a Dropout layer. The output layer includes two neurons, corresponding to the normal drilling state and the drill bit breakage state, respectively, and uses a Softmax activation function to output the probability distribution of the corresponding state labels.

9. The method for detecting broken micro-drill bits based on multimodal information fusion according to claim 1, characterized in that, Step S4, online detection deployment and prediction uncertainty assessment, includes the following steps: S41 Monte Carlo Dropout Model Enablement: Configure the online state classification model output from step S33 to keep it active during online prediction; S42 Online Diagnosis and Random Forward Propagation: The real-time data stream of the full-band acoustic time-series signal and real-time spindle power time-series data acquired in step S11 is subjected to a complete preprocessing and feature enhancement process according to step S2, continuously generating a real-time multi-channel fusion sequence dataset. S43 Bayesian approximation and dual-threshold decision-making: Parallel uncertainty assessment and decision-making are performed on the real-time data in step S42. Monte Carlo sampling and probability distribution calculation are performed to output multiple different state label probability distributions. Statistical indicators are calculated based on the state label probability distributions to calculate uncertainty indicators. The average needle breakage probability and prediction uncertainty indicator are output in real time. Then, a dual-pre-support decision-making and alarm fusion mechanism is executed to determine the current state of the drill bit and output the corresponding early warning signal.

10. The method for detecting broken micro-drill bits based on multimodal information fusion according to claim 9, characterized in that, The Monte Carlo sampling and probability distribution calculation process is as follows: a real-time six-channel fusion time series matrix and the corresponding real-time operating condition label are input into the online state classification model configured by S41, and V random forward propagation are performed continuously. The Dropout layer activated by S41 is used to output multiple different state label probability distributions. The uncertainty index calculation process is as follows: collect the probability distributions of multiple generated state labels, calculate their statistical indicators, and output two core metrics, namely the average needle breakage probability and the prediction uncertainty indicator; the average needle breakage probability is obtained by calculating the mean of multiple probability distributions, representing the model's average judgment on the occurrence of needle breakage; the prediction uncertainty indicator is obtained by calculating the variance of multiple probability distributions, quantifying the model's confidence in the current judgment.

11. The method for detecting broken micro-drill bits based on multimodal information fusion according to claim 9, characterized in that, The dual-threshold decision and alarm fusion mechanism is as follows: When the average needle breakage probability calculated in real time is greater than a preset high-confidence needle breakage threshold, the system determines that a needle breakage has occurred regardless of the prediction uncertainty and immediately triggers an emergency stop signal for the machine tool. When the average needle breakage probability is within a preset medium-level threshold range, but its prediction uncertainty indicator is significantly higher than the uncertainty baseline threshold, the system determines that the drill bit is in an unstable early degradation state that the model has not seen, and outputs a tool change warning signal.

12. The micro-drill bit broken needle detection method based on multimodal information fusion according to claim 11, characterized in that, The uncertainty baseline threshold is obtained through uncertainty baseline calibration. Includes the following processes: From the multi-channel fusion sequence dataset in step S24, all samples of normal drilling status are selected. For each normal sample, the probability distribution calculation and uncertainty index calculation process in step S43 above are performed to obtain its predicted uncertainty indicator value. After calculating the uncertainty index of all normal samples, its overall distribution is statistically analyzed, and a high quantile is set as the uncertainty baseline threshold.

Images