Milling multi-modal chatter detection method and system for small sample scenarios

Abstract

The milling multi-modal chatter detection method and system for small sample scene solve the problem of how to realize stable and reliable cutting chatter state discrimination under the condition of limited labeled data, and belong to the technical field related to intelligent manufacturing and robot machining.The present application comprises: acquiring real time-frequency data;building a discriminator and a generator, the generator containing learnable architecture parameters, which are used to adaptively adjust the contribution weight of the multi-scale feature extraction branch;building a multi-modal classifier.Three-stage collaborative training: (1) update the generator and discriminator weights through adversarial training, and fix the architecture parameters; (2) train the classifier with real and synthetic data; (3) fix the weights, and use the loss of the classifier on the real data as the optimization objective to update the architecture parameters in reverse, so that the generator structure evolves towards the direction conducive to chatter recognition.Cycle to convergence, and obtain the classifier for online detection.

Description

Technical Field

[0001] This application relates to a method and system for detecting multimodal chatter in milling for small sample scenarios, belonging to the technical field of intelligent manufacturing and robotic processing. Background Technology

[0002] With the continuous development of intelligent manufacturing technology, robotic milling, due to its advantages such as large workspace, high flexibility, and relatively low system cost, has gradually become an important supplement to traditional CNC machine tool milling in the processing of complex structures and large components. However, the multi-degree-of-freedom serial structure of robots leads to problems such as low rigidity, making them more prone to chatter instability during milling compared to CNC machine tools, significantly affecting machining accuracy, surface quality, and tool life. Therefore, chatter detection in robotic milling is crucial for ensuring machining quality and improving equipment safety.

[0003] The invention patent with publication number CN108296881A, entitled "An Online Monitoring Method for Milling Chatter," proposes a chatter monitoring method based on wavelet transform to decompose the cutting force signal and establish a multivariate regression model to achieve online monitoring of chatter during the milling process. The invention patent with publication number CN112529099A, entitled "A Chatter Identification Method for Robot Milling," proposes a chatter identification method for robot milling. This method involves acquiring images of the workpiece surface morphology during robot milling and inputting them into a chatter type prediction model to obtain the robot chatter type. The chatter type prediction model is trained using a public dataset and can identify chatter types including stable, excessive, regular, and irregular chatter. However, it cannot distinguish between low-frequency and high-frequency chatter dominated by different modes and is difficult to achieve online detection of chatter in robot milling.

[0004] However, in real-world robotic milling scenarios, chatter samples are often sporadic and condition-dependent, leading to high acquisition costs and difficult annotation. This small sample size problem has thus become a key bottleneck restricting the application of multimodal chatter detection methods. To alleviate the limitations of model training under small sample conditions, generative models, especially Generative Adversarial Networks (GANs), have been gradually introduced into chatter detection and fault diagnosis in recent years for data augmentation and sample expansion. However, in small sample scenarios, this generative strategy, which focuses solely on data augmentation, cannot guarantee the effectiveness of synthesized samples near the discrimination boundary. Summary of the Invention

[0005] To address the problem of how to achieve stable and reliable cutting chatter state discrimination under limited labeled data conditions, this application provides a milling multimodal chatter detection method and system for small sample scenarios.

[0006] This application discloses a method and system for detecting multimodal chatter in milling operations in small-sample scenarios, comprising:

[0007] Obtain real samples of multi-source signals and perform preprocessing to obtain a real time-frequency dataset;

[0008] A discriminator and a multi-scale conditional generator with learnable architecture parameters are constructed, wherein the learnable architecture parameters are used to adaptively adjust the structural weights of the multi-scale feature extraction branches within the generator;

[0009] A multimodal signal fusion classifier was built to simultaneously perform two tasks: signal mode discrimination and flutter recognition.

[0010] The model is optimized using a three-stage collaborative training strategy, which includes:

[0011] Phase 1: Adversarial training is performed on the discriminator and the multi-scale conditional generator, the weight parameters of the discriminator and the multi-scale conditional generator are updated, the learnable architecture parameters are fixed, and a synthetic time-frequency dataset is obtained.

[0012] The second stage involves training the multimodal signal fusion classifier using both the real time-frequency dataset and the synthetic time-frequency dataset, and then updating the weight parameters of the multimodal signal fusion classifier.

[0013] The third stage: fix the weight parameters of the multi-scale conditional generator and the weight parameters of the multi-modal signal fusion classifier, take the classification loss of the multi-modal signal fusion classifier on the real dataset as the optimization objective, and inversely update the learnable architecture parameters of the multi-scale conditional generator so that the generator structure evolves into a form that is conducive to flutter recognition.

[0014] The above three stages are repeated until convergence, resulting in the final multimodal signal fusion classifier, which is used for online flutter detection.

[0015] Preferably, the multimodal signal fusion classifier includes a shared feature extraction module, a modal discrimination branch, and a flutter recognition branch;

[0016] The shared feature extraction module extracts shared features from the real time-frequency dataset and the synthetic time-frequency dataset, and these shared features are simultaneously input to the modality discrimination branch and the flutter recognition branch.

[0017] The modality discrimination branch is used to determine the sensing mode type of the input signal;

[0018] The flutter recognition branch is used to process the shared features and the intermediate features extracted from the intermediate layer of the modality discrimination branch, so that the flutter recognition branch can perceive the differences in feature expression under different sensing modalities.

[0019] As a preferred embodiment, the multi-scale condition generator includes an upsampling block, a multi-scale feature extraction unit, and an output block;

[0020] Upsampling blocks are used to align features and improve resolution between the input noise vector and the conditional vector in both spatial and channel dimensions. The feature map is gradually reorganized and expanded through sub-pixel convolutional layers.

[0021] The multi-scale feature extraction unit is used to simultaneously extract local fine time-frequency structure and global spectral distribution features in the reconstructed extended feature map by setting convolutional branches of different scales in parallel, and to introduce learnable architecture parameters to adaptively adjust the contribution weight of convolutional branches at each scale to obtain multi-scale time-frequency features.

[0022] The output block is used to map multi-scale time-frequency features into synthetic time-frequency data under corresponding conditions.

[0023] As a preferred approach, the multi-scale condition generator employs reverse updating, which involves updating the multi-scale condition generator parameters. Under fixed conditions, for learnable architectural parameters Conduct training:

[0024]

[0025] In the formula, This represents the updated learnable architecture parameters. Indicates the learnable architecture parameters The generated data is used to train the classifier network weights. Represents classification loss, This represents a real dataset consisting of actual time-frequency data and corresponding conditional vectors.

[0026] Preferably, the multi-scale feature extraction unit adopts a feature shrinkage-expansion structure, which includes three parallel convolutional branches, using small-scale convolutional kernels, medium-scale convolutional kernels and large-scale convolutional kernels respectively, to obtain compressed multi-scale time-frequency features;

[0027] Subpixel convolutional layers are used to restore the spatial resolution of compressed multi-scale time-frequency features, and learnable architecture parameters are introduced to adaptively adjust the contribution weights of convolutional branches at each scale to obtain multi-scale time-frequency features.

[0028] As a preferred option, the upsampling block uses subpixel convolutional layers instead of transposed convolutions.

[0029] Preferably, the discriminator includes an upsampling block, a convolution block, and an output block;

[0030] The upsampling block is used to perform feature mapping and upsampling on the time-frequency data and the conditional vector respectively, and to fuse the two features in the channel dimension to obtain the fused features; the time-frequency data is synthetic time-frequency data generated by the multi-scale conditional generator or real time-frequency data;

[0031] Convolutional blocks are used to further extract and discriminate relevant features from the fused features;

[0032] The output block is used to map the discriminative features to the probability that the time-frequency data originates from the real data distribution; the discriminator calculates the adversarial loss by comparing the output probabilities of the real time-frequency data and the synthetic time-frequency data, thereby forcing the multi-scale conditional generator to generate synthetic time-frequency data that is closer to the real distribution.

[0033] Preferably, the loss function of the multimodal signal fusion classifier during training is:

[0034]

[0035] In the formula, To represent a real dataset consisting of actual time-frequency data and corresponding conditional vectors, The high-quality dataset generated by the multi-scale condition generator. Represents classification loss, Weight parameters of a multimodal signal fusion classifier This represents the weight parameters of the updated multimodal signal fusion classifier. , This represents the contribution weight of real time-frequency data and synthetic time-frequency data to the training of the multimodal signal fusion classifier.

[0036] Preferably, the discriminator and the multi-scale condition generator are subjected to adversarial training to update the weight parameters of the discriminator and the multi-scale condition generator:

[0037]

[0038] in, , This indicates the changes in learnable architecture parameters before and after the update. Weight parameters of a fixed-scale conditional generator , This represents the weight parameters of the discriminator before and after the update. To represent a real dataset consisting of actual time-frequency data and corresponding conditional vectors, It signifies resistance to loss.

[0039] This application also provides a milling multimodal chatter detection system for small sample scenarios, including a storage device, a processor, and a computer program stored in the storage device and executable on the processor. The processor executes the computer program to implement the steps of the above-described milling multimodal chatter detection method for small sample scenarios.

[0040] The beneficial effects of this application are that by placing sample generation and chatter detection within a unified collaborative learning system, it achieves stable identification of cutting chatter states under limited labeled data conditions, providing an effective solution for chatter detection during robotic milling.

[0041] This application proposes a low-data-volume cutting state monitoring model that combines a conditional generative network (GDN) and a learnable network architecture. This effectively alleviates the data shortage problem under small sample conditions and significantly improves the recognition ability in such situations. The application employs a training strategy where the classifier loss is used to back-update the generator's learnable network architecture. This ensures that the update of the learnable network architecture parameters does not depend on a single input sample, but is driven by the classifier task during global training. This allows the generator to gradually update to a structure conducive to cutting state recognition, improving the quality of synthetic data and its distribution approximation ability. Furthermore, this application constructs a multi-task classifier that jointly performs signal domain discrimination and cutting state recognition, enabling the sharing and fusion of multimodal features and improving the model's cross-modal generalization ability. Simultaneously, this application explicitly introduces intermediate features extracted from the modality discrimination branch into the cutting state recognition branch, allowing the classifier to perceive differences in feature representation under different sensing modalities when performing stability discrimination, thereby improving the discrimination robustness under multi-sensor modal conditions. Attached Figure Description

[0042] Figure 1 This is a flowchart of the method described in this application;

[0043] Figure 2 This is a schematic diagram of the multi-scale condition generator in this application;

[0044] Figure 3 This is a schematic diagram of the discriminator in this application;

[0045] Figure 4 This is a schematic diagram of the multimodal signal fusion classifier in this application;

[0046] Figure 5 This is a simplified diagram of the device according to an embodiment of this application;

[0047] Figure 6 The loss curves and accuracy curves are shown in the training and validation sets of the ablation experiments in this application.

[0048] Figure 7 This illustrates the identification of vibration signals in the embodiments of this application.

[0049] Figure 8 This illustrates the identification of force signals in the embodiments of this application. Detailed Implementation

[0050] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0051] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other.

[0052] The present application will be further described below with reference to the accompanying drawings and specific embodiments, but this is not intended to limit the scope of the application.

[0053] This embodiment of the milling multimodal chatter detection method and system for small sample scenarios includes:

[0054] Step 1: Obtain real samples of multi-source signals and perform preprocessing to obtain a real time-frequency dataset;

[0055] Specifically, the vibration signal and force signal are acquired by two independent acquisition systems. Since their sampling start times and trigger delays cannot be fully synchronized, affecting the correspondence of cutting state characteristics, cross-device signal timing alignment is first implemented. To ensure the accuracy of the alignment results, a timing synchronization method based on cross-correlation is used to accurately align the vibration signal and force signal. Assuming the vibration signal is x(t) and the force signal is y(t), although they are sampled asynchronously, they both reflect the same cutting characteristics in the physical process. Therefore, the time offset between them can be calculated from the location of the maximum value of the cross-correlation function. The cross-correlation function is defined as follows:

[0056]

[0057] In the formula For time delay. When When the maximum value is reached, it indicates that the signal... and The similarity is the highest at this time. This is the optimal alignment offset between the two signals, i.e.:

[0058]

[0059] Obtain the optimal alignment offset Then, the force signal can be offset to achieve time alignment with the vibration signal. It should be noted that the sampling frequencies of the force signal and the vibration signal are different. Therefore, the force signal needs to be upsampled before the offset is calculated. To ensure the continuity of the waveform, the invention uses linear interpolation to construct a mapping function to realize the upsampling of the force signal. The original force signal is time-offset by the offset between the upsampled force signal and the vibration signal.

[0060] After alignment, a sliding window is used to extract signal segments from the vibration and force signals. To further extract the local frequency characteristics during the cutting process, short-time Fourier transforms (STFTs) are performed on the aligned vibration and force signals to obtain their time-frequency distributions. The mathematical expression for STFT is:

[0061]

[0062] In the formula The window function is centered at t, controlling the integration range. The characteristics of each signal in both time and frequency dimensions are obtained through STFT, and further extraction is performed. As a time-frequency graph of the signal, construct a real time-frequency dataset.

[0063] Step 2: First, build a conditional adversarial generative network, including a discriminator and a multi-scale conditional generator with learnable architecture parameters. The learnable architecture parameters are used to adaptively adjust the structural weights of the multi-scale feature extraction branch inside the multi-scale conditional generator. Then, build a multi-modal signal fusion classifier to perform two tasks simultaneously: signal mode discrimination and flutter recognition.

[0064] Step 21: Multi-scale condition generator It consists of trainable parameters Differentiable functions composed of, where , , These are the generated data space, the conditional vector space, and the noise vector space, respectively. These are the weight parameters for the multi-scale conditional generator. Given a prior noise distribution... and condition vector The multi-scale condition generator can be defined as follows:

[0065]

[0066] In the formula, x, y, and z represent the generated data, condition vector, and noise vector of the multi-scale condition generator, respectively.

[0067] Discriminator The trainable weight parameters are The network architecture is fixed and can be defined as:

[0068]

[0069] In the formula The output of the discriminator represents the probability that the sample comes from the true distribution.

[0070] The training objective of the multi-scale conditional generator is to generate generated data that approximates the true distribution as closely as possible to "deceive" the discriminator. The discriminator's objective is to distinguish between real data and generated data as much as possible. The two interact and ultimately improve the quality of data generation.

[0071] The inputs to the multi-scale condition generator include Follows a standard Gaussian distribution , a noise vector z of size (512,1,1), and a condition vector consisting of signal modes (0: vibration signal, 1: force signal), cutting state labels (0: stable cutting, 1: unstable cutting), and normalized cutting parameters (rotation speed, feed rate, depth of cut).

[0072] The multi-scale condition generator in this embodiment includes an upsampling block, a multi-scale feature extraction unit, and an output block. The upsampling block performs feature alignment and resolution enhancement on the input noise vector and condition vector in both spatial and channel dimensions, and gradually reconstructs and expands the feature map through sub-pixel convolutional layers. The multi-scale feature extraction unit extracts local fine time-frequency structure and global spectral distribution features from the reconstructed and expanded feature map by setting convolutional branches of different scales in parallel, and introduces learnable architecture parameters to adaptively adjust the contribution weights of convolutional branches at each scale to obtain multi-scale time-frequency features. The output block maps the multi-scale time-frequency features to synthetic time-frequency data under the corresponding conditions.

[0073] The upsampling block comprises two parallel branches, used to align the input random noise vector z and the conditional vector y in the spatial and channel dimensions, respectively. Each branch consists of five sub-pixel convolutional layers, with a basic structure of Convolution–Pixel Shuffle–Batch Normalization–ReLU, used to progressively reorganize and expand features in the spatial and channel dimensions. In SConv, the kernel size of the convolutional layers is set to (3,3), the stride is 1, and the padding is 1; the upsampling ratio of the pixel shuffle operation is 2. Compared to transposed convolution, SConv can effectively fuse channel information while expanding the spatial resolution of the feature map, and avoids the checkerboard artifacts that may be introduced by transposed convolution, thereby improving the stability and continuity of the learned features.

[0074] The multi-scale feature extraction unit, serving as the backbone network of the multi-scale conditional generator, learns local fine-grained time-frequency structures and global spectral distribution features simultaneously by setting convolutional branches with different receptive fields in parallel. This enables feature extraction of multi-modal milling signals across multiple time and frequency scales. The multi-scale feature extraction unit adopts a feature contraction-expansion structure. In the feature contraction stage, the noise vector and conditional vector, after being fused by upsampling blocks, enter three parallel convolutional branches. These branches use small-scale (conv_220), medium-scale (conv_421), and large-scale (conv_622) convolutional kernels to compress the multi-scale time-frequency features. The basic structure of each branch is Convolution–BatchNormalization–ReLU. The last three digits in conv_220, conv_421, and conv_622 represent the kernel size, stride, and padding parameters, respectively. The feature expansion stage is similar to the contraction stage, using SConv instead of conventional convolutional layers to restore the spatial resolution of the compressed multi-scale time-frequency features.

[0075] This application introduces structure-level learnable parameters into the multi-scale feature extraction unit. This is used to allocate the contribution levels of features at different scales. The above multi-scale feature extraction design enables the multi-scale condition generator to fully characterize the key time-frequency modes related to cutting stability, laying a structural foundation for generating high-quality multimodal signal samples.

[0076] The final output block consists of five convolutional layers. In the last convolutional layer, the ReLU activation function is replaced with Sigmoid, which maps the extracted features to the [0,1] interval, and finally outputs time-frequency data of (3,32,32).

[0077] The discriminator in this embodiment includes an upsampling block, a convolutional block, and an output block. The upsampling block performs feature mapping and upsampling on the time-frequency data and the conditional vector, respectively, and fuses the two features in the channel dimension to obtain the fused features. The time-frequency data is either synthetic time-frequency data generated by the multi-scale conditional generator or real time-frequency data. The convolutional block further extracts discriminative features from the fused features. The output block maps the discriminative features to the probability that the time-frequency data originates from the real data distribution. The discriminator calculates the adversarial loss by comparing the output probabilities of the real time-frequency data and the synthetic time-frequency data, thereby forcing the multi-scale conditional generator to generate synthetic time-frequency data that more closely approximates the real distribution.

[0078] Specifically, the structure of the upsampling block is similar to that of the multi-scale conditional generator, containing two parallel branches: for the conditional vector y, a 5-layer transposed convolutional module (Transposed Convolution–Batch Normalization–ReLU) is used to upsample it in both the spatial and channel dimensions; while for the input time-frequency data x, feature mapping is performed through convolutional layers. The feature maps from the two branches are fused in the channel dimension and then input into a convolutional feature extraction block (Conv block) consisting of three convolutional layers to further extract discriminative features.

[0079] Furthermore, the fused features are mapped via a linear output block (Flatten–Dropout–Linear) to finally output the probability that the input sample originates from the real data distribution. The training objective of the multi-scale conditional generator is to generate synthetic samples that approximate the real data distribution as closely as possible to confuse the discriminator's discrimination results; while the discriminator strives to distinguish between real samples and generated samples as accurately as possible under given constraints. Through the adversarial training process between the multi-scale conditional generator and the discriminator, the model gradually improves the authenticity and discriminative consistency of the generated data, thereby improving the overall data generation quality. Step 22: In this implementation, a multi-modal signal fusion classifier is used to simultaneously perform two tasks: sensor modality discrimination and flutter recognition.

[0080] The multimodal signal fusion classifier of this embodiment includes a shared feature extraction module, a modal discrimination branch, and a flutter recognition branch. The shared feature extraction module extracts shared features from the real time-frequency dataset and the synthetic time-frequency dataset, and these shared features are simultaneously input to the modal discrimination branch and the flutter recognition branch. The modal discrimination branch determines the sensing mode type of the input signal. The flutter recognition branch processes the intermediate features extracted from the shared features and the intermediate layer of the modal discrimination branch, enabling the flutter recognition branch to perceive the differences in feature expression under different sensing modes.

[0081] Specifically, the modal discrimination task is used to distinguish whether the input signal originates from a vibration or force sensor, while the chatter recognition task is used to determine whether the current cutting process is stable.

[0082] The multimodal signal fusion classifier in this embodiment For input data Through learnable parameters The task of achieving two probability outputs has a fixed network architecture and can therefore be defined as:

[0083]

[0084] In the formula Let represent the output probabilities of Task 1 (sensor mode discrimination) and Task 2 (flutter recognition), respectively. , , Represent θ C The parameters are specific to Task 1, specific to Task 2, and shared. These are intermediate parameters for Task 1 introduced in Task 2.

[0085] The multimodal signal fusion classifier in this embodiment completes two tasks through several convolutional and linear layers. Task 1 determines the signal mode of the input data, i.e., vibration or force signal, to distinguish multimodal data. Task 2 is used to determine whether the current signal cutting state is stable (stable: 0, unstable: 1). To fully utilize modal information, this embodiment explicitly introduces the intermediate features extracted from the modal discrimination branch into the cutting state recognition branch, enabling the classifier to perceive the differences in feature expression under different sensing modes when performing stability discrimination, thereby improving the discrimination robustness under multi-sensor modal conditions.

[0086] Step 3: Optimize the model using a three-stage collaborative training strategy. The three-stage collaborative training strategy includes:

[0087] This step involves training through a collaborative learning process of generation and recognition, namely "adversarial generation-multimodal classification-inverse update of multi-scale condition generator architecture," to achieve coordinated optimization of data generation and cutting state recognition.

[0088] Phase 1: Adversarial training is performed on the discriminator and the multi-scale conditional generator to update the weight parameters of the discriminator and the multi-scale conditional generator, fix the learnable architecture parameters, and obtain a synthetic time-frequency dataset; specifically, based on the comprehensive changes of multi-scale time-frequency features exhibited by cutting stability in multi-modal signals, and combined with the given signal modes, cutting states, and processing parameters, corresponding multi-modal time-frequency data is generated to enhance the diversity of training samples;

[0089] Specifically, the training objective of the multi-scale conditional generator network is to generate synthetic data that approximates the true distribution. This process can be described using a minimax optimization problem, with the following calculation formula:

[0090]

[0091] In the formula, Indicates the learnable architecture parameters The weight parameters of the multi-scale conditional generator are fixed for updating. This represents the weight parameters of the updated discriminator network. This represents a real dataset consisting of actual time-frequency data and corresponding conditional vectors.

[0092] in, Optimization through adversarial generation aims to increase the similarity between generated and real samples. The parameters... This is used to characterize the structural weights of each scale branch within the MFEC in the multi-scale conditional generator, so as to adaptively adjust the relative contribution of multi-scale features in the generation process. It is important to emphasize that... The updates do not depend on individual input samples, but evolve continuously during global training, thus reflecting the relative importance of features at different scales in the cutting state recognition task. Parameters The optimization objective is not to directly minimize the difference between the generated samples and the real data distribution, but rather to be driven by the performance feedback of the downstream cutting state recognition task. Through this task-guided structural adaptation mechanism, the multi-scale condition generator can gradually evolve into a network structure that is more conducive to the discrimination of cutting stable states, thereby significantly improving the actual discriminative value of the generated data in small-sample learning scenarios.

[0093] The second stage involves training a multimodal signal fusion classifier using both real and synthetic time-frequency datasets, and updating the weight parameters of the multimodal signal fusion classifier. This stage includes two tasks: signal mode discrimination and cutting state recognition. By explicitly introducing intermediate features for mode discrimination into the cutting state recognition branch, multimodal signal constraints are provided for cutting state recognition. The classifier is then trained using synthetic data synthesized by a multi-scale condition generator.

[0094] Specifically, a multimodal signal fusion classifier is used for two tasks: signal mode discrimination and flutter recognition. The modal discrimination feature extraction is explicitly introduced into the flutter recognition branch, and it is trained together with a multi-scale conditional generator. The calculation formula is as follows:

[0095]

[0096] In the formula, For real datasets, High-quality datasets generated for multi-scale conditional generators. γ represents the updated classifier network weights, and γ1 and γ2 represent the contribution weights of real data and synthetic data to model training.

[0097] The third stage involves fixing the weight parameters of the multi-scale conditional generator and the multi-modal signal fusion classifier. Using the classification loss of the multi-modal signal fusion classifier on the real dataset as the optimization objective, the learnable architecture parameters of the multi-scale conditional generator are updated in reverse, causing the multi-scale conditional generator structure to evolve into a form conducive to flutter recognition. This is achieved by introducing structural-level learnable architecture parameters. The parameters of this part are adaptively adjusted to adjust the contribution of features at different scales when generating multimodal data. The update of these parameters is driven by the classifier task in global training, so that the multi-scale condition generator is gradually updated to a structure that is conducive to the recognition of cutting state.

[0098] The multi-scale condition generator uses inverse updates, which are based on the weight parameters of the multi-scale condition generator. Under fixed conditions, for learnable architectural parameters The goal of training is to learn the architectural parameters. The structure has been updated to better facilitate cutting state recognition tasks, and the calculation formula is as follows:

[0099]

[0100] In the formula, Indicates the updated learnable architecture parameters , Indicates the learnable architecture parameters The generated data is used to train the multi-scale conditional generator network weights.

[0101] The above three stages are repeated until convergence, resulting in the final multimodal signal fusion classifier, which is used for online flutter detection.

[0102] This embodiment provides a milling multimodal chatter detection system for small sample scenarios, including a storage device, a processor, and a computer program stored in the storage device and executable on the processor. When the processor executes the computer program, it implements the steps of the above-described milling multimodal chatter detection method for small sample scenarios. Specifically, after the processor loads and executes the program, it sequentially performs the following steps: acquiring real samples of multi-source signals and preprocessing them to obtain a real time-frequency dataset; constructing a multi-scale conditional generator, discriminator, and multimodal signal fusion classifier with learnable architecture parameters; iterating cyclically until convergence according to a three-stage collaborative training strategy (adversarial training to update the weights of the generator and discriminator, joint training of the classifier, and inverse updating the learnable architecture parameters of the generator with classification loss); and finally, using the trained multimodal signal fusion classifier to determine the chatter state of the real-time acquired milling signals and outputting the detection results. This system, through a combination of hardware and software, achieves stable and accurate detection of multimodal chatter in robot milling under small sample conditions.

[0103] Example: To obtain real samples of multi-source signals, signals from the industrial robot milling process are collected and a dataset is constructed. A robot milling machining system is then built, and a simplified diagram of the equipment is shown below. Figure 5 As shown.

[0104] The hardware components mainly include: Robot 1 (KUKA KR500 robot), electric spindle 2, Kistler 9257B force gauge 6, PCB triaxial piezoelectric accelerometer 3, and workpiece 5. The robot's end effector, the electric spindle 2, acts as the actuator. The accelerometer 3 is fixed to the electric spindle 2 and performs milling on the Al 5083 workpiece 5, which is fixed to the force gauge 6, measuring vibration and force signals during the cutting process. A 20mm diameter three-tooth carbide end mill 4 is used for slot milling. The sampling frequencies of the force gauge 6 and the accelerometer 3 are set to 5kHz and 10kHz, respectively. The cutting parameters are shown in Table 1.

[0105] Table 1 Cutting parameters

[0106]

[0107] Preprocessing is performed according to step 1. Signal segments are extracted from the aligned vibration signal and force signal using a sliding window. The sliding window length and step size for the vibration signal are 1024 and 256, respectively, and the sliding window length and step size for the force signal are 512 and 128, respectively, to obtain the real dataset.

[0108] In practice, a three-stage generation-recognition collaborative training process is conducted to achieve coordinated optimization of data generation and cutting state recognition. The first step involves training the conditional adversarial generative network. Using real-world conditional information and time-frequency data, the multi-scale conditional generator and discriminator are trained adversarially. During this stage, the learnable architecture parameters are temporarily fixed. Only update the network weights θ of the multi-scale conditional generator and discriminator. G With θ D The first step involves training a multi-scale conditional generator to gradually approximate the real time-frequency data distribution under given constraints. The second step trains a multi-modal signal fusion classifier by synthesizing a certain number of generated samples using the current multi-scale conditional generator, which are then used together with real time-frequency samples to form the classifier network weights θ. C Training; the third stage involves reverse updating of the learnable architecture parameters. The trained classifier is evaluated on a real dataset, and the classification loss is calculated to update the learnable architecture parameters. This allows for adaptive adjustment of the contribution of multi-scale features in the generation process, enabling the multi-scale condition generator structure to gradually evolve into a form more conducive to cutting state recognition.

[0109] To verify the contribution of each key module in the proposed method to the model performance, this application designed ablation experiments to analyze the training process, test set classification results, and generated data quality under different structural configurations. Four models were constructed for comparison, as shown in Table 2, as detailed below:

[0110] (1) MFECGAN-GR+MFSC is a complete model that includes a multimodal signal fusion classifier, a multi-scale generation structure based on MFEC, and a generation-recognition collaborative learning mechanism.

[0111] (2) MFECGAN+MFSC retains the multi-scale generative structure and multi-modal classifier, but does not introduce a generative-recognition collaborative mechanism;

[0112] (3) CGAN+MFSC retains the multimodal classifier, but the multiscale condition generator does not adopt the MFEC multiscale structure, nor does it perform collaborative updates. It uses the basic CNN as the main body of the multiscale condition generator.

[0113] (4) CGAN+CNN further replaces the multimodal fusion classifier with the basic CNN classification model as the baseline model.

[0114] The above four models constitute a comparative system from complete design to gradual simplification, which is conducive to separating and analyzing the impact of three types of designs—multi-scale generation, collaborative learning, and multi-modal fusion—on the final recognition performance.

[0115] Table 2 Comparison of Ablation Experimental Model Configurations

[0116]

[0117] like Figure 6 As shown, the loss and accuracy curves of different models on the training and validation sets exhibit significant differences. Overall, all four models can gradually update parameters and improve performance during training, but the complete model shows better convergence and stability during the training phase. In contrast, as key modules are gradually removed, the performance improvement of the model weakens in the later stages of training, and the fluctuations are more pronounced during the validation phase. Specifically, CGAN+CNN converges slower than other models. Under the same training time of 500 epochs, its loss curve descent gradient on the training set is significantly smaller, and the accuracy of Task 2 still shows an increasing trend, failing to converge to a stable value, indicating that the MFSC classifier can significantly improve the model's convergence speed. All four models achieve recognition accuracy close to 1 on Task 1, indicating that Task 1 is a simple task. Therefore, subsequent evaluations will mainly focus on the recognition metrics of Task 2. Furthermore, the introduction of MFEC and GR has been validated in improving the accuracy of Task 2 on the validation set. This indicates that multi-scale generative structures, generative-recognition collaborative mechanisms, and multimodal feature fusion all contribute to improving the model optimization process, enabling it to learn more stable and discriminative feature representations during the training phase. Since this application focuses on the model's generalization performance in real-world cutting state recognition tasks, further analysis of the actual recognition capabilities of each model will be conducted using test set results.

[0118] Table 3 presents the quantitative performance comparison results of the four models on the test set Task 2. It can be seen that the complete model MFECGAN-GR+MFSC achieves the best results in all four metrics (ACC and F1-score), reaching 0.9605 and 0.9596 respectively, representing improvements of 6.12% and 6.86% compared to the baseline. MFECGAN+MFSC is second, with corresponding metrics of 0.9458 and 0.9444. The performance of CGAN+MFSC and CGAN+CNN further declines. These results indicate that MFECGAN-GR+MFSC outperforms other structures in overall recognition accuracy, class balance recognition ability, and classification consistency.

[0119] Table 3. Ablation Experiment Test Set Indicators

[0120]

[0121] Furthermore, to verify the effectiveness of the proposed model in identifying cutting stability states during actual milling processes, this application processed and analyzed the original vibration and force signals from the complete milling process. First, a sliding window strategy consistent with the aforementioned training data was used to extract signal segments, where the sliding window length and step size for vibration signals were 1024 and 256, respectively, and for force signals, the sliding window length and step size were 512 and 128, respectively. The time-domain data within each sliding window was converted to a time-frequency domain image via STFT. To ensure the accuracy and consistency of the identification results, signals from the air-cutting stage were removed, and only signals actually involved in the cutting stage were input into the model for classification. The model was trained using a dataset containing 512 samples.

[0122] Specifically, Figure 7 The results of identifying the cutting state from the vibration signal are presented. It can be observed that the model achieves nearly 100% accuracy in identifying signal modes and possesses extremely high discrimination ability between stable and unstable states of the vibration signal. The unstable cutting state was identified as occurring in the interval from 2.93s to 29.86s, which is highly consistent with the original vibration signal and the characteristics such as amplitude abrupt changes and high-frequency energy enhancement in the corresponding time-frequency plot. This indicates that the model can effectively extract time-frequency modes related to the cutting state from the vibration signal, verifying the model's effectiveness in identifying the cutting state in the vibration domain.

[0123] Specifically, regarding the identification of force signals, as follows: Figure 8As shown, the instability state was detected at 2.95 s, which is 0.02 s later than the start time detected by the vibration signal, while the instability state was detected at 29.84 s, which is 0.02 s earlier than the vibration signal. This indicates that the model is more "aggressive" in identifying instability states based on force signals. The two types of signal outputs maintained a high degree of consistency, indicating that the model can effectively extract the characteristic patterns of unstable cutting from the physical signals of two typical modes, and has good cross-modal generalization ability.

[0124] This application addresses the challenges of scarce chatter samples, difficult annotation, and strong non-stationarity of multimodal signals during robotic milling. It proposes a multimodal chatter detection framework for small-sample scenarios. By placing the generative model and the chatter detection model within a unified collaborative learning system, this framework achieves stable identification of cutting chatter states even with limited labeled data, providing an effective solution for chatter detection in robotic milling.

[0125] While this application has been described herein with reference to specific embodiments, it should be understood that these embodiments are merely examples of the principles and applications of this application. Therefore, it should be understood that many modifications can be made to the exemplary embodiments, and other arrangements can be designed without departing from the spirit and scope of this application as defined by the appended claims. It should be understood that different dependent claims and features herein can be combined in ways different from those described in the original claims. It is also understood that features described in conjunction with individual embodiments can be used in other embodiments.

Claims

1. A method for detecting multimodal chatter in milling operations for small sample scenarios, characterized in that, include: Obtain real samples of multi-source signals and perform preprocessing to obtain a real time-frequency dataset; A discriminator and a multi-scale conditional generator with learnable architecture parameters are constructed, wherein the learnable architecture parameters are used to adaptively adjust the structural weights of the multi-scale feature extraction branches within the generator; A multimodal signal fusion classifier was built to simultaneously perform two tasks: signal mode discrimination and flutter recognition. The model is optimized using a three-stage collaborative training strategy, which includes: Phase 1: Adversarial training is performed on the discriminator and the multi-scale conditional generator, the weight parameters of the discriminator and the multi-scale conditional generator are updated, the learnable architecture parameters are fixed, and a synthetic time-frequency dataset is obtained. The second stage involves training the multimodal signal fusion classifier using both the real time-frequency dataset and the synthetic time-frequency dataset, and then updating the weight parameters of the multimodal signal fusion classifier. The third stage: fix the weight parameters of the multi-scale conditional generator and the weight parameters of the multi-modal signal fusion classifier, take the classification loss of the multi-modal signal fusion classifier on the real dataset as the optimization objective, and inversely update the learnable architecture parameters of the multi-scale conditional generator so that the generator structure evolves into a form that is conducive to flutter recognition. The above three stages are repeated until convergence, resulting in the final multimodal signal fusion classifier, which is used for online flutter detection.

2. The milling multimodal chatter detection method for small sample scenarios according to claim 1, characterized in that, The multimodal signal fusion classifier includes a shared feature extraction module, a modality discrimination branch, and a flutter recognition branch; The shared feature extraction module extracts shared features from the real time-frequency dataset and the synthetic time-frequency dataset, and these shared features are simultaneously input to the modality discrimination branch and the flutter recognition branch. The modality discrimination branch is used to determine the sensing mode type of the input signal; The flutter recognition branch is used to process the shared features and the intermediate features extracted from the intermediate layer of the modality discrimination branch, so that the flutter recognition branch can perceive the differences in feature expression under different sensing modalities.

3. The milling multimodal chatter detection method for small sample scenarios according to claim 1, characterized in that, The multi-scale condition generator includes an upsampling block, a multi-scale feature extraction unit, and an output block; Upsampling blocks are used to align features and improve resolution between the input noise vector and the conditional vector in both spatial and channel dimensions. The feature map is gradually reorganized and expanded through sub-pixel convolutional layers. The multi-scale feature extraction unit is used to simultaneously extract local fine time-frequency structure and global spectral distribution features in the reconstructed extended feature map by setting convolutional branches of different scales in parallel, and to introduce learnable architecture parameters to adaptively adjust the contribution weight of convolutional branches at each scale to obtain multi-scale time-frequency features. The output block is used to map multi-scale time-frequency features into synthetic time-frequency data under corresponding conditions.

4. The milling multimodal chatter detection method for small sample scenarios according to claim 3, characterized in that, The multi-scale condition generator employs a reverse update mechanism, which updates the multi-scale condition generator parameters. Under fixed conditions, for learnable architectural parameters Conduct training: ； In the formula, This represents the updated learnable architecture parameters. Indicates the learnable architecture parameters The generated data is used to train the classifier network weights. Represents classification loss, This represents a real dataset consisting of actual time-frequency data and corresponding conditional vectors.

5. The milling multimodal chatter detection method for small sample scenarios according to claim 3, characterized in that, The multi-scale feature extraction unit adopts a feature shrinkage-expansion structure. The multi-scale feature extraction unit includes three parallel convolutional branches, which respectively use small-scale convolutional kernels, medium-scale convolutional kernels, and large-scale convolutional kernels to obtain compressed multi-scale time-frequency features. Subpixel convolutional layers are used to restore the spatial resolution of compressed multi-scale time-frequency features, and learnable architecture parameters are introduced to adaptively adjust the contribution weights of convolutional branches at each scale to obtain multi-scale time-frequency features.

6. The milling multimodal chatter detection method for small sample scenarios according to claim 3, characterized in that, The upsampling block uses subpixel convolutional layers instead of transposed convolutions.

7. The milling multimodal chatter detection method for small sample scenarios according to claim 1, characterized in that, The discriminator includes an upsampling block, a convolution block, and an output block; The upsampling block is used to perform feature mapping and upsampling on the time-frequency data and the conditional vector respectively, and to fuse the two features in the channel dimension to obtain the fused features; the time-frequency data is synthetic time-frequency data generated by the multi-scale conditional generator or real time-frequency data; Convolutional blocks are used to further extract and discriminate relevant features from the fused features; The output block is used to map the discriminative features to the probability that the time-frequency data originates from the real data distribution; the discriminator calculates the adversarial loss by comparing the output probabilities of the real time-frequency data and the synthetic time-frequency data, thereby forcing the multi-scale conditional generator to generate synthetic time-frequency data that is closer to the real distribution.

8. The milling multimodal chatter detection method for small sample scenarios according to claim 1, characterized in that, The loss function for the multimodal signal fusion classifier during training is: ； In the formula, To represent a real dataset consisting of actual time-frequency data and corresponding conditional vectors, The high-quality dataset generated by the multi-scale condition generator. Represents classification loss, The weight parameters of a multimodal signal fusion classifier This represents the weight parameters of the updated multimodal signal fusion classifier. , This represents the contribution weight of real time-frequency data and synthetic time-frequency data to the training of the multimodal signal fusion classifier.

9. The milling multimodal chatter detection method for small sample scenarios according to claim 1, characterized in that, Adversarial training is performed on the discriminator and the multi-scale condition generator to update the weight parameters of the discriminator and the multi-scale condition generator: ； in, , This indicates the changes in learnable architecture parameters before and after the update. Weight parameters of a fixed-scale conditional generator , This represents the weight parameters of the discriminator before and after the update. To represent a real dataset consisting of actual time-frequency data and corresponding conditional vectors, It signifies resistance to loss.

10. A milling multimodal chatter detection system for small sample scenarios, comprising a storage device, a processor, and a computer program stored in the storage device and executable on the processor, characterized in that, The processor executes the computer program to implement the steps of the milling multimodal chatter detection method for small sample scenarios as described in any one of claims 1 to 9.

Images