A Sensing and Measurement Method for Material Removal Rate in Belt Abrasive Grinding Integrating Audiovisual Information

Abstract

This invention discloses a method for sensing and measuring the material removal rate in belt grinding by integrating audiovisual information. The method includes: 1) acquiring visual spark images and sound signals under a single working condition; 2) preprocessing the acquired signals; 3) establishing a machine learning ensemble tree model for sensing and measuring the material removal rate for sparse samples under the single working condition and a deep convolutional neural network model for sensing and measuring the material removal rate for dense samples; transferring the established single-working-condition model to multi-working-condition conditions; 4) establishing a hybrid transfer model based on feature and model transfer for sparse samples under multi-working-condition conditions, and establishing an adversarial adaptive deep transfer learning model for the lack of samples under multi-working-condition conditions. The method of this invention exhibits good transfer performance, saves the work of data acquisition and labeling under multiple working conditions, reduces the cost of measuring the material removal rate in belt grinding under multiple working conditions, and improves the generalization ability and robustness of the material removal rate sensing and measuring model.

Description

Technical Field

[0001] This invention relates to a method for sensing and measuring the material removal rate of belt grinding that integrates audiovisual information, belonging to the field of sensing and measuring the material removal rate of belt grinding. Background Technology

[0002] Belt grinding, as a precision and semi-precision machining process, is widely used in the machining of complex curved surfaces. Due to factors such as the elastic contact between the abrasive belt and the workpiece, the nonlinear wear of the abrasive belt, and mechanical errors, accurately controlling the material removal rate in belt grinding is one of the key issues restricting the process accuracy. Furthermore, accurately obtaining the material removal rate is the foundation and key to precise control.

[0003] Currently, there are two methods for obtaining the material removal rate in belt grinding: theoretical formula modeling and experimental data modeling. Theoretical formula modeling allows for the generation of an expression between grinding parameters and the material removal rate, enabling the calculation of the corresponding material removal rate. However, the material removal rate model obtained through theoretical formulas does not consider the nonlinear wear characteristics of the abrasive belt and the coupling between various grinding parameters, making accurate measurement difficult.

[0004] Chinese patent application (application number 201910906218.6) discloses a method for identifying the material removal rate in belt grinding based on spark images. This method only considers spark images for constructing the removal rate model. The inventors discovered the following problems during the implementation of this solution:

[0005] (1) Data-driven modeling can establish a relationship model between relevant information generated during grinding and the real-time material removal rate, thereby achieving accurate monitoring of the material removal rate. However, information from a single sensor is easily affected by external environmental interference during monitoring.

[0006] (2) The test set and training set of the material removal rate monitoring method based on data-driven modeling usually need to meet the same data distribution, that is, the operating conditions should be highly consistent. The data obtained by the same processing equipment under different operating conditions, and by different processing equipment under the same operating conditions, often have different distributions. This leads to a sharp decline in the generalization ability of existing models for different operating conditions. Summary of the Invention

[0007] The technical problem to be solved by this invention is to provide a method for sensing and measuring the material removal rate of belt grinding that integrates audiovisual information, so as to realize the intelligent detection of the nonlinear time-varying material removal rate during belt grinding, and solve the problems of large data requirements, poor generalization ability and high promotion cost of existing material removal rate monitoring methods based on data-driven modeling.

[0008] The technical solution adopted in this invention is: a method for sensing and measuring the material removal rate of belt grinding that integrates audiovisual information, the method comprising the following steps:

[0009] Step 1: Use a CCD industrial camera to capture visual spark images generated during the grinding process, and simultaneously use an acoustic sensor connected to a signal acquisition device to collect sound signals during the grinding process.

[0010] Among them, visual spark image signals and grinding sound information constitute audiovisual information;

[0011] Step 2: Preprocess the acquired spark images to filter out interfering background information; perform line-of-sight correction on the acquired spark images according to the change in the size of the contact wheel to obtain spark images obtained at the same line-of-sight, and then remove the spark background to remove background information and obtain only the information containing the spark image.

[0012] Step 3: Remove background noise from the collected audio, retaining the valid audio signal;

[0013] The collected sound information is corrected for propagation distance, and then background noise is removed to obtain sound information related to grinding.

[0014] Step 4: Establish a material removal rate sensing and measurement model based on a single working condition (same sanding belt specifications and same workpiece material);

[0015] Step 4.1: Divide the single-condition samples into sparse samples and dense samples based on the number of samples of collected audiovisual information;

[0016] Step 4.2: Establish an ensemble tree model based on machine learning algorithms for sparse audiovisual information samples;

[0017] Step 4.2.1: Extract features from the visual spark image after preprocessing in Step 2, including color features, texture features, shape features, and spatial relationship features.

[0018] Step 4.2.2: Extract time-domain, frequency-domain, and time-frequency-domain features from the effective grinding sound signal in Step 3;

[0019] Step 4.2.3: Perform feature selection and optimization on the visual spark image and grinding sound signal features extracted in Step 4.2.1 and Step 4.2.2 to obtain the feature subset most relevant to the material removal rate;

[0020] Step 4.2.4: Input the feature subset of audiovisual information obtained in Step 4.2.3 into the LightGBM-based ensemble tree model, adjust the learning rate, number of leaves, maximum depth of the model tree, and the number of feature selections and random resampling, and train the tree model.

[0021] Step 4.2.5: Compare the predicted material removal rate from the integrated tree model obtained in Step 4.2.4 with the measured material removal rate, and calculate the corresponding evaluation parameters;

[0022] Step 4.2.6: Using the values ​​of the evaluation parameters as the optimization objective, obtain the material removal rate perception measurement model based on audiovisual information of single-condition sparse samples.

[0023] Step 4.3: Establish a material removal rate perception measurement model based on deep learning algorithm for audiovisual information samples of dense samples under single working conditions;

[0024] Step 4.3.1: Convert the grinding sound signal obtained in Step 2 into a time-spectrum graph using Short Time Fourier Transform (SFT);

[0025] Step 4.3.2: Using the visual spark image obtained in Step 2 and the time-spectrum diagram of the grinding sound in Step 3.3.1 as raw data, input them into the self-constructed convolutional neural network model;

[0026] The training method for the self-constructed convolutional neural network model is as follows: A feature matrix map constructed from the features of spark images and sound signals under dense samples is input into the input layer. Through a series of operations such as convolution, normalization, activation functions, and pooling, the automatically extracted features are connected to the fully connected layer. The final material removal rate is obtained through a regression layer. The specific training process of the model is as follows:

[0027] (3) Construction of the four-dimensional array of the input layer

[0028] The training and test images are loaded in the form of a digitTrain4D-ArrayData four-dimensional array. In the process of constructing the four-dimensional array dataset, the first three columns are the length, width and height of the image, and the last column is the label material removal rate of the data. It should be defined as a cell array.

[0029] (4) Network training process

[0030] In constructing the convolutional neural network model, the stochastic gradient descent algorithm was used, whose momentum was driven by the trained optimization function. The batch size is the amount of data used in each iteration of network training, and its initial value was set to 30. MaxEpochs is the maximum number of training rounds, and its initial value was 200. The initial learning rate InitialLearnRate was set to 0.001. The dataset of single-condition samples was used for training.

[0031] (3) Training parameter optimization

[0032] Further fine-tuning of the batch size, MaxEpochs, and InitialLearnRate parameters in the training model was performed, with the batch size set to 1-100, MaxEpochs set to 100-5000, and the initial learning rate set to 0.0001-0.1. Based on the optimization process of the model training parameters, the dataset of single-condition samples was trained, and the performance parameters of different models were calculated until the optimal model parameters were found.

[0033] Step 4.3.3: Based on the input audiovisual information, perform convolution and pooling calculations using a self-constructed convolutional neural network model, then adjust the number of network layers and weight parameters, compare the perceived material removal rate with the measured material removal rate, and calculate the loss function.

[0034] Step 4.3.4: Using the loss function as the optimization objective, we obtain a deep learning material removal rate perception measurement model for audiovisual information based on dense samples.

[0035] Step 5: Establish a material removal rate perception and measurement model based on transfer learning for cross-working conditions (different abrasive belt specifications and different workpiece materials);

[0036] The material removal rate model under a single operating condition is referred to as the source domain model, and the material removal rate sensing and measurement model across operating conditions is referred to as the target domain model. The specific steps for transferring the material removal rate model from the source domain model to the target domain model are as follows:

[0037] Step 5.1: Divide the target domain samples into sparse samples and lacking samples based on the number of target domain samples across different working conditions;

[0038] Step 5.2: Establish a material removal rate sensing and measurement model for the target domain across operating conditions based on an integrated tree model for sparse samples, the steps are the same as in 4.2;

[0039] Step 5.3: Fine-tune the weight coefficients of the model tree in step 4.2.4 to obtain a cross-condition material removal rate sensing measurement model based on ensemble tree transfer learning under sparse samples;

[0040] Step 5.4: Establish a cross-condition adversarial adaptive deep transfer learning model based on the lack of samples. The specific method is as follows:

[0041] Step 5.4.1: For the lack of sample set, firstly, a generator and discriminator based on GAN network are established, and a generative adversarial network is established. Through continuous adversarial interaction between the generator and discriminator, simulated samples that are almost indistinguishable from real samples are generated.

[0042] Step 5.4.2: By continuously generating samples to reach a certain number of samples, establish a mixed sample set based on real samples and simulated samples;

[0043] Step 5.4.3: Take the mixed sample set obtained in step 5.3.2 as input and the convolutional neural network model constructed in step 3.3.3 as the source domain model;

[0044] Step 5.4.4: Fix the corresponding network in the convolutional network model with fixed depth, fine-tune the parameters of the input layer, output layer and fully connected layer to obtain the cross-condition material removal rate sensing measurement model based on transfer learning of convolutional neural network under the lack of samples;

[0045] Step 5.4.5: Using the cross-condition material removal rate sensing and measurement model obtained in Step 4.4.4, compare the sensing and measurement values ​​under cross-conditions with the measured material removal rate values, and calculate the corresponding evaluation parameters.

[0046] Step 5.4.6: Using the values ​​of the evaluation parameters as the optimization objective, we obtain a cross-condition material removal rate sensing measurement model based on transfer learning under the condition of lack of samples.

[0047] This invention provides a method for perceiving and measuring the material removal rate in cross-condition belt grinding based on audiovisual information and transfer learning, overcoming the problem of weak generalization ability of data-driven material removal rate perception and measurement models in belt grinding. It innovatively fuses visual spark image signals and auditory signals from the grinding process, establishing material removal rate perception and detection models based on audiovisual information fusion according to the number of data samples. Furthermore, based on the amount of labeled data in the target domain, it establishes a cross-condition transfer material removal rate perception and measurement model based on a hybrid of sparse sample feature transfer and model transfer, and a cross-condition adversarial adaptive deep transfer model for material removal rate under conditions of insufficient samples. By using transfer learning to establish a perception and measurement model for cross-condition (target domain) belt grinding from a single-condition (source domain) model, the generalization ability of audiovisual information-based material removal rate perception is greatly improved, enhancing the model's stability and robustness.

[0048] The beneficial effects of this invention are as follows: Compared with the prior art, this invention integrates audiovisual information from a single working condition with knowledge from a material removal perception and measurement model in belt grinding. In scenarios where sufficient samples are lacking across working conditions, a hybrid transfer method combining feature transfer and model transfer, along with the powerful learning capabilities of deep learning transfer, can be used to establish a material removal rate perception and measurement model across working conditions. This reduces the cost and time of data acquisition and data labeling across working conditions, resolves the contradiction between the large sample size required in data-driven modeling and the small sample size in actual production, and reduces the cost and time of establishing a material removal rate perception and measurement model under cross-working-condition belt grinding. This is of great significance for the application and promotion of audiovisual information monitoring of material removal rate perception and measurement models in belt grinding. Attached Figure Description

[0049] Figure 1 This is an overall flowchart of a cross-condition abrasive belt grinding material removal rate sensing and measurement method based on audiovisual information and transfer learning in Example 1;

[0050] Figure 2 This is a flowchart of feature extraction based on audiovisual information in Example 1;

[0051] Figure 3 It is a two-dimensional convolutional neural network structure based on audiovisual information. Detailed Implementation

[0052] The present invention will be further described below with reference to the accompanying drawings and specific embodiments.

[0053] Example 1: As Figures 1-3 As shown, a method for sensing and measuring the material removal rate of belt grinding that integrates audiovisual information includes the following steps:

[0054] Step 1: Collection of audiovisual information and material removal rate

[0055] A CCD industrial camera is used to capture visual spark images generated during the grinding process, while an acoustic sensor is connected to a signal acquisition device to collect grinding sound signals during the grinding process.

[0056] An industrial CCD camera is installed on the side of the grinding area to capture images of grinding sparks. The spark images captured by the CCD are then stored in a computer.

[0057] A sound sensor is mounted on the workbench and moves with it while remaining stationary relative to the workpiece. The sound sensor signal is sent to the data acquisition card, and then stored in the computer after digital-to-analog conversion.

[0058] The weight of the workpiece before and after grinding was measured using an electronic balance, and the material removal rate was calculated using formula (1):

[0059]

[0060] Z w The material removal rate is calculated, Δm is the change in workpiece weight before and after grinding, B is the width of the abrasive belt, and Δt is the grinding time.

[0061] The labels for the spark image and sound information within Δt are the actual calculated material removal rate Zw;

[0062] Given different parameters such as belt speed, workpiece feed speed, and grinding pressure (theoretical depth of cut), both visual and auditory signals were collected, and the material removal rate for each stroke was measured. This parameter variation constitutes a single working condition, with 40 sets of experiments under a single working condition. Among them, the theoretical depth of cut was 0.15mm-0.35mm, with an interval of 0.01mm, for a total of 20 sets; the workpiece feed speed was 4mm / s-8mm / s, with an interval of 0.5mm / s, for a total of 10 sets; and the belt speed was 20m / s-30m / s, with an interval of 1m / s, for a total of 10 sets.

[0063] Step 2: Preprocess the acquired visual spark images and audio information:

[0064] The acquired spark images are corrected for viewing distance based on the change in the size of the contact wheel to obtain spark images obtained at the same viewing distance. Then, the spark background is removed to remove background information and obtain only the information with spark images.

[0065] The acquired sound signal is corrected for propagation distance, and then background noise is removed by wavelet decomposition and reconstruction techniques to obtain the sound signal related to grinding information.

[0066] Step 3: Establish a single-condition material removal sensing measurement model based on audiovisual information fusion;

[0067] Machine learning and deep learning are both effective methods for building data-driven models. However, when the dataset is too small, deep learning cannot demonstrate its significant advantages. In building a single-condition material removal model based on audiovisual information fusion, the samples are divided into sparse and dense sample sets, and modeled separately for each.

[0068] We perform statistical analysis on the preprocessed signal and samples. Let the number of labeled samples be Ns, and the number of parameters to be trained in deep learning be N. D When Ns≤10N D When the material removal rate is at the optimal level, an ensemble tree from machine learning is used to build a material removal rate perception measurement model; otherwise, a convolutional neural network from deep learning is used to build a material removal rate perception measurement model.

[0069] Therefore, the specific steps for establishing a single-condition material sensory removal measurement model based on audiovisual information fusion are as follows:

[0070] 3.1) Establishment of a machine learning ensemble tree model for sensing and measuring material removal rate using sparse samples under single operating conditions; the specific steps are as follows:

[0071] 3.1.1) When Ns≤10N D At that time, feature extraction is performed on Ns samples, such as Figure 2 As shown. There are a total of 27 features;

[0072] 3.1.2) Calculate the Pierre coefficient between sample characteristics and material removal rate, as shown in formulas (2) and (3).

[0073]

[0074]

[0075] Where Cov(x,y) is the covariance of variables x and y; x and y are the means of variables x and y; N is the total number of samples; x i y i Let be the i-th data of variables x and y; σ x , σ y ρ represents the standard deviation of variables x and y. x,y The value of Pearson's coefficient is closer to 1, indicating a stronger linear correlation between the two variables.

[0076] Analyze the correlation between each feature and the material removal rate, and select the feature with the strongest correlation to form the optimal feature subset {X};

[0077] 3.1.3) Using the optimal feature subset {X} obtained in step 3.1.2 as input and the corresponding material removal rate output label as output, train the LightGBM model based on the ensemble tree, and adjust the hyperparameters in the model such as learning rate, maximum tree depth, and number of leaf nodes.

[0078] 3.1.4) Using 80% of the samples as the training set and 20% of the data as the test set, the material removal rate perception measurement model established in step 3.1.3 was evaluated. The evaluation parameters are shown in Table 3 below:

[0079] Table 1 Model Performance Evaluation Indicators

[0080]

[0081] Where n is the number of test samples, y i y i ' are the measured value and predicted value of the i-th sample, respectively;

[0082] 3.1.5) Using the model evaluation parameters in Table 1 as the basis for adjusting the model parameters, establish a perception measurement model for the material removal rate of single-condition belt grinding under sparse samples.

[0083] 3.2) Establishment of a deep convolutional neural network-based material removal rate sensing and measurement model for dense samples under single operating conditions; the specific steps are as follows:

[0084] 3.2.1) When Ns ≥ 10N D At that time, the time spectrum of the spark image and sound information after preprocessing in step two is directly used as input to train the self-constructed convolutional neural network model, and the size and number of parameters of the input layer, convolutional layer, pooling layer, fully connected layer and regression layer are adjusted.

[0085] The self-constructed convolutional neuron network model, such as Figure 3 As shown.

[0086] The input layer takes a feature matrix map constructed from visual spark images and sound signal features from dense samples. Through a series of operations including convolution, normalization, activation functions, and pooling, the automatically extracted features from the convolution are connected to a fully connected layer. The final material removal rate is then obtained through a regression layer. Table 2 provides detailed information on the parameters of each layer.

[0087] Table 2. Detailed information on the self-constructed convolutional neural network model.

[0088]

[0089] The training process of the model is as follows:

[0090] (1) Construction of the four-dimensional array of the input layer

[0091] Using convolutional neural networks to build a material removal rate sensing measurement model falls under the category of regression problems. Unlike classification problems, regression problems require constructing a four-dimensional array, meaning that in addition to the length, width, and height of the input image, each input image needs to be labeled with the material removal rate.

[0092] This invention loads training and test images in the form of a digitTrain4D-ArrayData four-dimensional array. In constructing the four-dimensional array dataset, the first three columns represent the image's length, width, and height, respectively, and the last column represents the data's label removal rate, defined as a cell array.

[0093] (2) Network training process

[0094] During the training of a convolutional neural network, the model's training parameters have a significant impact on its performance. The accuracy of the model can be improved by adjusting the batch size, learning rate, and number of epochs during training.

[0095] In constructing the convolutional neural network model, the stochastic gradient descent algorithm was used. Its momentum is driven by a pre-trained optimization function, exhibiting superior convergence. Its typical value is set to 0.9 and remains constant. The batch size is the amount of data used in each iteration of network training, initially set to 30. MaxEpochs is the maximum number of training epochs, initially set to 200. The initial learning rate, InitialLearnRate, is set to 0.001, and the dataset contains single-case samples for training.

[0096] (3) Training parameter optimization

[0097] To improve the accuracy and efficiency of the model, further tuning of the batch size, MaxEpochs, and InitialLearnRate parameters in the training model is required. The optimization range based on the given default values ​​is shown in Table 3.

[0098] Table 2 Optimization Range of Convolutional Neural Network Model Parameters

[0099]

[0100] The batch size was set to 1-100; the MaxEpochs to 100-5000; and the initial learning rate to 0.0001-0.1. Following the optimization process for the model training parameters described above, the dataset of single-condition samples was used for training. The performance evaluation parameters for the training results with different model parameters are shown in Table 1. The performance parameters of different models were calculated; the smaller the performance parameters, the better the model, until the optimal model parameters were found.

[0101] Step 3.3.3: Based on the input audiovisual information, perform convolution and pooling calculations using a self-constructed convolutional neural network model, then adjust the number of network layers and weight parameters, compare the perceived material removal rate with the actual measured material removal rate, and calculate the loss function.

[0102] Step 3.3.4: Using the loss function as the optimization objective, we obtain a deep learning material removal rate perception measurement model for audiovisual information based on dense samples;

[0103] 3.2.2) Using 80% of the sample size as the training set and 20% of the data as the test set, the material removal rate perception measurement model established in step 3.2.1 was evaluated. The evaluation parameters are shown in Table 1.

[0104] 3.2.3) Using the model evaluation parameters in Table 1 as the basis for adjusting the model parameters, a deep learning perception measurement model for the material removal rate of belt grinding under dense sample single working condition is established.

[0105] Step 4: Establish a transfer learning model for sensing and measuring the material removal rate of belt grinding under cross-operating conditions;

[0106] Transfer learning is divided into traditional transfer learning and deep transfer learning. The method of this invention performs knowledge transfer of the material removal rate perception measurement model of single working condition (i.e., source domain) according to the number of samples in the cross working condition (i.e., target domain). According to the number of samples in the target domain, the task to be transferred is divided into cross working condition material removal rate perception measurement model based on ensemble tree transfer learning under sparse samples and cross working condition material removal rate perception measurement model based on transfer learning under lack of samples.

[0107] Let the target domain D be T The number of samples in the sample is N T In machine learning, the model to be trained with LightGBM integration has N parameters. M When N T ≤3N M When, then the target domain D T A sample is considered to be lacking in samples, while a sample is considered to be sparse.

[0108] When N T ≥3N M Let D be the domain of the machine learning perception measurement model for sparse samples established under a single working condition. s1 The source task is T s1 The domain and task of the objective function to be transferred are respectively represented by the target domain D. T1 and target task T T1 express;

[0109] 4.1) Establish a cross-condition material removal rate sensing and measurement model based on ensemble tree transfer learning under sparse samples; the specific steps are as follows:

[0110] 4.2): Establish a material removal rate perception and measurement model for sparse samples across the target domain based on an integrated tree model. The steps are the same as those for the single-condition material removal perception and measurement model based on audiovisual information fusion (i.e., 3.1).

[0111] 4.3): Fine-tune the weight coefficients of the model tree in step 3.1.3) to obtain a cross-condition material removal rate sensing measurement model based on ensemble tree transfer learning under sparse samples;

[0112] Step 5: Establish a cross-condition material removal rate sensing and measurement model based on transfer learning;

[0113] Step 5.4.4: Fix the corresponding network in the convolutional network model with fixed depth, fine-tune the parameters of the input layer, output layer and fully connected layer to obtain the cross-condition material removal rate sensing measurement model based on transfer learning of convolutional neural network under the lack of samples;

[0114] Step 5.4.5: Using the cross-condition material removal rate sensing and measurement model obtained in Step 4.4.4, compare the sensing and measurement values ​​under cross-conditions with the measured material removal rate values, and calculate the corresponding evaluation parameters.

[0115] Step 5.4.6: Using the values ​​of the evaluation parameters as the optimization objective, we obtain a cross-condition material removal rate sensing measurement model based on transfer learning under the condition of lack of samples.

[0116] Evaluate the number of samples in the target domain, when N T ≤3N M Let D be the domain of the machine learning perception measurement model for sparse samples established under a single working condition. s2 The source task is T s2 The domain and task of the objective function to be transferred are respectively represented by the target domain D. T2 and target task T T2 The specific steps are as follows:

[0117] 5.1) Due to the limited number of samples in the target domain, a Generative Adversarial Network (GAN) model is established to simulate sample generation. The specific steps are as follows:

[0118] 5.1.1) Design a generative network model G to generate spark images. It receives a random noise z and generates spark images by setting a noise, denoted as G(z).

[0119] 5.1.2) Design a discriminant network model D to distinguish between the generated image and the real spark image. The input parameter of the discriminant model is x, which represents a spark image. The output D(x) represents the probability that x is a real spark image. If the output is 1, it means that it is 100% a real spark image, while if the output is 0, it means that it is impossible for it to be a real spark image.

[0120] 5.1.3) Through continuous training, the goal of the generator network G is to generate realistic spark images, which are then used by the discriminator network D for discrimination. The goal of D is to distinguish the images generated by G from real images as much as possible. Thus, G and D constitute a dynamic "game." The loss function for transfer learning also consists of two parts: the source classification loss L during network training. C Domain discrimination loss L D As shown in equation (7):

[0121] L = L C (DS ,Y S )+λL D (D S D T (7)

[0122] 5.1.4) A GAN model that can be used to generate spark images is obtained. G can generate near-realistic spark images G(z). For the discriminator D, it is difficult to determine whether the spark images generated by G are real or not, so D(G(z)) = 0.5. Thus, a generative model G is obtained to generate spark images;

[0123] 5.2) Mix the simulated samples and real samples generated in step 5.1 to form a new target domain. Stop generating simulated samples when the number of samples in the target domain exceeds 10 times the training parameters of the deep convolutional neural network model to be transferred.

[0124] 5.3) At this point, the source domain is the deep convolutional neural network material removal rate sensing measurement model for single-condition dense samples established in step 3.2;

[0125] 5.4) Divide the data in the target domain into training and validation datasets, with 70% as the training set and 30% as the validation set. Configure the fully connected and regression layers of the network in the source domain. Increase the learning factor of the fully connected layers to accelerate the learning rate in the new final layer. This combination of learning rate settings will only speed up the learning speed in the new layer, while slowing down the learning speed for other layers. This results in a relatively small number of training epochs required for transfer learning.

[0126] 5.5) Preserve features from the shallower layers of the pre-trained network (transferred layer weights). To slow down the learning process in the transferred layers, set the initial learning rate to a small value;

[0127] 5.6) This yields a sample-limited, cross-condition material removal rate sensing measurement model based on transfer learning, consisting of a transfer layer and a new layer.

[0128] The following benefits can be obtained by transferring the material removal rate perception measurement model of audiovisual fusion under single working conditions to cross working conditions: (1) it saves the time and cost of data acquisition under different working conditions of belt grinding; (2) it effectively improves the generalization ability and robustness of the single working condition model; (3) it effectively solves the problem of insufficient deep learning samples.

[0129] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of protection of the claims.

Claims

1. A method for sensing and measuring the material removal rate of belt grinding by integrating audiovisual information, characterized in that: The method includes the following steps: Step 1: Acquisition of visual spark images and grinding sound information during belt grinding; Among them, visual spark image signals and grinding sound information constitute audiovisual information; Step 2: Preprocessing of audiovisual information; Step 3: Based on the processed audiovisual information, establish a perception and measurement model for the removal rate of audiovisual fusion materials under a single working condition; Step 4: Establish a transfer learning material removal rate perception measurement model based on audiovisual fusion under multiple working conditions: Transfer the audiovisual fusion material removal rate perception measurement model based on single working conditions to the transfer learning material removal rate perception measurement model based on audiovisual fusion under multiple working conditions. The method for establishing the perception and measurement model of material removal rate based on audiovisual fusion under a single working condition in step three is as follows: Step 3.1: Divide the single-condition samples into sparse samples and dense samples based on the number of samples of collected audiovisual information; Step 3.2: Establish an ensemble tree model based on machine learning algorithms for the audiovisual information of sparse samples; Step 3.3: Establish a material removal rate perception measurement model based on deep learning algorithms for the audiovisual information of dense samples; The method for establishing an ensemble tree model based on machine learning algorithms for sparse sample audiovisual information in step 3.2 is as follows: Step 3.2.1: Extract features from the visual spark image after preprocessing in Step 2, including color features, texture features, shape features, and spatial relationship features; Step 3.2.2: Extract time-domain, frequency-domain, and time-frequency-domain features from the effective grinding sound information in Step 2; Step 3.2.3: Perform feature selection and optimization on the visual spark image and grinding sound information features extracted in Steps 3.2.1 and 3.2.2 to obtain the feature subset most relevant to the material removal rate; Step 3.2.4: Input the feature subset of audiovisual information obtained in Step 3.2.3 into the LightGBM-based ensemble tree model, adjust the learning rate, number of leaves, maximum depth of the model tree, and the number of feature selections and random resampling, and train the tree model. Step 3.2.5: Based on the material removal rate value obtained in Step 3.2.4 based on the integrated tree model, compare it with the measured material removal rate value, and calculate the corresponding model evaluation parameters; Step 3.2.6: Using the values ​​of the evaluation parameters as optimization objectives, we obtain a material removal rate perception measurement model based on audiovisual information from a single working condition with sparse samples.

2. The method for sensing and measuring the material removal rate of belt grinding by integrating audiovisual information according to claim 1, characterized in that: The grinding sound information is collected by a sound sensor. The sound sensor is installed on the worktable and moves with it while remaining stationary relative to the workpiece. The sound sensor signal is sent to the data acquisition card and then stored in the computer after digital-to-analog conversion.

3. The method for sensing and measuring the material removal rate of belt grinding by integrating audiovisual information according to claim 1 or 2, characterized in that: In step two, the collected spark images are corrected for viewing distance based on the change in the size of the contact wheel to obtain spark images obtained at the same viewing distance. Then, the spark background is removed to remove background information and obtain information with spark images. The collected sound information is corrected for propagation distance and then background noise is removed to obtain sound information related to grinding information.

4. The method for sensing and measuring the material removal rate of belt grinding by integrating audiovisual information according to claim 1, characterized in that: Step 3.3 establishes a material removal rate perception measurement model based on deep learning algorithms for the audiovisual information of dense samples. The specific method is as follows: Step 3.3.1: Convert the grinding sound signal obtained in Step 2 into a time-spectrum graph using Short Time Fourier Transform (SFT); Step 3.3.2: Using the visual spark image obtained in Step 2 and the time-spectrum diagram of the grinding sound in Step 3.3.1 as raw data, input them into the self-constructed convolutional neural network model; Step 3.3.3: Based on the input audiovisual information, perform convolution and pooling calculations using a self-constructed convolutional neural network model, then adjust the number of network layers and weight parameters, compare the perceived material removal rate with the actual measured material removal rate, and calculate the loss function. Step 3.3.4: Using the loss function as the optimization objective, we obtain a deep learning material removal rate perception measurement model for audiovisual information based on dense samples.

5. The method for sensing and measuring the material removal rate of belt grinding by integrating audiovisual information according to claim 4, characterized in that: The training method of the self-constructed convolutional neural network model is as follows: the feature matrix map constructed from the features of spark images and sound signals under dense samples is input through the input layer. Through a series of operations such as convolution, normalization, activation function and pooling, the features automatically extracted by convolution are connected to the fully connected layer. The final material removal rate is obtained through the regression layer.

6. The method for sensing and measuring the material removal rate of belt grinding by integrating audiovisual information according to claim 5, characterized in that: The specific training process of the model is as follows: (1) Construction of the four-dimensional array of the input layer The training and test images are loaded as a four-dimensional array called digitTrain4D-ArrayData. In the process of constructing the four-dimensional array dataset, the first three columns are the length, width and height of the image, and the last column is the label material removal rate of the data. It should be defined as a cell array. (2) The training process of the network In constructing the convolutional neural network model, the stochastic gradient descent algorithm was used, whose momentum was driven by the trained optimization function. The batch size is the amount of data used in each iteration of network training, and its initial value is set to 30. MaxEpochs is the maximum number of training rounds, and its initial value is 200. The initial learning rate InitialLearnRate is set to 0.

001. The dataset of single-condition samples is used for training. (3) Optimization of training parameters Further fine-tuning of the batch size, MaxEpochs, and InitialLearnRate parameters in the training model was performed, with the batch size set to 1-100, MaxEpochs set to 100-5000, and the initial learning rate set to 0.0001-0.

1. Based on the optimization process of the model training parameters, the dataset of single-condition samples was trained, and the performance parameters of different models were calculated until the optimal model parameters were found.

7. A method for sensing and measuring the material removal rate of belt grinding that integrates audiovisual information according to claim 1 or any one of 4-6, characterized in that: The single working condition is the same sanding belt specification and the same workpiece material.

8. The method for sensing and measuring the material removal rate of belt grinding by integrating audiovisual information according to claim 1 or any one of 4-6, characterized in that: The method for establishing the transfer learning material removal rate perception measurement model based on audiovisual fusion under cross-working conditions in step four is as follows: Let the material removal rate sensing and measurement model under a single working condition be called the source domain model, and the material removal rate sensing and measurement model under multiple working conditions be called the target domain model; The specific steps for transferring the material removal rate sensing measurement model in the source domain model to the material removal rate sensing measurement model in the target domain are as follows: Step 4.1: Divide the target domain samples into sparse samples and lacking samples based on the number of target domain samples across different working conditions; Step 4.2: Establish a material removal rate sensing and measurement model for the target domain across operating conditions based on an integrated tree model for sparse samples, following the same steps as in 3.2; Step 4.3: Fine-tune the weight coefficients of the model tree in step 3.2.4 to obtain a cross-condition material removal rate sensing measurement model based on ensemble tree transfer learning under sparse samples; Step 4.4: Establish a cross-condition material removal rate sensing measurement model based on transfer learning under the condition of insufficient samples. The specific method is as follows: Step 4.4.1: For the lack of sample set, firstly, a generator and discriminator based on GAN network are established, and a generative adversarial network is established. Through continuous adversarial interaction between the generator and discriminator, simulated samples that are almost indistinguishable from real samples are generated. Step 4.4.2: By continuously generating samples to reach a certain number of samples, establish a mixed sample set based on real samples and simulated samples; Step 4.4.3: Take the mixed sample set obtained in step 4.4.2 as input and the convolutional neural network model constructed in step 3.3.3 as the source domain model; Step 4.4.4: Fix the corresponding network in the convolutional network model with fixed depth, fine-tune the parameters of the input layer, output layer and fully connected layer to obtain the cross-condition material removal rate sensing measurement model based on transfer learning of convolutional neural network under the condition of lack of samples; Step 4.4.5: Using the cross-condition material removal rate sensing and measurement model obtained in Step 4.4.4, compare the sensing and measurement values ​​under cross-conditions with the measured material removal rate values, and calculate the corresponding evaluation parameters. Step 4.4.6: Using the values ​​of the evaluation parameters as the optimization objective, we obtain a cross-condition material removal rate sensing measurement model based on transfer learning under the condition of lack of samples.

Images