Lightweight abrasive belt grinding spark image restoration method based on edge guidance

Abstract

The invention discloses a lightweight abrasive belt grinding spark image restoration method based on edge guidance. The method comprises the following steps: (1) acquiring a spark image; (2) performing data enhancement on the spark image acquired in the step (1); and (3) carrying out image restoration module training by adopting the spark image subjected to data enhancement in the step (2). And (4) repairing the damaged / missing spark image area in the step (1) by using the image repairing module in the step (3) so as to obtain a spark image with complete information. Limited spark image data are enhanced by adopting data enhancement, the enhanced spark image data set is input into the repair module for training, and finally the data set is used for repairing a damaged / missing spark image area occurring in the abrasive belt grinding process on line, so that the robustness and generalization ability of an MRR model are improved, and the abrasive belt grinding efficiency is improved. And the monitoring precision of the material removal rate is effectively improved.

Description

Technical Field

[0001] The present invention relates to the technical field of spark image processing for abrasive belt grinding, and particularly to a lightweight abrasive belt grinding spark image restoration method based on edge guidance. Background Technique

[0002] Abrasive belt grinding is an efficient finishing machining technology, which is widely used in complex curved surfaces and difficult-to-machine materials. The material removal during grinding is a non-linear process coupled by multiple factors. Therefore, accurately monitoring material removal is a huge challenge. The method of using spark images to identify the material removal rate (MRR) of abrasive belt grinding has been proven to be effective, which can effectively avoid the influence of multiple factors and their coupling. The literature "Ren L, Zhang G, Wang Y, et al. A new in-process material removal rate monitoring approach in abrasive belt grinding [J]. The International Journal of Advanced Manufacturing Technology, 2019, 104: 2715 - 2726 (Ren Lijuan, Zhang Guangpeng, Wang Yuan, etc., A new method for monitoring the material removal rate during abrasive belt grinding [J]. International Journal of Advanced Manufacturing Technology, 2019, 104: 2715 - 2726)" proposed a new method for monitoring the material removal rate of abrasive belt grinding with spark images based on machine learning, and the measurement accuracy is as high as 0.95; the literature "Ren L, Wang N, Pang W, et al. Modeling and monitoring the material removal rate of abrasive belt grinding based on vision measurement and the gene expression programming (GEP) algorithm [J]. The International Journal of Advanced Manufacturing Technology (International Journal of Advanced Manufacturing Technology), 2022, 120(1): 385 - 401 (Ren Lijuan, Wang Nina, Pang Wanjing, etc., Modeling and monitoring the material removal rate of abrasive belt grinding based on vision measurement and gene expression programming (GEP) algorithm)" uses the gene expression programming (GEP) algorithm to realize the online prediction of spark images, and the prediction accuracy is as high as 0.98. However, these MRR models based on spark image recognition are all based on the ideal state of planar machining.Considering the complex situation of grinding curved surface workpieces in actual industrial scenarios, due to the change of abrasive belt attitude, the subjective measurement direction of sparks is distorted, and there is a phenomenon that the workpiece, equipment or tool in the spark image captured by the vision sensor blocks the spark image. This phenomenon makes the collected image have a spark damage area, resulting in the loss of effective information in the spark image. The effective information of the spark image features affects the recognition accuracy of the MRR model. Therefore, in order to ensure the integrity of the spark image and improve the robustness of the MRR model, the spark image is repaired. Summary of the Invention

[0003] The technical problem to be solved by the present invention is to provide a lightweight abrasive belt grinding spark image repair method based on edge guidance, which can realize the repair of the spark image, ensure the integrity of the spark image, and improve the monitoring accuracy of the material removal rate and the robustness of the MRR model.

[0004] The technical solution adopted by the present invention is: a lightweight abrasive belt grinding spark image repair method based on edge guidance, which includes the following steps:

[0005] (1) Collect the spark image;

[0006] (2) Perform data augmentation on the spark image collected in step (1);

[0007] (3) Use the spark image after data augmentation in step (2) to train the image repair module to obtain a trained image repair module;

[0008] (4) Use the trained image repair module in step (3) to repair the damaged / missing spark image area in step (1) to obtain a spark image with complete information.

[0009] Further, the image repair module in step (3) includes a two-stage GAN network, namely the edge network f e and the repair network f i , through a two-level network architecture, an edge synthesis and an edge-guided structure reconstruction are constructed to repair the spark image, where (G1, D1) and (G2, D2) are the generators and discriminators of the edge network and the repair network respectively;

[0010] In the edge network stage, the damaged edge map is input into the generator G1. The generator G1 is composed of an encoder and a decoder. The encoder processes the damaged edge map After two downsamplings, it is processed by 9 lightweight-FFC residual blocks, and then the decoder upsamples the processed image back to the original image size; in the image inpainting network, the image processed in the edge network stage, the mask image, and the original image are input into the generator G2, which consists of an encoder and a decoder. After the encoder downsamples the image processed in the edge network stage, it is then processed by 9 lightweight-FFC residual blocks, and then the decoder upsamples the image back to the original image size to achieve the inpainting of the spark image, that is The discriminator uses the PatchGAN architecture.

[0011] Furthermore, the above lightweight-FFC residual block includes a residual mapping layer and a shortcut connection layer. Let X be the input of the lightweight-FFC residual block, and the output target mapping be H(X). F(X) is the residual mapping obtained by convolving X with FFC, that is, F(X) = H(X) - X. Through a feedforward neural network with a shortcut connection, the original mapping H(X) is written as F(X) + X. Therefore, the output is Y = F(X, {W i}+ X), and F(X, W i ) represents the residual mapping to be learned.

[0012] Furthermore, the above FFC consists of a local path and a global path. The local path performs ordinary convolution on the input features, and the global path works in the frequency domain through the Fourier transform (FFT) to consider the global context. The update process is as follows:

[0013] Let be the input feature map of the FFC, where H×W and C are the spatial resolution and the number of channels respectively. At the input end of the FFC, it is first split along the dimension of the feature channels into X = {X l , X g}. The local part learns from the local neighborhood, and the global part aims to capture the remote context information. Among them, α in ∈[0, 1] represents the percentage of the feature channels allocated to the global path. Assuming that the input and output are the same, similarly, is the output tensor, Y = {Y l , Y g} is the local and global segmentation. The scale of the global part of the output tensor is controlled by the parameter α out ∈[0, 1]. Therefore:

[0014] Y l = Y l→l + Y g→l = f l (X l ) + f g→l (Xg ) (2)

[0015] Y g = Y g→g + Y l→g = f g (X g ) + f l→g (X l ) (3)

[0016] Wherein, Y l , Y g→l and Y l→g are components obtained through traditional convolution, Y g→l and Y l→g are components obtained through conversion between paths, Y g is calculated using a spectral transformer.

[0017] Furthermore, a joint loss is introduced into the model of the above image inpainting module, and the joint loss includes L1 loss, high receptive field perception loss, feature matching loss, and adversarial loss. Let I O represent the real image, I edge represent the complete edge map, I gray represent the grayscale image, M is a 0-1 mask, 1 represents the masked area, is the damaged image, represents the damaged edge map, and are the inpainted image and the edge image. The output of the generator is

[0018] The L1 loss represents the distance between I0 and , and is defined as:

[0019]

[0020] Wherein, ⊙ is element-wise multiplication;

[0021] The high receptive field perception loss L HRF evaluates the distance between the features extracted from the original image and the predicted image through a basic pre-trained network, and is defined as:

[0022]

[0023] Wherein, φ HRF (*) is the activation map in the pre-trained network layer;

[0024] The feature matching loss L FM function can effectively solve the problem that the generator and discriminator in GAN cannot compete with each other, and is defined as:

[0025]

[0026] where D f (*) is the output of the middle layer of the discriminator;

[0027] The adversarial loss L adv is to ensure that the inpainting model generates natural-looking local details, which is defined as:

[0028] where D f (*) is the output of the middle layer of the discriminator, Ε(*) is the expected value of the distribution function. Therefore, the final loss L final of the image inpainting module is expressed as follows:

[0029]

[0030] where λ L1 , λ adv , λ FM and λ HRF are weights, which are set to 10, 10, 100, and 30.

[0031] Furthermore, the mask generation strategy of the above image inpainting module is as follows: two types of masks are generated for the training and testing of the image inpainting module. The first type is regular and irregular masks, which are generated by combining rectangles or irregular shapes of random sizes. There are a total of 20,000 masks, and the mask rate is between 0% and 70%. The second type is segmentation masks, which are generated by using Labelme to label and segment the occluded areas of real spark images. There are a total of 100 masks, and the mask rate is between 10% and 40%.

[0032] Advantages of the present invention: Compared with the prior art, the present invention uses data augmentation to enhance the limited spark image data, inputs the enhanced spark image dataset into the inpainting module for training, and finally uses it to online repair the damaged / missing spark image areas that occur during the abrasive belt grinding process. It can achieve the repair of spark images, ensure the integrity of spark images, improve the robustness and generalization ability of the MRR model, and effectively improve the monitoring accuracy of the material removal rate;

[0033] A lightweight spark image restoration method based on edge guidance (L-EGIM) proposed by the present invention. The L-EGIM method includes a data augmentation module and an image restoration module. The data augmentation module is used to augment the limited spark image data, and the image restoration module is a two-stage generative adversarial network (GAN), which reconstructs and restores the spark image by constructing an edge synthesis and an edge-guided structure. In addition, a hybrid block combining fast Fourier convolution (FFC) and lightweight residual blocks is introduced into the GAN, which can obtain a global receptive field while reducing the module size. Finally, the performance of the proposed L-EGIM method is evaluated on a spark image dataset. The results show that the performance of this method is 21.38% higher than that of the strong baseline method. The MRR accuracy of the restored spark image recognition is 0.913, with a difference of 3.38% from the recognition performance of the real spark image. Therefore, the proposed L-EGIM method can improve the robustness and generalization ability of the MRR model. Description of the Drawings

[0034] Figure 1 Schematic diagram of the belt grinding process;

[0035] Figure 2 Spark distribution area map of different MRR spark images;

[0036] Figure 3 Schematic diagram of the occlusion of the spark image of the belt grinding curved workpiece;

[0037] Figure 4 MRR monitoring result curve of the GEP-MRR model;

[0038] Figure 5 Edge map of the spark image;

[0039] Figure 6 Flow schematic diagram of the lightweight belt grinding spark image restoration method based on edge guidance;

[0040] Figure 7 Schematic diagram of the segmentation mask generation strategy;

[0041] Figure 8 Sample map generated by different mask strategies;

[0042] Figure 9 Qualitative comparison result map of the spark image dataset;

[0043] Figure 10 Restoration result map of the segmentation mask;

[0044] Figure 11 Quantitative comparison result map under different mask rates;

[0045] Figure 12Quantitative comparison results of the mask width ratio, where (a) is FID and (b) is LPIPS;

[0046] Figure 13 Quantitative comparison result diagram of segmentation masks with different mask rates, where (a) is FID and (b) is LPIPS;

[0047] Figure 14 Qualitative results of 1600x1200 high-resolution spark images, where the first and second rows are segmentation masks, and the third row is an irregular mask. (All images have been resized);

[0048] Figure 15 Quantitative comparison result diagram of the method of the present invention migrated to high resolution and other methods;

[0049] Figure 16 Monitoring result diagram of the GEP-MRR model for the repaired image;

[0050] Figure 17 Qualitative comparison result diagram of model training using different spark image datasets based on the L-EGIM method;

[0051] Figure 18 Quantitative comparison result diagram of model training based on the spark image dataset after data augmentation under different convolutional blocks. Detailed implementation manners

[0052] The present invention will be further explained below in conjunction with the accompanying drawings of the specification to facilitate better understanding by those skilled in the art.

[0053] Principle of spark generation in belt grinding:

[0054] As Figure 1 shown, during the belt grinding process, the abrasive belt is a tool that is tensioned and moves at high speed with the help of a tensioning wheel and a driving wheel. Under a certain pressure, the material is removed during the high-speed relative movement between the abrasive grains and the workpiece.

[0055] Steel is a metallic material made from iron and carbon through a series of processing methods. When steel is ground to a certain temperature, carbon will burn due to the heat. Therefore, when grinding steel, the hot iron filings of the workpiece and the sanding belt will be thrown out along the tangent direction of the rotation of the sanding belt. After that, the iron filings will rub against the air, and the temperature will continue to rise. The iron filings will burn directly, resulting in strong oxidation and melting. However, the FeO film on the surface of the iron filings is easily reactive with carbon at high temperatures. The reduction reaction formula is FeO + C → Fe + CO. The reduced iron will be oxidized again and then reduced again. The oxidation-reduction reaction continues to cycle, continuously generating CO gas. When the FeO film on the surface of the iron filings cannot control the generated CO gas, the iron filings will explode. This phenomenon causes the iron filings to fly out along the tangent direction of the belt speed, forming visible sparks.

[0056] Damage principle of the spark image: As Figure 2 shown, the sparks generated during the grinding process are distributed in a fan-shaped area tangent to the contact wheel. When grinding complex curved surface workpieces in a real industrial scenario, due to the change of the sanding belt attitude, the spark area will be blocked by the workpiece or equipment. This phenomenon leads to the damaged area of the captured spark image. Figure 3 is a schematic diagram of sanding belt grinding a blade, where the red and blue areas are the spark area and the damaged area respectively. Figure 3 a is the processing method perpendicular to the blade axis, Figure 3 b is the processing method parallel to the blade axis. Figure 4 is the MRR monitoring result of the existing GEP model of the inventor team. Among them, the measured data is the actual measured MRR in the sanding belt grinding experiment, the predicted data is the MRR predicted using the original spark image, and the damaged data is the MRR predicted from the damaged spark image. The results show that the model has good accuracy for the MRR monitored by the spark image. However, the spark damage area causes the loss of effective information of the spark image, which brings deviation to the online recognition of MRR.

[0057] The sanding belt grinding experimental device generally includes a sanding belt grinding system, a spark image acquisition system, and a monitoring system. The spark image acquisition system includes two CCD industrial cameras and two computers (PCs). The two cameras are respectively placed on the side and top of the machine tool to capture the spark images in two directions during grinding. The spark image acquisition frequency is 100 per second. The CCD cameras communicate with the PCs through a network interface, and the PCs use Mindvision to acquire and process the spark images. The resolution of the spark image is 1600×1200.

[0058] Example 1: Aiming at the problems existing in the acquisition process of the above sanding belt grinding spark image, a lightweight edge-guided image restoration method for sanding belt grinding sparks (L-EGIM) is proposed. The proposed method is to restore the color spark image I of the mask M with unknown pixels O, the damaged spark image is wherein In the grinding experiment, it is observed that the captured spark image has strong edge information. The edge map of the spark image is as Figure 5 shown. Figure 6 is the overall architecture of the proposed method, which consists of two modules: the data augmentation module and the image inpainting module.

[0059] A proposed lightweight abrasive belt grinding spark image inpainting method based on edge guidance includes the following steps:

[0060] (1) Collect spark images;

[0061] (2) Perform data augmentation on the spark images collected in step (1) using the data augmentation module;

[0062] The purpose is to solve the small sample problem of spark images caused by sparse labels in industrial scenarios and improve the generalization ability and effect of the image inpainting method. Therefore, a single-sample supervised data augmentation module is introduced, which includes geometric transformation methods and color transformation methods. Geometric transformations are rotation, shearing, and scaling. Color transformations are adding Gaussian noise and color perturbations;

[0063] (3) Use the data-augmented spark images in step (2) to train the image inpainting module to obtain a trained image inpainting module;

[0064] The image inpainting module includes a two-stage GAN network, namely the edge network f e and the inpainting network f i , through a two-level network architecture, construct an edge synthesis and edge-guided structure reconstruction to restore the spark image, where (G1, D1) and (G2, D2) are the generators and discriminators of the edge network and the inpainting network respectively;

[0065] In the edge network stage, the damaged edge map is input into the generator G1. The generator G1 consists of an encoder and a decoder. The encoder performs two downsamplings on the damaged edge map and then processes it through 9 lightweight-FFC residual blocks, and then the decoder upsamples the processed image back to the original image size; in the image inpainting network, the image processed in the edge network stage, the mask image, and the original image are input into the generator G2 together. The generator G2 consists of an encoder and a decoder. After the encoder downsamples the image processed in the edge network stage, it is then processed through 9 lightweight-FFC residual blocks, and then the decoder upsamples the image back to the original image size to achieve the inpainting of the spark image, that is The discriminator uses the PatchGAN architecture.

[0066] Among them, the lightweight-FFC residual block includes a residual mapping layer and a shortcut connection layer. Let X be the input of the lightweight-FFC residual block, and the output target mapping be H(X). F(X) is the residual mapping obtained by convolving X with FFC, that is, F(X) = H(X) - X. Through a feed-forward neural network with shortcut connections, the original mapping H(X) is written as F(X) + X. Therefore, the output is Y = F(X, {W i}+X), F(X, W i ) represents the residual mapping to be learned; compared with the serial structure of ordinary networks, shortcut connections can obtain deeper features.

[0067] Table 1 Steps of Spectrum Deep Learning

[0068]

[0069] Compared with ordinary convolution, FFC can not only obtain the receptive field of the entire image, but also perform cross-scale information fusion inside the convolution, which consists of a local path and a global path. The local path performs ordinary convolution on the input features, and the global path works in the frequency domain through Fourier transform (FFT) to consider the global context. Each path can obtain complementary information with different receptive fields. The process of updating in FFC can be described by the following formula:

[0070] Let be the input feature map of FFC, where H×W and C are the spatial resolution and the number of channels respectively. At the input end of FFC, it is first split along the dimension of the feature channels into X = {X l , X g}, the local part learns from the local neighborhood, and the global part aims to capture remote context information, where α in ∈[0, 1] represents the percentage of the feature channels allocated to the global path. Assuming that the input and output are the same, similarly, is the output tensor, Y = {Y l , Y g} is the local and global segmentation, and the scale of the global part of the output tensor is controlled by the parameter α out ∈[0, 1]. Therefore:

[0071] Y l = Y l→l + Y g→l = f l (X l ) + f g→l (X g ) (2)

[0072] Y g = Y g→g + Yl→g = f g (X g ) + f l→g (X l ) (3)

[0073] In the formula, Y l , Y g→l and Y l→g are components obtained through traditional convolution, Y g→l and Y l→g are components obtained through the conversion between paths, and Y g is calculated using a spectral transformer.

[0074] A joint loss is introduced into the model of the image inpainting module to ensure that the model generates visually realistic and semantically reasonable results. The joint loss includes L1 loss, high receptive field perception loss, feature matching loss, and adversarial loss. Let I O represent the real image, I edge represent the complete edge map, I gray represent the grayscale image, M is a 0-1 mask (1 represents the masked area), is the damaged image, I O represents the color spark image, represents the damaged edge map, and are the inpainted image and the edge image. The output of the generator is

[0075] The L1 loss represents the distance between I0 and , and is defined as:

[0076]

[0077] In the formula, ⊙ is element-wise multiplication;

[0078] The high receptive field perception loss LHRF evaluates the distance between the features extracted from the original image and the predicted image through a basic pre-trained network, and is defined as:

[0079]

[0080] In the formula, φ HRF (*) is the activation map in the pre-trained network layer;

[0081] The feature matching loss L FM function can effectively solve the problem that the generator and discriminator in GAN cannot compete with each other, and is defined as:

[0082]

[0083] In the formula, Df (*) is the output of the middle layer of the discriminator;

[0084] The adversarial loss L adv is to ensure that the inpainting model generates natural-looking local details, which is defined as:

[0085] where D f (*) is the output of the middle layer of the discriminator, Ε(*) is the expected value of the distribution function. Therefore, the final loss L of the image inpainting module final is expressed as follows:

[0086]

[0087] where λ L1 , λ adv , λ FM and λ HRF are weights, which are set to 10, 10, 100, and 30.

[0088] The mask generation method for the image inpainting module is as follows: Two types of masks are generated for the training and testing of the image inpainting module. The first type is regular and irregular masks, which are generated by combining rectangles or irregular shapes of random sizes. There are a total of 20,000 masks, and the mask rate is between 0% and 70%. The first type of masks generated is to increase the diversity of images. However, randomly generated masks rarely have the same shape and size as real-world objects. Therefore, in order to ensure that the conclusions drawn using irregular masks are also applicable to real-world objects, the second type of segmentation masks is generated. Figure 7 is the strategy for generating segmentation masks, which is generated by using Labelme to label and segment the occluded areas of real spark images. There are a total of 100 masks, and the mask rate is between 10% and 40%. Figure 8 is an example of the mask in the experiment, with a pixel size of 256×256.

[0089] (4) Use the trained image inpainting module in step (3) to repair the damaged / missing spark image area in step (1) to obtain a spark image with complete information.

[0090] To illustrate the effect of the present invention, the following simulation experiment is carried out:

[0091] 1.1 Dataset Selection: The method of the present invention was evaluated on the spark image dataset obtained from the experiment. The spark image dataset contains a total of 34,300 images. The training set consists of 30,000 (image, mask) pairs, and the test set consists of 6,000 (image, mask) pairs and 900 validation set (image, mask) pairs. The masks of the model include segmentation masks and irregular masks, with a total of 100 segmentation masks and 12,000 irregular masks. The algorithm was trained on a single Nvidia 1050Ti (GPU), and the Adam optimizer was used. The fixed learning rates of the repair network and the discriminator network are 0.001 and 0.0001, respectively. All spark images and mask pixels are 256×256, and the batch size is 2.

[0092] 1.2 The proposed model was compared with strong baseline image inpainting methods (including Lama-Fourier, Edgeconnect, MAT, CoModGAN, and the present invention). For fair comparison, all methods were used for inpainting on the spark image dataset described above.

[0093] 1.2.1. Qualitative Comparison

[0094] Figure 9 These are the qualitative comparison results of some strong baseline algorithms, namely Lama-Fourier, Edgeconnect, MAT, CoModGAN, and the method of the present invention. The picture pixels are 256×256 (the pictures are all scaled down). Figure 9 These are the qualitative comparison results shown for regular masks and irregular masks. In the first three rows, as the mask holes become larger, the inpainting ability of all algorithms decreases. The baseline algorithms have artifacts in inpainting spark images. For the MAT algorithm, which is the current state-of-the-art based on Vision Transformer (ViT), the inpainting results are visually poor. The inpainting results of Edgeconnect are visually similar to our method for small masks, but there are unhealed regions and artifacts in the inpainting results for larger mask holes and irregular masks. The last row shows the inpainting details of the blue area in the figure. Visually, the method of the present invention also has a better inpainting effect in terms of details. In summary, in the inpainting results for regular and irregular masks, the method of the present invention has a clearer effect visually than other results.

[0095] Figure 10 Shown are the inpainting results under the segmentation mask, and there are also visually better results under the segmentation mask.

[0096] 1.2.2. Quantitative Comparison

[0097] The model proposed in this invention is quantitatively compared with these strong baseline image inpainting methods. Three metrics, namely the Fréchet Inception Distance (FID), Learned Perceptual Image Patch Similarity (LPIPS), and Structural Similarity Index Measure (SSIM), are used to quantitatively evaluate the model. The detailed information of these three evaluation parameters is listed in Table 2. Among them, "↓" indicates that the smaller the value, the better the inpainting performance, and "↑" is the opposite. Table 3 shows the evaluation results of the segmentation mask and regular / irregular masks. Our model always outperforms most baseline methods and has fewer parameters. Compared with the best Edgeconnect, the performance is improved by 21.38%.

[0098] Table 2. Evaluation Parameters of the Image Restoration Model

[0099]

[0100] Table 3. Quantitative Comparison Results of the Spark Image Dataset

[0101]

[0102] Note: The table shows the comparison results of the Lama-fourier, Edgeconnect, MAT, CoModGAN, and the model of this invention. ▲ indicates deterioration, and ▼ indicates improvement, which are the scores compared with the L-EGIM model of this invention (shown in the first row). Table 3 shows the metrics of different test mask generation strategies (i.e., segmentation mask and irregular mask).

[0103] The method of this invention is also verified in terms of the mask rate and mask width ratio. Figure 11 It is the comparison result of the method of this invention and the strong baseline algorithm under irregular masks with different mask rates from 0% to 70%. (a), (b), and (c) are the three parameters FID, LPIPS, and SSIM respectively, and (d) is the model training time and image inpainting time. Among them, the Edgeconnect method is the most competitive method. However, when the mask ratio is higher than 50%, the inpainting performance of the Edgeconnect method significantly decreases. Compared with our method, the training time of lama is very short, but the inpainting effect is poor. The method of this invention does not require much training time and achieves inpainting at a speed of 15.95 frames per second. The inpainting effects of the remaining strong baseline methods are poor. Therefore, the method of this invention always outperforms most baseline methods.

[0104] Figure 12The following are the comparison results between the method of the present invention and the strong baseline algorithm under different mask width ratios. (a) and (b) are the parameters FID and LPIPS respectively. The abscissa is the mask width ratio. When the mask width ratio is from 0% to 10%, the method of the present invention achieves an FID value of 0.89. When the mask width ratio is from 10% to 20%, the FID value of our method is reduced by 4.25% compared with Edgeconnect. When the mask width ratio is greater than 40%, the repair performance of the method of the present invention is significantly greater than that of the Edgeconnect method.

[0105] Finally, Figure 13 The following are the comparison results between the method of the present invention and the strong baseline algorithm under the split mask with the mask ratio from 10% to 40%. The method of the present invention has the best repair effect.

[0106] To sum up, the average performance of the method of the present invention is better, and it can accurately repair the spark images during the grinding process. Since the actual collected spark image pixels of the present invention are 1600×1200, therefore, the model of the present invention is migrated to a higher resolution.

[0107] 1.3. Migration to a higher resolution

[0108] Training a repair model with high-resolution images is time-consuming and computationally intensive. The model of the present invention is trained by cropping and shrinking the high-resolution spark images to a size of 256×256. However, the actual collected spark images are of high resolution 1600×1200. Therefore, the model of the present invention is migrated to a higher resolution. The model of the present invention is evaluated on spark images with a higher resolution. As Figure 14 is a qualitative comparison of the high-resolution spark image repair results under different material removal rates, Figure 15 is the quantitative comparison result of migrating to a high resolution and other methods. The method of the present invention has higher quality and consistency in repairing spark images.

[0109] 1.4. Monitoring the material removal rate

[0110] Taking the experimental data of a 60# abrasive belt with a theoretical grinding depth of 0.2 as an example. The MRR performance of the spark image recognition after being repaired by the method of the present invention is verified. The mask ratio of the spark image is randomly assigned, and the range is 0% to 70%. Figure 16 is the R value of the GEP-MRR model predicting MRR with the mask rate from 0 to 70%, 2 and the value is 0.913, with a difference of 3.35% from the original spark image. Therefore, the method of the present invention can effectively improve the robustness and generalization ability of the MRR model and achieve accurate monitoring of the material removal rate.

[0111] The comparison results of training the model without using the data augmentation module and using the data augmentation module are discussed.Figure 17 Table 4 shows the qualitative and quantitative comparison results of model training using different spark image datasets based on the proposed method (L-EGIM). The results indicate that the model trained with the data augmentation module has better repair ability for spark images.

[0112] Table 4 presents the quantitative comparison results of model training using different spark image datasets based on the L-EGIM method.

[0113]

[0114] The comparison results of models using the original convolutional block and the lightweight-FFC residual block respectively based on the spark image dataset after data augmentation are discussed. Figure 18 Table 5 shows the qualitative and quantitative comparison results of the two models. The results indicate that the lightweight-FFC residual block has more accurate recognition and better repair performance for spark images.

[0115] Table 5 presents the quantitative comparison results of model training under different convolutional blocks based on the spark image dataset after data augmentation.

[0116]

[0117] In summary, in view of the problem that in the actual industrial scenario of polishing blades or complex curved surface parts, due to the change of the abrasive belt attitude, the subjective observation direction spark is distorted, and there is a phenomenon that the workpiece, equipment or tool blocks the spark image when the vision sensor captures the spark image, resulting in the loss of effective information of the spark image, which in turn affects the deviation of the online recognition of the material removal rate and makes the existing MRR model unable to continue monitoring, the present invention proposes a lightweight method for repairing abrasive belt grinding spark images. This method consists of a data augmentation module and an image repair module, and the specific advantages are as follows:

[0118] 1) To solve the small sample problem of spark images caused by sparse labels in industrial scenarios and improve the generalization ability and effect of the image repair method, an image enhancement module is introduced.

[0119] 2) In the image repair module, a two-stage GAN network combining edge guidance and image repair is adopted, and FFC and lightweight residual blocks are introduced into the network to solve the problems of limited network receptive field and excessive network weight.

[0120] 3) To solve the problem that the generated irregular mask is not the same as the shape and size of objects in the real world, a segmentation mask strategy is proposed to ensure that the conclusions drawn using the irregular mask are also applicable to objects in the real world.

[0121] 4) Improve the robustness and generalization ability of the existing MRR model.

Claims

1. A lightweight abrasive belt grinding spark image restoration method based on edge guidance, characterized in that The method includes the following steps: (1) Collect spark images; (2) Perform data augmentation on the spark images collected in step (1); (3) Use the spark images after data augmentation in step (2) to train an image inpainting module to obtain a trained image inpainting module; (4) Use the trained image inpainting module in step (3) to repair the damaged / missing spark image regions in step (1) to obtain a spark image with complete information.

2. A lightweight abrasive belt grinding spark image restoration method based on edge guidance according to claim 1, characterized in that, In step (3), the image restoration module includes a two-stage GAN network, namely the edge network f e and the restoration network f i , and through a two-level network architecture, an edge generation and edge-guided structure reconstruction are constructed to restore the spark image, where (G1, D1) and (G2, D2) are the generators and discriminators of the edge network and the restoration network respectively; In the edge network stage, the damaged edge map is input into the generator G1. The generator G1 consists of an encoder and a decoder. The encoder performs two downsamplings on the damaged edge map and then processes it through 9 lightweight-FFC residual blocks. Then the decoder upsamples the processed image back to the original image size. In the image inpainting network, the image processed in the edge network stage, the mask image, and the original image are input into the generator G2 together. The generator G2 consists of an encoder and a decoder. After the encoder downsamples the damaged image, it is then processed through 9 lightweight-FFC residual blocks. Then the decoder upsamples the image back to the original image size to achieve the inpainting of the spark image, that is The discriminator uses the PatchGAN architecture.

3. The method for repairing the spark image of the lightweight abrasive belt grinding based on edge guidance according to claim 2, wherein The lightweight-FFC residual block includes a residual mapping layer and a shortcut connection layer. Let X be the input of the lightweight-FFC residual block, the output target mapping be H(X), and F(X) be the residual mapping obtained by convolving X with FFC, i.e., F(X) = H(X) - X. Through a feedforward neural network with a shortcut connection, the original mapping H(X) is written as F(X) + X. Therefore, the output is Y = F(X, {W i}+ X), and F(X, W i ) represents the residual mapping to be learned.

4. A lightweight abrasive belt grinding spark image restoration method based on edge guidance according to claim 3, characterized in that The FFC consists of a local path and a global path. The local path performs ordinary convolution on the input features, and the global path works in the frequency domain through Fourier transform to consider the global context. The update process is as follows: Let be the input feature map of the FFC, where H×W and C are the spatial resolution and the number of channels respectively. At the input end of the FFC, first split it along the dimension of the feature channels into X = {X l , X g}, where the local part learns from the local neighborhood, and the global part aims to capture the long-range context information. Among them, α in ∈[0, 1] represents the percentage of the feature channels assigned to the global path. Assuming that the input and output are the same, similarly, is the output tensor, Y = {Y l , Y g} is the local and global segmentation, and the scale of the global part of the output tensor is controlled by the parameter α out ∈[0, 1]. Therefore: Y l = Y l→l + Y g→l = f l (X l ) + f g→l (X g ) (2) Y g = Y g→g + Y l→g = f g (X g ) + f l→g (X l ) (3) Wherein, Y l , Y g→l and Y l→g are components obtained by traditional convolution, Y g→l and Y l→g are components obtained by conversion between paths, Y g is calculated using a spectral transformer.

5. A lightweight abrasive belt grinding spark image restoration method based on edge guidance according to claim 3, characterized in that Introduce a joint loss into the model of the image inpainting module. The joint loss includes L1 loss, high receptive field perception loss, feature matching loss, and adversarial loss. Let I O represent the real spark image, I edge represent the complete edge map, I gray represent the grayscale image, M is a 0-1 mask (1 represents the masked area), is the damaged image, represents the damaged edge map, and are the inpainted image and edge image. The output of the generator is The L1 loss represents the distance between I0 and and is defined as: In the formula, ⊙ is element-wise multiplication; High receptive field perception loss L HRF Evaluating the distance between the features extracted from the original image and the predicted image through the basic pre-trained network, which is defined as: Where, φ HRF (*) is the activation map in the pre-trained network layer; Feature matching loss L FM The function is defined as: where D f (*) is the output of the intermediate layer of the discriminator; Adversarial loss L adv is defined as: where D f (*) is the output of the intermediate layer of the discriminator, and Ε(*) is the expected value of the distribution function. Therefore, the final loss L final of the image inpainting module is expressed as follows: In the formula, λ adv , λ FM and λ HRF are weights, set to 10, 10, 100, and 30.

6. A lightweight abrasive belt grinding spark image restoration method based on edge guidance according to claim 3, characterized in that The mask generation strategy of the image inpainting module is as follows: Generate two types of masks for the training and testing of the image inpainting module. The first type is regular and irregular masks, which are generated by combining rectangles or irregular shapes of random sizes. There are a total of 20,000 masks, and the mask rate is between 0% and 70%. The second type is segmentation masks, which are generated by using Labelme to label and segment the occluded regions of real spark images. There are a total of 100 masks, and the mask rate is between 10% and 40%.

Images