Gear grinding burn segmentation method based on deep convolutional neural network

Abstract

The application discloses a gear grinding burn segmentation method based on a deep convolutional neural network, which comprises the following steps: acquiring a gear tooth side surface image of a gear to be detected, and performing data preprocessing; using a data enhancement technology to expand the data after preprocessing, and preparing a gear tooth side surface grinding burn segmentation data set; constructing a gear grinding burn segmentation model based on an improved U-Net network and training the same; inputting test data, extracting a gear tooth side surface grinding burn area and visualizing the same, and determining the classification and level of the gear tooth side surface grinding burn according to the color depth and area proportion of the grinding burn area. The method constructs an end-to-end gear grinding burn segmentation model, fuses shallow high-resolution local features extracted by a convolutional neural network and sequence global features extracted by a self-attention mechanism, better solves the problems of data lack and classification imbalance in a segmentation task, can output a higher-resolution segmentation result image, and improves the segmentation accuracy of the gear tooth side surface grinding burn area.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of machine vision and machine learning, and particularly relates to a gear grinding burn segmentation method based on a deep convolutional neural network. BACKGROUND

[0002] The final link of the gear manufacturing process is generally finished by the gear grinding process. When the grinding force is large or the grinding temperature is high, grinding burn is extremely easy to occur. Grinding burn, as a typical thermal damage problem, has a great impact on the surface quality and service life of the gear. The gear grinding burn detection methods mainly include color observation method, acid etching method, surface microhardness test method, residual stress measurement method, metallographic detection method and magnetic elastic method. In actual production, the commonly used gear grinding burn detection method is chemical etching test. The burn area will show a specific appearance after being treated by a specified etching solution, and the classification and level of the burn can be determined according to the color depth and area ratio. However, the detection solution of this lossy detection method is easy to corrode the gear and pollute the environment.

[0003] The machine vision detection method uses a camera to replace the naked eye to collect the image of the workpiece to be detected, and uses a corresponding model or algorithm to distinguish and extract the target feature information, which is a rapid detection method with great development potential. Yi Xie et al. once proposed a workpiece surface burn detection method based on color CCD image, which comprehensively considered the color depth and distribution of the burn, and used the HSV color model to calculate the burn degree index to quantitatively evaluate the burn degree. However, this method has the following shortcomings: (1) When facing color CCD images under different light sources and shooting angles, the changes of the HSV quantities are different, and the established image model is only for several specific optical conditions, and the generalization ability is poor; (2) The model can only analyze the whole burn area image, and when detecting the gear grinding burn, the obtained image often contains random background area and multiple gear tooth side surfaces. Starting from only one variable of color, the established HSV image model is not comprehensive for the classification and grading of burn defects.

[0004] Nowadays, deep learning, as a powerful artificial intelligence algorithm, gradually highlights its advantages in solving machine vision detection tasks. Among them, semantic segmentation is one of the key technologies in the field of machine vision. A deep learning model is used to learn autonomously from a data set, construct a deep neural network and extract multiple levels of semantic features, and then recover the segmentation information layer by layer to establish the relationship between the input image and the output segmentation result. The U-Net network is a commonly used segmentation method in the field of image segmentation. Although it can improve the image segmentation effect, it has the problems of data deficiency and classification imbalance, and cannot fully extract image features, resulting in low segmentation accuracy. SUMMARY

[0005] Invention purposes: The application aims to provide a gear grinding burn segmentation method based on a deep convolutional neural network, which can improve segmentation accuracy and generalization ability.

[0006] Technical solutions: The gear grinding burn segmentation method based on the deep convolutional neural network comprises the following steps:

[0007] (1) Obtain the tooth side surface image of the gear to be detected, and perform data preprocessing;

[0008] (2) Use data enhancement technology to expand the preprocessed data, then outline the image burn area and define the class label, and make a tooth side surface grinding burn segmentation data set;

[0009] (3) Construct a gear grinding burn segmentation model based on an improved U-Net network, use the U-shaped structure of the U-Net network as the main body, use the residual neural network as the backbone network in the encoder part to extract the feature maps of different scales of the input image, convert the feature maps into a sequence and attach the position encoding, input the ViT network to extract the global information and obtain a one-dimensional feature vector; the decoder part performs upsampling on the self-attention one-dimensional feature vector, and connects the feature maps of different scales through a jump connection to output a pixel-level tooth side surface grinding burn segmentation result image;

[0010] (4) Divide the segmentation data set obtained in step (2) into a training set, a validation set and a test set, and use the training set and the validation set for model training;

[0011] (5) Input the test set data into the trained segmentation model, extract the tooth side surface grinding burn area and visualize it, and determine the classification and level of the tooth side surface grinding burn according to the color depth and area ratio of the grinding burn area.

[0012] Preferably, when the tooth side surface image of the gear to be detected is obtained in step (1), factors such as changes in brightness and lighting conditions of the environment will interfere with the image, and a filtering algorithm is used to remove noise; gears made of different materials have different colors and textures on the side surface, and a gray scale transformation is used to enhance the image; due to the influence of the shooting angle and the lens depth of field, the edge of the tooth surface appears inclined, and a geometric transformation is used to correct the tooth surface inclination of the image.

[0013] Preferably, the data expansion in step (2) comprises using traditional image processing methods such as geometric transformation, color space transformation and / or pixel point operation to preliminarily expand the original data set, then inputting the preliminarily expanded data set into a generative adversarial network (GAN) to generate images, further expanding the gear grinding burn data set through a deep learning method, saving all the generated images, and obtaining the expanded data set.

[0014] Preferably, the defining category label in step (2) comprises classifying the burn area according to the color depth of the tooth flank surface grinding burn area into two categories of mild burn and severe burn, and each category is classified into three levels according to the area ratio of the burn area.

[0015] Preferably, the mild burn is a dark gray pixel part, classified as B, and the severe burn is a black pixel part, classified as D; the three levels are that the maximum area ratio of the burn area is less than 10%, and the burn level suffix code is 1; the maximum area ratio of the burn area is 10% to 25%, and the burn level suffix code is 2; the maximum area ratio of the burn area is greater than 25%, and the burn level suffix code is 3.

[0016] Preferably, the residual neural network in step (3) adopts a ResNet50 network.

[0017] Preferably, the ViT network in step (3) comprises a plurality of layers of multi-head self-attention layers and multi-layer perceptrons.

[0018] Preferably, the model training process in step (4) comprises the following steps:

[0019] (4.1) initializing the weight parameters of the burn segmentation model;

[0020] (4.2) performing the i-th round of training, loading the tooth flank surface grinding burn segmentation data set, reading the tooth surface image as a matrix and the corresponding burn area and category label image as a matrix wherein H and W are the height and width of the tooth surface image, C I represents the number of channels of the tooth surface image, and C T is the number of channels of the label image;

[0021] (4.3) inputting the data into the burn segmentation model, and outputting the predicted label image matrix as

[0022] (4.4) calculating the loss function value after the i-th round of training, using the weighted superposition value of the cross-entropy loss and the Dice coefficient loss:

[0023] Loss = ξ1*Loss CE + ξ2*Loss Dice

[0024]

[0025]

[0026] wherein ξ1 and ξ2 are weight variables for adjusting Loss CEand Loss Dice between the stable; Loss CE is cross-entropy loss; Loss Dice is Dice coefficient loss; epsilon is a smoothing coefficient used when calculating Loss Dice .

[0027] (4.5) The burn segmentation model back-propagates to calculate the partial derivative of the learnable parameters, uses a stochastic gradient descent optimizer to process the gradient of the loss function, and uses the learning rate to update the learnable parameters of the burn segmentation model;

[0028] (4.6) Let i = i + 1, repeat steps (4.1)-(4.5) for a specified number of training periods, and store the burn segmentation model parameter file for the specified number of training periods.

[0029] Advantages: Compared with the prior art, the present application has the following significant advantages: 1. Based on the U-Net network structure and combined with the ResNet50 network and the ViT network, an end-to-end gear grinding burn segmentation model is constructed, which combines the shallow high-resolution local features extracted by the CNN network and the sequence global features extracted by the self-attention mechanism, better solves the problems of data lack and classification imbalance in the segmentation task, and can output higher resolution segmentation result images, improving the segmentation accuracy of the gear side surface grinding burn area; 2. By enhancing the number and diversity of the data set, the network generalization ability is improved. BRIEF DESCRIPTION OF DRAWINGS

[0030] Figure 1 is the workflow diagram of the present application;

[0031] Figure 2 is a comparison diagram of the present application and the U-Net network segmentation result. DETAILED DESCRIPTION

[0032] The technical solutions of the present application will be further described below in conjunction with the drawings.

[0033] As Figure 1 shown, the gear grinding burn segmentation method based on a deep convolutional neural network according to the present application comprises the following steps:

[0034] (1) Obtain the gear tooth side surface image of the gear to be detected and perform data preprocessing.

[0035] When the CCD camera is used to acquire the tooth flank surface image of the gear to be detected, the brightness and illumination conditions of the environment and other factors will interfere with the image, and the median filter algorithm is used to remove noise; gears made of different materials exhibit different colors and textures on the side surface, and the image is enhanced using grayscale transformation to adjust the contrast of the image and enhance the gear surface and grinding burn features in the experimental image, making the image information more obvious; due to the influence of the shooting angle and the lens depth of field, the tooth surface edge appears to be inclined by 1-2°, and the Radon transform is used to correct the tooth surface inclination of the image.

[0036] (2) The data after preprocessing is expanded using data enhancement technology to make a gear tooth flank surface grinding burn segmentation data set, specifically including the following steps:

[0037] (2.1) Preliminary expansion of the original gear tooth flank surface grinding burn data set;

[0038] The horizontal flip, vertical flip and random rotation method of spatial geometric transformation is used for preliminary expansion to obtain a gear tooth flank surface grinding burn pre-expansion data set;

[0039] (2.2) Further expansion using GAN network;

[0040] The preliminary expansion data set is input into the GAN network to generate images, and the gear grinding burn data set is further expanded by deep learning method. The GAN network includes two main parts: the generation network G (generator) and the discrimination network D (discriminator). The generation network starts with a random noise vector to generate images, and the discrimination network discriminates the authenticity of the generated images. The training process is alternately fixed for one side and updated for the other side until Nash equilibrium is reached. After training, the GAN network can learn complete image information, including the relative position of the tooth surface, tooth top surface and tooth bottom surface, the color and texture of the tooth surface, etc. When collecting the original data set, the light-dark relationship and the relative position of the tooth surface in the picture are artificially controlled to be consistent, and only the grinding burn defect feature exists between the pictures, so the training process is relatively stable;

[0041] (2.3) Making a gear tooth flank surface grinding burn segmentation data set;

[0042] All the images generated in the above steps are saved, the burn area is outlined using a labeling tool, and a class label is defined to produce a gear side surface grinding burn segmentation dataset. The dataset contains 400 images of gear side surfaces with grinding burn defects. According to the color depth of the gear side surface grinding burn area, the gear surface grinding burn sample image is divided into two categories: light burn and heavy burn. The light burn is the dark gray pixel part, classified as B, and the heavy burn is the black pixel part, classified as D. Each category is divided into three levels according to the area ratio of the burn area: the maximum area ratio of the burn area is less than 10%, and the burn level suffix code is 1; the maximum area ratio of the burn area is 10% to 25%, and the burn level suffix code is 2; the maximum area ratio of the burn area is greater than 25%, and the burn level suffix code is 3. The image resolution is 512x512 pixels, and it is a RGB three-channel color image.

[0043] (3) Construct a gear grinding burn segmentation model based on an improved U-Net network, which includes the following steps:

[0044] (3.1) In the encoder part of the burn segmentation model, ResNet50 network is used as the backbone network to extract different scale feature maps of the input image, which is divided into three modules, each module outputs a set of feature maps, and uses pre-activation operation, which is divided into three stages and the stacking number of residual blocks in each stage is 3, 4 and 9 respectively.

[0045] (3.2) The feature maps are flattened into a series of image blocks, converted into a sequence and attached with position encoding, input into the ViT network composed of several layers of multi-head self-attention layers and multi-layer perceptron to extract global information and get a one-dimensional feature vector. Among them, the sequence slice size is set to 16, the basic model of ViT network is used, the number of heads of multi-head self-attention module is 12 and the module is stacked 12 times.

[0046] (3.3) In the decoder part of the burn segmentation model, the self-attention one-dimensional feature vector is subjected to multiple upsampling operations and is connected to the feature maps of different scales by jumping, and the pixel-level gear side surface grinding burn segmentation result image is output. Among them, the 3 layers of feature maps of different scales extracted by CNN network are connected by jumping for 3 times, and then spliced with the up-sampling results of the corresponding layers of the decoder.

[0047] (4) Select 1120 images from the 1400 images in the gear surface image segmentation dataset obtained in step (2) as the training set, 140 images as the validation set, and 140 images as the test set, and use the training set and the validation set for model training. The training process includes the following steps:

[0048] (4.1) Set the training parameters, where the number of training epochs is 150, the number of image batches used by the burn segmentation model for each training round is 8, and the total number of iterations is 21000 rounds. Initialize the weight parameters of the burn segmentation model, where the initial weight parameters of the ViT network come from the pre-trained model parameters of the public dataset Imagenet21k.

[0049] (4.2) Perform the ith training round, load the gear side surface grinding burn segmentation dataset, read the tooth surface image as a matrix and the corresponding burn area and class label image as a matrix where H and W are the height and width of the tooth surface image, C I represents the number of channels of the tooth surface image, C T is the number of channels of the label image.

[0050] (4.3) Input the data into the burn segmentation model, and output the predicted label image matrix as

[0051] (4.4) Calculate the loss function value Loss after the ith training round, using the average of the cross-entropy loss and the Dice coefficient loss:

[0052] Loss = 0.5 * Loss CE + 0.5 * Loss Dice

[0053]

[0054]

[0055] where Loss CE is the cross-entropy loss; Loss Dice is the Dice coefficient loss; ε is the smoothing coefficient used when calculating Loss Dice .

[0056] (4.5) The burn segmentation model calculates the partial derivative of the learnable parameters through backpropagation, uses the stochastic gradient descent optimizer to process the gradient of the loss function, and sets the initial learning rate to 0.01. Subsequently, it is updated using discrete decay, which is used to update the learnable parameters of the burn segmentation model.

[0057] (4.6) Let i = i + 1, repeat steps (4.2)-(4.5) for the specified number of times according to the set number of training epochs, and store the burn segmentation model parameter file for the specified number of training epochs.

[0058] (5) adjusting the resolution of the to-be-tested tooth surface image to be consistent with the training data set used by the burn segmentation model; loading the specified burn segmentation model parameter file, inputting the adjusted to-be-tested tooth surface image into the trained burn segmentation model, outputting a mask image of the grinding burn area on the tooth surface, extracting and visualizing the grinding burn area on the tooth surface; according to the color depth and area proportion of the grinding burn area, the classification and level of the grinding burn on the tooth surface are determined. Among them, the white pixel part is the background area, the light gray pixel part is the tooth surface area, the dark gray pixel part is the mild burn, classified as B, and the black pixel part is the severe burn, classified as D. Mild burn B and severe burn D correspond to 3 levels according to the burn area proportion, respectively. The maximum proportion of the burn area is less than 10%, and the burn level suffix code is 1; the maximum proportion of the burn area is 10% to 25%, and the burn level suffix code is 2; the maximum proportion of the burn area is greater than 25%, and the burn level suffix code is 3.

[0059] Preferably, the hardware environment of the training process of the embodiment is Intel(R) Xeon(R) Gold 6248R CPU, using one NVIDIA GeForce RTX 3090 GPU for acceleration.

[0060] In order to evaluate the effect of the gear grinding burn segmentation model based on the improved U-Net network on the segmentation of the tooth surface grinding burn, the same training samples and initial parameters are selected to train the U-Net network segmentation model.

[0061] Randomly select 40 tooth surface images in the training set to input into the U-Net segmentation model and the burn segmentation model of the present application, respectively, to obtain two groups of segmentation result images, as shown in Figure 2 . Among them, Figure 2 (a) is the to-be-tested tooth surface image, Figure 2 (b) is the semantic segmentation label image of the tooth surface area and the grinding area labeled in step (2.3), Figure 2 (c) is the segmentation result image output by the U-Net segmentation model, Figure 2 (d) is the mild burn segmentation result image output by the U-Net segmentation model, Figure 2 (e) is the severe burn segmentation result image output by the U-Net segmentation model, Figure 2 (f) is the segmentation result image output by the burn segmentation model of the present application, Figure 2 (g) is the semantic segmentation result image output by the burn segmentation model of the present application.

[0062] By comparing the segmentation result images of the two models, the burn segmentation model provided by the application can extract more accurate grinding burn areas on the gear side surface, and compared with the U-Net segmentation model, better semantic segmentation results can be obtained, and the edge profile of the burn area is closer to the label image.

[0063] The segmentation results are evaluated by two indicators of Dice value and IoU value. In the research of image segmentation based on deep learning, Dice and IoU are main indicators for measuring the accuracy of image segmentation, and are used to measure the correlation between actual value and prediction result. The higher the correlation is, the larger the value is.

[0064] According to the following table, the statistical results of the burn segmentation model of the application are basically higher than those of the U-Net segmentation model, especially in the multi-classification semantic segmentation task, the accuracy is greatly improved. Therefore, the application can better solve the problems of data lack and classification imbalance in the segmentation task, can output higher resolution segmentation result images, and improve the segmentation accuracy of the grinding burn area of the gear side surface.

[0065] Table 1 Comparison of evaluation index results of two segmentation models

[0066]

Claims

1. A method for segmenting gear grinding burns based on a deep convolutional neural network, characterized in that, Includes the following steps: (1) Obtain images of the tooth side surfaces of the gear to be inspected and perform data preprocessing; (2) Data augmentation techniques were used to augment the preprocessed data, and then the burn area of ​​the image was delineated and category labels were defined to create a gear tooth side surface grinding burn segmentation dataset. (3) Construct a gear grinding burn segmentation model based on the improved U-Net network. The U-shaped structure of the U-Net network is used as the main body. The encoder part uses the residual neural network as the backbone network to extract feature maps of different scales of the input image. The feature maps are converted into sequences and attached with position codes. The ViT network is input to extract global information to obtain a one-dimensional feature vector. The decoder upsamples the self-attention one-dimensional feature vector, skips connections between feature maps of different scales, and outputs a pixel-level segmentation result of grinding and burning on the tooth side surface. The residual neural network mentioned in step (3) adopts the ResNet50 network; the ViT network includes several layers of multi-head self-attention layers and multilayer perceptrons; (4) Divide the segmented dataset obtained in step (2) into a training set, a validation set and a test set, and use the training set and the validation set to train the model; The model training process described in step (4) includes the following steps: (4.1) Initialize the weight parameters of the burn segmentation model; (4.2) Perform the i-th round of training, load the gear tooth side surface grinding burn segmentation dataset, and read the tooth surface image as a matrix. The matrix consists of images of the corresponding burn areas and category labels. Where H and W are the height and width of the tooth surface image, and C I C represents the number of channels in the tooth surface image. T The number of channels in the label image; (4.3) Input the data into the burn segmentation model, and output the predicted label image matrix as follows: ; (4.4) Calculate the loss function value after the i-th round of training, using the weighted sum of the cross-entropy loss and the Dice coefficient loss: Where ξ1 and ξ2 are weight variables used for adjustment. and Stability between them; Cross-entropy loss; For Dice coefficient loss; For calculation The smoothing coefficient used at that time; (4.5) The partial derivatives of the learnable parameters are calculated by backpropagation of the burn segmentation model. The gradient of the loss function is processed by the stochastic gradient descent optimizer and used together with the learning rate to update the learnable parameters of the burn segmentation model. (4.6) Let i = i+1, repeat steps (4.1)-(4.5) according to the set number of training periods, and store the burn segmentation model parameter file for the specified number of training periods; (5) Input the test set data into the trained segmentation model, extract the grinding burn area on the tooth side surface and visualize it, and determine the classification and level of grinding burn on the tooth side surface.

2. The gear grinding burn segmentation method according to claim 1, characterized in that, The data preprocessing described in step (1) includes using a filtering algorithm to remove noise, using grayscale transformation to enhance the image, and using geometric transformation to correct the tooth surface tilt of the image, to obtain the original dataset.

3. The gear grinding burn segmentation method according to claim 1, characterized in that, The data augmentation in step (2) includes performing preliminary augmentation on the original dataset using geometric transformation, color space transformation and / or pixel point operations, and then using a generative adversarial network to continue augmentation to obtain the augmented dataset.

4. The gear grinding burn segmentation method according to claim 1, characterized in that, The category label defined in step (2) includes two categories based on the color depth of the grinding burn area on the tooth side surface: mild burn and severe burn. Each category is further divided into three levels based on the area ratio of the burn area.

5. The gear grinding burn segmentation method according to claim 4, characterized in that, The term "mild burn" refers to dark gray pixels, classified as B, while "severe burn" refers to black pixels, classified as D. The three levels are as follows: burn area with a maximum percentage of less than 10% is designated as level 1; burn area with a maximum percentage of 10% to 25% is designated as level 2; and burn area with a maximum percentage of more than 25% is designated as level 3.

Images