A deep learning-based part surface defect classification system

Abstract

The application discloses a kind of parts surface defect classification systems based on deep learning, it belongs to image classification technical field.The present deep learning model is optimized channel quantity when the main technical scheme that the present application solves is that each channel in convolution kernel is respectively regarded as an independent candidate, according to control variable and the index of candidate, respectively, the index of each candidate is mapped to an indicator value with 0 or 1 value, and then channel selection is carried out according to the mapping result.The present deep learning model is optimized channel quantity when the present application method can solve the problem that the optimization time and storage overhead are large.The classification accuracy of the deep learning model trained can be significantly improved by optimizing the number of channels using the present application method.The present application method can be applied to the field of image classification.

Description

Technical Field

[0001] This invention belongs to the field of image classification technology, specifically relating to a deep learning-based system for classifying surface defects in parts. Background Technology

[0002] Parts include die-cast parts, plastic parts, rubber parts, and hardware parts. In the industrial field, classifying and detecting surface defects is essential for improving the quality of machined parts. However, currently, the main method for classifying and detecting surface defects is still manual inspection. With the development of deep learning technology, new possibilities have emerged for the development of surface defect classification and detection technologies.

[0003] In image classification of parts surfaces, the convolution kernel (or convolution matrix, or mask) is a small matrix used to achieve effects such as blurring, sharpening, embossing, and edge detection. This is achieved by convolving the kernel with the image. A single convolution can be represented as follows:

[0004]

[0005] Where g and f represent the convolved image and the original image, respectively, (x, y) are the pixel indices in the image, and (i, j) are the indices of the convolution kernel. When multiple feature maps appear in an image or during the convolution process, multi-channel convolution can also be performed.

[0006]

[0007] Therefore, the number of channels is one of the most important parameters of the convolution kernel.

[0008] Currently, optimizing the number of channels in convolution kernels using differential methods faces several challenges:

[0009] I. Integrating hyperparameters into the specific computational process of deep learning forward propagation.

[0010] The actual component involved in the forward propagation calculation is each individual channel, not the hyperparameter of the number of channels. However, the relevant information about the channels is already included in the calculation results; it's just that it cannot be directly optimized through backpropagation.

[0011] Second, measure the value of each instance on the candidate set.

[0012] Each channel contributes differently to the features, and the most valuable channels need to be selected through continuous iteration in deep learning. A common approach is to create a new selection variable for each channel to indicate whether the channel is needed, and to make the selection variable differentiable through relaxation. After multiple iterations, the value of each channel is determined based on the relative magnitude of the selection variable for each channel.

[0013] Third, reduce the error caused by approximation or relaxation operations in the algorithm.

[0014] After relaxation, the calculated features will contain a small amount of information about the unselected channels. This information cannot be too much, as too much will lead to large errors in the relaxation operation, resulting in a significant deviation between the observed performance and the actual performance of the current hyperparameter configuration. However, this information is also essential, as it contains crucial information guiding the selection variables of the channels. During backpropagation, this information is used to provide feedback to the selection variables of each channel.

[0015] Fourth, cover as much of the search space as possible with high-dimensional hyperparameters.

[0016] Deep multilayer perceptrons may have thousands of channels per layer, and the relaxed selection variables need to cover as large a search space as possible. If a substantial change in the number of channels represented by the relaxed selection variables requires multiple deep learning iterations, then a 200-epoch training session can only guarantee coverage of a very small portion of the search space.

[0017] In summary, existing deep learning models require significant optimization time and storage overhead when optimizing the number of channels, and the classification accuracy of deep learning models after optimizing the number of channels is relatively low. Summary of the Invention

[0018] The purpose of this invention is to address the problems of high optimization time and storage overhead required for optimizing the number of channels in existing deep learning models, and low classification accuracy of deep learning models after channel optimization. Therefore, this invention proposes a deep learning-based part surface defect classification system.

[0019] The technical solution adopted by the present invention to solve the above-mentioned technical problems is as follows:

[0020] A deep learning-based system for classifying surface defects in machine parts, the system comprising an image acquisition module, an image processing module, and a neural network module, wherein:

[0021] The image acquisition module is used to acquire images of the part surface and send the acquired images to the image processing module;

[0022] The image processing module is used to process the acquired image and send the processed image to the neural network module;

[0023] The neural network module is used to output image classification results;

[0024] The channel number optimization process during the training of the neural network module is as follows:

[0025] 1) Treat each channel in the convolution kernel as an independent candidate, and denote the index of the candidate as 1, 2, ..., n, where n is the number of channels;

[0026] 2) Throughout the iterative training process, the control variables are continuously updated. For each candidate, its index is mapped to a selection variable value of 0 or 1 based on the control variables.

[0027] Channel selection is performed based on the mapping results obtained from the last iteration of training.

[0028] The beneficial effects of this invention are:

[0029] This invention treats each channel in the convolutional kernel as an independent candidate. Based on control variables and the candidate's index, each candidate's index is mapped to an indicator value of 0 or 1. Channel selection is then performed based on the mapping results. This method addresses the problem of high optimization time and storage overhead in existing deep learning models when optimizing the number of channels.

[0030] Moreover, by optimizing the number of channels using the method of this invention, the classification accuracy of the trained deep learning model can be significantly improved.

[0031] Meanwhile, experimental results show that when using the method of this invention for network architecture search, it can accelerate by 20% compared to the current state-of-the-art FBNetV2, greatly shortening the optimization time. Attached Figure Description

[0032] Figure 1 This is a structural block diagram of a deep learning-based part surface defect classification system according to the present invention.

[0033] Figure 2 The image shows the optimization results based on DARTS.

[0034] The horizontal axis represents the channel number;

[0035] Figure 3This is a schematic diagram illustrating the optimization of the number of convolutional kernel channels using the method (SoftStep) of this invention. Detailed Implementation

[0036] The present application will now be described in further detail with reference to specific embodiments and accompanying drawings. Obviously, the described embodiments are merely a part of the embodiments of the present invention, and not all of them. Other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are all within the scope of protection of the present invention.

[0037] Specific Implementation Method 1: Combination Figure 1 This embodiment describes a deep learning-based part surface defect classification system. The system includes an image acquisition module, an image processing module, and a neural network module, wherein:

[0038] The image acquisition module is used to acquire an image containing the surface of the part and send the acquired image to the image processing module;

[0039] The image processing module is used to process the acquired image and send the processed image to the neural network module;

[0040] The neural network module is used to output image classification results; the classification results include whether the part has surface defects, and what type of defect it is;

[0041] The channel number optimization process during the training of the neural network module is as follows:

[0042] 1) Treat each channel in the convolution kernel as an independent candidate, and denote the index of the candidate as 1, 2, ..., n, where n is the number of channels;

[0043] 2) Throughout the iterative training process, the control variables are continuously updated. For each candidate, its index is mapped to a selection variable (indicator) value of 0 or 1 based on the control variables.

[0044] Channel selection is performed based on the mapping results obtained from the last iteration of training.

[0045] The method of this invention classifies the surface defects of parts in the image to be inspected, which can accurately detect whether there are surface defects in the image and process them in a timely manner.

[0046] Specific Implementation Method Two: This implementation method differs from Specific Implementation Method One in that the mapping is implemented using the SoftStep function, which is specifically as follows:

[0047]

[0048] Where x represents the index, S ～γ (x) represents the selection variable value mapped to the x-th index, σ(·) represents the Sigmoid function, and γ represents the control variable.

[0049] A first-order convolution with ReLU activation can be represented as:

[0050]

[0051] in It has C l-1 The input features of the l-th layer of each channel These are the kernel weights of the l-th layer. It has C l The output features of the l-th layer of each channel. We compare each output feature with the output features from S. ～γ [K] The transformations of the (x) sampled indicators can be summarized by the following formula:

[0052]

[0053] in, It is the input feature of the (l-1)th layer with j channels. It is the input feature of the (l-1)th layer with i channels. These are the kernel weights of the l-th layer. It is the bias of the l-th layer with j channels.

[0054] Compare each output feature with S ～γ (x) The multiplication of the indicator values ​​obtained from sampling is used as the output of this stage.

[0055] The other steps and parameters are the same as in Specific Implementation Method 1.

[0056] Specific Implementation Method 3: This implementation method differs from Specific Implementation Method 1 or 2 in that the control variable γ is updated using a gradient descent method.

[0057] This implementation method uses gradient descent to automatically fine-tune the control variables.

[0058] Other steps and parameters are the same as in specific implementation method one or two.

[0059] Specific Implementation Method Four: This implementation method differs from Specific Implementation Methods One to Three in that the control variable γ is updated using the Momentum optimizer.

[0060] The other steps and parameters are the same as those in one of the specific implementation methods one to three.

[0061] Specific Implementation Method Five: This implementation method differs from Specific Implementation Methods One to Four in that the mapping result obtained in the last iteration of training satisfies the constraint that: among the n candidates, only k candidates have a selection variable value of 1, and the selection variable values ​​of the other candidates are 0.

[0062] A value of 1 for the selected variable indicates that the current candidate is selected, meaning that the channel corresponding to the current candidate is selected.

[0063] The other steps and parameters are the same as those in one of the specific implementation methods one to four.

[0064] Specific Implementation Method Six: This implementation method differs from Specific Implementation Methods One to Five in that the deep learning model used in the neural network module is MobileNetV2.

[0065] The other steps and parameters are the same as those in one of the specific implementation methods one to five.

[0066] Specific Implementation Method Seven: This implementation method differs from Specific Implementation Methods One to Six in that the processing of the acquired image specifically involves:

[0067] The acquired image is segmented into target and background. Then, the size of the segmented target image is normalized. The image after size normalization is the processed image.

[0068] The other steps and parameters are the same as those in one of the specific implementation methods one to six.

[0069] The traditional DARTS algorithm can be theoretically abstracted into two logical steps. The first is a Max function mapping from [1,n] to {0,1}, and the second is the relaxation of the Max function, i.e., the SoftMax function. In the Max function mapping, candidates are sequentially labeled from 1 to n, each mapped to an indicator with a value of 0 or 1. These indicators are constrained to have only one indicator that can be 1, and the rest that are 0. At the end of training, the candidate with the largest indicator is selected as the optimal search result. The relaxation step aims to make the indicators differentiable while preserving the constraints as much as possible. DARTS assigns each candidate an independent and unconstrained control variable and generates a constrained indicator by applying the SoftMax function to the control variable. SoftMax maintains the constraints by ensuring that the indicator is positive and the sum of the relaxation indicators is 1. In the supernet, each candidate is multiplied by its own relaxation indicator during forward propagation, which is equivalent to taking a weighted sum as the output of that stage. Therefore, we can use gradient descent to update the control variable and the indicator and automatically select the best candidate.

[0070] Figure 2 The example shows 10 candidates. These candidates are indexed from 1 to 10, with the 5th candidate being selected. In general, DARTS effectively extracts a relaxed indicator from the SoftMax distribution using the candidate indices.

[0071] The visualization process of the method of this invention is as follows: Figure 3 As shown, the input image with 3 channels is first convolved into features with 8 channels. Then each channel is convolved with an indicator S. j Element-wise multiplication is performed. Finally, some channels are weakened by an indicator with a value of 0. According to the SoftStep proposed by this invention, the index is less than γ. (l) The feature that is γ will receive an indicator of approximately 1, while other features will receive an indicator of approximately 0. During search training, the control variable γ... (l) The gradient descent algorithm will be used for updating. This is equivalent to automatically optimizing the control variable γ. (l) This invention optimizes the channel. Furthermore, the method (SoftStep) can be applied to more hyperparameter optimizations, such as the size of the convolutional kernel and the number of neurons in an MLP.

[0072] The flowchart of the method of the present invention is shown in Table 1:

[0073] Table 1

[0074]

[0075] Line 2 indicates that we use the scheduler to update the learning rate of the model parameters; the learning rate for the architecture parameters is a fixed value. In line 4, we calculate the control variable γ′ used based on γ as the implementation of the regularization term. In line 5, the model parameters are updated using weight decay and basic gradient descent. In line 6, the architecture parameters are also updated using weight decay and gradient descent. Practice shows that the Momentum optimizer can perform better. Finally, we use the obtained γ... * Generate sub-models for hypernetwork recommendations. For example, if the search space of a certain convolutional layer channel is C, the optimal value of the corresponding control variable is 1 / C≤γ. * If ≤1, then generate This is the optimal channel for the convolutional layer.

[0076] The above examples of the present invention are merely illustrative of the computational model and process of the present invention, and are not intended to limit the implementation of the present invention. Those skilled in the art will recognize that other variations or modifications can be made based on the above description. It is impossible to exhaustively list all possible implementations here. Any obvious variations or modifications derived from the technical solutions of the present invention are still within the scope of protection of the present invention.

Claims

1. A deep learning-based system for classifying surface defects in parts, characterized in that, The system includes an image acquisition module, an image processing module, and a neural network module, wherein: The image acquisition module is used to acquire images of the part surface and send the acquired images to the image processing module; The image processing module is used to process the acquired image and send the processed image to the neural network module; The neural network module is used to output image classification results; The channel number optimization process during the training of the neural network module is as follows: 1) Treat each channel in the convolution kernel as an independent candidate, and denote the indices of the candidates as 1, 2, ... n , n It refers to the number of channels; 2) Throughout the iterative training process, the control variables are continuously updated. For each candidate, its index is mapped to a selection variable value of 0 or 1 based on the control variables. The mapping is implemented using the SoftStep function, which is specifically as follows: in, x Represents an index. Representing the x Each index maps to the value of the selection variable. Represents the Sigmoid function. Represents control variables; Channel selection is performed based on the mapping results obtained from the last iteration of training. The mapping result obtained in the last iteration of training satisfies the following constraint: n Of the candidates, only k The selection variable value for one candidate is 1, and the selection variable value for the other candidates is 0; A value of 1 for the selected variable indicates that the current candidate is selected, meaning that the channel corresponding to the current candidate is selected.

2. The deep learning-based part surface defect classification system according to claim 1, characterized in that, The control variable The update is performed using gradient descent.

3. The deep learning-based part surface defect classification system according to claim 1, characterized in that, The control variable Updates are performed using the Momentum optimizer.

4. A deep learning-based part surface defect classification system according to claim 2 or 3, characterized in that, The neural network module uses the MobileNetV2 deep learning model.

5. A deep learning-based part surface defect classification system according to claim 4, characterized in that, The specific processing of the acquired image is as follows: The acquired image is segmented into target and background. Then, the size of the segmented target image is normalized. The image after size normalization is the processed image.

Images