An image classification system based on a CNN model

Abstract

The application discloses a kind of CNN model-based image classification system, it belongs to image classification technical field.The application solves the problem that the accuracy of image classification of the deep learning model obtained by using existing learning rate planning method is poor.The main technical scheme adopted by the application is as follows: a part surface image is acquired using an image acquisition module, and the acquired image is sent to an image processing module;the acquired image is processed using the image processing module, and the processed image is sent to a CNN model;the CNN model outputs the classification result of the image;when the CNN model is trained, the learning rate is updated according to the updated learning rate gradient after updating the gradient of the learning rate according to the model parameters, which can significantly improve the classification accuracy of the trained deep learning model.The application method can be applied to the field of image classification technology.

Description

Technical Field

[0001] The present invention belongs to the technical field of image classification, and in particular relates to an image classification system based on a CNN model. Background Art

[0002] Parts include die-castings, plastics, rubber parts, and hardware. In the industrial sector, classifying and detecting surface defects on parts is essential for improving the quality of machined parts. However, manual inspection remains the primary method for classifying and detecting surface defects. The development of deep learning technology has opened up new possibilities for the development of surface defect classification and detection technology.

[0003] The learning rate plays a decisive role in the training process of neural networks. HD (hypergradient descent) achieves learning rate adaptation by calculating the approximate derivative of the learning rate with respect to the loss function and using gradient descent. Therefore, HD better understands the learning rate needs the loss function, that is, when to increase and when to decrease. We can interpret the idea of ​​HD as follows: if the gradient of the parameter indicates the same direction in the last two iterations, the learning rate will be expanded, and if the direction is opposite, the learning rate will be reduced. However, there are often cases of HD misunderstanding, that is, after HD adapts the learning rate, the value of the loss function unexpectedly increases, and this misunderstanding is precisely due to the lack of foresight, that is, the lack of understanding of the future distribution. Under the control of HD, for batch training and mini-batch training, the learning rate is adjusted using formula (1) in each iteration.

[0004]

[0005] Intuitively, if the step direction indicated by the parameter gradient is the same as the step direction of the previous iteration, it can be understood that the current position is on a smooth descent surface, and acceleration is a better strategy for approaching the extreme point. However, experience-based HD methods suffer from two fatal flaws: misunderstanding the future distribution and being unsuitable for small batch training.

[0006] In HD, such as Figure 2a 、 Figure 2b and Figure 2c The situation shown in the figure always exists, that is, a simple convex optimization starts from x0, and x1 is very close to the extreme point after the first iteration. According to the learning rate update rule of HD, increasing the step size to a larger value causes HD to miss, that is,

[0007] Mini-batch training is more complicated than batch training because there are multiple cost functions. The adaptive rule of HD for learning rate is actually:

[0008]

[0009] Gradient used and are not on the same distribution, in other words, To L t It is meaningless.

[0010] In short, existing learning rate adaptation methods lack sufficient sensitivity and application prospects, and the current mainstream learning rate planning methods all use manual planning, such as the cosine annealing algorithm and step-change learning rate. These methods do not fully utilize the gradient of the learning rate to plan the learning rate, which greatly inhibits the further improvement of neural network performance.

[0011] In summary, the accuracy of deep learning models for image classification obtained using existing learning rate planning methods is still poor. Summary of the Invention

[0012] The purpose of this invention is to solve the problem of poor accuracy of image classification of deep learning models obtained by using existing learning rate planning methods, and to propose an image classification system based on the CNN model.

[0013] The technical solution adopted by the present invention to solve the above technical problems is:

[0014] According to one aspect of the present invention, an image classification system based on a CNN model includes an image acquisition module, an image processing module, and a CNN model, wherein:

[0015] The image acquisition module is used to acquire the surface image of the part and send the acquired image to the image processing module;

[0016] The image processing module is used to process the acquired image and send the processed image to the CNN model;

[0017] The CNN model is used to output image classification results;

[0018] During the CNN model training process, the learning rate adjustment process is as follows:

[0019] Step 1. Set the global loss function to L(W), initialize the model parameter W to W0, initialize the learning rate α to α0, initialize the adaptive period to T, and initialize the total number of iterations to N;

[0020] And let the number of iterations t = 1;

[0021] Step 2: Calculate the model parameter W1 after the tth iteration:

[0022]

[0023] in, L(W t-1 )’s gradient;

[0024] Calculate the loss function L(W t-1 ) with respect to the gradient of the learning rate α:

[0025]

[0026] in, L(W t )’s gradient;

[0027] Step 3: determine whether t%T=0 is satisfied;

[0028] If satisfied, go to step 4, otherwise not satisfied, go to step 5;

[0029] Step 4: According to Update the learning rate and express the updated learning rate as α t ;

[0030] Update the learning rate and then execute step 5;

[0031] Step 5: Determine whether t is less than N. If t is less than N, set t = t + 1 and return to step 2. If t is equal to N, end and obtain the final learning rate.

[0032] According to another aspect of the present invention, an image classification system based on a CNN model includes an image acquisition module, an image processing module, and a CNN model, wherein:

[0033] The image acquisition module is used to acquire the surface image of the part and send the acquired image to the image processing module;

[0034] The image processing module is used to process the acquired image and send the processed image to the CNN model;

[0035] The CNN model is used to output image classification results;

[0036] During the CNN model training process, the learning rate adjustment process is as follows:

[0037] Step 1: Set the loss functions for training with n batches of small batch samples to be L1, ..., L n ; Set the total number of iterations to N; initialize the learning rate α to α0;

[0038] Step 2: Initialize the number of iterations to 1;

[0039] Step 3, initialize batch t = 1, initialize model parameter W as W0, initialize adaptive period as T, initialize learning rate gradient to 0;

[0040] Step 4, calculate model parameter W after training input t-th batch t

[0041]

[0042]

[0043]

[0044] wherein, represents the gradient of L t (W t-1 ),

[0045] When t < n, then update learning rate gradient and G t-1 according to L1(W

[0046] When t = -n, then update learning rate gradient n and G n according to L1(W

[0047] Determine whether t%T = 0 is satisfied, if satisfied, jump to step 5, if not satisfied, jump to step 6;

[0048] Step 5, update learning rate a according to updated learning rate gradient;

[0049] After updating the learning rate, initialize learning rate gradient to 0, and jump to step 6;

[0050] Step 6, if t < n, then let t = t + 1, return to step 3;

[0051] If t = n, then return to step 3 after increasing the iteration number by 1;

[0052] Until the maximum iteration number N is reached, the updated learning rate of the last iteration is taken as the final learning rate.

[0053] The beneficial effects of the present application are:

[0054] After updating the gradient of the learning rate according to the model parameter, the learning rate is updated according to the updated learning rate gradient, which solves the problem of poor accuracy of the deep learning model obtained by using the existing learning rate planning method for image classification. ​BRIEF DESCRIPTION OF THE DRAWINGS

[0055] Figure 1 This is a structural block diagram of an image classification system based on a CNN model of the present invention;

[0056] Figure 2a It is a schematic diagram of the learning rate initialization stage;

[0057] In the figure, the horizontal axis represents the model parameters. The horizontal axis is the independent variable x, and the vertical axis is the dependent variable y. The relationship between the two variables is y=x 2 ; At this time, the model has only one model parameter;

[0058] Figure 2b This is a schematic diagram of the optimization results of the first stage of the learning rate;

[0059] Figure 2c This is a schematic diagram of the second stage optimization results of the learning rate under the HD method;

[0060] Figure 3a This is a comparison chart of the method of the present invention and the SGD (stochastic gradient descent) method. Figure 1 ;

[0061] In the figure, ADS represents the method of the present invention, the horizontal axis W represents the parameters of the neural network, and the vertical axis represents the loss function;

[0062] Figure 3b FIG2 is a comparison diagram of the method of the present invention and the SGD (stochastic gradient descent) method. DETAILED DESCRIPTION

[0063] The present application will be further described in detail below through specific embodiments in conjunction with the accompanying drawings. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, other embodiments obtained by ordinary technicians in this field without making any creative work are within the scope of protection of the present invention.

[0064] Specific implementation method 1. Combination Figure 1 This embodiment describes an image classification system based on a CNN model, the system comprising an image acquisition module, an image processing module, and a CNN model, wherein:

[0065] The image acquisition module is used to acquire the surface image of the part and send the acquired image to the image processing module;

[0066] The image processing module is used to process the acquired image and send the processed image to the CNN model;

[0067] The CNN model is used to output image classification results; the classification results are whether there are defects on the surface of the part and what kind of defects they are;

[0068] During the CNN model training process, the learning rate adjustment process is as follows:

[0069] Step 1. Set the global loss function to L(W), initialize the model parameter W to W0, initialize the learning rate α to α0, initialize the adaptive period to T, and initialize the total number of iterations to N;

[0070] And let the number of iterations t = 1;

[0071] Step 2: Calculate the model parameter W after the tth iteration t :

[0072]

[0073] in, L(W t-1 )’s gradient;

[0074] Calculate the loss function L(W t-1 ) with respect to the gradient of the learning rate α:

[0075]

[0076] in, L(W t )’s gradient;

[0077] Step 3: determine whether t%T=0 (meaning that t divided by T has a remainder of 0);

[0078] If satisfied, go to step 4, otherwise not satisfied, go to step 5;

[0079] Step 4: According to Update the learning rate and express the updated learning rate as α t ;

[0080] Update the learning rate and then execute step 5;

[0081] Step 5: Determine whether t is less than N. If t is less than N, set t = t + 1 and return to step 2. If t is equal to N, end and obtain the final learning rate.

[0082] In this embodiment, the learning rate is updated once every T iterations, and the learning rate remains unchanged between two updates.

[0083] If W T-1 is regarded as a natural assignment of parameter W, then should be 0, and the gradient of α collapses to We call this first-order detection. Similarly, second-order detection corresponds to The gradient of α is Here we take a simple approximation and ignore the second-order error term

[0084] This embodiment is suitable for batch training. The algorithm of this embodiment is summarized in Table 1, where the second row is the normal SGD backpropagation. If the learning rate needs to be adjusted, the first-order or second-order detection of ADS will be applied on α, corresponding to rows 3 to 6.

[0085] Table 1

[0086]

[0087] Specific embodiment 2: This embodiment differs from specific embodiment 1 in that the acquired image is processed as follows:

[0088] The acquired image is segmented into target and background, and then the target image obtained by segmentation is size normalized. The size normalized image is the processed image.

[0089] Other steps and parameters are the same as those in the first embodiment.

[0090] Specific implementation method three: This implementation method is different from specific implementation methods one or two in that the Update the learning rate. The update process is as follows:

[0091]

[0092] Among them, δ is the step size of the learning rate, α t-1 is the learning rate for the t-1th iteration (if the learning rate is updated at the t-1th iteration, then α t-1 is the updated learning rate at the t-1th iteration. If the learning rate is not updated at the t-1th iteration, then α t-1 is the learning rate after the last update), α t is the updated learning rate.

[0093] Other steps and parameters are the same as those in the first or second embodiment.

[0094] Specific implementation method 4: Combination Figure 1 This embodiment describes an image classification system based on a CNN model, the system comprising an image acquisition module, an image processing module, and a CNN model, wherein:

[0095] The image acquisition module is used to acquire the surface image of the part and send the acquired image to the image processing module;

[0096] The image processing module is used to process the acquired image and send the processed image to the CNN model;

[0097] The CNN model is used to output image classification results;

[0098] During the CNN model training process, the learning rate adjustment process is as follows:

[0099] Step 1: Set the loss functions for training with n batches of small batch samples to be L1, ..., L n ; Set the total number of iterations to N; initialize the learning rate α to α0;

[0100] Step 2: Initialize the number of iterations to 1;

[0101] Step 3: Initialize the learning rate gradient is 0; initialize the model parameter W to W0;

[0102] Step 4: Initialize batch t=1 and initialize the adaptive period to T;

[0103] Step 5: Calculate the model parameters W after inputting the tth batch for training t ;

[0104]

[0105]

[0106]

[0107] in, Represents L t (W t-1 ),

[0108] When t<n, according to and G t-1 Update the learning rate gradient

[0109] When t=n, according to L1(W n ) and G n Update the learning rate gradient

[0110] Determine whether t%T=0 is satisfied. If so, jump to step 6. If not, jump to step 7.

[0111] Step 6: Update the learning rate α according to the updated learning rate gradient;

[0112] After updating the learning rate, initialize the learning rate gradient is 0, and jump to step 7;

[0113] Step 7: If t < n, set t = t + 1 and return to step 5;

[0114] If t=n, then increase the number of iterations by 1 and return to step 3;

[0115] When the maximum number of iterations N is reached, the learning rate after the last iteration update is used as the final learning rate.

[0116] In this embodiment, the learning rate remains unchanged between two updates of the learning rate. The learning rate is continuously updated throughout the training process, and between two updates of the learning rate, all learning rates used are the learning rates after the previous update.

[0117] This embodiment is suitable for mini-batch training. The algorithm of this embodiment is summarized in Table 2. Rows 1 to 16 are a normal mini-batch training. Row 3 indicates that this period will be used to adjust the learning rate. Row 4 indicates that Represents the gradient of the learning rate, which accumulates with batch iterations. Line 10 corresponds to the incremental term of the formula. Line 11 completes the adaptation of the learning rate α. Finally, the parameters are restored to W0, because we regard W0 as the natural assignment of parameters, and W t It is just a temporary variable when adjusting the learning rate.

[0118] Table 2

[0119]

[0120] Specific embodiment 5: This embodiment differs from specific embodiment 4 in that the acquired image is processed as follows:

[0121] The acquired image is segmented into target and background, and then the target image obtained by segmentation is size normalized. The size normalized image is the processed image.

[0122] Other steps and parameters are the same as those in the fourth embodiment.

[0123] Specific embodiment 6: This embodiment is different from specific embodiment 4 or 5 in that the and G t-1 Update the learning rate gradient The specific update method is:

[0124]

[0125] Among them, “←” represents updating the parameters.

[0126] Other steps and parameters are the same as those in the fourth or fifth embodiment.

[0127] Specific embodiment seven: This embodiment differs from any one of the specific embodiments four to six in that the n ) and G n Update the learning rate gradient The specific update method is:

[0128]

[0129] Among them, “←” represents updating the parameters.

[0130] The other steps and parameters are the same as those in any one of the fourth to sixth embodiments.

[0131] Specific embodiment eight: This embodiment differs from any one of specific embodiments four to seven in that the learning rate α is updated according to the updated learning rate gradient. The specific updating method is:

[0132]

[0133] Here, δ is the learning rate step size.

[0134] The other steps and parameters are the same as those in any one of the fourth to seventh embodiments.

[0135] pass Figure 3a and Figure 3b It can be seen that the method of the present invention is significantly superior to the existing SGD method. Optimizing the learning rate using the method of the present invention can significantly improve the accuracy of defect classification in part surface images, facilitating the timely detection of part surface defects.

[0136] The above examples are merely illustrative of the calculation model and process of the present invention and are not intended to limit the embodiments of the present invention. Persons skilled in the art will readily appreciate that other variations or modifications based on the above description are possible. This list of embodiments is not exhaustive; however, any obvious variations or modifications derived from the technical solution of the present invention remain within the scope of protection of the present invention.

Claims

1. An image classification system based on a CNN model, characterized in that: The system includes an image acquisition module, an image processing module and a CNN model, wherein: The image acquisition module is used to acquire the surface image of the part and send the acquired image to the image processing module; The image processing module is used to process the acquired image and send the processed image to the CNN model; The CNN model is used to output image classification results; During the CNN model training process, the learning rate adjustment process is as follows: Step 1. Set up the application The loss functions for training with batches of small batch samples are ; Set the total number of iterations to N; Initialize the learning rate for ; Step 2: Initialize the number of iterations to 1; Step 3: Initialize the learning rate gradient 0; initialize model parameters for ; Step 4: Initialize the batch t=1 and the adaptive period to T; Step 5: Calculate the model parameters after inputting the tth batch for training ; in, represent The gradient, = ; When t< When and Update the learning rate gradient ; When t= When and Update the learning rate gradient ; Determine whether it is satisfied %T=0, if satisfied, jump to step 6, if not satisfied, jump to step 7; Step 6: Adjust the learning rate according to the updated learning rate gradient Make updates; After updating the learning rate, initialize the learning rate gradient is 0, and jump to step 7; Step 7: If t < , then let t=t+1 and return to step 5; If t= , then increase the number of iterations by 1 and return to step 3; When the maximum number of iterations N is reached, the learning rate after the last iteration update is used as the final learning rate.

2. The image classification system based on the CNN model according to claim 1, characterized in that: The processing of the acquired image is specifically as follows: The acquired image is segmented into target and background, and then the target image obtained by segmentation is size normalized. The size normalized image is the processed image.

3. The image classification system based on the CNN model according to claim 2, characterized in that: The basis and Update the learning rate gradient , the specific update method is: / n in," " represents updating the parameters.

4. The image classification system based on the CNN model according to claim 2, characterized in that: The basis and Update the learning rate gradient , the specific update method is: / n in," " represents updating the parameters.

5. The image classification system based on the CNN model according to claim 3 or 4, characterized in that: The learning rate is adjusted according to the updated learning rate gradient To update, the specific update method is: in, is the learning rate step size.

Images