Steel product quality defect identification method based on multi-objective evolution deep learning

Through multi-objective evolution deep learning technology, the quality defect recognition model of steel products is automatically generated and optimized, which solves the problem of artificial experience dependence in the existing technology, achieves a balance between high recognition accuracy and low model complexity, and improves the surface of steel products Efficiency and accuracy of quality inspection.

CN119068240AActive Publication Date: 2024-12-03NORTHEASTERN UNIV CHINA

Patent Information

Application Number
CN202411086279.X
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-08
Publication Date
2024-12-03
Estimated Expiration
2044-08-08

AI Technical Summary

Technical Problem

The prior art relies on manual experience to define neural network model structure in surface quality detection of steel products, making it difficult to balance model complexity and recognition accuracy, resulting in insufficient recognition accuracy or low generalization ability.

Method used

The multi-objective evolution deep learning method is adopted to randomly generate the steel product quality defect recognition model through automatic neural network deepening technology, and the multi-objective differential evolution algorithm is used to optimize the model structure to balance the recognition accuracy and model complexity.

Benefits of technology

We successfully got rid of the dependence of artificial experience and obtained a quality defect recognition model that performed better in terms of recognition accuracy and model complexity, which improved detection efficiency and accuracy.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119068240A_ABST
    Figure CN119068240A_ABST
Patent Text Reader

Abstract

The invention provides a steel product quality defect identification method based on multi-objective evolution deep learning, and relates to the technical field of steel product quality identification. The method comprises the following steps: firstly, constructing an original steel product quality defect identification model by adopting an automatic neural network deepening technology; then pruning the obtained optimal original steel product quality defect identification model by adopting a multi-objective differential evolution algorithm to reduce the network calculation complexity; then, on the basis of the preference vector, determining a preference Knee solution on a Pareto leading edge obtained after pruning processing of a multi-target differential evolution algorithm; and finally, determining a steel product quality defect identification model according to the obtained Knee solution, carrying out training again to obtain a corresponding accuracy rate on the test data set, and verifying the effectiveness of the model. The method successfully gets rid of defining a network structure depending on artificial experience, and the obtained quality defect identification model shows ideal performance in the aspects of identification accuracy and model complexity.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of steel product quality identification, and in particular to a steel product quality defect identification method based on multi-objective evolutionary deep learning. Background Art

[0002] Steel products are one of the most important and widely used products in industrial production and daily life, with the characteristics of large-scale production, convenient processing, diverse and reliable performance, low price, convenient use and recyclability. In the manufacturing process of steel products, various types of defects such as inclusions, scars, scratches, cracks and spots on the surface greatly reduce important performance indicators such as corrosion resistance, wear resistance and fatigue strength of the products. Therefore, identifying and testing the surface quality of steel products is a key means to determine their overall quality.

[0003] In the field of surface quality inspection of steel products, it is particularly important to correctly identify surface defects. Affected by the combined influence of various factors such as heating system, production process, cooling speed, heat treatment method, etc., the surface defects of steel products are diverse and complex. Traditional online detection methods mainly rely on manual observation and experience to judge the type of surface defects, which not only wastes labor, but also makes it difficult to achieve efficient and accurate detection. Deep learning models combine low-level features to form more abstract high-level representations to describe attribute categories or features, thereby discovering distributed feature representations of data. They have strong robustness and generalization capabilities. The use of deep learning technology for steel surface image feature extraction and defect detection has gradually become a research hotspot.

[0004] Many patents have been published on methods for identifying surface defects of steel products. For example, patent CN112164040A uses a semi-supervised deep learning algorithm to detect surface defects of steel products, enabling the neural network model to adapt to unlabeled sample data, greatly reducing the labeled sample data required by the classifier and reducing the cost of classifier training. Patent CN109242825 A provides a method for detecting the surface quality of steel products based on deep learning technology and builds a matching identification device. Its advantage is that the neural network model can be distributedly deployed and applied according to the equipment performance and model scale, and the surface quality of steel products can be detected in real time and with high precision.

[0005] The above patents all use a fixed-structure neural network model to detect the surface quality of steel products, requiring the user to manually define the model structure in advance. However, for specific problems, an overly simple model structure may lead to insufficient accuracy, while an overly complex model may reduce the generalization ability of the model. This reliance on manual experience limits the further application of deep learning technology in the field of steel product surface quality detection. Summary of the invention

[0006] The technical problem to be solved by the present invention is to provide a steel product quality defect recognition method based on multi-objective evolutionary deep learning in view of the deficiencies of the above-mentioned prior art, which integrates the advantages of multi-objective evolutionary algorithm and deep learning technology, uses automatic deepening technology to randomly generate an original steel product quality defect recognition model with high recognition accuracy, and then uses a multi-objective differential evolution algorithm to optimize the automatically generated original quality defect recognition model structure to balance the two optimization goals of model complexity and recognition accuracy. Through experimental testing, the proposed method successfully got rid of the reliance on artificial experience to define the network structure, and the obtained quality defect recognition model showed relatively ideal performance in terms of recognition accuracy and model complexity.

[0007] In order to solve the above technical problems, the technical solution adopted by the present invention is: a method for identifying quality defects of steel products based on multi-objective evolutionary deep learning, comprising the following steps:

[0008] Step 1: Collect product surface defect images during steel production and processing to obtain sample image data.

[0009] Step 2: Label the defect types of the sample image data obtained in step 1 and divide the data set;

[0010] Step 2.1: label the sample image data obtained in step 1 according to the type of surface defects of the product;

[0011] Step 2.2: Divide the sample image data annotated in step 2.1 into a training data set, a validation data set, and a test data set in proportion;

[0012] Step 3: Use automatic neural network deepening technology to build a quality defect recognition model for original steel products;

[0013] Step 3.1: Model initialization process, randomly generate a residual block as the initial steel product quality defect recognition model to identify steel product quality defects, and obtain the evaluation index on the verification data set by training the initial steel product quality defect recognition model, that is, the recognition accuracy, and record its accuracy as the model accuracy;

[0014] Step 3.1.1: randomly generate a residual block as an initial steel product quality defect recognition model; the randomly generated residual block consists of several residual layers, each residual layer includes two convolutional layers and two batch normalization layers, and the number of convolution kernels of all convolutional layers in the same residual block is the same; each residual block is represented by three parameters, namely: the number of residual layers, the number of convolution kernels, and whether to downsample, where the number of residual layers represents the number of residual layers in the current residual block; the number of convolution kernels represents the number of filters of all convolutional layers in the current residual block; and whether to downsample represents whether the current residual block performs a downsampling operation on its input; the initial steel product quality defect recognition model is generated by randomly taking values ​​of the three parameters;

[0015] Step 3.1.2: Train the initial steel product quality defect recognition model generated in step 3.1.1 to obtain the recognition accuracy on the validation data set, which is recorded as the model accuracy;

[0016] Step 3.2: Use automatic neural network deepening technology to perform multiple rounds of neural network automatic deepening process on the initial quality defect recognition model; in each round of deepening process, generate a random number of residual blocks, and then add the randomly generated residual blocks to the initial steel product quality defect recognition model in turn, and obtain the accuracy based on the verification data set through training. After each round of deepening process, return the best model and the corresponding highest accuracy based on the verification data set;

[0017] Step 3.2.1: In a round of automatic neural network deepening, randomly generate multiple residual blocks with different network structures according to the method of step 3.1.1;

[0018] Step 3.2.2: Add the multiple residual blocks with different network structures generated in step 3.2.1 to the initial steel product quality defect recognition model in turn, obtain multiple new steel product quality defect recognition models and train them, so as to obtain the new network with the highest recognition accuracy on the verification data set as the initial steel product quality defect recognition model for the next automatic neural network deepening. Each time a new residual block is added to the initial steel product quality defect model, a new steel product quality defect recognition model is retrained, and its model parameter settings are consistent with step 3.1.2; by training the new steel product quality defect recognition model, an evaluation indicator based on the verification data set, namely, the accuracy, is obtained, and the accuracy is recorded as the test accuracy;

[0019] Step 3.2.3: In the process of adding residual blocks, if the residual block added is the first randomly generated residual block in the current round of deepening, the test accuracy obtained by training the model with the added residual block is assigned to the highest accuracy obtained on the validation data set in the current round, and the model with the added residual block is recorded as the best model obtained by the current round of automatic neural network deepening;

[0020] Step 3.2.4: If the added residual block is not the first residual block randomly generated in the current round of deepening, but its test accuracy is greater than the highest accuracy obtained after the current round of automatic neural network deepening, then the test accuracy is regarded as the highest accuracy, and the new steel product quality defect recognition model after adding the residual block is recorded as the best model obtained by the current round of automatic neural network deepening;

[0021] Step 3.2.5: Determine whether the new steel product quality defect recognition model after adding the residual block is the last model tried in the current round. If so, the current round of automatic neural network deepening process ends, and the best model obtained in the current round and the corresponding highest accuracy based on the verification data set are returned, and the obtained best model is used as the initial steel product quality defect recognition model for the next round of deepening process; if not, return to step 3.2.2;

[0022] Step 3.3: Compare the highest accuracy based on the validation set obtained in the automatic neural network deepening process in step 3.2 with the model accuracy obtained in step 3.1. If the former is greater than the latter, the number of residual blocks that do not improve the model performance after joining the model is recorded as 0, and the former is assigned to the latter; otherwise, the number of residual blocks that do not improve the model performance after joining the model is increased by 1;

[0023] Step 3.4: In a neural network automatic deepening process, if adding residual blocks to the initial steel product quality defect recognition model for two consecutive rounds does not bring about model performance improvement, then the current round of automatic neural network deepening process is stopped, and the best model obtained in the current round is recorded as the original steel product quality defect recognition model, and the model and corresponding accuracy rate that have not been improved in the two rounds are recorded; otherwise, return to step 3.2 and continue to execute a new round of neural network model deepening process;

[0024] Step 3.5: After an automatic deepening process stops, determine whether this is the last automatic neural network deepening process set. If not, return to step 3.1 to perform a new automatic neural network deepening process to obtain a new original steel product quality defect recognition model; if this is the last automatic neural network deepening process, return the best model of all automatic neural network deepening processes;

[0025] Step 4: Use a multi-objective differential evolution algorithm to prune the best original steel product quality defect recognition model obtained in step 3, and retrain it to reduce the network calculation complexity while maintaining the accuracy of the model; select the error rate of the steel product quality defect recognition model on the verification data set, that is, the difference between the accuracy and 1, and the number of floating-point operations per second FLOPs of the model as two optimization targets;

[0026] Step 4.1: First, encode the original steel product quality defect recognition model, using a string of 0 / 1 codes to represent the quality defect recognition model, where 0 represents discarding the convolution filter and 1 represents retaining the convolution filter;

[0027] Step 4.2: Based on the encoded original steel product quality defect recognition model and the preset population size, an initial population for the pruning process of the multi-objective differential evolution algorithm is generated;

[0028] Step 4.3: Train each individual in the initial population and obtain the evaluation index accuracy based on the validation data set and the total floating-point operations per second (FLOPs) of the model. The total floating-point operations per second (FLOPs) of the model is the sum of the floating-point operations per second (FLOPs_1) of the two-dimensional convolution structure of the model and the floating-point operations per second (FLOPs_2) of the fully connected layer of the model.

[0029] Step 4.4: Use a multi-objective differential evolution algorithm to prune the steel quality defect recognition model. The operation process includes differential mutation, crossover and selection of retained individuals. Through these operations, individuals in the population are gradually optimized.

[0030] Step 4.4.1: Determine the mutation operator used in this evolution according to the preset initial mutation operator and the current evolution generation; use the operator to perform differential mutation on the individuals in the parent population to obtain the mutant sub-population;

[0031] The differential mutation strategy expression used is as follows:

[0032]

[0033] in, represents the i-th differential mutation vector generated in the t-th iteration, F is the mutation operator, represents three different individuals randomly selected from the population, o1, o2 and o3 are random integers in the interval [0, P) that are not equal to i, and they are different from each other;

[0034] An adaptive mutation operator strategy is adopted to balance the relationship between search efficiency and global optimal solution accuracy during the search process by dynamically adjusting the value of F. The formula is as follows:

[0035] F=F0×2 λ (2)

[0036]

[0037] In the formula, F0 represents the initial mutation operator, G prune represents the maximum evolutionary generation, G represents the current evolutionary generation;

[0038] Step 4.4.2: Perform a crossover operation on the parent population and the mutant sub-population according to the set crossover probability to achieve information exchange, thereby generating a new offspring population;

[0039] Step 4.4.3: Use the training data set to train the offspring population, and then use the validation data set to evaluate the model performance to obtain the corresponding model error rate and total FLOPs of the model;

[0040] Step 4.4.4: According to the Pareto dominance principle of the multi-objective optimization algorithm, the individuals of the parent population and the individuals of the child population are dominated; if one of the individuals dominates the corresponding individual of the other, the dominated individual is recorded; otherwise, if the two individuals do not dominate each other, both individuals are recorded at the same time;

[0041] Step 4.4.5: After the operation in step 4.4.4 is completed, the recorded individual set is returned;

[0042] Step 4.4.6: Compare the size of the individual set obtained in step 4.4.5 with the size of the population P. If the two are the same, then the individual set is used as the parent population of the next generation; otherwise, select individuals from the individual set as the parent population of the next generation based on the Pareto rank and crowding distance;

[0043] Step 4.5: If the current evolutionary generation is less than the maximum evolutionary generation G prune , return to step 4.4 to continue executing this step; otherwise, take the individual set of the population size obtained in step 4.4 as the final population;

[0044] Step 5: Perform non-dominated sorting on the final population obtained in step 4.5 to obtain the Pareto front, and select the boundary solution and the knee solution in the Pareto front;

[0045] According to the obtained Pareto frontier, according to the two fitness values ​​of each individual, namely error rate and FLOPs, the heavy boundary solution, light boundary solution and preferred Knee solution are selected, where the heavy boundary solution represents the solution with the highest accuracy, the light boundary solution represents the solution with the smallest FLOPs, and the preferred Knee solution is selected based on the preference vector r; the weighted induced norm of each individual in the Pareto frontier is calculated through the preference vector r, and the individual with the smallest weighted induced norm value is selected as the preferred Knee solution;

[0046] Step 6: Retrain the steel product quality defect recognition model corresponding to the Knee solution obtained in step 5 to obtain the corresponding accuracy on the test data set to verify the effectiveness of the model.

[0047] The beneficial effect of adopting the above technical solution is that: the steel product quality defect recognition method based on multi-objective evolutionary deep learning provided by the present invention is aimed at the current field of steel product quality image recognition, and the neural network model structure adopted by the deep learning technology needs to be manually pre-defined. This method is highly dependent on the professional knowledge level and practical experience of the technicians, and it is difficult to achieve a balance between model complexity and recognition accuracy in practice. In view of the above problems, the present invention combines the advantages of multi-objective evolutionary algorithm and deep learning technology, and proposes a steel product quality image recognition algorithm based on multi-objective evolutionary deep learning. First, the original steel product quality defect recognition model with high recognition accuracy is randomly generated by automatic neural network deepening technology, and then the multi-objective differential evolution algorithm is used to optimize the automatically generated original quality defect recognition model structure to balance the two optimization goals of model complexity and recognition accuracy. Through experimental testing, the proposed method successfully gets rid of the reliance on artificial experience to define the network structure, and the obtained quality defect recognition model shows a relatively ideal performance in terms of recognition accuracy and model complexity. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 A flowchart of a method for identifying quality defects of steel products based on multi-objective evolutionary deep learning provided by an embodiment of the present invention;

[0049] Figure 2 A sample image of a steel strip surface defect provided by an embodiment of the present invention;

[0050] Figure 3 A flowchart of the automatic deepening process provided by an embodiment of the present invention;

[0051] Figure 4 A schematic diagram of a network structure of automatic neural network deepening provided in an embodiment of the present invention, wherein (a) is a schematic diagram of a residual network structure obtained in the automatic deepening process, and (b) is a schematic diagram of a residual layer structure used in the automatic deepening process;

[0052] Figure 5 A flow chart of the differential pruning process provided by an embodiment of the present invention;

[0053] Figure 6 The final population graph returned by the pruning process of the multi-objective differential evolution algorithm provided in the embodiment of the present invention;

[0054] Figure 7 A graph showing how the accuracy of the model provided in the embodiment of the present invention on the validation set changes with the increase in the number of deepening rounds. DETAILED DESCRIPTION

[0055] The specific implementation of the present invention is further described in detail below in conjunction with the accompanying drawings and examples. The following examples are used to illustrate the present invention, but are not intended to limit the scope of the present invention.

[0056] This embodiment takes the strip steel rolling process as an example, and adopts the steel product quality defect identification method based on multi-objective evolutionary deep learning of the present invention to identify defects in the strip steel rolling process.

[0057] In this embodiment, the steel product quality defect recognition method based on multi-objective evolutionary deep learning automatically generates a steel product quality defect recognition model with high recognition accuracy and low model complexity through automatic neural network deepening technology and differential pruning. The overall process is as follows: Figure 1 As shown in the figure. Firstly, the automatic deepening technology is used to continuously add randomly generated residual blocks to generate a quality defect recognition model, and a model with better performance is obtained by judging the recognition accuracy of the model; then, the multi-objective differential evolution algorithm is used to prune the obtained quality defect recognition model. The whole process selects model complexity and recognition accuracy as two optimization goals, so as to achieve a balance between multiple goals and obtain a steel product quality defect recognition model.

[0058] In this implementation, the steel product quality defect recognition model based on multi-objective evolutionary deep learning includes the following steps:

[0059] Step 1: Collect product surface defect images during steel production and processing to obtain sample image data.

[0060] This example mainly collects common defects in the strip rolling process, including inclusions, scratches, pressed oxide scale, cracks, pits and plaques. Figure 2 As shown, each defect contains 300 pictures;

[0061] Step 2: Label the defect types of the sample image data obtained in step 1 and divide the data set;

[0062] Step 2.1: The sample image data obtained in step 1 is labeled with defect types according to the types of surface defects of the product, namely, cracks, inclusions, plaques, pits, indentations, and scratches;

[0063] Step 2.2: Divide the sample image data annotated in step 2.1 into a training data set, a validation data set, and a test data set in a ratio of 7:2:1;

[0064] Step 3: Use automatic neural network deepening technology to build a quality defect recognition model for raw steel products. The specific process is as follows: Figure 3 As shown;

[0065] Step 3.1: Model initialization process, randomly generate a residual block as the initial steel product quality defect recognition model to identify steel product quality defects, and obtain the evaluation index on the verification data set by training the initial steel product quality defect recognition model, that is, the recognition accuracy, and record its accuracy as the model accuracy;

[0066] Step 3.1.1: First, a residual block is randomly generated as the initial steel product quality defect recognition model, such as Figure 4 As shown in . The randomly generated residual block consists of several residual layers, such as Figure 4 (a) Each residual layer includes two convolutional layers with a convolution kernel size of 3x3 and two batch normalization layers. The number of convolution kernels in all convolutional layers in the same residual block is the same. When the number of input channels and output channels of the residual layer is different, the dotted line structure in the figure will take effect. The structural diagram is shown in Figure 4 (b). Each residual block is represented by three parameters, namely: the number of residual layers, the number of convolution kernels, and whether to downsample. The number of residual layers represents the number of residual layers in the current residual block, and its range is set to [4, 12]. The number of convolution kernels represents the number of filters in all convolution layers in the current residual block, and is set to [8, 64]. Whether to downsample indicates whether the current residual block performs downsampling operations on its input. The initial steel product quality defect recognition model is generated by randomly taking values ​​for the three parameters.

[0067] Step 3.1.2: Train the initial steel product quality defect recognition model generated in step 3.1.1 to obtain the recognition accuracy on the validation data set, which is recorded as the model accuracy.

[0068] In this embodiment, during the initial steel product quality defect recognition model training process, the learning rate is set to 0.001, the cross entropy loss function and the stochastic gradient descent method are used for training, and a strategy of evaluating on the validation set immediately after a round of training is adopted. If the accuracy of the validation set does not improve for five consecutive times, the training is stopped.

[0069] Step 3.2: Use automatic neural network deepening technology to perform multiple rounds of neural network automatic deepening process on the initial quality defect recognition model; in each round of deepening process, generate a random number of residual blocks, and then add the randomly generated residual blocks to the initial steel product quality defect recognition model in turn, and obtain the accuracy based on the verification data set through training. After each round of deepening process, return the best model and the corresponding highest accuracy based on the verification data set;

[0070] Step 3.2.1: In a round of automatic neural network deepening, randomly generate multiple residual blocks with different network structures according to the method of step 3.1.1;

[0071] Step 3.2.2: Add the multiple residual blocks with different network structures generated in step 3.2.1 to the initial steel product quality defect recognition model in turn, obtain multiple new steel product quality defect recognition models and train them to obtain the new network with the highest recognition accuracy on the verification data set as the initial steel product quality defect recognition model for the next automatic neural network deepening. Each time a new residual block is added to the initial steel product quality defect model, a new steel product quality defect recognition model is retrained, and its model parameter settings are consistent with step 3.1.2. By training the new steel product quality defect recognition model, an evaluation indicator based on the verification data set, namely, the accuracy, is obtained, and this accuracy is recorded as the test accuracy;

[0072] Step 3.2.3: In the process of adding residual blocks, if the residual block added is the first randomly generated residual block in the current round of deepening, the test accuracy obtained by training the model with the added residual block is assigned to the highest accuracy obtained on the validation data set in the current round, and the model with the added residual block is recorded as the best model obtained by the current round of automatic neural network deepening;

[0073] Step 3.2.4: If the added residual block is not the first residual block randomly generated in the current round of deepening, but its test accuracy is greater than the highest accuracy obtained after the current round of automatic neural network deepening, then the test accuracy is regarded as the highest accuracy, and the new steel product quality defect recognition model after adding the residual block is recorded as the best model obtained by the current round of automatic neural network deepening;

[0074] Step 3.2.5: Determine whether the new steel product quality defect recognition model after adding the residual block is the last model tried in the current round. If so, the current round of automatic neural network deepening process ends, and the best model obtained in the current round and the corresponding highest accuracy based on the verification data set are returned, and the obtained best model is used as the initial steel product quality defect recognition model for the next round of deepening process; if not, return to step 3.2.2;

[0075] Step 3.3: Compare the highest accuracy based on the validation set obtained in the automatic neural network deepening process in step 3.2 with the model accuracy obtained in step 3.1. If the former is greater than the latter, the number of residual blocks that do not improve the model performance after joining the model is recorded as 0, and the former is assigned to the latter; otherwise, the number of residual blocks that do not improve the model performance after joining the model is increased by 1;

[0076] Step 3.4: In a neural network automatic deepening process, if adding residual blocks to the initial steel product quality defect recognition model for two consecutive rounds does not bring about model performance improvement, then the current round of automatic neural network deepening process is stopped, and the best model obtained in the current round is recorded as the original steel product quality defect recognition model, and the model and corresponding accuracy rate that have not been improved in the two rounds are recorded; otherwise, return to step 3.2 and continue to execute a new round of neural network model deepening process;

[0077] Step 3.5: After an automatic deepening process stops, determine whether this is the last automatic neural network deepening process set. If not, return to step 3.1 to perform a new automatic neural network deepening process to obtain a new original steel product quality defect recognition model; if this is the last automatic neural network deepening process, return the best model of all automatic neural network deepening processes.

[0078] In this embodiment, the automatic neural network deepening process is carried out 5 times in total, and the 5 original steel product quality defect recognition models after the automatic neural network deepening are tested on the test data set to obtain the highest recognition accuracy, the average recognition accuracy and the corresponding total FLOPs of the model, and the best model with the highest recognition accuracy is used as the best original steel product quality defect recognition model for differential pruning in step 4.

[0079] Step 4: Use a multi-objective differential evolution algorithm to prune the optimal original steel product quality defect recognition model obtained in step 3, such as Figure 5 As shown, the model is retrained to reduce the network calculation complexity while maintaining the accuracy. In this embodiment, the error rate of the steel product quality defect recognition model on the verification data set, that is, the difference between the accuracy and 1, and the number of floating point operations per second (FLOPs) of the model are selected as two optimization targets.

[0080] For the two-dimensional convolution structure in the original steel product quality defect recognition model, the convolution kernel size is k, and the input feature map Output feature map Among them, B is the batch size of the model input data set, C in , C out Respectively represent the number of input and output feature map channels, W in , H in With W out , H out Represent the width and height of the input feature map and the width and height of the output feature map respectively. The calculation formula for the number of floating-point operations per second FLOPs_1 of the two-dimensional convolution structure of the model is as follows:

[0081] FLOPs_1=(2×Cin ×k 2 -1)×W out ×H out ×C out (1)

[0082] For the fully connected layer, the number of input and output feature map channels are recorded as C' in , C' out , then the calculation formula for the number of floating-point operations per second FLOPs_2 of the fully connected layer of the model is described as:

[0083] FLOPs_2=(2×C i ' n -1)×C′ out (2)

[0084] Step 4.1: First, encode the original steel product quality defect recognition model, using a string of 0 / 1 codes to represent the quality defect recognition model, where 0 represents discarding the convolution filter and 1 represents retaining the convolution filter;

[0085] Step 4.2: Based on the encoded original steel product quality defect recognition model and the preset population size, generate an initial population for the multi-objective differential evolution algorithm pruning process. In this embodiment, the population P is set to 30;

[0086] Step 4.3: Train each individual in the initial population and obtain the evaluation index accuracy based on the validation data set and the total floating-point operations per second FLOPs of the model. The total floating-point operations per second FLOPs of the model is the sum of FLOPs_1 and FLOPs_2.

[0087] In this embodiment, the cross entropy loss function is combined with the stochastic gradient descent method for training, where the learning rate is set to 0.01 and the training step size is 3;

[0088] Step 4.4: Use a multi-objective differential evolution algorithm to prune the steel quality defect recognition model. The operation process includes differential mutation, crossover and selection of retained individuals. Through these operations, individuals in the population are gradually optimized.

[0089] Step 4.4.1: Determine the mutation operator used in this evolution according to the preset initial mutation operator and the current evolution generation; use the operator to perform differential mutation on the individuals in the parent population to obtain the mutant sub-population;

[0090] The differential mutation strategy expression used in this embodiment is as follows:

[0091]

[0092] in, represents the i-th differential mutation vector generated in the t-th iteration, Represents three different individuals randomly selected from the population, o1, o2 and o3 are random integers in the interval [0, P) that are not equal to i, and they are different from each other.

[0093] In the differential evolution algorithm search process, when the mutation operator F takes a large value, the algorithm search efficiency is reduced and the accuracy of the global optimal solution is not high. When the F value is small, the population diversity is weakened and the "premature" phenomenon is prone to occur. To meet this challenge, this embodiment adopts an adaptive mutation operator strategy to dynamically adjust the value of F to balance the relationship between search efficiency and global optimal solution accuracy during the search process. The formula is as follows:

[0094] F=F0×2 λ (4)

[0095]

[0096] In the formula, F0=4, which indicates the initial mutation operator, G prune =40, indicating the maximum evolutionary generation, and G indicating the current evolutionary generation.

[0097] In the initial stage of the algorithm, the adaptive mutation operator F has a large value to maintain the initial population diversity and prevent the "premature" phenomenon. As the algorithm continues to iterate, the mutation operator gradually decreases and finally approaches the initial value F0. This strategy retains excellent individuals and increases the probability of searching for the global optimal solution.

[0098] Step 4.4.2: Perform a crossover operation on the parent population and the mutant sub-population according to the set crossover probability to achieve information exchange, thereby generating a new sub-population. The crossover operation is shown in the following formula;

[0099]

[0100] In the formula, as well as They represent the values ​​of the j-th position of the i-th differential mutation vector, parent vector and offspring vector generated in the t-th evolution process respectively. Cr=0.1 represents the crossover probability. sn represents a random integer selected from the crossover position set.

[0101] Step 4.4.3: Use the training data set to train the offspring population, and then use the validation data set to evaluate the model performance to obtain the corresponding model error rate and total FLOPs of the model.

[0102] Step 4.4.4: According to the Pareto dominance principle of the multi-objective optimization algorithm, the individuals of the parent population and the individuals of the child population are dominated; if one of the individuals dominates the corresponding individual of the other, the dominated individual is recorded; otherwise, if the two individuals do not dominate each other, both individuals are recorded at the same time;

[0103] Step 4.4.5: After the operation in step 4.4.4 is completed, the recorded individual set is returned;

[0104] Step 4.4.6: Compare the size of the individual set obtained in step 4.4.5 with the size of the population P. If the two are the same, then the individual set is used as the parent population of the next generation; otherwise, select individuals from the individual set as the parent population of the next generation based on the Pareto rank and crowding distance;

[0105] Step 4.5: If the current evolutionary generation is less than the maximum evolutionary generation G prune , return to step 4.4 to continue executing this step; otherwise, take the individual set of the population size obtained in step 4.4 as the final population;

[0106] Step 5: Perform non-dominated sorting on the final population obtained in step 4.5 to obtain the Pareto frontier, and select the boundary solution and the knee solution in the Pareto frontier. Figure 6 The figure shows the final population returned after pruning by the multi-objective differential evolution algorithm. The triangles in the figure represent the boundary solution and knee solution of the final population. From left to right, they are the heavy boundary solution, the preferred knee solution, and the light boundary solution. The line segment connecting the boundary solution and the knee solution is the Pareto frontier of the final population.

[0107] According to the obtained Pareto frontier, according to the two fitness values ​​of each individual, namely, error rate and FLOPs, the heavy boundary solution, the light boundary solution and the preferred Knee solution are selected, where the heavy boundary solution represents the solution with the highest accuracy, the light boundary solution represents the solution with the smallest FLOPs, and the preferred Knee solution is selected based on the preference vector. The weighted induced norm of each individual in the Pareto frontier is calculated by the preference vector r, and the individual with the smallest weighted induced norm value is selected as the preferred Knee solution. In this embodiment, the preference vector r is set to [1, 2]. Each individual F k The weighted induced norm of is shown in the following formula:

[0108]

[0109] Φ1=2-r1+1 (10)

[0110] Φ2=2-r2+1 (11)

[0111] Where r1 and r2 are the two components of the preference vector r, r1 represents the preference of the first target (the total FLOPs of the model); r2 represents the preference of the second target (the model error rate); f1(F k ) and f2(F k ) represents each individual F k The total FLOPs and model error rate of the model; min(f1) and min(f2) represent the total FLOPs and model error rate of the model with light boundary solution, respectively; max(f1) and max(f2) represent the total FLOPs and model error rate of the model with heavy boundary solution, respectively.

[0112] Step 6: Retrain the steel product quality defect recognition model corresponding to the Knee solution obtained in step 5 to obtain the corresponding accuracy on the test data set to verify the effectiveness of the model, where the learning rate is 0.0001 and the number of steps is 200.

[0113] In this embodiment, a steel product quality defect recognition model is obtained by conducting experiments on the sample image data obtained in step 1. First, an original steel product quality defect recognition model is obtained through training, where Figure 7 The figure shows the curve of the accuracy of the model based on the validation set as the residual block increases during an automatic deepening process. It can be seen from the figure that before stopping deepening, two consecutive additions of residual blocks did not improve the model performance. The obtained original quality defect recognition model was pruned using a multi-objective differential evolution algorithm, and the experimental results are shown in Table 1. As can be seen from the table, the automatic deepening neural network technology of this embodiment can build a steel quality defect recognition model without relying on artificial experience, but the calculation complexity of the result obtained by automatic deepening is high, and there is a network structure with more redundant. By pruning and retraining the network model obtained by the automatic deepening technology using a multi-objective differential evolution algorithm, the model accuracy can be made close to or exceed the accuracy before pruning, thereby achieving the premise of maintaining the accuracy, reducing the network calculation complexity, and using the results of the multi-objective differential evolution algorithm to prune, it can be found that the calculation complexity is lower than that of the fixed structure convolutional neural network, and the residual neural network with higher accuracy. It can be seen that by combining the automatic deepening technology with the differential pruning technology in this method, it is possible to build a network model with both ideal detection accuracy and model complexity without relying on artificial experience.

[0114] Table 1 Final results of this embodiment

[0115]

[0116] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope defined by the claims of the present invention.

Claims

1. A method for identifying quality defects of steel products based on multi-objective evolutionary deep learning, characterized in that: The following steps are involved: Step 1: Collect product surface defect images during steel production and processing to obtain sample image data; Step 2: Label the defect types of the sample image data obtained in step 1 and divide the data set; Step 3: Use automatic neural network deepening technology to build a quality defect recognition model for original steel products; Step 4: Use a multi-objective differential evolution algorithm to prune the optimal original steel product quality defect recognition model obtained in step 3, and retrain it to reduce the network calculation complexity while maintaining the accuracy of the model; Step 5: Perform non-dominated sorting on the final population obtained by the multi-objective differential evolution algorithm in step 4 to obtain the Pareto frontier, and select the boundary solution and the Knee solution in the Pareto frontier; Step 6: Retrain the steel product quality defect recognition model corresponding to the Knee solution obtained in step 5 to obtain the corresponding accuracy on the test data set to verify the effectiveness of the model.

2. The method for identifying quality defects of steel products based on multi-objective evolutionary deep learning according to claim 1 is characterized in that: The step 2 labels the sample image data obtained in step 1 according to the types of surface defects of the product; and divides the labeled sample image data into a training data set, a verification data set and a test data set in proportion.

3. The method for identifying quality defects of steel products based on multi-objective evolutionary deep learning according to claim 2 is characterized in that: The step 3 comprises: Step 3.1: Model initialization process, randomly generate a residual block as the initial steel product quality defect recognition model to identify steel product quality defects, and obtain the evaluation index on the verification data set by training the initial steel product quality defect recognition model, that is, the recognition accuracy, and record its accuracy as the model accuracy; Step 3.2: Use automatic neural network deepening technology to perform multiple rounds of neural network automatic deepening process on the initial quality defect recognition model; in each round of deepening process, generate a random number of residual blocks, and then add the randomly generated residual blocks to the initial steel product quality defect recognition model in turn, and obtain the accuracy based on the verification data set through training. After each round of deepening process, return the best model and the corresponding highest accuracy based on the verification data set; Step 3.3: Compare the highest accuracy based on the validation set obtained in the automatic neural network deepening process in step 3.2 with the model accuracy obtained in step 3.

1. If the former is greater than the latter, the number of residual blocks that do not improve the model performance after joining the model is recorded as 0, and the former is assigned to the latter; otherwise, the number of residual blocks that do not improve the model performance after joining the model is increased by 1; Step 3.4: In a neural network automatic deepening process, if adding residual blocks to the initial steel product quality defect recognition model for two consecutive rounds does not bring about model performance improvement, then the current round of automatic neural network deepening process is stopped, and the best model obtained in the current round is recorded as the original steel product quality defect recognition model, and the model and corresponding accuracy rate that have not been improved in the two rounds are recorded; otherwise, return to step 3.2 and continue to execute a new round of neural network model deepening process; Step 3.5: After an automatic deepening process stops, determine whether this is the last automatic neural network deepening process set. If not, return to step 3.1 to perform a new automatic neural network deepening process to obtain a new original steel product quality defect recognition model; if this is the last automatic neural network deepening process, return the best model of all automatic neural network deepening processes.

4. The method for identifying quality defects of steel products based on multi-objective evolutionary deep learning according to claim 3 is characterized in that: The step 3.1 comprises: Step 3.1.1: randomly generate a residual block as an initial steel product quality defect recognition model; the randomly generated residual block consists of several residual layers, each residual layer includes two convolutional layers and two batch normalization layers, and the number of convolution kernels of all convolutional layers in the same residual block is the same; each residual block is represented by three parameters, namely: the number of residual layers, the number of convolution kernels, and whether to downsample, where the number of residual layers represents the number of residual layers in the current residual block; the number of convolution kernels represents the number of filters of all convolutional layers in the current residual block; and whether to downsample represents whether the current residual block performs a downsampling operation on its input; the initial steel product quality defect recognition model is generated by randomly taking values ​​of the three parameters; Step 3.1.2: Train the initial steel product quality defect recognition model generated in step 3.1.1 to obtain the recognition accuracy on the validation data set, which is recorded as the model accuracy.

5. The method for identifying quality defects of steel products based on multi-objective evolutionary deep learning according to claim 4 is characterized in that: The step 3.2 comprises: Step 3.2.1: In a round of automatic neural network deepening, randomly generate multiple residual blocks with different network structures according to the method of step 3.1.1; Step 3.2.2: Add the multiple residual blocks of different network structures generated in step 3.2.1 to the initial steel product quality defect recognition model in turn, obtain multiple new steel product quality defect recognition models and train them to obtain the new network with the highest recognition accuracy on the verification data set as the initial steel product quality defect recognition model for the next automatic neural network deepening; each time a new residual block is added to the initial steel product quality defect model, retrain the new steel product quality defect recognition model, and its model parameter setting is consistent with step 3.1.2; obtain the evaluation index based on the verification data set, that is, the accuracy, by training the new steel product quality defect recognition model, and record the accuracy as the test accuracy; Step 3.2.3: In the process of adding residual blocks, if the residual block added is the first randomly generated residual block in the current round of deepening, the test accuracy obtained by training the model with the added residual block is assigned to the highest accuracy obtained on the validation data set in the current round, and the model with the added residual block is recorded as the best model obtained by the current round of automatic neural network deepening; Step 3.2.4: If the added residual block is not the first residual block randomly generated in the current round of deepening, but its test accuracy is greater than the highest accuracy obtained after the current round of automatic neural network deepening, then the test accuracy is regarded as the highest accuracy, and the new steel product quality defect recognition model after adding the residual block is recorded as the best model obtained by the current round of automatic neural network deepening; Step 3.2.5: Determine whether the new steel product quality defect recognition model after adding the residual block is the last model attempted in the current round. If so, the current round of automatic neural network deepening process ends, and the best model obtained in the current round and the corresponding highest accuracy based on the verification data set are returned, and the obtained best model is used as the initial steel product quality defect recognition model for the next round of deepening process; if not, return to step 3.2.

2.

6. The method for identifying quality defects of steel products based on multi-objective evolutionary deep learning according to claim 5, characterized in that: The step 4 selects the error rate of the steel product quality defect recognition model on the verification data set and the number of floating-point operations per second (FLOPs) of the model as two optimization objectives of the multi-objective differential evolution algorithm.

7. The method for identifying quality defects of steel products based on multi-objective evolutionary deep learning according to claim 6, characterized in that: The step 4 comprises: Step 4.1: First, encode the original steel product quality defect recognition model, using a string of 0 / 1 codes to represent the quality defect recognition model, where 0 represents discarding the convolution filter and 1 represents retaining the convolution filter; Step 4.2: Based on the encoded original steel product quality defect recognition model and the preset population size, an initial population for the pruning process of the multi-objective differential evolution algorithm is generated; Step 4.3: Train each individual in the initial population and obtain the evaluation index accuracy based on the validation data set and the total floating-point operations per second (FLOPs) of the model. The total floating-point operations per second (FLOPs) of the model is the sum of the floating-point operations per second (FLOPs_1) of the two-dimensional convolution structure of the model and the floating-point operations per second (FLOPs_2) of the fully connected layer of the model. Step 4.4: Use a multi-objective differential evolution algorithm to prune the steel quality defect recognition model. The operation process includes differential mutation, crossover and selection of retained individuals. Through these operations, individuals in the population are gradually optimized. Step 4.5: If the current evolutionary generation is less than the maximum evolutionary generation G prune , return to step 4.4 to continue executing this step; otherwise, the set of individuals with the population size obtained in step 4.4 is taken as the final population.

8. The method for identifying quality defects of steel products based on multi-objective evolutionary deep learning according to claim 7, characterized in that: The step 4.4 comprises: Step 4.4.1: Determine the mutation operator used in this evolution according to the preset initial mutation operator and the current evolution generation; use the operator to perform differential mutation on the individuals in the parent population to obtain the mutant sub-population; Step 4.4.2: Perform a crossover operation on the parent population and the mutant sub-population according to the set crossover probability to achieve information exchange, thereby generating a new offspring population; Step 4.4.3: Use the training data set to train the offspring population, and then use the validation data set to evaluate the model performance to obtain the corresponding model error rate and total FLOPs of the model; Step 4.4.4: According to the Pareto dominance principle of the multi-objective optimization algorithm, the individuals of the parent population and the individuals of the child population are dominated; if one of the individuals dominates the corresponding individual of the other, the dominated individual is recorded; otherwise, if the two individuals do not dominate each other, both individuals are recorded at the same time; Step 4.4.5: After the operation in step 4.4.4 is completed, the recorded individual set is returned; Step 4.4.6: Compare the size of the individual set obtained in step 4.4.5 with the size of the population P. If the two are the same, then the individual set is used as the parent population of the next generation; otherwise, individuals are selected from the individual set as the parent population of the next generation based on the Pareto rank and crowding distance.

9. The method for identifying quality defects of steel products based on multi-objective evolutionary deep learning according to claim 8, characterized in that: The differential mutation strategy used in 4.4.1 is expressed as follows: in, represents the i-th differential mutation vector generated in the t-th iteration, F is the mutation operator, represents three different individuals randomly selected from the population, o1, o2 and o3 are random integers in the interval [0, P) that are not equal to i, and they are different from each other; An adaptive mutation operator strategy is adopted to balance the relationship between search efficiency and global optimal solution accuracy during the search process by dynamically adjusting the value of F. The formula is as follows: F=F0×2 λ (2) In the formula, F0 represents the initial mutation operator, G prune represents the maximum evolutionary generation, and G represents the current evolutionary generation.

10. The method for identifying quality defects of steel products based on multi-objective evolutionary deep learning according to claim 1, characterized in that: In step 5, according to the obtained Pareto frontier, according to the two fitness values ​​of each individual, namely, the error rate and the total number of floating-point operations per second (FLOPs) of the model, a heavy boundary solution, a light boundary solution and a preferred Knee solution are selected, wherein the heavy boundary solution represents the solution with the highest accuracy, the light boundary solution represents the solution with the smallest FLOPs, and the preferred Knee solution is selected based on the preference vector r; the weighted induced norm of each individual in the Pareto frontier is calculated by the preference vector r, and the individual with the smallest weighted induced norm value is selected as the preferred Knee solution.

Citation Information

Patent Citations

  • A method and a device for recognizing surface defects of steel based on depth learning technology

    CN109242825A

  • Steel surface defect identification method based on semi-supervised deep learning algorithm

    CN112164040A

  • Steel material quality analysis method based on evolutionary multi-scale feature learning

    CN117392450A

  • Metal fracture type identification method based on deep learning

    CN117994786A

  • Face recognition method based on evolutionary convolutional neural network

    US20210295019A1

Cited By

  • Industrial equipment state monitoring and early warning system and method based on machine learning

    CN119493374A