A Scrap Steel Recognition and Segmentation Method Based on Segformer

Abstract

The present invention discloses a scrap steel recognition and segmentation method based on Segformer, belonging to the technical field of scrap steel. The steps of the present invention are as follows: S1: Take pictures of the steel piles carried by the scrap steel transport vehicle from multiple angles to obtain steel pile sample photos; S2: Denoise, preprocess and data augment the steel pile sample photos to obtain preprocessed steel pile photos; S3: Label the preprocessed steel pile photos; S4: Input the labeled steel pile photos into the Segformer semantic segmentation model for deep learning model training to extract training weights; S5: During the scrap steel unloading operation, collect photos of the unloading area for each suction action; S6: Collect the parts with prediction errors and missing predictions, supplement them into the data set according to a ratio and train iteratively. The present invention is easy to operate, solves the problem of difficult segmentation caused by the disorderly stacking of scrap steel piles, separately segments each scrap steel block, and lays a foundation for the next step of independent classification and thickness measurement of single-piece scrap steel.

Description

Technical Field

[0001] The present invention relates to the technical field of scrap steel, and specifically provides a method for identifying and segmenting scrap steel based on Segformer. Background Art

[0002] Scrap steel is currently the only green and environmentally friendly renewable resource that can replace iron ore and can be recycled multiple times. However, due to the large amount of scrap steel used in steel enterprises, there are generally problems such as mixed loading and doping of multiple material types, a large variety of scrap steel, and complex actual detection scenarios. Conventional classification of scrap steel mainly relies on manual visual inspection, which is greatly affected by human factors, has low accuracy and classification efficiency, and affects the subsequent production rhythm.

[0003] In the existing technology, the Chinese patent with the application number 201910201422.8 discloses "a method and device for automatically identifying the types of scrap steel based on CNN". This method for automatically identifying the types of scrap steel based solely on CNN can, although, automatically identify specific types of scrap steel, but is limited to scrap steel with regular shapes and specific shapes; for scrap steel with mixed loading of multiple material types, on the one hand, the identification and classification effect of some scrap steel is poor, such as incomplete identification or failure to identify some scrap steel; on the other hand, relying solely on CNN identification will introduce a lot of noise, bringing more interference to subsequent work. Summary of the Invention

[0004] The purpose of the present invention is to provide a method for identifying and segmenting scrap steel based on Segformer. By using the deep learning method of Segformer, the problem of segmenting dense scrap steel piles is solved, laying a foundation for the next step of classifying single-piece scrap steel and measuring its thickness; by adopting k-fold cross-validation as the training method for the Segformer model, the model is effectively optimized, making the model have better generalization ability, and solving the generalization problem and evaluation problem of the model in complex scenarios.

[0005] To achieve the above purpose, the present invention provides the following technical solutions:

[0006] A method for identifying and segmenting scrap steel based on Segformer, comprising the following steps:

[0007] S1: Take pictures of the steel piles carried by the scrap steel transport vehicle from multiple angles to obtain steel pile sample photos;

[0008] S2: Denoise, preprocess the annotation and perform data augmentation on the steel pile sample photos to obtain preprocessed steel pile photos;

[0009] S3: Annotate the preprocessed steel pile photos and use the k-fold cross-validation method to split them into x training sets, validation sets and test sets;

[0010] S4: Input the marked steel pile photos into the Segformer semantic segmentation model for deep learning model training, and extract the training weights;

[0011] S5: During the scrap steel unloading operation, collect the photos of the unloading area for each suction action. Input the steel pile photos into the obtained semantic segmentation model, make predictions on the steel pile photos, and obtain the photos of the unloading area for semantic segmentation;

[0012] S6: Collect the parts with prediction errors and missing predictions, supplement them into the dataset according to a certain proportion and train iteratively; evaluate the weights obtained by iteration through k-fold cross-validation, select the one with the best evaluation, compare the predictions with the original weights for the new dataset, and publish the better weights.

[0013] Further, the specific steps of S1 are as follows:

[0014] S101: Take photos of different steel piles on different vehicles respectively;

[0015] S102: Take photos of the unloading areas of the steel piles on the same vehicle each time.

[0016] Further, the specific steps of S2 are as follows:

[0017] S201: Use a Gaussian filter to filter out the granular noise in the imaging pictures. For the overexposure phenomenon in some photos, reduce the information loss and improve the signal-to-noise ratio through methods such as histogram equalization and template matching;

[0018] S202: Apply the Mosaic data augmentation algorithm to the imaging picture set, perform operations such as migration, occlusion, and blurring on the data main body to enhance and augment the data, and ensure the robustness of the training model.

[0019] Further, the specific steps of S3 are as follows:

[0020] S301: Through the k-fold cross-validation method, divide the preprocessed steel pile photos into a training set, a validation set, and a test set according to a certain proportion, divide them x times, and obtain x sets of pictures containing the training set and the validation set and y sets of test set pictures;

[0021] S302: For the training sets in the x sets of pictures, label the training sets. Among them, the steel blocks are divided into m categories, and x corresponding json files are obtained.

[0022] Further, the specific steps of S4 are as follows:

[0023] S401: Modify the Segformer semantic segmentation model according to the usage scenario. The specific modifications are as follows: 1. Since the scrap steel block area is small, the 4-layer Transformer encoder of the Segformer model generates too much low-resolution coarse feature information. Add a new fully-connected MLP encoder with a size of 1 / 2 of the original resolution as the head network, and correspondingly modify the decoder part. The size of the modified decoder part is as follows:

[0024]

[0025] At the same time, the Softmax function is used as the activation function in the attention mechanism of the Segformer model, which is likely to suppress the edge impact information in the steel block feature information. Therefore, it is modified to:

[0026]

[0027] S402: Import x original steel block images and label images into the Segformer deep learning semantic segmentation model for training. Each training is performed E times to obtain x sets of training weights.

[0028] S403: Use the x sets of training weights and the Segformer deep learning semantic segmentation model to predict y corresponding test sets, and take the average of the y prediction results.

[0029] S404: According to the y prediction result values, adjust hyperparameters such as the learning rate, intersection over union threshold, and loss function calculation method. Import all the labeled images and corresponding labels into the Segformer semantic segmentation model for training to obtain the final training weights.

[0030] Furthermore, the specific steps of S5 are as follows:

[0031] S501: Send the photo of the scrap steel unloading area into the Segformer model for prediction.

[0032] S502: Extract the prediction results, segment the predicted single-body closed area, and obtain the single-piece scrap steel image.

[0033] Furthermore, the specific steps of S6 are as follows:

[0034] S601: Supplement the incorrectly predicted and missing prediction images into the original dataset, and split the new dataset into a training set, a validation set, and a test set according to the original ratio.

[0035] S602: Use k-fold cross-validation to split the new dataset into k parts, and use the Segformer model to train and evaluate the optimal weights for the k parts of the dataset.

[0036] S603: Evaluate the optimal weight and the original weight using a new test set, and release the one with better evaluation.

[0037] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0038] The present invention is easy to operate. During the automatic grading process of scrap steel, based on the Segformer deep learning vision algorithm, it solves the problem of difficult segmentation caused by the disorderly stacking of scrap steel piles, separates each piece of scrap steel block individually, and lays a foundation for the next step of independent classification and thickness measurement of single-piece scrap steel. Description of the Drawings

[0039] Figure 1 It is a schematic flow chart of the present invention. Detailed Embodiments

[0040] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0041] Please refer to Figure 1 , a scrap steel recognition and segmentation method based on Segformer is provided in the embodiments of the present invention, including the following steps:

[0042] S1: Take pictures of the steel pile carried by the scrap steel transport vehicle from multiple angles to obtain steel pile sample photos;

[0043] S2: Denoise, preprocess the annotation and perform data augmentation on the steel pile sample photos to obtain preprocessed steel pile photos;

[0044] S3: Annotate the preprocessed steel pile photos, and use the k-fold cross-validation method to split them into x training sets, validation sets and test sets;

[0045] S4: Input the annotated steel pile photos into the Segformer semantic segmentation model for deep learning model training to extract training weights;

[0046] S5: During the scrap steel unloading operation, collect photos of the unloading area for each suction action, input the steel pile photos into the obtained semantic segmentation model, predict the steel pile photos, and obtain the unloading area photos for semantic segmentation;

[0047] S6: Collect the parts with incorrect prediction and missing prediction, supplement them into the data set according to a certain proportion and train iteratively; evaluate the weights obtained by iteration according to k-fold cross-validation, select the one with the best evaluation and compare it with the original weight for prediction on a new data set, and release the better weight.

[0048] The specific steps of S1 are as follows:

[0049] S101: Take pictures of different steel stacks of different vehicles respectively;

[0050] S102: Take pictures of the unloading areas of the steel stacks of the same vehicle each time.

[0051] The specific steps of S2 are as follows:

[0052] S201: Use a Gaussian filter to filter out the granular noise in the imaging pictures. For the overexposure phenomenon in some photos, methods such as histogram equalization and template matching are used to reduce information loss and improve the signal-to-noise ratio;

[0053] S202: Apply the Mosaic data augmentation algorithm to the imaging picture set, and perform operations such as migration, occlusion, and blurring on the data main body to enhance and augment the data, ensuring the robustness of the training model.

[0054] The specific steps of S3 are as follows:

[0055] S301: Through the k-fold cross-validation method, divide the preprocessed steel stack photos into a training set, a validation set, and a test set according to a certain proportion, divide them x times, and obtain x sets of pictures containing the training set and the validation set respectively and y sets of test set pictures;

[0056] S302: For the training sets in the x sets of pictures, label the training sets. Among them, the steel blocks are divided into m categories, and x corresponding json files are obtained.

[0057] The specific steps of S4 are as follows:

[0058] S401: Modify the Segformer semantic segmentation model according to the usage scenario. The specific modifications are as follows: 1. Since the scrap steel block area is small, the 4-layer Transformer encoder of the Segformer model generates too much low-resolution coarse feature information. Add a new fully MLP encoder with a size of 1 / 2 of the original resolution as the head network, and correspondingly modify the decoder part. The size of the modified decoder part is as follows:

[0059]

[0060] At the same time, in the attention mechanism of the Segformer model, the Softmax function is used as the activation function, which is prone to suppressing the edge impact information in the steel block feature information. Therefore, it is modified to:

[0061]

[0062] S402: Import the original steel block images and label images in x portions into the Segformer deep learning semantic segmentation model for training. Each portion is trained E times to obtain x sets of training weights.

[0063] S403: Use the x sets of training weights and the Segformer deep learning semantic segmentation model to predict y portions of the corresponding test sets, and take the average of the y prediction results.

[0064] S404: According to the y prediction result values, adjust hyperparameters such as the learning rate, intersection over union threshold, and loss function calculation method. Import all the labeled images and the corresponding labels into the Segformer semantic segmentation model for training to obtain the final training weights.

[0065] The specific steps of S5 are as follows:

[0066] S501: Send the photo of the scrap steel unloading area into the Segformer model for prediction.

[0067] S502: Extract the prediction results, segment the predicted single-block closed areas, and obtain the images of single pieces of scrap steel.

[0068] The specific steps of S6 are as follows:

[0069] S601: Supplement the images with prediction errors and missing predictions into the original data set, and split the new data set into a training set, a validation set, and a test set according to the original ratio.

[0070] S602: Use k-fold cross-validation to split the new data set into k portions, and use the Segformer model to train and evaluate the optimal weights for the k portions of the data sets.

[0071] S603: Evaluate the optimal weights and the original weights using the new test set, and release the one with better evaluation.

[0072] In summary, a scrap steel recognition and segmentation method based on Segformer of the present invention is easy to operate. By using the Segformer deep learning method, the segmentation problem of dense scrap steel piles is solved, laying a foundation for the next step of single-piece scrap steel classification and thickness measurement problems. By adopting k-fold cross-validation as the training method of the Segformer model, the model is effectively optimized, making the model have better generalization ability, and solving the generalization problem and evaluation problem of the model in complex scenarios.

[0073] The above is only a preferred specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present invention, according to the technical solution and inventive concept of the present invention, makes equivalent substitutions or changes, and all should be covered within the protection scope of the present invention.

Claims

1. A scrap steel recognition and segmentation method based on Segformer, characterized in that: Including the following steps: S1: Take pictures of the steel piles carried by the scrap steel transport vehicle from multiple angles to obtain sample photos of the steel piles; S2: Denoise, preprocess the annotation and perform data augmentation on the sample photos of the steel piles to obtain preprocessed steel pile photos; S3: Annotate the preprocessed steel pile photos and use the k-fold cross-validation method to split them into x training sets, validation sets and test sets; S4: Input the annotated steel pile photos into the Segformer semantic segmentation model for deep learning model training to extract training weights, specifically including the following steps: S401: Modify the Segformer semantic segmentation model according to the usage scenario. The specific modifications are as follows:

1. Since the scrap steel block area is small, the 4-layer Transformer encoder of the Segformer model generates too much low-resolution coarse feature information. Add a new fully MLP encoder with a size of 1 / 2 of the original resolution as the head network, and correspondingly modify the decoder part. The size of the modified decoder part is as follows: At the same time, the Softmax function is used as the activation function in the attention mechanism of the Segformer model, which is prone to suppressing the edge impact information in the steel block feature information. Therefore, it is modified to: S402: Import the x original steel block pictures and label pictures into the Segformer deep learning semantic segmentation model for training, and train each portion E times to obtain x training weights; S403: Use the obtained x training weights and the Segformer deep learning semantic segmentation model to predict the y corresponding test sets, and take the average of the y prediction results; S404: According to the y prediction result values, adjust hyperparameters such as the learning rate, intersection over union threshold, and loss function calculation method. Import all the annotated pictures and corresponding labels into the Segformer semantic segmentation model for training to obtain the final training weights; S5: During the scrap steel unloading operation, collect photos of the unloading area for each suction action. Input the steel pile photos into the obtained semantic segmentation model to predict the steel pile photos and perform semantic segmentation on the unloading area photos; S6: Collect the parts with incorrect predictions and missing predictions, supplement them into the dataset according to a certain proportion and train iteratively; Evaluate the weights obtained by iteration according to k-fold cross-validation, select the one with the best evaluation and compare the prediction with the original weights for the new dataset, and publish the better weights.

2. The scrap steel recognition and segmentation method based on Segformer according to claim 1, characterized in that: The specific steps of S1 are as follows: S101: Take pictures of different steel piles on different vehicles respectively; S102: Take pictures of the unloading areas of the steel piles on the same vehicle each time.

3. The scrap steel recognition and segmentation method based on Segformer according to claim 1, characterized in that: The specific steps of S2 are as follows: S201: Use a Gaussian filter to filter out the granular noise in the imaging pictures. For the overexposure phenomenon in some photos, reduce the information loss and improve the signal-to-noise ratio through methods such as histogram equalization and template matching; S202: Apply the Mosaic data augmentation algorithm to the imaging picture set, and perform operations such as migration, occlusion, and blurring on the data main body to enhance and augment the data to ensure the robustness of the training model.

4. A scrap steel recognition and segmentation method based on Segformer according to claim 1, characterized in that: The specific steps of S3 are as follows: S301: Divide the preprocessed steel stack photos into a training set, a validation set, and a test set in proportion through k-fold cross-validation, and divide them x times to obtain x sets of pictures containing the training set and the validation set respectively and y sets of test set pictures; S302: For the training sets in the x sets of pictures, label the training sets, where the steel blocks are divided into m categories, and obtain x corresponding json files.

5. A scrap steel recognition and segmentation method based on Segformer according to claim 1, characterized in that: The specific steps of S5 are as follows: S501: Send the photo of the scrap steel unloading area into the Segformer model for prediction; S502: Extract the prediction results, segment the predicted single-block closed area, and obtain pictures of single pieces of scrap steel.

6. A scrap steel recognition and segmentation method based on Segformer according to claim 1, characterized in that: The specific steps of S6 are as follows: S601: Supplement the pictures with prediction errors and missing predictions into the original data set, and split the new data set into a training set, a validation set, and a test set according to the original ratio; S602: Use k-fold cross-validation to split the new data set into k parts, and use the Segformer model to train and evaluate the optimal weights for the k parts of the data set; S603: Evaluate the optimal weights and the original weights using the new test set, and release the one with better evaluation.

Images