Deep learning-based billet detection method

Abstract

The invention discloses a steel billet detection method based on deep learning, belongs to the technical field of steel billet detection, and is mainly used for identifying and positioning steel billets in the discharging process of a hot continuous rolling heating furnace. According to the method, the CSA module, the SPPELAN network and the improved PANET network are embedded in the multi-layer feature extraction structure, the complex environment where the steel billet is located is fully considered, end-to-end network construction from detection picture input to prediction result output is achieved, and the problems that a traditional method is low in precision and poor in real-time performance are solved. And the image features are extracted by using the residual network, so that the network complexity and the feature effectiveness are well balanced, and the attention of the model to the target and the prediction accuracy are improved. Meanwhile, the improved path aggregation network also enables the model to pay more attention to the characteristics of billets with different scales, and the detection precision is improved. Therefore, based on the method, the automation degree and the intelligence degree are high, and the industrial application prospect is good.

Description

Technical Field

[0001] The present invention relates to the technical field of steel billet detection, and in particular to a steel billet detection method based on deep learning. Background Art

[0002] As the cornerstone of modern industry, steel has been widely used in many key fields such as construction, machinery manufacturing, automobiles, aerospace, etc. Its importance is self-evident and it has become the core material supporting the vigorous development of various industries. The steel production process is complex, and the hot rolling process is one of the key links. In this process, the billet needs to be fully heated in the heating furnace. After meeting the specific process requirements, it is pulled out of the heating furnace with the help of a robotic arm and placed on the roller table, and then smoothly transported to the subsequent process. However, this process faces many problems. On the one hand, the positioning accuracy of the robotic arm is greatly reduced due to long-term frequent operation or the influence of factors such as equipment aging and inaccurate parameter calibration, making it difficult to accurately grasp and stably transfer the billet; on the other hand, the environment around the heating furnace is harsh and complex, and the interference factors such as high temperature, strong light, dust and vibration caused by equipment operation are intertwined, making the spatial environment of the billet full of uncertainty, which can easily lead to deviations between the actual position of the billet and the preset route.

[0003] Once the position accuracy of the billet is out of control, it will lead to a series of serious consequences, such as frequent billet roll drop and roll-off accidents. This will not only cause a large loss of steel, delay production progress, and increase production costs, but may also cause irreversible damage to the equipment and even endanger the personal safety of operators. In view of the important impact of accurate control of the billet position on the smoothness, stability and safety of the entire hot rolling production process, real-time and accurate billet position detection is imminent. Summary of the invention

[0004] In view of this, the present application provides a steel billet detection method based on deep learning, which mainly aims to accurately track and locate the position of the steel billet by leveraging the powerful advantages of deep learning technology, and solve the problems of low accuracy in steel billet identification and real-time position detection, poor ability to cope with complex environments, and low degree of automation in the current hot rolling process.

[0005] To achieve the above objectives, the present application provides a steel billet detection method based on deep learning, comprising:

[0006] S1. Build an initial deep learning network model based on YOLOv8; the model includes Backbone layer, Neck layer and Head layer; the Backbone layer network is a multi-layer feature extraction structure, whose input is a preprocessed image and output is a feature map; the input of the Neck layer network is a feature map and the output is a feature fusion map; the input of the Head layer network is a feature fusion map and the output is the predicted billet position information, prediction box and confidence;

[0007] S2, a fusion CSA module that embeds the initial C2f module and the hybrid attention Shuffle Attention module in the multi-layer feature extraction structure;

[0008] S3, embed the SPPELAN network in the Backbone layer;

[0009] S4, improve the traditional path aggregation network in the model;

[0010] S5, obtaining training samples containing steel billet feature data, and dividing the samples into a training set and a validation set; the feature data includes a steel billet real frame for indicating the position and range of the steel billet;

[0011] S6, input the characteristic data of the training set into the model for model training. After the first round of training is completed, the model loss value is calculated using the characteristic data of the verification set for model verification, and then the loss value is used to perform a gradient descent operation to update the parameters of the model; repeat the model training and model verification process until the loss value no longer decreases or tends to a dynamic balance, the training is completed, and the billet detection network model is obtained;

[0012] S7. Input the characteristic data of the steel billet to be detected into the steel billet detection network model for prediction; the output of the steel billet detection network model is the predicted steel billet position information, prediction box and confidence.

[0013] In some embodiments, the multi-layer feature extraction structure has five layers, and the embedding position of the CSA module is the output end of the third, fourth and fifth layers of the multi-layer feature extraction structure, its input is a feature map, and its output is an adaptive feature map.

[0014] In some embodiments, the input of the SPPELAN network is an adaptive feature map, and the output is a feature representation map.

[0015] In some embodiments, the improved traditional path aggregation network can perform sampling operations and fusion on the adaptive feature map and the feature representation map, so that the model pays more attention to the steel billet features of different scales.

[0016] In some embodiments, a residual network structure is used in a multi-layer feature extraction structure to extract image features.

[0017] In some embodiments, the steel billet real frame is used to select an anchor frame using a center point based method.

[0018] In some embodiments, the loss value is calculated by a loss function consisting of CIOU+DFL.

[0019] In some embodiments, step S6 further includes evaluating the trained steel billet detection network model; the evaluation is:

[0020] The test set data is input into the steel billet detection network model for prediction, the predicted results are compared with the actual results, and the detection performance of the steel billet detection network model is analyzed through the Precision, Recall and mAP evaluation indicators.

[0021] According to the above technical scheme, the deep learning-based steel billet detection method provided by the present application has a high degree of automation, and can directly realize the end-to-end network establishment from the detection image input to the prediction result output, thereby solving the problems of low detection accuracy and poor real-time performance of traditional methods. At the same time, the present application also fully considers the complex environment in which the steel billet is located when it comes out of the heating furnace, and improves the network model according to the actual working conditions, such as using the residual network structure to extract image features to balance the network complexity and feature effectiveness, and using (improved PANET network) to improve the model's attention to steel billet features of different scales, thereby improving the detection accuracy and recognition accuracy of difficult samples.

[0022] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] By reading the detailed description of the preferred embodiment below, various other advantages and benefits will become clear to those skilled in the art. The accompanying drawings are only used to illustrate the preferred embodiment and are not considered to be limiting of the present application, wherein:

[0024] Figure 1 A schematic flow chart of a steel billet detection method based on deep learning provided in an embodiment of the present application;

[0025] Figure 2 A schematic diagram of an initial deep learning network model provided in an embodiment of the present application;

[0026] Figure 3 A CSA network diagram of an initial deep learning network model provided in an embodiment of the present application;

[0027] Figure 4 A schematic diagram of the SPPELAN network of the initial deep learning network model provided in the embodiment of the present application;

[0028] Figure 5 A schematic diagram of an improved path aggregation network (improved PANET network) of the initial deep learning network model provided in an embodiment of the present application. DETAILED DESCRIPTION

[0029] The following describes the embodiments of the present invention through specific examples. Those skilled in the art can easily understand other advantages and effects of the present application from the contents disclosed in this specification. The present application can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present application. Among them, the drawings are only used for exemplary descriptions, and only schematic diagrams are shown, not physical drawings, and cannot be understood as limitations on the present application; in order to better illustrate the embodiments of the present application, it is understandable to those skilled in the art that some well-known structures or steps in the drawings and their descriptions may be omitted.

[0030] Reference Figure 1-Figure 5 , a steel billet detection method based on deep learning involved in an implementation manner of the present application is described.

[0031] In some embodiments, Figure 1 As shown, a steel billet detection method based on deep learning provided by the present application includes:

[0032] Step 101, build a deep learning network model based on YOLOv8, and after the Backbone and Neck layer networks, three feature fusion maps of different scales can be output to extract the feature information of the steel billet. Among them: Step 101-1, the Backbone network of the initial deep learning network model is a multi-layer feature extraction structure, whose input is the preprocessed image and output is the feature map; Step 101-2, the input of the Neck network of the initial deep learning network model is the feature map output in the above step 101-1, and its output is a feature fusion map, including a first feature fusion map, a second feature fusion map and a third feature fusion map, and the scales of each fusion map are different.

[0033] In some embodiments, the preprocessed image may refer to an image obtained after processing the input image, which is convenient for subsequent use. For example, the input is 640*640*3 image data, which can be preprocessed through a convolution layer. By changing the number of channels, the input 640*640*3 image is transformed into a 320*320*64 feature map through a 3*3 convolution kernel, a step size of 1, BatchNorm2d and SiLU activation function. Among them, the size of the convolution kernel of the first layer is K=3, the step size S=1, the number of input channels in channel=3, and the number of output channels out channel=64. In this application, the input image refers to an image containing a steel billet. For example, the input image may be a steel billet position image with a size of 640*640*3; and the feature map refers to a map containing features related to steel obtained after processing the input image once or multiple times, such as setting the size of the first feature map to 160*160*128, the size of the second feature map to 80*80*256, the size of the third feature map to 40*40*512, and the size of the fourth feature map to 20*20*1024.

[0034] The present application aims to fully cope with the complex environment in which the steel billet is located when it comes out of the heating furnace. To this end, multiple key modules are embedded in the multi-layer feature extraction structure of the network structure in step 101. The first is the CSA module, which integrates the original network C2f module and the hybrid attention Shuffle Attention (SA) module. Its function is to improve the model's ability to extract feature maps in channels and space. With the attention mechanism, the model can quickly lock the detection target and accurately focus on the features of the steel billet in a complex environment. The second is the SPPELAN network, which focuses on improving the global receptive field of the model. By calculating the correlation between feature maps, reasonably allocating weights to different regions, and splicing the feature maps after pooling at each scale, local attention enhancement is achieved, thereby effectively improving the detection accuracy of the steel billet. The next is the improved path aggregation network (improved PANET network). On the one hand, it inherits the advantages of the traditional PANET network in fusing features of different scales, fuses features of different scales, enhances the model's ability to extract feature information of defects of different sizes, and makes the model more sensitive to the position information of the billet; on the other hand, it adds a feature layer, further optimizes the model performance, increases the model's control over the overall feature information, and improves the accuracy of billet detection. These modules work together to build a deep learning network for billet detection that adapts to the complex heating furnace environment.

[0035] In some embodiments, Figure 2As shown in the figure, the multi-layer feature extraction structure of the Backbone network includes the first layer, the second layer, the third layer, the fourth layer and the fifth layer; the input of the first layer is the preprocessed image, and the output is the initial feature map; wherein the feature is related to the steel billet. For example, the first layer can transform the input information of 640*640*3 into 320*320*64 through convolution, wherein the convolution kernel k=3*3, the step size s=2, the padding=1, the output channel out=64, and the BatchNorm2d and SiLU activation functions are used. The input of the second layer is the initial feature map, and the output is the first feature map; the input of the third layer is the first feature map, and the output is the second feature map; the input of the fourth layer is the second feature map, and the output is the third feature map; the input of the fifth layer is the third feature map, and the output is the fourth feature map.

[0036] In some embodiments, Figure 2 As shown in FIG. 1 , the multi-layer feature extraction structure is embedded with a CSA module, and its embedded position is the output end of the third layer, the fourth layer and the fifth layer of the multi-layer feature extraction structure. The schematic diagram of the CSA network is shown in FIG. Figure 3 As shown, its inputs are the aforementioned second feature map, third feature map and fourth feature map, and its outputs are the first adaptive feature map, the second adaptive feature map and the third adaptive feature map.

[0037] In some embodiments, the CSA fusion module integrates the Shuffle Attention (SA) mechanism into the Bottleneck layer in the C2f module of the initial network model. The specific integration method is as follows: Figure 2 As shown in the figure, the first layer of the attention module adopts a block operation to divide the input feature X into x1, x2...xg sub-feature channels; the second layer of the attention module adopts a split operation on the sub-feature channels to extract spatial and channel attention from the sub-feature channels respectively; the third layer of the attention module adopts a splicing operation to splice the spatial attention and channel attention obtained after the second layer of the attention module through the Sigmoid activation function; the fourth layer of the attention module is a fusion operation, which fuses all the sub-feature channels together after the splicing operation.

[0038] In some embodiments, Figure 4 As shown in the figure, the SPPELAN network includes the first layer, the second layer, the third layer, the fourth layer, the fifth layer and the sixth layer; the first layer adopts 1*1 convolution operation, normalization and SiLU activation function; the second, third and fourth layers adopt 5*5 maximum pooling layer operation with padding of 2; the fifth layer adopts splicing operation to splice the outputs of the first to fifth layers according to the channel dimension; the sixth layer adopts 1*1 convolution operation, normalization and SiLU activation function.

[0039] In some embodiments, the input of the SPPELAN network is set to be the aforementioned third adaptive feature map, and the output is a feature representation map. Figure 5 As shown, its input is set to the aforementioned first adaptive feature map, second adaptive feature map, third adaptive map and feature representation map, and its output is the first feature fusion map, second feature fusion map and third feature fusion map.

[0040] In some embodiments, an improved path aggregation network (improved PANET network) is used to perform sampling operations and fusion on adaptive feature maps and feature representation maps, including: performing a 2-fold upsampling operation on the feature representation map, and fusing the upsampled feature representation map with the second adaptive feature map to obtain a second initial feature fusion map; performing a 2-fold upsampling operation on the second initial feature fusion map, and fusing the upsampled feature representation map with the first adaptive feature map to obtain a first initial feature fusion map, i.e., a first feature fusion map; performing a convolution operation on the first feature fusion map, and fusing and convolving the first feature fusion map with the second adaptive feature map and the second initial feature fusion map to obtain a second feature fusion map; performing a convolution operation on the second feature fusion map, and fusing it with the feature representation map and the third adaptive feature map, and convolving the second feature fusion map to obtain a third feature fusion map.

[0041] In some embodiments, the residual network of YOLOv8 is implemented through the architecture of ResNet (residual neural network); the residual network directly connects the output of the previous layer to certain layers of the subsequent layers by introducing skip connections, so that the information flow can flow directly on the existing layers, thereby accelerating the optimization process; in the YoloV8 network model, this residual network is used for feature extraction and processing at each stage, thereby reducing the problems of gradient disappearance and gradient explosion.

[0042] Step 102, obtaining a training sample including billet position information data. In some embodiments, the training sample is an image, and the billet information data label at least includes a billet real frame, which is used to indicate the position and range of the billet.

[0043] Step 103, based on the training samples, train the initial deep learning network model to obtain a trained deep learning network model. In some embodiments, the image data is collected billet pictures, and these pictures and corresponding labels are divided into a training set and a validation set according to the principle of 8:2. The more pictures there are, the better the effect of model training will be.

[0044] In some embodiments, a trained deep learning network model is obtained, and the training parameters are set as follows:

[0045] Learning rate: Initial 1e-2;

[0046] Epoch: 300 epochs in total;

[0047] Input image size: 640*640*3;

[0048] Batch: 64;

[0049] momentum: 0.937;

[0050] weight_decay: 5e-4;

[0051] Optimizer: SGD;

[0052] Mosaic data augmentation:1.

[0053] In this embodiment, a total of 1328 photos are collected, including 1063 training data and 178 verification data. The specific training process includes: selecting anchor frames from the real frames using a center point-based method; inputting the training samples in the training set into the initial deep learning network model for model training; after an epoch of training is completed, the performance of the model is verified by the loss value of the verification set data. The loss function is composed of CIoU+DFL, as shown in the following formulas (1) and (2), and the loss value of the initial deep learning network model is calculated; the loss value is used to perform a gradient descent operation to update the parameters of the deep learning network model; the model training and model verification process are repeated until the loss value no longer decreases or tends to a dynamic equilibrium, the training is completed, and the deep learning network model is obtained.

[0054]

[0055] Where: represents the total loss, Indicates the position loss of BOX, It is used to calculate the difference between the predicted feature points and the real feature points. IoU represents the intersection over union ratio between the real box and the predicted box. gt represents the real box, b represents the predicted box, Represents the Euclidean distance between the true box and the predicted box, c represents the diagonal length of the minimum rectangle containing the true box and the predicted box, V represents the horizontal and vertical distance between the predicted box and the true box, and α represents the weight parameter.

[0056] In some embodiments, after the deep learning network model training is completed, the test data and corresponding labels are input into the trained deep learning network model, and the prediction results of the deep learning network model are compared with the true labels to evaluate the final detection performance of the deep learning network model. After the deep learning network model training is completed, the deep learning model with the best verification set effect is selected as the final deep learning network model, and then the test set data image data is packaged into a folder. The model traverses each picture in the file and predicts them separately. Since the model will predict each small square of the feature map, many prediction boxes will be generated, and those prediction boxes with low confidence need to be filtered out by the non-maximum suppression method, and the threshold is set to 0.5. The predicted results are compared with the true results, and the detection performance of the model is analyzed by the following evaluation indicators:

[0057]

[0058] Where: TP is the number of positive samples correctly identified as positive samples, FP is the number of negative samples considered as positive samples, TN is the number of positive samples considered as negative samples, and FN is the number of negative samples considered as positive samples. Precision is the prediction accuracy, which indicates the ratio of the number of samples predicted correctly to the number of samples predicted as positive. The larger the value, the lower the false positive rate. Recall is the recall rate, which indicates the ratio of the number of positive samples predicted correctly to the total number of positive samples. The larger the value, the lower the missed detection rate. AP is the average prediction accuracy, with Recall as the horizontal axis and Precision as the vertical axis. The larger the value, the better the prediction performance. mAP is the average accuracy of all. This value reflects the overall performance of the model. The larger the value, the better the model performance.

[0059] Step 104, input the billet image into the deep learning network model, and the model outputs the predicted billet position information, prediction box and confidence.

[0060] The billet image refers to the billet image that needs to be predicted. In some embodiments, the billet image can be obtained by a monitoring device of a production line or other image acquisition device. The billet position refers to the position where the billet exists. The confidence level can refer to the probability that the model determines whether the billet is correct.

[0061] The terms "including" and "having" and any variations thereof in the specification and claims of the present application and the above-mentioned drawings are intended to cover non-exclusive inclusions. For example, a process, method, system, product or apparatus comprising a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to these processes, methods, products or apparatuses.

[0062] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the purpose and scope of the technical solution, which should be included in the scope of the claims of the present invention.

Claims

1. A steel billet detection method based on deep learning, characterized in that: include: S1. Build an initial deep learning network model based on YOLOv8; the model includes Backbone layer, Neck layer and Head layer; the Backbone layer network is a multi-layer feature extraction structure, whose input is a preprocessed image and output is a feature map; the input of the Neck layer network is the feature map and the output is a feature fusion map; the input of the Head layer network is the feature fusion map and the output is the predicted billet position information, prediction box and confidence; S2, embedding a fusion CSA module of an initial C2f module and a hybrid attention Shuffle Attention module in the multi-layer feature extraction structure; S3, embedding the SPPELAN network in the Backbone layer; S4, improving the traditional path aggregation network in the model; S5, obtaining training samples containing steel billet feature data, and dividing the samples into a training set and a validation set; the feature data includes a steel billet real frame for indicating the position and range of the steel billet; S6, inputting the characteristic data of the training set into the model for model training, after the first round of training is completed, using the characteristic data of the verification set to calculate the model loss value for model verification, and then using the loss value to perform a gradient descent operation to update the parameters of the model; repeating the model training and model verification process until the loss value no longer decreases or tends to a dynamic balance, the training is completed, and a steel billet detection network model is obtained; S7. Input the characteristic data of the steel billet to be detected into the steel billet detection network model for prediction; the output of the steel billet detection network model is the predicted steel billet position information, prediction box and confidence.

2. A steel billet detection method based on deep learning according to claim 1, characterized in that: The multi-layer feature extraction structure has five layers, and the embedding position of the CSA module is the output end of the third layer, the fourth layer and the fifth layer of the multi-layer feature extraction structure. Its input is the feature map and its output is an adaptive feature map.

3. A steel billet detection method based on deep learning according to claim 2, characterized in that: The input of the SPPELAN network is the adaptive feature map, and the output is a feature representation map.

4. A steel billet detection method based on deep learning according to claim 3, characterized in that: The improved traditional path aggregation network can perform sampling operations and fusion on the adaptive feature graph and the feature representation graph, so that the model pays more attention to the steel billet features of different scales.

5. A steel billet detection method based on deep learning according to claim 1, characterized in that: In the multi-layer feature extraction structure, a residual network structure is used to extract image features.

6. A steel billet detection method based on deep learning according to claim 1, characterized in that: The anchor frame of the real frame of the steel billet is selected using a center point-based method.

7. A steel billet detection method based on deep learning according to claim 1, characterized in that: The loss value is calculated by a loss function consisting of CIOU+DFL.

8. A steel billet detection method based on deep learning according to claim 1, characterized in that: The step S6 also includes evaluating the trained steel billet detection network model; the evaluation is: The test set data is input into the steel billet detection network model for prediction, the prediction result is compared with the actual result, and the detection performance of the steel billet detection network model is analyzed through the Precision, Recall and mAP evaluation indicators.

Images