A billet detection method based on deep learning

Abstract

The application discloses a billet detection method based on deep learning, and belongs to the technical field of billet detection, which is mainly used for the identification and positioning of billets in the process of discharging from a hot continuous rolling heating furnace. The multilayer feature extraction structure of the method is embedded with a CSA module, a SPPELAN network and an improved PANET network, the complex environment in which the billets are located is fully considered, end-to-end network construction from input of a detection picture to output of a prediction result is realized, and the problems of low precision and poor real-time performance of traditional methods are solved. Moreover, a residual network is used to extract image features, the network complexity and feature effectiveness are well balanced, and the model attention to the target and the prediction accuracy are improved. Meanwhile, the improved path aggregation network also makes the model pay more attention to billet features of different scales, and improves the detection precision. Therefore, the automation and intelligence degree of the method is high, and the industrial application prospect is good.

Description

Technical Field

[0001] This invention relates to the field of billet inspection technology, and more specifically, to a billet inspection method based on deep learning. Background Technology

[0002] Steel, as the cornerstone of modern industry, permeates numerous key fields such as construction, machinery manufacturing, automobiles, and aerospace. Its importance is self-evident, and it has become a core material supporting the vigorous development of various industries. The steel production process is complex, with hot continuous rolling being a crucial step. In this process, steel billets need to be fully heated in a furnace to meet specific process requirements. Then, a robotic arm pulls the billet from the furnace and places it on a roller conveyor, where it is smoothly transported to subsequent processes. However, this process faces many challenges. On the one hand, the positioning accuracy of the robotic arm is significantly reduced due to long-term frequent operation or factors such as equipment aging and inaccurate parameter calibration, making it difficult to accurately grasp and stably transfer the billet. On the other hand, the environment around the furnace is harsh and complex, with high temperatures, strong light, dust, and vibrations from equipment operation creating uncertainty in the billet's spatial environment, easily leading to deviations between the billet's actual position and the preset route.

[0003] If the billet position accuracy becomes uncontrolled, it will trigger a series of serious consequences, such as frequent billet drop and roll detachment accidents. This will not only cause significant steel loss, delay production progress, and increase production costs, but may also cause irreversible damage to equipment and even endanger the personal safety of operators. Given the crucial impact of accurate billet position control on the smoothness, stability, and safety of the entire hot continuous rolling production process, real-time and accurate billet position detection is urgently needed. Summary of the Invention

[0004] In view of this, this application provides a billet detection method based on deep learning, which mainly aims to leverage the powerful advantages of deep learning technology to accurately track and locate the position of the billet, thereby solving the problems of low accuracy in real-time billet identification and position detection, poor ability to cope with complex environments, and low degree of automation in the current hot rolling process.

[0005] To achieve the above objectives, this application provides a deep learning-based billet inspection method, comprising:

[0006] S1. Build an initial deep learning network model based on YOLOv8; the model includes a Backbone layer, a Neck layer, and a Head layer; the Backbone layer network is a multi-layer feature extraction structure, its input is a preprocessed image, and its output is a feature map; the Neck layer network's input is a feature map, and its output is a feature fusion map; the Head layer network's input is the feature fusion map, and its output is the predicted billet position information, prediction box, and confidence score;

[0007] S2. Embed a fusion CSA module, which combines the initial C2f module and the hybrid attention Shuffle Attention module, into the multi-layer feature extraction structure;

[0008] S3. Embed SPPELAN network in the Backbone layer;

[0009] S4. Improve the traditional path aggregation network in the model;

[0010] S5. Obtain training samples containing billet feature data and divide them into training set and validation set; the feature data includes a real frame of the billet used to indicate the position and range of the billet.

[0011] S6. Input the feature data of the training set into the model for model training. After the first round of training is completed, use the feature data of the validation set to calculate the model loss value for model validation. Then use the loss value to perform gradient descent operation to update the model parameters. Repeat the model training and model validation process until the loss value no longer decreases or tends to dynamic equilibrium. The training is completed, and the billet detection network model is obtained.

[0012] S7. Input the feature data of the billet to be detected into the billet detection network model for prediction; the output of the billet detection network model is the predicted billet position information, prediction box and confidence score.

[0013] In some embodiments, the multi-layer feature extraction structure has five layers, and the CSA module is embedded at the output of the third, fourth, and fifth layers of the multi-layer feature extraction structure. Its input is a feature map, and its output is an adaptive feature map.

[0014] In some embodiments, the input to the SPPELAN network is an adaptive feature map, and the output is a feature representation map.

[0015] In some embodiments, the improved traditional path aggregation network can perform sampling operations and fusion on the adaptive feature map and feature representation map, making the model more focused on the billet features at different scales.

[0016] In some embodiments, a residual network structure is used in the multi-layer feature extraction structure to extract image features.

[0017] In some embodiments, the anchor frame is selected using a center-point-based method for the actual frame of the billet.

[0018] In some embodiments, the loss value is calculated using a loss function consisting of CIOU+DFL.

[0019] In some embodiments, step S6 further includes evaluating the trained billet detection network model; the evaluation is as follows:

[0020] The test set data is input into the billet detection network model for prediction. The prediction results are compared with the actual results, and the detection performance of the billet detection network model is analyzed through evaluation indicators such as Precision, Recall, and mAP.

[0021] Based on the above technical solution, the deep learning-based billet detection method provided in this application has a high degree of automation and can directly realize the end-to-end network establishment from the input of the detection image to the output of the prediction result, solving the problems of low detection accuracy and poor real-time performance of traditional methods. At the same time, this application also fully considers the complex environment in which the billet is located when it exits the heating furnace, and improves the network model according to the actual working conditions. For example, it adopts a residual network structure to extract image features to balance network complexity and feature effectiveness, and uses (improved PANET network) to improve the model's attention to the features of billets at different scales, thereby improving the detection accuracy and recognition accuracy of difficult samples.

[0022] The above description is only an overview of the technical solution of this application. In order to better understand the technical means of this application, it can be implemented according to the contents of the specification. In order to make the above and other objects, features and advantages of this application more obvious and understandable, specific embodiments of this application are given below. Attached Figure Description

[0023] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the scope of this application, wherein:

[0024] Figure 1 A schematic diagram of a deep learning-based billet inspection method provided for embodiments of this application;

[0025] Figure 2 A schematic diagram of the initial deep learning network model provided in the embodiments of this application;

[0026] Figure 3 A schematic diagram of the CSA network for the initial deep learning network model provided in the embodiments of this application;

[0027] Figure 4 A schematic diagram of the SPPELAN network for the initial deep learning network model provided in the embodiments of this application;

[0028] Figure 5 A schematic diagram of the improved path aggregation network (improved PANET network) of the initial deep learning network model provided in the embodiments of this application. Detailed Implementation

[0029] The following specific examples illustrate the embodiments of the present invention. Those skilled in the art can easily understand other advantages and effects of this application from the content disclosed in this specification. This application can also be implemented or applied through other different specific embodiments, and various details in this specification can be modified or changed based on different viewpoints and applications without departing from the spirit of this application. The accompanying drawings are for illustrative purposes only, representing schematic diagrams rather than actual objects, and should not be construed as limiting the application. To better illustrate the embodiments of this application, it is understandable that some well-known structures or steps and their descriptions may be omitted in the drawings.

[0030] Reference Figures 1-5 This paper describes a deep learning-based billet detection method according to the embodiments of this application.

[0031] In some embodiments, such as Figure 1 As shown, the steel billet detection method based on deep learning provided in this application includes:

[0032] Step 101: A deep learning network model is built based on YOLOv8. After passing through the Backbone and Neck layers, three feature fusion maps of different scales are output to extract billet feature information. Specifically: Step 101-1: The Backbone network of the initial deep learning network model is a multi-layer feature extraction structure. Its input is a preprocessed image, and its output is a feature map. Step 101-2: The input of the Neck network of the initial deep learning network model is the feature map output in Step 101-1, and its output is a feature fusion map, including a first feature fusion map, a second feature fusion map, and a third feature fusion map, each with a different scale.

[0033] In some embodiments, a preprocessed image can refer to an image obtained after processing the input image for convenient subsequent use. For example, if the input is a 640*640*3 image data, it can be preprocessed through a convolutional layer. By changing the number of channels, the 640*640*3 input image is transformed into a 320*320*64 feature map through a 3*3 convolutional kernel with a stride of 1, using BatchNorm2d and SiLU activation functions. Here, the kernel size K = 3, the stride S = 1, the number of input channels (in channel) = 3, and the number of output channels (out channel) = 64. In this application, the input image refers to an image containing a steel billet. For example, the input image can be a billet location image with a size of 640*640*3; while the feature map refers to the image containing steel-related features obtained after processing the input image once or multiple times. For example, the size of the first feature map can be set to 160*160*128, the size of the second feature map to 80*80*256, the size of the third feature map to 40*40*512, and the size of the fourth feature map to 20*20*1024.

[0034] This application aims to fully address the complex environment of steel billets when they exit the heating furnace. To this end, several key modules are embedded in the multi-layer feature extraction structure of the network in step 101. First is the CSA module, which integrates the original C2f module with a hybrid attention module (ShuffleAttention (SA). Its role is to improve the model's ability to extract feature maps in both channels and space. Through the attention mechanism, the model can quickly lock onto the detection target and accurately focus on the features of the steel billet in complex environments. Second is the SPPELAN network, which focuses on improving the model's global receptive field. By calculating the correlation between feature maps, it reasonably allocates weights to different regions and concatenates the pooled feature maps at various scales, thereby achieving local attention enhancement and effectively improving the detection accuracy of the steel billet. Furthermore, there is the improved path aggregation network (improved PANET network). On the one hand, it inherits the advantage of the traditional PANET network in fusing features at different scales, enhancing the model's ability to extract feature information of defects of different sizes, making the model more sensitive to billet location information. On the other hand, it adds a feature layer, further optimizing model performance, increasing the model's control over overall feature information, and improving the accuracy of billet detection. These modules work together to construct a deep learning network for billet detection that adapts to the complex heating furnace environment.

[0035] In some embodiments, such as Figure 2As shown, the multi-layer feature extraction structure of the Backbone network includes layers 1, 2, 3, 4, and 5. The input of the first layer is a preprocessed image, and the output is an initial feature map; the features are related to the steel billet. For example, the first layer can transform the 640*640*3 input information into 320*320*64 through convolution, where the convolution kernel k = 3*3, stride s = 2, padding = 1, and output channel out = 64, using the BatchNorm2d and SiLU activation functions. The input of the second layer is the initial feature map, and the output is the first feature map; the input of the third layer is the first feature map, and the output is the second feature map; the input of the fourth layer is the second feature map, and the output is the third feature map; the input of the fifth layer is the third feature map, and the output is the fourth feature map.

[0036] In some embodiments, such as Figure 2 As shown, the multi-layer feature extraction structure embeds a CSA module, which is embedded at the outputs of the third, fourth, and fifth layers of the multi-layer feature extraction structure. A schematic diagram of the CSA network is shown below. Figure 3 As shown, its inputs are the aforementioned second feature map, third feature map, and fourth feature map, and its outputs are the first adaptive feature map, second adaptive feature map, and third adaptive feature map.

[0037] In some embodiments, the CSA fusion module incorporates the ShuffleAttention (SA) attention mechanism into the Bottleneck layer of the C2f module in the initial network model. The specific integration method is as follows: Figure 3 As shown, the first layer of the attention module uses a block operation to divide the input feature X into x1, x2...xg sub-feature channels; the second layer of the attention module uses a split operation to extract spatial and channel attention from the sub-feature channels respectively; the third layer of the attention module uses a concatenation operation to concatenate the spatial and channel attention obtained after the second layer operation using a sigmoid activation function; the fourth layer of the attention module is a fusion operation, which merges all the sub-feature channels together after the concatenation operation.

[0038] In some embodiments, such as Figure 4 As shown, the SPPELAN network consists of six layers: a first layer, a second layer, a third layer, a fourth layer, a fifth layer, and a sixth layer. The first layer uses a 1x1 convolution operation, normalization, and the SiLU activation function. The second, third, and fourth layers use a 5x5 max pooling operation with a padding of 2. The fifth layer uses a concatenation operation, which concatenates the outputs of the first to fifth layers according to the channel dimension. The sixth layer uses a 1x1 convolution operation, normalization, and the SiLU activation function.

[0039] In some embodiments, the input to the SPPELAN network is the aforementioned third adaptive feature map, and the output is a feature representation map. A schematic diagram of the improved path aggregation network (improved PANET network) is shown below. Figure 5 As shown, its input is set to the aforementioned first adaptive feature map, second adaptive feature map, third adaptive map and feature representation map, and the output is the first feature fusion map, second feature fusion map and third feature fusion map.

[0040] In some embodiments, the improved path aggregation network (improved PANET network) is used to perform sampling operations and fusion on the adaptive feature map and the feature representation map, including: performing a 2x upsampling operation on the feature representation map and fusing the upsampled feature representation map with the second adaptive feature map to obtain a second initial feature fusion map; performing a 2x upsampling operation on the second initial feature fusion map and fusing the upsampled feature representation map with the first adaptive feature map to obtain a first initial feature fusion map, i.e., a first feature fusion map; performing a convolution operation on the first feature fusion map and then fusing and convolving it with the second adaptive feature map and the second initial feature fusion map to obtain a second feature fusion map; performing a convolution operation on the second feature fusion map and then fusing it with the feature representation map and the third adaptive feature map, and then convolving it to obtain a third feature fusion map.

[0041] In some embodiments, the residual network of YOLOv8 is implemented using the ResNet (Residual Neural Network) architecture. The residual network directly connects the output of the previous layer to certain layers of the subsequent layer by introducing skip connections, allowing information to flow directly on the existing layers, thereby accelerating the optimization process. In the YOLOv8 network model, this residual network is applied to feature extraction and processing at various stages, thereby reducing the problems of gradient vanishing and gradient exploding.

[0042] Step 102: Obtain training samples including billet location information data. In some embodiments, the training samples are images, and the billet information data labels include at least a true bounding box of the billet, used to indicate the location and extent of the billet.

[0043] Step 103: Based on the training samples, train the initial deep learning network model to obtain the trained deep learning network model. In some embodiments, the image data consists of collected images of steel billets. These images and their corresponding labels are divided into a training set and a validation set according to an 8:2 ratio. The more images there are, the better the model training effect will be.

[0044] In some embodiments, a trained deep learning network model is obtained, with the training parameters set as follows:

[0045] Learning rate: Initially 1e-2;

[0046] Epoch: A total of 300 epochs;

[0047] Input image size: 640*640*3;

[0048] Batch: 64;

[0049] momentum: 0.937;

[0050] weight_decay: 5e-4;

[0051] Optimizer: SGD;

[0052] Mosaic data augmentation: 1.

[0053] This embodiment collected a total of 1328 photos, including 1063 training data and 178 validation data. The specific training process includes: selecting anchor boxes based on center points from the ground truth bounding boxes; inputting the training samples from the training set into the initial deep learning network model for model training; after one epoch of training, validating the model's performance using the loss value of the validation set data. The loss function consists of CIoU + DFL, as shown in formulas (1) and (2) below, to calculate the initial deep learning network model loss value; using the loss value to perform gradient descent operations to update the parameters of the deep learning network model; repeating the model training and model validation process until the loss value no longer decreases or tends to a dynamic equilibrium, the training is completed, and the deep learning network model is obtained.

[0054]

[0055] In the formula: Indicates the total loss. This indicates the positional loss of the BOX. Used to calculate the difference between predicted and ground truth feature points, IoU represents the intersection-union ratio of the ground truth bounding box and the predicted bounding box, b gt 'b' represents the true bounding box, and 'b' represents the predicted bounding box. α represents the Euclidean distance between the ground truth bounding box and the predicted bounding box, c represents the diagonal length of the smallest rectangle containing both the ground truth bounding box and the predicted bounding box, v represents the aspect ratio of the predicted bounding box and the ground truth bounding box, and α represents the weight parameter.

[0056] In some embodiments, after the deep learning network model has been trained, test data and corresponding labels are input into the trained deep learning network model, and the prediction results of the deep learning network model are compared with the true labels to evaluate the final detection performance of the deep learning network model. After the deep learning network model is trained, the deep learning model with the best performance on the validation set is selected as the final deep learning network model. Then, the test set image data is packaged into a folder, and the model iterates through each image in the file and makes predictions for them individually. Since the model predicts every small square in the feature map, many prediction boxes are generated. Non-maximum suppression is used to filter out prediction boxes with low confidence, with a threshold set to 0.5. The prediction results are compared with the true results, and the detection performance of the model is analyzed using the following evaluation metrics:

[0057]

[0058] In the formula: TP represents the number of positive samples correctly identified as positive samples, FP represents the number of negative samples considered as positive samples, TN represents the number of positive samples considered as negative samples, and FN represents the number of negative samples considered as positive samples. Precision is the prediction accuracy, which represents the proportion of correctly predicted samples out of all predicted positive samples; a higher value indicates a lower false positive rate. Recall is the recall rate, which represents the proportion of correctly predicted positive samples out of the total number of positive samples; a higher value indicates a lower false negative rate. AP represents the average prediction accuracy, calculated by plotting Recall on the horizontal axis and Precision on the vertical axis; a higher value indicates better prediction performance. mAP represents the overall average accuracy, reflecting the overall performance of the model; a higher value indicates better model performance.

[0059] Step 104: Input the billet image into the deep learning network model. The model outputs the predicted billet location information, prediction box, and confidence score.

[0060] A billet image refers to an image of a billet that needs to be predicted. In some embodiments, the billet image can be obtained through production line monitoring equipment or other image acquisition equipment. The billet position refers to the location where the billet exists. Confidence level can refer to the probability that the model correctly identifies the billet as such.

[0061] The terms “comprising” and “having”, and any variations thereof, in the specification, claims, and accompanying drawings of this application are intended to cover non-exclusive inclusion, for example, a process, method, system, product, or apparatus that includes a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such process, method, product, or apparatus.

[0062] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A deep learning-based method for inspecting steel billets, characterized in that, include: S1. Build an initial deep learning network model based on YOLOv8; the model includes a Backbone layer, a Neck layer, and a Head layer; the Backbone layer network is a multi-layer feature extraction structure, its input is a preprocessed image, and its output is a feature map; the Neck layer network's input is the feature map, and its output is a feature fusion map; the Head layer network's input is the feature fusion map, and its output is the predicted billet position information, prediction box, and confidence score; S2. After integrating the ShuffleAttention module into the Bottleneck layer of the initial C2f module, a fused CSA module is obtained. The integration method includes: the first layer of the attention module uses a block operation to divide the input feature X into x1, x2...xg sub-feature channels; the second layer of the attention module uses a split operation to extract spatial and channel attention from the sub-feature channels respectively; the third layer of the attention module uses a concatenation operation to concatenate the spatial and channel attention obtained after the second layer operation of the attention module through a Sigmoid activation function; the fourth layer of the attention module is a fusion operation to fuse all sub-feature channels together after the concatenation operation. The multi-layer feature extraction structure has five layers. The CSA module is embedded at the output of the third, fourth, and fifth layers of the multi-layer feature extraction structure. Its input is the feature map, and its output is an adaptive feature map. S3. Embed the SPPELAN network in the Backbone layer; the input of the SPPELAN network is the adaptive feature map, and the output is the feature representation map; S4. Improve the traditional path aggregation network in the model; the improved traditional path aggregation network can sample and fuse the adaptive feature map and the feature representation map, making the model more focused on the billet features at different scales; the improvement includes: performing a 2x upsampling operation on the feature representation map, and fusing the upsampled feature representation map with the second adaptive feature map to obtain a second initial feature fusion map; performing a 2x upsampling operation on the second initial feature fusion map, and fusing the upsampled feature representation map with the first adaptive feature map to obtain a first initial feature fusion map, i.e., the first feature fusion map; performing a convolution operation on the first feature fusion map, and then fusing and convolving it with the second adaptive feature map and the second initial feature fusion map to obtain a second feature fusion map; performing a convolution operation on the second feature fusion map, and then fusing it with the feature representation map and the third adaptive feature map, and then convolving it to obtain a third feature fusion map; S5. Obtain training samples containing billet feature data and divide them into training set and validation set; the feature data includes a real frame of the billet used to indicate the position and range of the billet. S6. Input the feature data of the training set into the model for model training. After the first round of training is completed, use the feature data of the validation set to calculate the model loss value for model validation. Then use the loss value to perform gradient descent operation to update the parameters of the model. Repeat the model training and model validation process until the loss value no longer decreases or tends to dynamic equilibrium. The training is completed, and the billet detection network model is obtained. S7. Input the feature data of the billet to be detected into the billet detection network model for prediction; the output of the billet detection network model is the predicted billet position information, prediction box and confidence score.

2. The deep learning-based billet inspection method according to claim 1, characterized in that, The image feature extraction is performed using a residual network structure in the multi-layer feature extraction structure.

3. The deep learning-based billet inspection method according to claim 1, characterized in that, The anchor frame is selected using a center-point-based method for the actual frame of the steel billet.

4. The deep learning-based billet inspection method according to claim 1, characterized in that, The loss value is calculated using a loss function consisting of CIoU+DFL.

5. A deep learning-based billet inspection method according to claim 1, characterized in that, Step S6 further includes evaluating the trained billet detection network model; the evaluation is as follows: The test set data is input into the billet detection network model for prediction. The prediction results are compared with the actual results, and the detection performance of the billet detection network model is analyzed through evaluation indicators such as Precision, Recall, and mAP.

Images