An automatic labeling system based on steel billet spray number recognition

Abstract

The present invention discloses an automatic labeling system based on steel billet spray number recognition, comprising: a printer, a labeling machine located at a labeling position, the printer and the labeling machine being communicatively connected to a secondary system, a camera installed in a stacking area, the camera being connected to a processor, and the processor being connected to the secondary system; after the steel billet arrives at the printer position, the printer is controlled to print the batch number of the H-shaped steel billet and associate the batch number with the basic information of the current steel billet; when the steel billet arrives at the stop block position in the stacking area, the camera is controlled to capture an image of the steel billet and send it to the processor, which identifies the batch number of the steel billet in the image and sends it to the secondary system, which controls the generation of label information based on the batch number and basic information of the steel section; when the steel billet is transported to the labeling position, the labeling machine generates a label corresponding to the label information and applies the label. Automatic labeling and storage of steel billets are achieved, and the information flow of steel billets corresponds to the physical flow, thereby improving the information level of heavy H-shaped steel.

Description

Technical Field

[0001] The present invention belongs to the technical field of information tracking, and more specifically, relates to an automatic labeling system based on steel billet spray number recognition. Background Art

[0002] In the steel and metallurgical industry, billet numbering is required to achieve full-process tracking and traceability of steel billets and prevent accidents such as incorrect shipments. A material tracking system tracks data from the entire process, including plate delivery, rolling mills, steel loading, and tapping. To achieve a high level of automation in hot-rolled billet production lines, unique billet numbering improves the efficiency of logistics tracking and quality management in steel companies.

[0003] The H-beam production line has complex processes, and there are problems with difficulty or inaccurate tracking of materials one by one in the finishing area, which results in the automatic labeling machine being unable to be put into use normally and the finished product warehouse area being unable to realize automatic warehousing.

[0004] In the existing finished product warehouse, incoming goods need to be manually labeled before being shipped. Manual labeling is not only labor-intensive, but also poses safety risks such as missed or incorrect labeling. Summary of the Invention

[0005] The present invention provides an automatic labeling system based on steel billet spray number recognition, aiming to improve the above problems.

[0006] The present invention is implemented as follows: an automatic labeling system based on steel billet spray number recognition, the system comprising:

[0007] The inkjet printer located behind the straightening machine and the labeling machine located at the labeling position are connected to the secondary system in communication. The camera installed in the stacking area is connected to the processor, and the processor is connected to the secondary system.

[0008] After the billet arrives at the coding position, the secondary system controls the inkjet printer to code the batch number of the H-shaped billet and associates the batch number with the basic information of the current billet.

[0009] When the billet reaches the stop block position in the stacking area, the camera is controlled to capture the billet image and send it to the processor. The processor identifies the billet batch number in the billet image and sends it to the secondary system. The secondary system controls the generation of label information based on the batch number and basic information of the steel section.

[0010] When the steel billet is transported to the labeling location, the labeling machine generates a label corresponding to the label information and labels it.

[0011] Furthermore, a billet batch number recognition model is constructed based on the improved Yolov3 network model. The billet image is input into the billet batch number recognition model, and the billet batch number recognition model outputs the batch number in the current billet image.

[0012] Furthermore, before the billet image is input into the billet batch number recognition model, the brightness of the billet image is corrected to balance the image brightness of the billet image.

[0013] Furthermore, the brightness correction process of the billet image is as follows:

[0014] (1) Use the Gaussian function to extract the illumination component I(x,y) in the billet image;

[0015] (2) Based on the illumination component I(x,y), the brightness value of the corresponding (x,y) pixel is corrected to balance the image brightness of the steel billet image.

[0016] Furthermore, the correction formula for the image brightness of the billet image is as follows:

[0017]

[0018] Where F′(x,y) is the brightness value of the billet image at the (x,y) position after correction, and F(x,y) is the brightness value of the billet image at the (x,y) position before correction. is the mean value of the illumination component in the billet image, and γ(x,y) is the brightness balance index at the (x,y) position.

[0019] Furthermore, the improved Yolov3 network model structure is as follows:

[0020] The first DBL module, Res×1 residual module, first Res×4 residual module, first Res×8 residual module, second Res×8 residual module, second Res×4 residual module, third Res×4 residual module, second DBL module and first Res×2 residual module are connected in sequence;

[0021] The output end of the first Res×2 residual module is connected to the first convolutional layer Conv, and the output end of the first convolutional layer Conv is the first output end;

[0022] The output end of the first Res×2 residual module is connected to the third DBL module and the first upsampling module in sequence. The output end of the second Res×4 residual module and the first upsampling module is connected to the seventh DBL module through the first tensor splicing module Concat. The seventh DBL module is connected to the second Res×2 residual module and the second convolutional layer Conv in sequence. The output end of the second convolutional layer Conv is the second output end.

[0023] The output end of the second Res×2 residual module is connected to the fourth DBL module and the second upsampling module in sequence. The output ends of the second Res×8 residual module and the second upsampling module are connected to the eighth DBL module through the second tensor splicing module Concat. The eighth DBL module is connected to the third Res×2 residual module and the third convolutional layer Conv in sequence. The output end of the third convolutional layer Conv is the third output end.

[0024] The output end of the third Res×2 residual module is connected to the fifth DBL module and the third upsampling module in sequence. The output ends of the first Res×8 residual module and the third upsampling module are connected to the ninth DBL module through the third tensor splicing module Concat. The ninth DBL module is connected to the fourth Res×2 residual module and the fourth convolutional layer Conv in sequence. The output end of the fourth convolutional layer Conv is the fourth output end.

[0025] The output end of the fourth Res×2 residual module is connected to the sixth DBL module and the fourth upsampling module in sequence. The output ends of the first Res×4 residual module and the fourth upsampling module are connected to the tenth DBL module through the fourth tensor splicing module Concat. The tenth DBL module is connected to the fifth Res×2 residual module and the fifth convolutional layer Conv in sequence. The output end of the fifth convolutional layer Conv is the fifth output end.

[0026] The output results from the first output terminal to the fifth output terminal are subjected to non-maximum suppression and then the billet batch number in the billet image is output.

[0027] Furthermore, the DBL module is a combination of the convolutional layer Conv, the BN layer and the activation function layer Relu.

[0028] Furthermore, the inkjet printer prints the batch number every 5 meters.

[0029] Furthermore, the inkjet printer sprays the batch number of the steel billet on the flange of the H-shaped steel billet.

[0030] The automatic labeling system based on steel billet spray number recognition provided by the present invention has the following beneficial technical effects:

[0031] (1) After the straightening machine, the batch number is printed on the steel billet with a printer, and a camera is installed at the manual inspection table in the stacking area to identify the batch number of the steel billet, so as to achieve the full-line tracking of heavy H-beam in the finishing area;

[0032] (2) Automatic labeling and storage of steel billets are realized, which greatly improves the problem of manual labeling, which is labor-intensive and has potential safety hazards, realizes the correspondence between the information flow and the physical flow of steel billets, and improves the information level of heavy H-beams. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1A schematic structural diagram of an automatic labeling system based on steel billet spray number recognition provided by an embodiment of the present invention;

[0034] Figure 2 A schematic diagram of the structure of the improved Yolov3 network model provided by an embodiment of the present invention;

[0035] Figure 3 A comparison diagram of the improvements before the target detection output layer of the Yolov3 network model provided by an embodiment of the present invention, where (a) is the existing Yolov3 network model and (b) is the improved Yolov3 network model;

[0036] Figure 4 Schematic diagram of the change of loss function value with the number of iterations provided by an embodiment of the present invention;

[0037] Figure 5 The following are the effect diagrams of identifying the spray number of a steel billet provided by an embodiment of the present invention, wherein (a) is Example 1 and (b) is Example 2;

[0038] Figure 6 A comparison diagram of the pretreatment effects of steel billets provided in an embodiment of the present invention, wherein (a) is an image of the pretreated steel billet, and (b) is an image of the steel billet after pretreatment. DETAILED DESCRIPTION

[0039] The specific implementation methods of the present invention will be further explained in detail below by describing the embodiments with reference to the accompanying drawings, so as to help those skilled in the art to have a more complete, accurate and in-depth understanding of the inventive concept and technical solution of the present invention.

[0040] Figure 1 The following is a schematic diagram of the structure of an automatic labeling system based on steel billet spray number recognition provided by an embodiment of the present invention. For ease of explanation, only the parts related to the embodiment of the present invention are shown. The system includes:

[0041] The inkjet printer located behind the straightening machine and the labeling machine located at the labeling position are connected to the secondary system in communication. The camera installed on the inspection stand is connected to the processor, and the processor is connected to the secondary system.

[0042] After the billet arrives at the coding position, the secondary system controls the inkjet printer to code the batch number on the flange of the H-shaped billet, and at the same time associates the batch number with the basic information of the current billet;

[0043] After that, it is moved horizontally to the cold saw area through rows of stands. The cold saw cuts the steel billets into fixed lengths and sends them to the stacking area. When the steel billets reach the stop block position in the stacking area, the camera is controlled to capture the image of the steel billet and send it to the processor. The processor identifies the batch number of the steel billet in the image and sends it to the secondary system. The secondary system controls the formation of label information based on the batch number and basic information of the steel section. When the steel billets are transported to the labeling position, the labeling machine generates a label corresponding to the label information and labels it. Finally, the steel billets are packed and shipped through stacking.

[0044] In an embodiment of the present invention, when subsequent steel billets are cold sawed, it is ensured that there is at least one complete batch number on the H-shaped steel after sawing. The inkjet printer sprays the batch number once every 5 meters. If there are two batch numbers on a steel billet, any one of the batch numbers can be selected to form the label information.

[0045] In the embodiment of the present invention, the process of identifying the billet number in the billet image is as follows:

[0046] A billet batch number recognition model is constructed, and the billet image captured by the camera is input into the billet batch number recognition model, which then outputs the batch number in the current billet image.

[0047] The present invention adopts the trained improved Yolov3 network model as the steel billet batch number recognition model. The structure of the improved Yolov3 network model is as follows: Figure 2 As shown, the improved Yolov3 network model includes:

[0048] The first DBL module, Res×1 residual module, first Res×4 residual module, first Res×8 residual module, second Res×8 residual module, second Res×4 residual module, third Res×4 residual module, second DBL module and first Res×2 residual module are connected in sequence;

[0049] The output end of the first Res×2 residual module is connected to the first convolutional layer Conv, and the first convolutional layer Conv outputs the first recognition result Y1;

[0050] The output end of the first Res×2 residual module is connected to the third DBL module and the first upsampling module in sequence. The output end of the second Res×4 residual module and the first upsampling module is connected to the seventh DBL module through the first tensor splicing module Concat. The seventh DBL module is connected to the second Res×2 residual module and the second convolutional layer Conv in sequence. The second convolutional layer Conv outputs the second recognition result Y2;

[0051] The output end of the second Res×2 residual module is connected to the fourth DBL module and the second upsampling module in sequence. The output ends of the second Res×8 residual module and the second upsampling module are connected to the eighth DBL module through the second tensor splicing module Concat. The eighth DBL module is connected to the third Res×2 residual module and the third convolutional layer Conv in sequence. The third convolutional layer Conv outputs the third recognition result Y3;

[0052] The output end of the third Res×2 residual module is connected to the fifth DBL module and the third upsampling module in sequence. The output ends of the first Res×8 residual module and the third upsampling module are connected to the ninth DBL module through the third tensor splicing module Concat. The ninth DBL module is connected to the fourth Res×2 residual module and the fourth convolutional layer Conv in sequence. The fourth convolutional layer Conv outputs the fourth recognition result Y4;

[0053] The output end of the fourth Res×2 residual module is connected to the sixth DBL module and the fourth upsampling module in sequence. The output ends of the first Res×4 residual module and the fourth upsampling module are connected to the tenth DBL module through the fourth tensor splicing module Concat. The tenth DBL module is connected to the fifth Res×2 residual module and the fifth convolutional layer Conv in sequence. The fifth convolutional layer Conv outputs the fifth recognition result Y5.

[0054] After performing non-maximum suppression on the first to fifth recognition results Y1 to Y5, the billet spray number in the billet image is output.

[0055] Among them, the DBL module is a combination of the convolution layer Conv, the BN layer and the activation function layer Relu.

[0056] Since the size of a single character printed on a steel billet accounts for less than 5% of the entire image, the difficulty of steel billet character recognition lies in accurately locating and classifying relatively small characters in complex scenes. The present invention improves the Yolov3 network model by (1) firstly enhancing the deep residual network to extract character features, then designing 5 convolutional feature maps of different scales, and fusing them with the feature maps of corresponding scales in the residual network to form a final feature pyramid to perform the steel billet character prediction task; (2) adding 2 residual units to the second residual block of Darknet53 in the Yolov3 network model to obtain accurate position information in the lower layer of the network; at the same time, performing an upsampling operation on the feature pyramid with a 2-fold step size, and fusing it with the previous deep residual network to form a deep fusion fast character detection model; (3) the target detection output layer of the Yolo v3 network contains 6 DBL units and a 1×1 convolution, such as Figure 3As shown in (a), in order to avoid gradient disappearance and enhance feature reuse, the first 6 DBL units in the output layer are converted into 1 DBL unit and 2 ResNet units, as shown in Figure 3 As shown in (b), while ensuring high detection accuracy, the model depth is reduced and the detection speed is improved.

[0057] In the training phase, the steel billets with printed characters photographed at the heavy H site are used as the training sample set. During the training process, the loss function value changes with the number of iterations as follows: Figure 4 As shown in the figure, the average loss function value stabilizes with the increase in the number of iterations, and after nearly 20,000 training iterations, the average loss function value drops to around 0.18. The improved Yolo 3 network model performs relatively well. Based on the feature extraction network structure of the Yolo v3 network model, two scale detections are added, which improves mAP and recognition accuracy, reaching 99.5%. At the same time, the network structure before the output layer is changed, which effectively reduces the detection time. The recognition time for a single image is only 135ms. Figure 5 To show the recognition results, Figure 5 (a) and (b) are both the recognition effect diagrams of the steel billet spray number.

[0058] Due to factors such as changes in ambient lighting conditions and shadows between objects, there will be significant differences in the brightness of the collected images. Unconstrained changes in billet brightness will cause differences in grayscale and contrast between the billet characters and the background, as well as differences in brightness distribution in the billet area, thereby reducing the distinction between characters and background, making it more difficult to extract stable features in the billet area. In order to improve the recognition stability of the billet batch number recognition model, after obtaining the billet image, the billet image is first processed, and the processed billet image is then input into the billet batch number recognition model for batch number recognition of the current billet. The preprocessing process of the billet image is described below:

[0059] (1) Use the Gaussian function to extract the illumination component I(x,y) in the billet image, where I(x,y) represents the illumination component in the xth row and yth column of the billet image;

[0060] The object image is produced when the light reflected from its surface reaches the imaging unit. The brightness F(x,y) of the billet image at the (x,y) position is composed of the product of the illumination component I(x,y) incident on the billet surface and the reflection component R(x,y) of the billet surface. The expression is as follows: F(x,y)=I(x,y)*R(x,y). In order to eliminate the interference of uneven illumination, the illumination component of the billet image must be extracted first. The Gaussian function can effectively extract the illumination component in the billet image. The Gaussian function form is: Where λ is the normalization constant, and c is the Gaussian function scaling factor. According to Retinex color theory, the Gaussian function scaling factor c controls the range of the convolution kernel. A larger c increases the range of the convolution kernel and is more sensitive to global effects. A smaller c decreases the range of the convolution kernel and is more sensitive to local effects. After extensive experimentation, we selected the parameter c = 2 to accommodate most images. Furthermore, we must ensure that the Gaussian function G(x,y) satisfies ∫∫G(x,y) = 1. Then, by convolving the Gaussian function with the image, we can extract the illumination component I(x,y) = F(x,y) * G(x,y).

[0061] (2) Based on the illumination component I(x,y), the brightness value of the corresponding (x,y) pixel is corrected to balance the image brightness of the steel billet image.

[0062] After the illumination component I(x,y) is extracted through the Gaussian function, the illumination balance of the billet image can be performed. It needs to be converted to the HSV color model. The two-dimensional gamma function is shown in the following formula:

[0063]

[0064]

[0065] Where F′(x,y) is the brightness value of the billet image at the (x,y) position after correction, and F(x,y) is the brightness value of the billet image at the (x,y) position before correction. is the mean value of the illumination component in the billet image, γ(x,y) is the brightness balance index at the (x,y) position, and the illumination is balanced based on the brightness balance index to obtain the corrected image brightness value.

[0066] The illumination equalization steel billet preprocessing based on the two-dimensional gamma function is effective for images with uneven illumination, such as night images and exposed images. The algorithm has a fast calculation speed and it only takes about 40ms to perform illumination equalization on a single image, effectively improving the image quality and the accuracy of subsequent character detection and recognition steps. The preprocessing results of the steel billet are shown below. Figure 6 As shown, (a) is the preprocessed billet image, and (b) is the preprocessed billet image.

[0067] The present invention has been described exemplarily. Obviously, the specific implementation of the present invention is not limited to the above-mentioned method. As long as various non-substantial improvements are made using the method concept and technical solution of the present invention, or the concept and technical solution of the present invention are directly applied to other occasions without improvement, they are all within the scope of protection of the present invention.

Claims

1. An automatic labeling system based on steel billet spray number recognition, characterized in that: The system comprises: The inkjet printer located behind the straightening machine and the labeling machine located at the labeling position are connected to the secondary system in communication. The camera installed in the stacking area is connected to the processor, and the processor is connected to the secondary system. After the billet arrives at the coding position, the secondary system controls the inkjet printer to code the batch number of the H-shaped billet and associates the batch number with the basic information of the current billet. When the billet reaches the stop block position in the stacking area, the camera is controlled to capture the billet image and send it to the processor. The processor identifies the billet batch number in the billet image and sends it to the secondary system. The secondary system controls the generation of label information based on the batch number and basic information of the steel section. When the billet is transported to the labeling location, the labeling machine generates a label corresponding to the label information and labels it; A billet batch number recognition model is constructed based on the improved Yolov3 network model. The billet image is input into the billet batch number recognition model, and the billet batch number recognition model outputs the batch number in the current billet image. Before inputting the billet image into the billet batch number recognition model, brightness correction is performed on the billet image to balance the image brightness of the billet image. The brightness correction process of the billet image is as follows: (1) Use the Gaussian function to extract the illumination component I(x,y) in the billet image; (2) Correcting the brightness value of the corresponding (x, y) pixel based on the illumination component I(x, y) to equalize the image brightness of the billet image; The correction formula for the image brightness of the billet image is as follows: Where F′(x,y) is the brightness value of the billet image at the (x,y) position after correction, F(x,y) is the brightness value of the billet image at the (x,y) position before correction, I is the mean value of the illumination component in the billet image, and γ(x,y) is the brightness balance index at the (x,y) position.

2. The automatic labeling system based on steel billet spray number recognition as claimed in claim 1, characterized in that: The improved Yolov3 network model structure is as follows: The first DBL module, Res×1 residual module, first Res×4 residual module, first Res×8 residual module, second Res×8 residual module, second Res×4 residual module, third Res×4 residual module, second DBL module and first Res×2 residual module are connected in sequence; The output end of the first Res×2 residual module is connected to the first convolutional layer Conv, and the output end of the first convolutional layer Conv is the first output end; The output end of the first Res×2 residual module is connected to the third DBL module and the first upsampling module in sequence. The output end of the second Res×4 residual module and the first upsampling module is connected to the seventh DBL module through the first tensor splicing module Concat. The seventh DBL module is connected to the second Res×2 residual module and the second convolutional layer Conv in sequence. The output end of the second convolutional layer Conv is the second output end. The output end of the second Res×2 residual module is connected to the fourth DBL module and the second upsampling module in sequence. The output ends of the second Res×8 residual module and the second upsampling module are connected to the eighth DBL module through the second tensor splicing module Concat. The eighth DBL module is connected to the third Res×2 residual module and the third convolutional layer Conv in sequence. The output end of the third convolutional layer Conv is the third output end. The output end of the third Res×2 residual module is connected to the fifth DBL module and the third upsampling module in sequence. The output ends of the first Res×8 residual module and the third upsampling module are connected to the ninth DBL module through the third tensor splicing module Concat. The ninth DBL module is connected to the fourth Res×2 residual module and the fourth convolutional layer Conv in sequence. The output end of the fourth convolutional layer Conv is the fourth output end. The output end of the fourth Res×2 residual module is connected to the sixth DBL module and the fourth upsampling module in sequence. The output ends of the first Res×4 residual module and the fourth upsampling module are connected to the tenth DBL module through the fourth tensor splicing module Concat. The tenth DBL module is connected to the fifth Res×2 residual module and the fifth convolutional layer Conv in sequence. The output end of the fifth convolutional layer Conv is the fifth output end. The output results from the first output terminal to the fifth output terminal are subjected to non-maximum suppression and then the billet batch number in the billet image is output.

3. The automatic labeling system based on steel billet spray number recognition as claimed in claim 2, characterized in that: The DBL module is a combination of the convolution layer Conv, the BN layer and the activation function layer Relu.

4. The automatic labeling system based on steel billet spray number recognition as claimed in claim 1, characterized in that: The inkjet printer prints the batch number every 5 meters.

5. The automatic labeling system based on steel billet spray number recognition as claimed in claim 1, characterized in that: The inkjet printer sprays the batch number of the steel billet on the flange of the H-shaped steel billet.

Images