Steel coil head detection and identification method

Abstract

The present invention proposes a steel coil head detection and recognition method, which combines digital image processing technology with the Faster RCNN target detection algorithm, obtains the gradient information of the steel coil head in the picture through the digital image processing technology, and adds the gradient information as prior knowledge to the original picture to strengthen the pixel value information of the steel coil head in the picture; the picture with prior knowledge is input into the Faster RCNN target detection algorithm, and the picture with prior knowledge can strengthen the feature extraction of the target detection algorithm on the steel coil head, and improve the accuracy of the algorithm in detecting and identifying the steel coil head. The method is stable and reliable, and can improve the detection accuracy to complete the predetermined work task.

Description

Technical Field

[0001] The invention belongs to the technical field of steel coil head detection and identification, and in particular relates to a steel coil head detection and identification method. Background Art

[0002] Steel coils are essential raw materials for the construction industry and important products for steel companies. During the rolling process of steel coils, the head of a bundle of coils is rolled and fed into the clamping roll. At this time, the cooling water tank is turned on, and the first few turns have not been treated with water, resulting in the growth of ferrite grains inside the steel coils, uneven internal organizational composition, and unstable mechanical properties such as tensile strength. When used, various indicators do not meet the required standards. Therefore, in the finishing process, the first few turns need to be cut off according to the company's regulations, and how to identify the head of the steel coil becomes the key in this process. At present, the mainstream steel coil head removal is still manually operated, and the specific operation process is as follows: bundles of steel coils are transported on C-hooks, and workers first observe and find the position of the steel coil heads. If no observation is found, the bundles of steel coils need to be sorted. After finding the steel coil heads, the steel coils are cut off according to the diameter specifications of the coils according to factory standards.

[0003] There are mainly the following problems when workers cut steel coils: First, this work in the factory is done entirely by workers manually, which is inefficient. As labor costs increase, it affects the production efficiency of the enterprise; second, this work is done manually by workers, which is the shortcoming of the automated production of the production line, and it is impossible to achieve full automation of the steel coil production process; finally, the factory environment is harsh, with high heat and high risk, and boring and repetitive, which is not suitable for workers to work continuously for a long time. Therefore, manual operation has affected the production efficiency and production benefits of the enterprise, and it needs to be solved urgently. Summary of the invention

[0004] In order to make up for the gaps and deficiencies in the prior art, the present invention proposes a steel coil head detection and recognition method. Starting from the actual engineering needs, the characteristic features of the steel coil head are analyzed, and the gradient information of the steel coil head is added as prior knowledge to the deep learning target detection algorithm to improve the precision and accuracy of the target detection algorithm in identifying the steel coil head. By identifying the position of the steel coil head in the image, its spatial coordinates can be located, and finally the grasping action of the steel coil head can be completed by guiding the manipulator.

[0005] In order to improve the automation level of the steel industry and realize the automation and intelligence of steel coil production, the present invention proposes a steel coil head detection and recognition method. This method combines digital image processing technology with the Faster RCNN target detection algorithm. The gradient information of the steel coil head in the picture is obtained by digital image processing technology, and the gradient information is added to the original picture as prior knowledge to strengthen the pixel value information of the steel coil head in the picture; the picture with prior knowledge is input into the Faster RCNN target detection algorithm. The picture with prior knowledge can strengthen the feature extraction of the target detection algorithm at the steel coil head and improve the accuracy of the algorithm in detecting and identifying the steel coil head. This method is stable and reliable and can improve the detection accuracy to complete the predetermined work tasks.

[0006] The present invention specifically adopts the following technical solutions:

[0007] A method for detecting and identifying a steel coil head is characterized in that it comprises the following steps: Step S1: Using the Sobel operator to extract the gradient information of the steel coil head from a steel coil image sample data set with a steel coil head, and adding the extracted steel coil head gradient information to the original image as prior knowledge; Step S2: Inputting the steel coil image containing prior knowledge into a deep learning target detection algorithm, and identifying the position of the steel coil head in the image according to the deep learning target detection algorithm.

[0008] Furthermore, the gradient information of the steel coil head is extracted using the Sobel operator, and the tilted coil gradient is filtered out by subtracting the y-direction gradient from the x-direction gradient, thereby obtaining a single steel coil head gradient and preliminarily locating the position of the steel coil head.

[0009] Furthermore, step S1 specifically includes the following steps:

[0010] Step S11: Use the Sobel operator to convolve the input image to extract the gradient information in the y direction and the x direction, and obtain two images containing the gradient information in the y direction and the x direction of the steel coil respectively;

[0011] Step S12: performing Otsu threshold processing on the extracted gradient information in the y direction and the x direction respectively, processing the two gradient information images into black and white images, subtracting the gradient image containing the y direction from the gradient image containing the x direction pixel by pixel, so as to remove the gradient information of the tilted steel coil in the y direction, and obtain an image containing the gradient information of the steel coil head and some noise;

[0012] Step S13: using a 3×3 kernel function to perform morphological opening processing on the obtained image to remove noise information in the image containing gradient information without losing the gradient information of the steel coil head; during the morphological opening processing, the image is first corroded and then dilated, the corrosion processing processes the target foreground into the background according to the size of the kernel function, and the dilation processing processes the background at the edge of the target foreground into the foreground according to the size of the kernel function;

[0013] Step S14: perform pixel-by-pixel addition processing on the obtained image containing the steel coil head information and the original image, and add the steel coil head gradient information into the original image as prior knowledge.

[0014] Furthermore, the deep learning target detection algorithm includes: a backbone feature extraction module, a region proposal network, a region of interest pooling module, and a classification and regression module;

[0015] The backbone feature extraction module is used to extract the features of the input steel coil image according to the selected convolutional neural network, and output a feature map; ResNet50 is used as the feature extraction backbone; the input steel coil image is passed through the feature extraction module with ResNet50 as the backbone to obtain a feature map with 512 channels;

[0016] The region proposal network is used to generate anchor frames of different sizes according to the set parameters, and map the generated anchor frames to the steel coil feature map extracted by the backbone feature extraction network, preliminarily predict whether each anchor frame is the head of the steel coil and the offset of each anchor frame relative to the ground truth, and divide the positive and negative samples at the same time; during training, the Proposal formed by the predicted anchor frame is screened to obtain the RoI;

[0017] The region of interest pooling module is used to pool anchor frames of different sizes mapped on the feature map into uniform size and fixed dimension;

[0018] The classification and regression module is used to predict and regress the position of the steel coil head, and calculate the loss function value of the following formula, and optimize the model by minimizing the loss function value:

[0019]

[0020] Where: i is the index of anchor; p i is the probability that the anchor is predicted as a target; is the category label, which is 1 when there is a target, otherwise it is 0; t i is the predicted value of position regression; is the label value of position regression; λ is the weight balance value; N cls and N regare the normalized values ​​of classification loss and regression loss respectively; L cls is the classification logarithmic loss; L reg is the position regression loss.

[0021] An electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steel coil head detection and identification method as described above when executing the program.

[0022] A computer-readable storage medium having a computer program stored thereon, characterized in that when the program is executed by a processor, the steel coil head detection and identification method as described above is implemented.

[0023] Compared with the prior art, the present invention and its preferred embodiment first obtain the gradient information of the steel coil head through digital image processing technology, and perform morphological opening operation to remove noise in the gradient image; secondly, enhance the pixel value of the steel coil head through the pixel-by-pixel addition method, thereby enhancing the features of the steel coil head; finally, input the image with enhanced features into the Faster RCNN target detection algorithm, identify the position of the steel coil head in the image through the algorithm model, and improve the detection accuracy of the algorithm. The advantage of this method is that the features of the steel coil head in the image are enhanced through digital image processing technology, thereby improving the accuracy of the deep learning target detection algorithm in detecting and identifying the steel coil head; the image gradient acquisition, adaptive threshold processing, and morphological operation in digital image processing are used in the image preprocessing of the target detection algorithm, and the detection and recognition accuracy of Faster RCNN is improved by enhancing the features of the target in the image. Experiments have shown that this method improves the detection accuracy, and this method can more accurately identify the steel coil head in the image, thereby locating the spatial position of the steel coil head.

[0024] So that the position information of the steel coil head can be transmitted to the robotic arm, assisting the robotic arm to better complete the grabbing of the steel coil head and the shearing of several circles of the steel coil head, helping enterprises to improve production efficiency, reduce production costs, reduce the labor intensity of workers, realize the full process automation of the steel coil production line, and improve the automation level of the steel industry. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 This is a flow chart of steel coil head detection and identification according to an embodiment of the present invention.

[0026] Figure 2 It is a schematic diagram of the flow chart of the steel coil head detection and identification method according to an embodiment of the present invention.

[0027] Figure 3 Schematic diagram of the Sobel operator according to an embodiment of the present invention.

[0028] Figure 4Schematic diagram for explaining the calculation of partial derivatives in the horizontal direction according to an embodiment of the present invention.

[0029] Figure 5 Schematic diagram of the ResNet50 structure of an embodiment of the present invention: Conv Block (left) and Identity Block (right). DETAILED DESCRIPTION

[0030] In order to make the features and advantages of this patent more obvious and easy to understand, the following embodiments are specifically described in detail as follows:

[0031] It should be noted that the following detailed descriptions are illustrative and are intended to provide further explanation of the present application. Unless otherwise specified, all technical and scientific terms used in this specification have the same meanings as those commonly understood by those skilled in the art to which the present application belongs.

[0032] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present application. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprise" and / or "include" are used in this specification, it indicates the presence of features, steps, operations, devices, components and / or combinations thereof.

[0033] The flowchart of the steel coil head detection and identification method in this embodiment is as follows: Figure 1 As shown, the two major components are Figure 2 As shown in the figure, the first part is to use the sobel operator to extract the gradient information of the steel coil head from the input image, and add the extracted gradient information of the steel coil head as prior knowledge to the original image; the second part is to input the steel coil image containing prior knowledge into the deep learning target detection algorithm, and identify the position of the steel coil head in the image according to the deep learning target detection algorithm.

[0034] The purpose of the first part of the operation is to obtain the gradient information of the steel coil head in the image by performing digital image processing on the image. Analysis of the steel coil image shows that there is a sudden change in the pixels of the steel coil head in the vertical direction of the image, that is, there is gradient information in the vertical direction of the steel coil head. At the same time, some tilted coils also have a sudden change in pixels in the vertical direction, so there is also gradient information in the vertical direction. The gradient of the tilted coil is filtered out by subtracting the gradient in the y direction from the gradient in the x direction, thereby obtaining a single gradient of the steel coil head and preliminarily locating the position of the steel coil head. The specific operation process is as follows:

[0035] 1. First of all, in mathematical terms, gradients need to calculate derivatives, and in actual calculations, image gradients are generally calculated by calculating the difference in pixel values ​​to obtain the approximate derivative value of the gradient. Therefore, the calculation of image gradients is generally achieved by convolution of operators and images. The input image is convolved with the Sobel operator to extract the gradient information in the y and x directions. At this time, two images containing the gradient information in the y and x directions of the steel coil are obtained. The Sobel operator is divided into a horizontal operator and a vertical operator, such as Figure 3 As shown, they are all a set of 3×3 matrices.

[0036] The principle of using the Sobel operator to calculate image gradient is as follows Figure 4 As shown, assuming that the image has 9 pixels, if we want to calculate the horizontal partial derivative A5 of pixel A5 x , we need to use the Sobel operator and the pixels in its field, and the calculation formula (1) is as follows:

[0037] A5 x =(A3-A1)+2×(A6-A4)+(A9-A7) (1)

[0038] That is, the pixel value on the right side of pixel A5 is subtracted from the pixel value on its left side. Since the middle pixels (A4 and A6) are closer to pixel A5, the weight of their pixel value interpolation is 2.

[0039] Similarly, the vertical partial derivative A5 of pixel A5 can be obtained y .

[0040] 2. Secondly, perform Otsu threshold processing on the extracted y-direction and x-direction gradient information respectively to process the two gradient information images into black and white images, and subtract the gradient image containing the y-direction from the gradient image containing the x-direction pixel by pixel. The purpose of this operation is to remove the gradient information of the tilted steel coil in the y-direction, and obtain an image containing the gradient information of the steel coil head and some noise. Otsu adaptive threshold processing is also known as the Otsu method. It finds the global threshold of the image by iteratively calculating the normalized histogram of the image. Its formula (2) (3) is as follows:

[0041] u 2 =w 0 ×u 0 +w 1 ×u 1 (2)

[0042] g=w 0 ×(u 0 -u 2 ) 2 +w 1 ×(u 1 -u 2 ) 2 (3)

[0043] Among them, the values ​​of the traversal histogram are i, w 0 with u 0 are the proportion of pixels with gray levels of 0 to i in the entire image and the average gray level of foreground pixels; w 1 with u 1 They are respectively the proportion of pixels with gray levels of i to 255 in the entire image and the average gray level of background pixels; at the end of the traversal, the maximum g value is taken as the global threshold of the image.

[0044] 3. Again, the obtained image is processed by morphological opening using a 3×3 kernel function, the purpose of which is to remove noise information in the image containing gradient information without losing the gradient information of the steel coil head. During the morphological opening operation, the image is first corroded and then dilated. The corrosion process converts the target foreground into the background according to the size of the kernel function, and the dilation process converts the background at the edge of the target foreground into the foreground according to the size of the kernel function.

[0045] 4. Finally, the obtained image containing the steel coil head information is added to the original image pixel by pixel, and the gradient information of the steel coil head is added to the original image as prior knowledge.

[0046] The second part is the Faster RCNN target detection algorithm in deep learning, which is mainly composed of four modules: backbone feature extraction module (Feature extraction), region proposal network (Region preposalnetwork), region of interest pooling module (RoI pooling), classification and regression module (Classification andregression).

[0047] 1. Backbone feature extraction module. The function of the backbone feature extraction module is to extract the features of the input steel coil image according to the selected convolutional neural network and output a feature map. Since there are few pixels in the head area of ​​the steel coil, in order to obtain a better feature map, ResNet50 is used as the backbone feature extraction module. ResNet50 is mainly composed of 4 Conv Blocks and 12 Identity Blocks. Conv Block and Identity Block are as follows: Figure 5 As shown in the figure, the input steel coil image is passed through a feature extraction module with ResNet50 as the backbone to obtain a feature map with 512 channels.

[0048] 2. Region Proposal Network. The role of the region proposal network is to generate anchor boxes of different sizes according to the set parameters, and map the generated anchor boxes to the steel coil feature map extracted by the backbone feature extraction network, preliminarily predict whether each anchor box is the head of the steel coil and the offset of each anchor box relative to the ground truth, and divide the positive and negative samples at the same time. During training, the Proposal formed by the predicted anchor box is screened to obtain the RoI.

[0049] 3. RoI Pooling module. Since the fully connected layer is used in the subsequent classification and regression modules, the fully connected layer requires the input to have a uniform fixed dimension, and the role of the region of interest pooling module is to pool the anchor boxes of different sizes mapped on the feature map into a uniform size and fixed dimension.

[0050] 4. Classification and regression module: The classification and regression module predicts and regresses the position of the steel coil head, and calculates the loss function value as shown in the following formula (4), and optimizes the model by minimizing the loss function value.

[0051]

[0052] Where: i is the index of anchor; p i is the probability that the anchor is predicted as a target; is the category label, which is 1 when there is a target, otherwise it is 0; t i is the predicted value of position regression; is the label value of position regression; λ is the weight balance value; N cls and N reg are the normalized values ​​of classification loss and regression loss respectively; L cls is the classification logarithmic loss; L reg is the position regression loss.

[0053] Through the above analysis, it can be seen that the second part of the target detection algorithm recognizes the steel coil head mainly based on the pixel information of the picture. The purpose of the first part of the processing is to obtain a picture containing prior knowledge of the steel coil head. The gradient information of the steel coil head is obtained by processing the picture with the Sobel operator and adding it to the original picture pixel by pixel, and then input it into the second part of the target detection algorithm. The purpose is to strengthen the feature extraction of the target detection feature extraction module at the steel coil head position by enhancing the pixel value of the steel coil head area, strengthen the features of the steel coil head in the picture, and improve the precision and accuracy of the target detection algorithm in detecting and identifying the steel coil head.

[0054] The above-mentioned program design scheme involving the algorithm provided in this embodiment can be stored in a computer-readable storage medium in a coded form, and implemented in the form of a computer program, and the basic parameter information required for the calculation is input through computer hardware, and the calculation result is output.

[0055] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, devices, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0056] The present invention is described with reference to flowcharts of methods, devices (apparatus), and computer program products according to embodiments of the present invention. It should be understood that each process in the flowchart, as well as the combination of processes in the flowchart, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the process in the flowchart. Figure 1 A device that specifies functions in a process or multiple processes.

[0057] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple functions specified in a flowchart.

[0058] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 The steps of a specified function in a process or multiple processes.

[0059] The above is only a preferred embodiment of the present invention, and does not limit the present invention in other forms. Any technician familiar with the profession may use the above disclosed technical content to change or modify it into an equivalent embodiment with equivalent changes. However, any simple modification, equivalent change and modification made to the above embodiment according to the technical essence of the present invention without departing from the technical solution of the present invention still belongs to the protection scope of the technical solution of the present invention.

[0060] This patent is not limited to the above-mentioned optimal implementation method. Anyone can derive other forms of steel coil head detection and identification methods under the inspiration of this patent. All equal changes and modifications made according to the scope of the patent application of the present invention should be covered by this patent.

Claims

1. A method for detecting and identifying a steel coil head, characterized in that: The method comprises the following steps: Step S1: using the Sobel operator to extract the gradient information of the steel coil head from a steel coil image sample data set with the steel coil head, and adding the extracted gradient information of the steel coil head into the original image as prior knowledge; Step S2: inputting the steel coil image containing prior knowledge into the deep learning target detection algorithm, and identifying the position of the steel coil head in the image according to the deep learning target detection algorithm; the gradient information of the steel coil head is extracted by using the sobel operator, and the inclined coil gradient is filtered out by subtracting the y-direction gradient from the x-direction gradient, so as to obtain a single steel coil head gradient, and preliminarily locate the steel coil head position; Step S1 specifically includes the following steps: Step S11: Use the Sobel operator to convolve the input image to extract the gradient information in the y direction and the x direction, and obtain two images containing the gradient information in the y direction and the x direction of the steel coil respectively; Step S12: performing Otsu threshold processing on the extracted gradient information in the y direction and the x direction respectively, processing the two gradient information images into black and white images, subtracting the gradient image containing the y direction from the gradient image containing the x direction pixel by pixel, so as to remove the gradient information of the tilted steel coil in the y direction, and obtain an image containing the gradient information of the steel coil head and some noise; Step S13: using a 3×3 kernel function to perform morphological opening processing on the obtained image to remove noise information in the image containing gradient information without losing the gradient information of the steel coil head; during the morphological opening processing, the image is first corroded and then dilated, the corrosion processing processes the target foreground into the background according to the size of the kernel function, and the dilation processing processes the background at the edge of the target foreground into the foreground according to the size of the kernel function; Step S14: performing pixel-by-pixel addition processing on the obtained image containing the steel coil head information and the original image, and adding the steel coil head gradient information into the original image as prior knowledge; The deep learning target detection algorithm includes: a backbone feature extraction module, a region proposal network, a region of interest pooling module and a classification and regression module; The backbone feature extraction module is used to extract the features of the input steel coil image according to the selected convolutional neural network, and output a feature map; ResNet50 is used as the feature extraction backbone; the input steel coil image is passed through the feature extraction module with ResNet50 as the backbone to obtain a feature map with 512 channels; The region proposal network is used to generate anchor frames of different sizes according to the set parameters, and map the generated anchor frames to the steel coil feature map extracted by the backbone feature extraction network, preliminarily predict whether each anchor frame is the head of the steel coil and the offset of each anchor frame relative to the ground truth, and divide the positive and negative samples at the same time; during training, the Proposal formed by the predicted anchor frame is screened to obtain the RoI; The region of interest pooling module is used to pool anchor frames of different sizes mapped on the feature map into a uniform size and fixed dimension; The classification and regression module is used to predict and regress the position of the steel coil head, and calculate the loss function value of the following formula, and optimize the model by minimizing the loss function value: Where: i is the index of anchor; p i is the probability that the anchor is predicted as a target; is the category label, which is 1 when there is a target, otherwise it is 0; t i is the predicted value of position regression; is the label value of position regression; λ is the weight balance value; N cls and N reg are the normalized values ​​of classification loss and regression loss respectively; L cls is the classification logarithmic loss; L reg is the position regression loss.

2. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the steel coil head detection and identification method as described in claim 1 is implemented.

3. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steel coil head detection and identification method as described in claim 1 is implemented.

Images