A method for automatic positioning and identification of steel coil numbers based on industrial vision

Abstract

The present invention relates to an automatic steel coil number positioning and recognition method based on industrial vision, comprising the following steps: constructing a training sample set: obtaining the training sample set; steel coil number positioning and region extraction: utilizing a segmented text detection method DBnet algorithm to locate the steel coil number, calculating the minimum enclosing matrix of the steel coil number, rotating the enclosing matrix to a horizontal direction, and extracting text regions of different shapes in a natural scene; coordinate system conversion: horizontally changing the steel coil number, and performing polar coordinate conversion using a multi-scale fusion polar coordinate conversion algorithm; steel coil number text recognition: utilizing a CPTN deep model to extract the steel coil number based on the coordinate conversion result; the steel coil number can be extracted more accurately, thereby solving the problem of low steel coil number extraction accuracy, effectively reducing production costs, and solving the problem of low manual recording efficiency, and is of great significance for improving the efficiency and accuracy of steel coil inventory management.

Description

Technical Field

[0001] The present invention relates to the technical field of metallurgical production, and in particular to an automatic steel coil number positioning and identification method based on industrial vision. Background Art

[0002] The coil number is a unique identifier on a steel coil, typically consisting of a string of numbers and letters. It uniquely identifies a specific coil and can include information such as manufacturer, origin, production date, steel grade, and specifications. Coil number identification and management are crucial for the steel manufacturing and processing industries, enabling tracking, inventory management, and production scheduling. Coil numbers are typically affixed to the coil in the form of a label or sign, but can also be automatically identified and managed using coil number recognition technology.

[0003] The background of steel coil number recognition technology is that with the rapid development of the steel industry and the expansion of production scale, the traditional manual recording and management methods can no longer meet production needs. The steel coil number is the unique identifier of the steel coil. Its accurate identification and management are crucial to improving production efficiency and reducing costs. Therefore, the automatic steel coil number recognition technology came into being. The application of this technology can improve the production efficiency and management level of the steel industry, reduce production costs and manual error rates, and become an indispensable part of production management in the steel industry. Summary of the Invention

[0004] The present invention provides a method for automatic positioning and identification of steel coil numbers based on industrial vision, which can realize automatic tracking and management of steel coils, including real-time monitoring of the position, status and production progress of steel coils, which can not only improve production efficiency and production quality, but also reduce the human error rate and errors in the production process.

[0005] In order to achieve the above object, the present invention adopts the following technical solutions:

[0006] A method for automatically locating and identifying steel coil numbers based on industrial vision includes the following steps:

[0007] S1. Constructing a training sample set: Obtaining an industry dataset constructed from production site scenarios as a training sample set;

[0008] S2. Coil Number Location and Region Extraction: Use the segmented text detection method DBnet algorithm to locate the coil number, calculate the minimum enclosing matrix of the coil number, rotate the enclosing matrix to the horizontal direction, extract text regions of different shapes in natural scenes, and output their position and size information;

[0009] S3. Coordinate system conversion: Based on the coil number positioning and area extraction results, the coil number is horizontally changed, and polar coordinate conversion is performed using a multi-scale fusion polar coordinate conversion algorithm. The converted images are fused into a single image through interpolation.

[0010] S4. Steel coil number text recognition: The image after coordinate system conversion is input into the deep neural network model CTPN for feature extraction to obtain a feature map, and the feature vector is obtained on the feature map. The obtained feature vector is input into the long short-term memory network LSTM to obtain the text candidate area text proposal. The graph-based text line construction algorithm is used to obtain the text segments one by one and merge them into text lines to obtain the final steel coil number.

[0011] Furthermore, the training sample set includes location information and category information of the steel coil number.

[0012] Furthermore, the extraction of text regions of different shapes in natural scenes is to extract the steel coil number region using a minimum enclosing matrix.

[0013] Furthermore, the multi-scale fusion polar coordinate transformation in step S3 is performed using x with different step lengths. step and y step Perform polar coordinate transformation, and then fuse the transformed images into one image by interpolation, where Then the polar coordinate transformation formula is as follows:

[0014]

[0015] The value of the i-th row and j-th column of the image matrix O is calculated using the following formula:

[0016] O step (r step ,θ step )=f(x'+r step cosθ step ,y'+r step sinθ step ) (3)

[0017] where x step and y step are the step lengths in the x-axis and y-axis directions, r step and θ step x step and y step are the polar coordinates obtained after step size conversion, (x', y') are the center coordinates of the polar coordinate space transformation, θ is the angle between a point (x, y) on the xoy plane in the Cartesian coordinate system and the x-axis, and r is the distance from a point (x, y) on the xoy plane in the Cartesian coordinate system to the center of the polar coordinate space transformation;

[0018] Use interpolation algorithm to add k O step Interpolation is performed to obtain the final polar coordinate image O(r,θ).

[0019] Furthermore, the step 4) specifically includes the following steps:

[0020] S4.1. Use GoogleNet as the backbone network to extract features and obtain the features as feature maps with a size of W × H × C.

[0021] S4.2. Then, a sliding window of size k×k is made on this feature map. Each window can obtain a feature vector, which will be used to predict the offset distance between it and the n anchor boxes.

[0022] S4.3. Input the features obtained in the previous step into a bidirectional long short-term memory (LSTM) network to obtain an output of length W × 256. This is then connected to a 512-bit fully connected layer to prepare for output.

[0023] S4.4. Obtain the horizontal translation of the densely predicted text proposal.

[0024] S4.5. Use the graph-based text line construction algorithm to merge the obtained text segments into text lines.

[0025] Compared with the prior art, the present invention has the following beneficial effects:

[0026] 1) A multi-scale fusion polar coordinate transformation algorithm is introduced to calculate the minimum enclosing matrix of the steel coil number. Then, the multi-scale fusion polar coordinate transformation algorithm is used to transform the arc-shaped steel coil number within the minimum enclosing matrix into a horizontal direction, allowing the steel coil number on the curved steel coil to be accurately detected. This method improves the quality of the steel coil number image after coordinate transformation, effectively improving the accuracy of subsequent steel coil number extraction and reducing the error rate, bringing the accuracy rate to over 97%;

[0027] 2) Using the DBnet algorithm to locate the coil number, optimizing the training process to simplify post-processing, and better detecting text regions of different shapes in natural scenes;

[0028] 3) It improves production efficiency and quality, reduces manual errors and errors in the production process, and is also of great significance for improving the efficiency and accuracy of inventory management;

[0029] 4) The steel coil identification technology of the present invention can also realize the automated management of steel coil inventory, including the rapid query and statistics of inventory quantity, type, status and other information, which is also of great significance for improving the efficiency and accuracy of inventory management. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 It is a flow chart of the method of the present invention.

[0031] Figure 2 This is the CPTN network structure diagram of the present invention.

[0032] Figure 3 It is a schematic diagram of the steel coil number positioning result of the present invention.

[0033] Figure 4 It is a schematic diagram of the steel coil number area extraction result of the present invention.

[0034] Figure 5 It is a schematic diagram of the steel coil number coordinate conversion result of the present invention.

[0035] Figure 6 It is a schematic diagram of the steel coil number extraction result of the present invention. DETAILED DESCRIPTION

[0036] The specific embodiments of the present invention will be further described below with reference to the accompanying drawings:

[0037] See Figure 1 , is a flow chart of the method of the present invention. The present invention provides a method for automatically locating and identifying steel coil numbers based on industrial vision, specifically comprising the following steps:

[0038] S1. Construct a training sample set: Set up a network camera at the steel coil management site for image acquisition. The site is well-lit, the camera has a resolution of 400W, and the camera is 5 meters away from the steel coil. Collect steel coil image data to obtain a data set containing 2,500 samples. Use the data set to complete the training of the steel coil number positioning model. The data set is an industry data set built based on the production site scene. The samples in each data set contain the location information and category information of the ladle number.

[0039] S2. Steel coil number positioning and region extraction: The segmented text detection method DBnet algorithm is used to locate the steel coil number. The network structure diagram is shown in Figure 2. The feature map pyramid network FPN is used to fuse multi-scale features, combined with the multi-resolution scale prediction of SSD and the multi-resolution feature fusion of U-Net. Through the differentiable binarization operation, the training process is optimized to simplify the post-processing, which can better extract text regions of different shapes in natural scenes. Figure 3 , output its position and size information;

[0040] Calculate the minimum enclosing matrix of the steel coil number and rotate the enclosing matrix to the horizontal direction, such as Figure 4 As shown in the text detection method based on segmentation, the input image is first processed into a pixel-level text segmentation probability map through the network, and then the probability map is converted into a binary map according to a preset threshold. Finally, the final text detection result is obtained by aggregation means, using a fixed threshold, as shown in formula (1):

[0041]

[0042] Among them, B i，j is a binary image, P i,j is the pixel value of the probability map, and t is the set threshold;

[0043] This standard binarization operation will lead to non-differentiable gradients during training. Since the function is not differentiable, it cannot participate in network optimization. The segmentation-based text detection method DBnet uses differentiable binarization to binarize the image and complete the segmentation of the text area. Differentiable binarization uses a function that approximates the standard binarization to enable the binarization operation to participate in network training, as shown in formula (4).

[0044]

[0045] in, is a differential binary graph, T i,j is the threshold image pixel value, k is a fixed factor that can expand the influence of the gradient and help correct the network model error. Here it is 40;

[0046] S3. Coordinate system conversion: After DBnet network detection using the segmented text detection method, the text position on the image has been displayed in the form of a four-point annotation box. As the production process of the steel coil number is known, the steel coil code on the steel coil is preferably in an arc state, and the detection error rate of the curved text is relatively high. Based on the segmentation positioning, the steel coil number is horizontally changed to obtain the horizontal steel coil number, such as Figure 5 As shown, the image is converted from the Cartesian coordinate system to the polar coordinate system using the standard conversion method. At a point (x, y) on the Cartesian coordinate plane xoy, with (x', y') as the center, the Cartesian coordinates are converted to polar coordinates using the following formula:

[0047]

[0048]

[0049] Where θ is the angle between a point (x, y) on the Cartesian coordinate system's xoy plane and the x-axis, and r is the distance from a point (x, y) on the Cartesian coordinate system's xoy plane to the center of the polar coordinate space transformation. Assuming the input image matrix is I, (x', y') represents the center of the polar coordinate space transformation, and the output image matrix is O, the one-to-one correspondence between polar coordinates and Cartesian coordinates is used to obtain each pixel value of O, that is:

[0050] O(r, θ)=f(x'+rcosθ,y'+rsinθ) (8)

[0051] When both θ and r are discretized with a step size of 1, due to the large transformation step size, the output image matrix O will lose a lot of information of the original image, resulting in blurred image after polar coordinate transformation, which is not conducive to the subsequent extraction of steel coil number. The present invention adopts a multi-scale fusion polar coordinate transformation algorithm, that is, using different step sizes of x step and y step Perform polar coordinate transformation, and then fuse the transformed images into one image by interpolation, where Then the polar coordinate transformation formula is as follows:

[0052]

[0053] The value of the i-th row and j-th column of the image matrix O is calculated using the following formula:

[0054] O step (r step ,θ step )=f(x'+r step cosθ step ,y'+r step sinθ step ) (11)

[0055] where x step and y step are the step lengths in the x-axis and y-axis directions, r step and θ step x step and y step is the polar coordinate obtained after step size conversion, (x', y') is the center coordinate of polar coordinate space transformation, θ is the angle between a point (x, y) on the xoy plane of the Cartesian coordinate system and the x-axis, and r is the distance from a point (x, y) on the xoy plane in the Cartesian coordinate system to the center of polar coordinate space transformation;

[0056] Use interpolation algorithm to add k O step Interpolation is performed to obtain the final polar coordinate image O(r,θ).

[0057] S4. Steel coil number text recognition: After DBNet network detection and multi-scale fusion polar coordinate conversion, the horizontal steel coil number is obtained. Next, the text content in these boxes is recognized. The deep neural network with CTPN as the benchmark framework is used. The network model consists of three parts: convolutional layer, bidirectional LSTM, and fully connected layer. The convolutional layer in the network structure uses VGG16. Due to the limited depth and width of the VGG16 network, the detection accuracy of CTPN small text is not high. The feature extraction network GoogleNet is used to replace the original convolutional network VGG16 of CTPN. It not only deepens the structure of the neural network, but also increases the width of the network. The Inception-v1 structure is introduced to replace the traditional convolutional layer plus activation function structure, expands the width of the network, and uses parallelization modules to reduce the amount of calculation, thereby reducing the size of the feature map, improving the problem of low accuracy of CPTN in small text recognition. The improved CPTN network structure is as follows Figure 2 As shown, the identification steps are as follows:

[0058] S4.1. First, use the feature extraction network GoogleNet as the backbone network to extract features and obtain the features as feature maps with a size of W×H×C.

[0059] S4.2. Then, a sliding window of 3×3 size is applied to this feature map. This means that each window will generate a feature vector of length 3×3×C. This feature vector will be used to predict the offset distances between the five anchor boxes. This means that five text candidate regions will be predicted at the center of each window.

[0060] S4.3. Input the features obtained in the previous step into a bidirectional long short-term memory (LSTM) network to obtain an output of length W × 256. This is then connected to a 512-bit fully connected layer to prepare for output.

[0061] S4.4. This method obtains densely predicted text proposals and then uses a standard non-maximum suppression algorithm to filter out redundant detection boxes.

[0062] S4.5. Finally, we use the graph-based text line construction algorithm to merge the obtained text segments into text lines. Figure 6 .

[0063] The above embodiments are implemented under the premise of the technical solution of the present invention, and detailed implementation methods and specific operation processes are given, but the protection scope of the present invention is not limited to the above embodiments. The methods used in the above embodiments are conventional methods unless otherwise specified.

Claims

1. A method for automatic positioning and identification of steel coil numbers based on industrial vision, characterized in that: The steps include: S1. Constructing a training sample set: Obtaining an industry dataset constructed from production site scenarios as a training sample set; S2. Coil Number Location and Region Extraction: Use the segmented text detection method DBnet algorithm to locate the coil number, calculate the minimum enclosing matrix of the coil number, rotate the enclosing matrix to the horizontal direction, extract text regions of different shapes in natural scenes, and output their position and size information; S3. Coordinate system conversion: Based on the coil number positioning and area extraction results, the coil number is horizontally changed, and polar coordinate conversion is performed using a multi-scale fusion polar coordinate conversion algorithm. The converted images are fused into a single image through interpolation. Multi-scale fusion polar coordinate transformation uses different step lengths of x step and y step Perform polar coordinate transformation, and then fuse the transformed images into one image by interpolation, where Then the polar coordinate transformation formula is as follows: The value of the i-th row and j-th column of the image matrix O is calculated using the following formula: The step (r step ,the step )=f(x'+r step cosθ step ,y'+r step sinth step ) (3) where x step and y step are the step lengths in the x-axis and y-axis directions, r step and θ step x step and y step is the polar coordinate obtained after step size conversion, (x', y') is the center coordinate of polar coordinate space transformation, θ is the angle between a point (x, y) on the xoy plane of the Cartesian coordinate system and the x-axis, and r is the distance from a point (x, y) on the xoy plane in the Cartesian coordinate system to the center of polar coordinate space transformation; Use interpolation algorithm to add k O step Interpolation is performed to obtain the final polar coordinate image O(r,θ); S4. Steel coil number text recognition: The image after coordinate system conversion is input into the deep neural network model CTPN for feature extraction to obtain a feature map, and the feature vector is obtained on the feature map. The obtained feature vector is input into the long short-term memory network LSTM to obtain the text candidate area text proposal, and the graph-based text line construction algorithm is used to obtain the text segments one by one and merge them into text lines to obtain the final steel coil number.

2. The method for automatic steel coil number positioning and identification based on industrial vision according to claim 1 is characterized in that: The training sample set includes location information and category information of the steel coil number.

3. The method for automatic steel coil number positioning and identification based on industrial vision according to claim 1 is characterized in that: The extraction of text regions of different shapes in natural scenes is to extract the steel coil number region using a minimum enclosing matrix.

4. The method for automatic steel coil number positioning and identification based on industrial vision according to claim 1 is characterized in that: The step S4 specifically includes the following steps: S4.

1. Use GoogleNet as the backbone network to extract features and obtain the features as feature maps with a size of W × H × C. S4.

2. Then, a sliding window of size k×k is made on this feature map. Each window can obtain a feature vector, which will be used to predict the offset distance between it and the n anchor boxes. S4.

3. Input the feature vector obtained in the previous step into a bidirectional long short-term memory (LSTM) network to obtain an output of length W × 256. This is then connected to a 512-bit fully connected layer for output. S4.

4. Obtain the horizontal translation of the densely predicted text proposal. S4.

5. Use the graph-based text line construction algorithm to merge the obtained text segments into text lines.

Images