Angle steel counting method based on scale adaptive convolutional neural network

Abstract

The present application relates to the technical field of steel bar counting, and particularly discloses a method for counting angle steel based on a scale adaptive convolutional neural network, which comprises the following steps: S1, preprocessing input steel bar image data; S2, respectively building three convolutional neural networks as a classifier, a linear regressor and a binary classifier; S3, filtering the binary classification result according to a distance clustering algorithm; and S4, counting the center coordinates of the filtered result and finally outputting the total number of angle steels. The steel bar counting method is composed of three neural networks and a clustering technology, and each network solves a specific task. The modular design not only allows portability, but also is replicable, thereby providing the ability to add new materials by using the same method. The steel bar type can be automatically classified, the center of each steel bar can be located, and an output containing the total number of steel bars can be provided. The method has robustness to natural changes in conditions in a warehouse.

Description

Technical Field

[0001] This invention relates to the field of rebar counting technology, and in particular to a method for counting angle steel based on a scale-adaptive convolutional neural network. Background Technology

[0002] Counting is a time-consuming task and a key factor in tracking any material inventory. This task becomes even more challenging when dealing with objects of varying shapes and sizes. In the steel industry, steel bars are among the most widely used construction and forging products in the world. During manufacturing, bars are typically counted using images, allowing for control over distance, lighting, and angle. However, once they leave the factory, these heavy and large materials must be stacked and stored in warehouses or retail stores, where an unfavorable environment prevails, making it difficult to track a stable and reliable inventory.

[0003] Traditional rebar counting is based on manual calculation, but due to limited mobility and low efficiency, manual counting is quite slow, labor-intensive, and inaccurate. Therefore, to improve the effectiveness and reliability of the rebar counting process, an automated system capable of calculating these materials regardless of physical conditions is needed.

[0004] Basically, the main approaches to the two computational tasks can be distinguished. Image processing techniques implement algorithms based on mathematical functions to transform images. Filters, thresholding, edge detection, and matching are commonly used techniques [1–4]. While these techniques are highly accurate, they are limited to specific conditions, such as constant lightning and background, or special camera requirements. Furthermore, they are limited to round steel bars of fixed shape and size, assuming their shape is quasi-circular, and therefore lack robustness. Note that some of these methods are limited to steel structure fabric production lines, where physical separation of materials is feasible.

[0005] Similarly, other image processing-based rebar counting algorithms are mainly based on area algorithms and template algorithms. Both methods are feasible, but they also have some drawbacks. The results of the first method cannot directly locate the rebars in the counting results, thus causing great inconvenience to the error analysis of the [8,9] algorithm.

[0006] Most existing technologies focus on calculating single object types with specific characteristics and constant backgrounds. While some machine learning-based works offer high performance, portable systems may be impractical due to the amount of computational resources required. To address these issues, this paper proposes a scale-adaptive convolutional neural network-based angle steel counting method, namely Scale-Adaptive Convolutional Neural Network Distance Clustering (SA-CNN-DC). Employing a compact design with minimal parameters, it is adaptable to various natural environments and counts steel bars based on their type and quantity. Summary of the Invention

[0007] The purpose of this invention is to address the shortcomings mentioned in the background art by proposing a scale-adaptive convolutional neural network-based method for counting angle steel.

[0008] The technical solution adopted in this invention is as follows:

[0009] A method for counting angle steel based on a scale-adaptive convolutional neural network is provided, including the following steps:

[0010] S1: Preprocess the input rebar image data;

[0011] S2: Construct three convolutional neural networks as a classifier, a linear regressor, and a binary classifier, respectively;

[0012] S3: Filter the binary classification results based on the distance clustering algorithm;

[0013] S4: Count the center coordinates of the filtered results and finally output the total number of angle steel.

[0014] As a preferred technical solution of the present invention: in S1, three density types, low, medium and high, are provided for preprocessing the input steel bar image.

[0015] As a preferred technical solution of the present invention: in S1, the size of the input steel bar image is adjusted, and according to the density type of the input steel bar image, five different sizes of crops are extracted from the center of the steel bar image and adjusted to be the same size as the input steel bar image.

[0016] As a preferred technical solution of the present invention: in S2, a classifier dataset, a regressor dataset, and a binary classifier dataset are constructed respectively.

[0017] As a preferred technical solution of the present invention: in S2, the classifier consists of a convolutional layer and a fully connected layer, and the input samples are classified and output as flat vectors by a softmax classifier. The l-th layer is set as a convolutional layer and the k-th layer is set as a fully connected layer.

[0018] The convolutional layer function is as follows:

[0019]

[0020] in, This represents the output value of the i-th neuron in the (l+1)-th layer, where f and M represent the activation function and the number of neurons, respectively. This represents the input value of the i-th neuron in the l-th layer. This represents the convolution operation. This represents the convolution kernel of the i-th neuron in layer l and the j-th neuron in layer l+1. This represents the bias value of the i-th neuron in the l-th layer;

[0021] The Softmax classifier function is as follows:

[0022]

[0023] in, This represents the output value of the J-th neuron in the (k+1)-th layer. This represents the weight of the J-th neuron in the k-th layer. This represents the output value of the J-th neuron in the k-th layer. This represents the bias value of the J-th neuron in the k-th layer.

[0024] As a preferred embodiment of the present invention: the linear regressor in S2 includes a hidden layer and an output layer, the input is a flat vector generated by the classifier, and the output layer outputs the value of fR; the calculation of the adjustable factor fR is as follows:

[0025]

[0026] Where k represents the size of the bar chart in which the steel bar image is labeled in pixels.

[0027] As a preferred embodiment of the present invention: the linear regressor performs data validation using a linear regression algorithm, as detailed below:

[0028]

[0029] Among them, R 2 To determine the coefficients, y true For the true value, y pred This is the network prediction value. This represents the average true value.

[0030] As a preferred embodiment of the present invention: in step S2, the binary classifier includes a hidden layer and an output layer containing a sigmoid activation function;

[0031] The sigmoid activation function σ(z) is as follows:

[0032]

[0033] Where e is a mathematical constant and z is the output value of the hidden layer; the binary classifier divides the candidate center set according to the classification results.

[0034] As a preferred embodiment of the present invention: In step S3, the Euclidean distance d between the coordinates of each candidate center is calculated based on a distance clustering algorithm.

[0035]

[0036] Where a and b are the coordinates of the candidate centers, and a, b = [(x1, y1), (x2, y2), ..., (x... N y N )] T And a≠b, where (x1, y1), (x2, y2), ... (x N y N () represents the candidate center set;

[0037] Set a distance threshold, perform the final division of candidate centers, and count based on the final division results.

[0038] As a preferred embodiment of the present invention, the counting accuracy is verified in step S4 based on the following performance indicators:

[0039]

[0040]

[0041]

[0042]

[0043] Where Precision represents accuracy, Recall represents recall, F1 represents score, Accuracy represents relative accuracy, TP represents correct identification (TP = true positive "1"), TN represents correct rejection (TN = true negative "0"), FP represents incorrect identification (TP = false positive "1"), FN represents incorrect rejection (TN = false negative "0"), FC represents final count, and GT represents true count.

[0044] The angle steel counting method based on scale-adaptive convolutional neural networks provided by this invention has the following advantages compared with existing technologies:

[0045] The present invention provides a method for counting rebars, the core of which consists of three neural networks and a clustering technique. Each network solves a specific task: bar classification, image resizing, and center localization. The modular design not only allows for portability but is also reproducible, thus providing the ability to add new materials using the same method. The method is adaptable to various natural environments, automatically classifies rebar types, locates the center of each rebar, and provides an output containing the total number of rebars. Attached Figure Description

[0046] Figure 1 This is a flowchart of a preferred embodiment of the present invention;

[0047] Figure 2 This is a block diagram of the SA-CNN-DC method in a preferred embodiment of the present invention;

[0048] Figure 3 This is a diagram of the classifier architecture in a preferred embodiment of the present invention;

[0049] Figure 4 This is a distribution diagram of the true and predicted values ​​of the circular material test set in a preferred embodiment of the present invention;

[0050] Figure 5 This is a distribution diagram of the true and predicted values ​​of the test set for the square material in a preferred embodiment of the present invention;

[0051] Figure 6 This is a diagram of the linear regressor architecture in a preferred embodiment of the present invention;

[0052] Figure 7 This is a diagram of the binary classifier architecture in a preferred embodiment of the present invention. Detailed Implementation

[0053] It should be noted that, unless otherwise specified, the embodiments and features described in this embodiment can be combined with each other. The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0054] Reference Figure 1 The preferred embodiment of the present invention provides a method for counting angle steel based on a scale-adaptive convolutional neural network, comprising the following steps:

[0055] S1: Preprocess the input rebar image data;

[0056] S2: Construct three convolutional neural networks as a classifier, a linear regressor, and a binary classifier, respectively;

[0057] S3: Filter the binary classification results based on the distance clustering algorithm;

[0058] S4: Count the center coordinates of the filtered results and finally output the total number of angle steel.

[0059] In S1, three density types—low, medium, and high—are provided for preprocessing the input rebar image.

[0060] In step S1, the size of the input rebar image is adjusted, and five different sizes of crops are extracted from the center of the rebar image according to the density type of the input rebar image, and these crops are adjusted to be the same size as the input rebar image.

[0061] In S2, classifier dataset, regressor dataset and binary classifier dataset are constructed respectively.

[0062] In S2, the classifier consists of convolutional layers and fully connected layers, and a softmax classifier is used to classify the input samples and output a flat vector. The l-th layer is set as a convolutional layer and the k-th layer is set as a fully connected layer.

[0063] The convolutional layer function is as follows:

[0064]

[0065] in, This represents the output value of the i-th neuron in the (l+1)-th layer, where f and M represent the activation function and the number of neurons, respectively. This represents the input value of the i-th neuron in the l-th layer. This represents the convolution operation. This represents the convolution kernel of the i-th neuron in layer l and the j-th neuron in layer l+1. This represents the bias value of the i-th neuron in the l-th layer;

[0066] The Softmax classifier function is as follows:

[0067]

[0068] in, This represents the output value of the J-th neuron in the (k+1)-th layer. This represents the weight of the J-th neuron in the k-th layer. This represents the output value of the J-th neuron in the k-th layer. This represents the bias value of the J-th neuron in the k-th layer.

[0069] The linear regressor in S2 includes a hidden layer and an output layer. The input is a flattened vector generated by the classifier, and the output layer outputs the value of fR. The calculation of the adjustable factor fR is as follows:

[0070]

[0071] Where k represents the size of the bar chart in which the steel bar image is labeled in pixels.

[0072] The linear regressor is validated using a linear regression algorithm, as detailed below:

[0073]

[0074] Among them, R 2 To determine the coefficients, y true For the true value, y pred This is the network prediction value. This represents the average true value.

[0075] In S2, the binary classifier includes a hidden layer and an output layer containing a sigmoid activation function;

[0076] The sigmoid activation function σ(z) is as follows:

[0077]

[0078] Where e is a mathematical constant and z is the output value of the hidden layer; the binary classifier divides the candidate center set according to the classification results.

[0079] In step S3, the Euclidean distance d between the coordinates of each candidate center is calculated based on a distance clustering algorithm:

[0080]

[0081] Where a and b are the coordinates of the candidate centers, and a, b = [(x1, y1), (x2, y2), ..., (x... N y N )] T And a≠b, where (x1, y1), (x2, y2), ... (x N y N () represents the candidate center set;

[0082] Set a distance threshold, perform the final division of candidate centers, and count based on the final division results.

[0083] In step S4, the counting accuracy is verified based on the following performance metrics:

[0084]

[0085]

[0086]

[0087]

[0088] Where Precision represents accuracy, Recall represents recall, F1 represents score, Accuracy represents relative accuracy, TP represents correct identification (TP = true positive "1"), TN represents correct rejection (TN = true negative "0"), FP represents incorrect identification (TP = false positive "1"), FN represents incorrect rejection (TN = false negative "0"), FC represents final count, and GT represents true count.

[0089] In this embodiment, refer to Figure 2 The core of the SA-CNN-DC method consists of three neural networks and a clustering technique. Each network solves a specific task: bar classification, image resizing, and center localization. The modular design not only allows for portability but is also reproducible, thus providing the ability to add new material using the same approach.

[0090] Since each neural network has a specific task in the proposed system, three corresponding datasets were constructed for the three neural networks: a classifier dataset, a regressor dataset, and a binary classifier dataset. In each network's dataset, 70% of the data was used for training, 20% for validation, and 10% for cross-validation of the results.

[0091] The input rebar image was resized to a size limit between 3500 and 1700 pixels. Based on its density type (low, medium, and high), five different sized crops were extracted from the center and resized to fit the CNN input. At high densities, the extracted crops were smaller, and vice versa for low densities.

[0092] The classifier dataset consists of photographs of four different sizes of rebar, collected manually from a typical warehouse. To ensure an appropriate representation of local conditions, images varying in lighting, framing, angle, and location were collected. Photographs were taken from each pile of materials in this manner. From these collected images, five crops of variable sizes (randomly increasing or decreasing) were extracted from different coordinates within the images, taking into account the density parameters mentioned above. Once extracted, they were resized to 64x64 pixels and converted to grayscale. This technique helps increase the amount of data and prevents distortion caused by resizing. The sizes of the five crops were selected based on density and image dimensions.

[0093] Reference Figure 3 The five crops are fed forward into a classifier to obtain the maximum output. The classifier consists of three convolutional layers and two fully connected layers, with 12 and 4 neurons respectively. It uses a softmax classifier to classify the input samples, specifically classifying the four types of steel bars.

[0094] Let the l-th (1≤l≤3) layer be a convolutional layer and the k-th (1≤k≤2) layer be a fully connected layer;

[0095] The convolutional layer function is as follows:

[0096]

[0097] in, This represents the output value of the i-th neuron in the (l+1)-th layer, where f and M represent the activation function and the number of neurons, respectively. This represents the input value of the i-th neuron in the l-th layer. This represents the convolution operation. This represents the convolution kernel of the i-th neuron in layer l and the j-th neuron in layer l+1. This represents the bias value of the i-th neuron in the l-th layer;

[0098] The Softmax classifier function is as follows:

[0099]

[0100] in, This represents the output value of the J-th neuron in the (k+1)-th layer. This represents the weight of the J-th neuron in the k-th layer. This represents the output value of the J-th neuron in the k-th layer. This represents the bias value of the J-th neuron in the k-th layer.

[0101] The confusion matrix was also used as a performance metric for the classification network. The network's class predictions for the test data were compared with the actual results.

[0102] Based on the five predicted categories obtained from the voting, the two categories with the most predictions, circle and square, are selected and generated after the convolutional layer. These categories contain all the information of the compressed image and are stored for the next step of optimal flatness.

[0103] The selection architecture of a linear regression network consists of a hidden layer with four neurons and an output layer with a single neuron, as shown in the reference. Figure 4 The input corresponds to a flattened vector generated by the convolutional encoder of the classifier, and the linear output is a variable factor f. R The value.

[0104]

[0105] Where k represents the size of the bar chart in which the steel bar image is labeled in pixels.

[0106] The loss is calculated using mean squared error (MAE), while mean absolute error (MAE) is the metric considered by the network.

[0107] Adjustable factor f R Used to adjust the size of the image. Adjustable factor f R The decision is made to resize the image so that each strip perfectly fits a fixed-size patch. The resulting image is then scaled for the next stage. The classifier's convolutional layers are used to train a simple multilayer perceptron, whose linear output serves as a regression variable for the resizing factor. This process, known as transfer learning, helps to significantly reduce the training data and time required for the regression network.

[0108] Validation and prediction of the test set are performed using a linear regression algorithm:

[0109]

[0110] Among them, R 2 To determine the coefficients, y true For the true value, y pred This is the network prediction value. This is the average true value;

[0111] The verification results for circles and squares are respectively referred to Figure 5 and Figure 6 .

[0112] Reference Figure 7 The binary classifier consists of a dense hidden layer with 6 neurons and an output layer with a single neuron and a sigmoid activation function.

[0113] The sigmoid activation function σ(z) is as follows:

[0114]

[0115] Where e is a mathematical constant and z is the output value of the hidden layer.

[0116] Thresholds A and B are set for two classes: circle and square. Output values ​​less than 0.5 correspond to one class, while values ​​equal to or greater than 0.5 correspond to the other class. If needed, the thresholds can be used as a hyperparameter, thus increasing the flexibility of material testing.

[0117] The binary classifier moves through a scale-adapted image using a fixed-size sliding window with a small stride. The binary output network classifies the image as soon as a result patch contains a bar. If a detection is positive, the center coordinates of the patch are stored as candidate centers.

[0118] The output of the binary classifier is grouped into candidate center coordinates using a distance clustering algorithm:

[0119]

[0120] Where a and b are the coordinates of the candidate centers, and a, b = [(x1, y1), (x2, y2), ..., (x... N y N )] T And a≠b, where (x1, y1), (x2, y2), ... (x N y N () represents the coordinates of the candidate center;

[0121] A distance threshold is set, and candidate centers are filtered based on their horizontal and vertical proximity to other candidate centers, i.e., their relationship to the distance threshold. If a candidate center is completely isolated, it is deleted. For each cluster, a center coordinate is stored, ideally the geometric center of the material. Groups containing closed elements are merged. The average point of each group of candidate centers is calculated by averaging the maximum and minimum coordinate values ​​of the group. Once clusters are found, the sum of the center points found for each cluster represents the total number of material centers.

[0122] The counting accuracy was verified based on the following performance metrics:

[0123]

[0124]

[0125]

[0126]

[0127] Where Precision represents accuracy, Recall represents recall, F1 represents score, Accuracy represents relative accuracy, TP represents correct identification (TP = true positive "1"), TN represents correct rejection (TN = true negative "0"), FP represents incorrect identification (TP = false positive "1"), FN represents incorrect rejection (TN = false negative "0"), FC represents final count, and GT represents true count.

[0128] The accuracy and precision of the counting can be measured by verifying the above performance indicators, and the counting process can be improved.

[0129] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the invention can be implemented in other specific forms without departing from its spirit or essential characteristics. Therefore, the embodiments should be considered in all respects as exemplary and non-limiting, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be included within the present invention. No reference numerals in the claims should be construed as limiting the scope of the claims.

[0130] Furthermore, it should be understood that although this specification describes embodiments, not every embodiment contains only one independent technical solution. This narrative style is merely for clarity. Those skilled in the art should consider the specification as a whole, and the technical solutions in each embodiment can also be appropriately combined to form other embodiments that can be understood by those skilled in the art.

Claims

1. A method for counting angle steel based on a scale-adaptive convolutional neural network, characterized in that: Includes the following steps: S1: Preprocess the input rebar image data; S2: Construct three convolutional neural networks as a classifier, a linear regressor, and a binary classifier, respectively; S3: Filter the binary classification results based on the distance clustering algorithm; S4: Count the center coordinates of the filtered results and finally output the total number of angle steels; In S2, the classifier consists of convolutional layers and fully connected layers, and uses a softmax classifier to classify the input samples and output a flat vector. The first... The layer is a convolutional layer, the first... The layer is a fully connected layer; The convolutional layer function is as follows: in, Indicates the first The first layer The output value of each neuron and These represent the activation function and the number of neurons, respectively. Indicates the first The first layer The input value of each neuron. This represents the convolution operation. Indicates the first The first layer The first neuron and the first The first layer The convolutional kernel of a neuron, Indicates the first The first layer The bias value of each neuron; The Softmax classifier function is as follows: in, Indicates the first The first layer The output value of each neuron Indicates the first The first layer The weights of each neuron, Indicates the first The first layer The output value of each neuron Indicates the first The first layer The bias value of each neuron; The linear regressor in S2 includes a hidden layer and an output layer. The input is a flat vector generated by the classifier, and the output layer outputs an adjustable factor. Value; Adjustable factor The specific calculation is as follows: in, The size of the bar chart, which represents the size of the rebar image as pixels; The linear regressor is validated using a linear regression algorithm, as detailed below: in, To determine the coefficients, For the true value, This is the network prediction value. This represents the average true value. In S2, the binary classifier includes a hidden layer and an output layer containing a sigmoid activation function; The sigmoid activation function Specifically as follows: in, It is a mathematical constant. The output value is the hidden layer value; the binary classifier divides the candidate center set according to the classification result.

2. The angle steel counting method based on scale-adaptive convolutional neural network according to claim 1, characterized in that: In S1, three density types—low, medium, and high—are provided for preprocessing the input rebar image.

3. The angle steel counting method based on scale-adaptive convolutional neural network according to claim 1, characterized in that: In step S1, the size of the input rebar image is adjusted, and five different sizes of crops are extracted from the center of the rebar image according to the density type of the input rebar image, and these crops are adjusted to be the same size as the input rebar image.

4. The angle steel counting method based on scale-adaptive convolutional neural network according to claim 1, characterized in that: In S2, classifier dataset, regressor dataset and binary classifier dataset are constructed respectively.

5. The angle steel counting method based on scale-adaptive convolutional neural network according to claim 1, characterized in that: In step S3, the Euclidean distance between the coordinates of each candidate center is calculated based on a distance clustering algorithm. : in, and These are the coordinates of the candidate center. ,in, For the candidate center set; Set a distance threshold, perform the final division of candidate centers, and count based on the final division results.

6. The angle steel counting method based on scale-adaptive convolutional neural network according to claim 1, characterized in that: In step S4, the counting accuracy is verified based on the following performance metrics: in, Indicates precision. Indicates the recall value. Representing fractions, Indicates relative accuracy. This indicates that the identification was correct. , This indicates that the rejection was accepted. , Indicates incorrect identification. , This indicates that the request was rejected incorrectly. , Indicates the final count, This represents the actual count value.

Images