A method for recognizing, counting and centering the end faces of a bundle of reinforcing steel bars and a memory therefor

Abstract

The present application relates to the technical field of machine vision, and particularly relates to a method for recognizing, counting and center positioning of bundled steel bar end faces and a memory thereof, which comprises the following steps: S1, capturing an image of the steel bar end face to obtain a to-be-recognized image after processing; S2, performing a data enhancement operation on the to-be-recognized image by using a first preset algorithm; S3, forming a final detection frame in the to-be-recognized image by using a second preset algorithm with a light-weight convolutional neural network, and calculating the number of the final detection frame; and S4, generating a counting result. The present application solves the problem that the existing steel bar end face recognition technology usually uses ordinary machine vision algorithms, resulting in inaccurate counting results and failing to meet the practical needs.

Description

Technical Field

[0001] This invention relates to the field of machine vision technology, and in particular to a method for identifying, counting, and centering the end faces of bundled steel bars, and its memory. Background Technology

[0002] Machine vision uses robots to replace human eyes for measurement and judgment. A machine vision system converts the captured target into an image signal using its vision products, which is then transmitted to a dedicated image processing system to obtain the target's shape information. Based on pixel distribution, brightness, color, and other information, this information is converted into a digital signal. The image system performs various calculations on these signals to extract the target's features, and then controls the on-site equipment based on the judgment results.

[0003] The YOLOv3 algorithm can be used to solve the problem of "how to detect two objects of the same kind that are very close to each other, or objects of different kinds that are very close to each other". It has good robustness to objects that are very close to each other or small objects.

[0004] Existing rebar end face recognition technology typically uses common machine vision algorithms. However, because these algorithms cannot produce accurate results, machine vision cannot yield accurate results in the field of rebar end face counting, thus failing to meet practical needs. Therefore, a method for recognizing, counting, and centering the end faces of bundled rebars, along with its memory, has emerged. Summary of the Invention

[0005] The present invention provides a method for identifying, counting, and centering the end faces of bundled steel bars and its memory, which mainly solves the problem that existing steel bar end face identification technologies usually use ordinary machine vision algorithms, resulting in inaccurate counting results and failing to meet practical needs.

[0006] This invention proposes a method for identifying, counting, and centering the end faces of bundled steel bars, comprising the following steps:

[0007] S1, capture an image of the end face of the steel bar, process it to obtain the image to be identified;

[0008] S2, the first preset algorithm is used to perform data augmentation on the image to be identified;

[0009] S3, a second preset algorithm with a lightweight convolutional neural network is used to form the final detection box in the image to be recognized, and the number of the final detection boxes is calculated;

[0010] S4 generates the counting results.

[0011] Preferably, step S3 specifically includes:

[0012] S31, a second pre-defined algorithm including a lightweight convolutional neural network is pre-formed;

[0013] S32, the second preset algorithm is used to form a final detection box in the image to be recognized, and the number of the final detection boxes is calculated.

[0014] Specifically, the second preset algorithm in step S31 involves improving the backbone feature extraction network by replacing the Darknet53 backbone feature extraction network in the original YOLOV3 network with the Shfflenetv2 backbone feature extraction network.

[0015] Preferably, step S31 specifically includes:

[0016] S311, perform clustering operations on the training images to form anchor boxes;

[0017] S312, divide the training image into multiple small blocks;

[0018] S313, Generate multiple rectangular frames in each small block, and the length and width of the rectangular frames are determined by the anchor frame;

[0019] S314, after fine-tuning multiple rectangular boxes under the same small block, a primary detection box is formed;

[0020] S315, determine whether any of the small blocks contains an object detection error. If so, calculate the IOU value between the multiple primary detection boxes in the current small block and the real boxes in the training image. If all the primary detection boxes are greater than a set threshold, select the primary detection box with the largest IOU value as a positive sample.

[0021] S316, After saving the frame shape of the positive sample, a second preset algorithm with a lightweight convolutional neural network is generated.

[0022] Preferably, step S314 specifically includes:

[0023] S314a, Obtain multiple parameter values ​​of the anchor frame;

[0024] S314b, The rectangle is adjusted according to the acquired parameter values ​​to form a primary detection box;

[0025] Step S314a further includes the offset of the anchor frame relative to the training image (c x c y );

[0026] In step S314b, adjusting the rectangular frame includes the following formula.

[0027] ;

[0028] ;

[0029] ;

[0030] ;

[0031] ;

[0032] Let the width of the rectangle be p. w The height of the rectangle is p h The true value of the coordinates of the rectangle is recorded. , These are preset coordinate values.

[0033] Preferably, after step S315 and before step S316, the following step is provided:

[0034] The following formula is used to calculate the loss.

[0035]

[0036]

[0037]

[0038] Wbox = 2.0 tw * th

[0039] Loss = lossbox + lossconf + lossclass

[0040] Among them, The training image size is S by S, and B represents the detection box. The value is 1 if there is an object in the detection box at coordinates [i,j], otherwise it is 0. The main calculation is three loss functions: the loss between the center coordinates and the width and height of the ground truth box, the loss for whether the predicted box contains the object, and the loss for the predicted category. Finally, the three losses are added together to form a loss function value of one level. The final loss value is the average of the three loss functions.

[0041] Preferably, in step S1, obtaining the image to be identified after processing includes randomly flipping the image, randomly scaling it, randomly cropping it, and randomly changing its brightness.

[0042] Preferably, in step S2, the data augmentation operation of the image to be identified using the first preset algorithm includes performing Fmix enhancement blending on datasets such as dirt or texture.

[0043] The present invention also proposes a computer-readable storage device comprising a stored computer program, wherein, when the computer program is executed, it controls the device in which the computer-readable storage device resides to perform the method as described in the preceding claims.

[0044] As can be seen from the above, the following beneficial effects can be obtained by applying the technical solution provided by the present invention:

[0045] In the method proposed in this invention, a second preset algorithm composed of a lightweight convolutional neural network is trained through model training and can be used to form detection boxes in a targeted manner, ensuring the accuracy of the technical results. Detailed Implementation

[0046] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0047] Existing rebar end face recognition technologies typically employ common machine vision algorithms, resulting in inaccurate counting results that fail to meet practical needs.

[0048] It should be emphasized that the counting and center positioning method proposed in this embodiment is not only applicable to the end face of the rebar, but can also be used in counting fields under different backgrounds and identification objects.

[0049] To address the aforementioned issues, this embodiment proposes a method for identifying, counting, and centering the end faces of bundled reinforcing bars, which mainly includes the following steps:

[0050] S1, capture an image of the end face of the steel bar, process it to obtain the image to be identified;

[0051] S2, the first preset algorithm is used to perform data augmentation on the image to be identified;

[0052] S3, a second preset algorithm with a lightweight convolutional neural network is used to form the final detection box in the image to be recognized, and the number of the final detection boxes is calculated;

[0053] S4 generates the counting results.

[0054] Preferably, but not limited to, the processing of the captured rebar end face image in step S1 of this embodiment includes random image flipping, random scaling, random cropping, and random changes in brightness, etc., to ensure that the edges of multiple rebar structures in the rebar end face are clear, which helps to accurately count them.

[0055] Preferably, but not limited to, in this embodiment, the first preset algorithm in step S2 performs data augmentation on the image to be recognized, including performing Fmix enhancement blending on datasets such as dirt or texture. The blending function is the Fmix function, and the Fmix function implementation process is as follows: 1) Randomly extract an image from the dirt dataset; 2) Obtain a mask by thresholding the low-frequency grayscale image sampled in Fourier space; 3) Perform mask blending between the randomly obtained image from the first step and the mask obtained in the second step. The Fourier transform and mask blending function include:

[0056] ;

[0057]

[0058] ;

[0059] That is, firstly, a random complex tensor with independent real and imaginary parts that follows a Gaussian distribution is sampled; then, through the parameters... Each component is scaled according to its frequency, so that Higher values ​​attenuate the increase of high-frequency information; after performing an inverse Fourier transform on the complex tensor and taking the real part, a grayscale image is obtained; finally, the image is converted into a binary mask by setting a threshold (the top proportion of the image), with values ​​above the threshold set to 1 and values ​​below the threshold set to 0.

[0060] More specifically, step S3 includes:

[0061] S31, a second pre-defined algorithm including a lightweight convolutional neural network is pre-formed;

[0062] S32, the second preset algorithm is used to form the final detection box in the image to be recognized, and the number of the final detection boxes is calculated.

[0063] Specifically, the second preset algorithm in step S31 involves improving the backbone feature extraction network by replacing the Darknet53 backbone feature extraction network in the original YOLOV3 network with the Shfflenetv2 backbone feature extraction network.

[0064] Preferably, the network in step S31 is an improved version of YOLOv3, primarily improving its backbone feature extraction network. Channel splitting is introduced to ensure the same number of input and output channels, thus reducing model memory access costs. Pointwise grouped convolutions (i.e., grouped convolutions with 1×1 kernels) are introduced to reduce computational cost. Pointwise convolutions reduce the number of parameters, while grouped pointwise convolutions further reduce computational cost. To address the issue of excessive grouping reducing network parallelism, channel shuffle is introduced, enriching the information obtained from each group and thus extracting more features.

[0065] Preferably, step S31 specifically includes:

[0066] S311, perform clustering operations on the training images to form anchor boxes;

[0067] S312, divide the training image into multiple small blocks;

[0068] S313, generate multiple rectangular boxes in each small block, and the length and width of the rectangular boxes are determined by the anchor box;

[0069] S314, after fine-tuning multiple rectangular boxes under the same small block, a primary detection box is formed;

[0070] S315, determine whether any small block contains a target object; if so, calculate the IOU value between multiple primary detection boxes in the current small block and the real boxes in the training image; if all primary detection boxes are greater than the set threshold, select the primary detection box with the largest IOU value as the positive sample.

[0071] S316, after saving the box shape of the positive samples, generates a second preset algorithm with a lightweight convolutional neural network.

[0072] Preferably, but not limited to, this embodiment includes a step before S3 to convert the preset data into a format that the convolutional network can process. In this embodiment, the preset original dataset is labeled in VOC format. In step S1, when the image size changes during data enhancement, the corresponding label coordinates also change. When feeding the data into the convolutional neural network, the image data and the labeled data need to be processed separately. The image data needs to be normalized as a whole and converted into NCHW format, i.e., number of channels, height, and width. The label data needs to be clustered using k-means to generate 9 cluster centers as the length and width of the anchor boxes, thus obtaining the anchor boxes used during training.

[0073] Preferably, but not limited to, in this embodiment, the small blocks in step S312 can be set to an N×N format, and the number of rectangular frames selected in step S313 is three, and the shapes of the three rectangular frames are different. Correspondingly, the anchor frame size is divided into three groups from small to large according to the network downsampling ratio, and each group has three rectangular frames from small to large.

[0074] In this embodiment, in step S315, if the initial detection value is less than the set threshold, it is considered that the part does not overlap with the real frame, and if the small square does not contain an object, it is set as a negative sample.

[0075] More specifically, step S314 includes:

[0076] S314a, retrieve multiple parameter values ​​of the anchor frame;

[0077] S314b, after adjusting the rectangle based on the acquired multiple parameter values, a primary detection box is formed;

[0078] Step S314a also includes the offset of the anchor frame relative to the training image (c x c y );

[0079] In step S314b, adjusting the rectangle includes the following formula:

[0080] ;

[0081] ;

[0082] ;

[0083] ;

[0084] ;

[0085] Let the width of the current rectangle be p. w The height of the rectangle is p h Let the true values ​​of the rectangle's coordinates be... The preset coordinates are .

[0086] More specifically, after step S315 and before step S316, the following steps are set:

[0087] SX, the loss is calculated using the following formula.

[0088]

[0089]

[0090]

[0091] Wbox = 2.0 tw * th

[0092] Loss = lossbox + lossconf + lossclass

[0093] Among them, The training image size is S by S, and B represents the detection box. The value is 1 if there is an object in the detection box at coordinates [i,j], otherwise it is 0. The main calculation is three loss functions: the loss between the center coordinates and the width and height of the ground truth box, the loss for whether the predicted box contains the object, and the loss for the predicted category. Finally, the three losses are added together to form a loss function value of one level. The final loss value is the average of the three loss functions.

[0094] In this embodiment, the coordinate values ​​obtained from multiple parameter values ​​can be used to calibrate the coordinates of the rebar within the current detection frame. The preset coordinate values ​​are determined coordinate values ​​obtained from the image of the current rebar end face through manual inspection or other methods. The difference between the two can be used to determine whether the deviation of the current primary detection frame is within the allowable range. If so, the current primary detection frame can be selected as the detection standard; otherwise, the primary detection frame is discarded.

[0095] It should be emphasized that the memory equipped with the aforementioned method is also within the protection scope of this embodiment.

[0096] In summary, this embodiment proposes a method for identifying, counting, and centering the end faces of bundled steel bars. It mainly obtains the most suitable detection box by iteratively replacing multiple detection boxes of different sizes, and determines the number of steel bar end faces in the current image based on the number of detection boxes identified in the current image, resulting in more accurate counting.

[0097] The embodiments described above do not constitute a limitation on the scope of protection of this technical solution. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the above embodiments should be included within the scope of protection of this technical solution.

Claims

1. A method for identifying, counting, and centering the end faces of bundled steel bars, characterized in that, Includes the following steps: S1, capture an image of the end face of the steel bar, process it to obtain the image to be identified; S2, the image to be identified is subjected to data augmentation operation using a first preset algorithm, wherein the first preset algorithm is Fmix enhancement mixing; S21, randomly select an image from the dirty dataset; S22, a mask is obtained by thresholding a low-frequency grayscale image sampled from Fourier space; S23, perform mask blending between the image randomly acquired in S21 and the mask obtained in S22; Convert the data into a format that convolutional networks can process; S3, a second preset algorithm with a lightweight convolutional neural network is used to form the final detection box in the image to be recognized, and the number of the final detection boxes is calculated; S4, generate the counting results; Step S3 specifically includes: S31, a second preset algorithm including a lightweight convolutional neural network is pre-formed. The second preset algorithm is to improve its backbone feature extraction network by replacing the Darknet53 backbone feature extraction network in the original YOLOV3 network with the Shfflenetv2 backbone feature extraction network and introducing channel split to make the number of input and output channels of the network the same. Step S31 specifically includes: S311, perform clustering operations on the training images to form anchor boxes; S312, divide the training image into multiple small blocks; S313, Generate multiple rectangular frames in each small block, and the length and width of the rectangular frames are determined by the anchor frame; S314, after fine-tuning multiple rectangular boxes under the same small block, a primary detection box is formed; S315, determine whether any of the small blocks contains a target object; if so, calculate the IOU value between the multiple primary detection boxes in the current small block and the real boxes of the training image; if all the primary detection boxes are greater than a set threshold, select the primary detection box with the largest IOU value as a positive sample. S316, After saving the frame shape of the positive sample, a second preset algorithm with a lightweight convolutional neural network is generated; S32, the second preset algorithm is used to form a final detection box in the image to be recognized, and the number of the final detection boxes is calculated; Specifically, the second preset algorithm in step S31 involves improving the backbone feature extraction network by replacing the Darknet53 backbone feature extraction network in the original YOLOV3 network with the Shfflenetv2 backbone feature extraction network.

2. The method for identifying, counting, and centering the end faces of bundled steel bars according to claim 1, characterized in that, Step S314 specifically includes: S314a, Obtain multiple parameter values ​​of the anchor frame; S314b, The rectangle is adjusted according to the acquired parameter values ​​to form a primary detection box; Step S314a further includes the offset of the anchor frame relative to the training image (c x c y ); In step S314b, adjusting the rectangular frame includes the following formula. ； ； ； ； ； Let the width of the rectangle be p. w The height of the rectangle is p h The true value of the coordinates of the rectangle is recorded. , These are preset coordinate values.

3. The method for identifying, counting, and centering the end faces of bundled steel bars according to claim 2, characterized in that, After step S315 and before step S316, the following steps are set: The following formula is used to calculate the loss. Wbox = 2.0 tw * th Loss = lossbox + lossconf + lossclass Among them, The training image size is S x S, and B represents the detection box. The value is 1 if there is an object in the detection box at coordinates [i,j], otherwise it is 0. The main calculation is three loss functions: the loss between the center coordinates and the width and height of the ground truth box, the loss for whether the predicted box contains the object, and the loss for the predicted category. Finally, the three losses are added together to form a loss function value of one level. The final loss value is the average of the three loss functions.

4. A method for identifying, counting, and centering the end faces of bundled reinforcing bars according to any one of claims 1 to 3, characterized in that: In step S1, obtaining the image to be identified after processing includes randomly flipping the image, randomly scaling it, randomly cropping it, and randomly changing its brightness.

5. A method for identifying, calculating, and centering the end faces of bundled reinforcing bars according to any one of claims 1 to 3, characterized in that: In step S2, the data augmentation operation of the image to be identified using the first preset algorithm includes performing Fmix enhancement blending on datasets such as dirt or texture.

6. A computer-readable storage device, characterized in that: The computer-readable storage device includes a stored computer program, wherein when the computer program is executed, it controls the device in which the computer-readable storage device is located to perform the method as described in any one of claims 1 to 5.

Images