Intelligent measuring method for hot-rolled bar flying shear cut piece

Abstract

The application discloses a kind of hot-rolled bar flying shear cutting piece intelligent measurement method.The described method includes the following steps: constructing target detection network model;The target detection network model constructed is trained;The image of waste is obtained by binocular stereo vision, and the region of interest in the image obtained is extracted using the trained target detection network model;The feature point information of waste in the two regions of interest is detected and matched to obtain matching points;The matching points are prior knowledge, the waste boundary in the region of interest is regionally matched, and the three-dimensional boundary is obtained by introducing the functional relationship between pixel points and three-dimensional coordinate points, and finally the three-dimensional size of waste is calculated.The application adopts shufflenet as the backbone of target detection algorithm, and the detection speed is fast and the accuracy is high, and the whole system is developed by cooperating with Spring Boots framework.

Description

Technical Field

[0001] This invention relates to a method for measuring material shearing using target detection and binocular stereo vision. Background Technology

[0002] The flying shear for hot-rolled rebar is located on the exit side of the roughing mill, before the descaling box in the finishing mill. Its function is to cut off the low-temperature and poorly shaped ends of the intermediate strip entering the finishing mill, allowing the billet to pass smoothly through the finishing mill and output roller table, preventing jamming and low-temperature ends from leaving roll marks on the rolls. Information on the scrap removed by the flying shear is of significant reference value for rebar quality monitoring. Summary of the Invention

[0003] The purpose of this invention is to provide a method for non-contact measurement of scrap material during flying shearing using binocular stereo vision combined with target detection, so as to measure the length of scrap material at the shear head and tail in real time, providing important information for the whole process quality monitoring of rebar.

[0004] To address the above problems, the present invention provides an intelligent measurement method for the flying shearing of hot-rolled bars, comprising the following steps:

[0005] Construct an object detection network model;

[0006] Train the constructed object detection network model;

[0007] Images of waste materials are acquired using binocular stereo vision.

[0008] The trained object detection network model is used to extract the region of interest from the acquired image.

[0009] Feature point information of waste in two regions of interest is detected and matched to obtain matching points;

[0010] Using the matching points as prior knowledge, the waste boundary in the region of interest is matched, and the three-dimensional boundary is obtained by substituting the function relationship between the pixel point and the three-dimensional coordinate point. Finally, the three-dimensional size of the waste is calculated.

[0011] Furthermore, the steps for training the constructed object detection network model are as follows:

[0012] 21) Collect samples using a camera;

[0013] 22) Sort samples and label all waste images;

[0014] 23) Use ShuffleNet to build a backbone for object detection;

[0015] 24) Use ShuffleNet to extract features from waste images;

[0016] 25) Use spatial pyramid pooling to transform feature maps of arbitrary size into feature vectors of fixed size;

[0017] 26) After transforming the feature map size using spatial pyramid pooling, it enters the FPN+PAN structure; the FPN structure performs upsampling from top to bottom, so that the bottom feature map contains stronger target semantic information; the PAN structure performs downsampling from bottom to top, so that the top feature contains strong target location information, and the two features are finally fused.

[0018] Furthermore, the steps for collecting samples include:

[0019] 31) Manually sort samples and retain only image files containing waste materials;

[0020] 32) Perform horizontal and vertical flipping, image brightness and contrast adjustments on the collected data to expand the dataset by more than four times.

[0021] The beneficial effects of this invention are as follows: This invention provides a non-contact detection system for flying shear waste based on target detection and binocular stereo vision. A target detection algorithm using ShuffleNet as the backbone is built, achieving fast detection speed and high accuracy. The entire system is then developed in conjunction with the Spring Boots framework. Attached Figure Description

[0022] Figure 1 This is a diagram of the target detection network structure of the present invention.

[0023] Figure 2 This is a schematic diagram of the binocular stereo vision of the present invention. Detailed Implementation

[0024] The embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0025] In the description of this invention, it should be understood that the terms "center", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing this invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this invention.

[0026] The terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this invention, unless otherwise stated, "a plurality of" means two or more.

[0027] In the description of this invention, it should be noted that, unless otherwise explicitly specified and limited, the terms "installation," "connection," and "joining" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal communication between two components. Those skilled in the art can understand the specific meaning of the above terms in this invention based on the specific circumstances.

[0028] The present invention provides an intelligent measurement method for the shearing of hot-rolled bars, comprising the following steps:

[0029] Construct an object detection network model;

[0030] Train the constructed object detection network model;

[0031] Images of waste materials are acquired using binocular stereo vision.

[0032] The trained object detection network model is used to extract the region of interest from the acquired image.

[0033] Feature point information of waste in two regions of interest is detected and matched to obtain matching points;

[0034] Using the matching points as prior knowledge, the waste boundary in the region of interest is matched, and the three-dimensional boundary is obtained by substituting the function relationship between the pixel point and the three-dimensional coordinate point. Finally, the three-dimensional size of the waste is calculated.

[0035] To more clearly illustrate the technical means and beneficial effects of the present invention, the present invention will be described in detail below with reference to the accompanying drawings and embodiments.

[0036] Step 1: Install camera equipment and collect samples. During data collection, to ensure the accuracy of model recognition, try to collect high-resolution, high-pixel images. Also, use a high-frequency camera with a fast shearing speed to ensure the quality of the captured images.

[0037] Step 2: Typically, the flying shear has a long work cycle, and most of the collected samples are from steel-cutting or idle states, with few samples actually cutting waste. To prevent imbalanced training samples for deep learning, it is necessary to manually select samples containing only waste. The samples are manually sorted, retaining only image files containing waste.

[0038] Step 3: Import all waste samples into the labelimg calibration software and calibrate the samples. To improve model accuracy, perform data augmentation on all samples.

[0039] To expand the quantity and diversity of the collected dataset, preliminary transformations were first performed, including horizontal and vertical image flipping, adjusting image brightness and contrast, etc., to expand the sample to more than four times, thus achieving data augmentation.

[0040] Secondly, a mosaic transformation is used to randomly crop and scale the four images, and then randomly arrange and stitch them together to form a single image. This enriches the dataset, increases the number of small sample targets, and improves the training speed of the network.

[0041] Using the mosaic transformation not only requires transforming and stitching the images, but also requires regenerating the calibration file based on the image transformation and the final stitching layout.

[0042] Step 4: Build the target detection model. To adapt to high-speed flying shear cutting motion, the model's detection speed must be fast. ShuffleNet is used as the backbone to extract image features, and the network input is a fixed color image with a size of 640×640×3.

[0043] After extracting image features using ShuffleNet, spatial pyramid pooling is first used to transform feature maps of arbitrary size into feature vectors of fixed size, thereby increasing the receptive field.

[0044] Even the same waste material can vary in scale. In the process of convolutional neural networks, the deeper the network layers, the stronger the feature information of the target, and the better the model predicts the target. However, at the same time, the positional information of the target will become weaker and weaker. Furthermore, in the continuous convolution process, information about small targets is easily lost. Therefore, it is necessary to make predictions at different scales.

[0045] After spatial pyramid pooling transforms the feature map size, it enters the FPN+PAN structure. Multi-scale feature fusion is performed on the image using FPN and PAN. The upper-layer feature maps, due to their deeper network layers, contain stronger target semantic information, while the lower-layer features, having undergone fewer convolutional layers, suffer less loss of target location information. The FPN structure performs top-down upsampling, ensuring that the lower-layer feature maps contain stronger target semantic information; the PAN structure performs bottom-up downsampling, ensuring that the top-layer features contain strong target location information. Finally, the two feature sets are fused, ensuring that feature maps of different sizes contain both strong target semantic and feature information, guaranteeing accurate prediction of target images of different sizes.

[0046] To further improve detection speed, an anchor-based approach is used for final detection and regression. To ensure detection performance, at the beginning of network training, cluster analysis is performed on all target sizes in the sample set to plan the sizes of the three anchors.

[0047] GIOU was chosen as the bounding box loss function for object detection, and aspect ratio similarity was added as an indicator. When the IOU value and the center point distance value are the same, the more similar the aspect ratio of the two boxes are, the better the comparison effect between the predicted box and the target box.

[0048] Step 5: Divide all the calibration data into training and testing sets in an 8:2 ratio and feed them into the model for training;

[0049] Step 6: Fabricate a pinhole camera calibration board, number the left and right cameras respectively, and acquire multi-pose images of the calibration board. Import the calibration images, calculate the intrinsic and extrinsic parameters of the left and right cameras, delete calibration images with large errors, and then recalibrate.

[0050] Step 7: After network training and stereo camera calibration are completed, a model parallel computing environment is created using Docker and packaged into an image. The specific steps are as follows: Create a Linux system kernel, install Python as the main programming language framework and PyTorch as the model computing framework, and install CUDA and cuDNN to implement GPU parallel computing model.

[0051] Write a script to build the aforementioned network model and train it. The script's specific content is as follows: Use OpenCV to acquire shearing images of the waste material captured by left and right cameras. First, use the model to detect the waste material's location in the left and right images. Then, input this information into a pinhole camera model to obtain the waste material's 3D information. Finally, use Flask to generate a web service connection to Spring Boot. Adjustable hyperparameters for training the model include: initial learning rate, final decaying annealing learning rate, learning rate decay weights, learning rate warm-up iterations, initial momentum of learning rate warm-up, learning rate warm-up weights, target box loss weights, classification loss weights, target loss weights, IOU threshold, and data augmentation parameters.

[0052] Step 8: Create a high-precision checkerboard calibration board and adjust the camera aperture and focal length to capture high-quality checkerboard images.

[0053] Adjust the chessboard's orientation and acquire images of the chessboard from different angles. To minimize calibration errors, ensure that the entire valid area of ​​the chessboard is within the image and occupies more than one-third of the total image area. To improve calibration reliability, at least twenty pairs of calibration samples need to be captured.

[0054] The Zhang Zhengyou calibration method was used to calibrate the binocular camera. After the initial calibration, images with large pixel errors were deleted, and the camera was recalibrated until the calibration accuracy met the requirements of the project.

[0055] The intrinsic and extrinsic parameters of the binocular camera are obtained through calibration. Based on the pinhole imaging model, the functional relationship between the pixel points and the three-dimensional coordinate points of the measured object is derived from the decoding matrix.

[0056] Step 9: Write a script to call the final trained model via PyTorch and capture images using a stereo camera via OpenCV. Extract the left and right waste images using an object detection algorithm, and segment the region of interest (ROI) based on the object detection results. Use the big rule algorithm to perform semantic segmentation on the ROI, separating the waste from the background. Use the SIFT feature point detection algorithm to detect feature point information of the waste in the two ROI images, and use brute-force matching to obtain matching points.

[0057] Using the matching points as prior knowledge, the waste boundary in the region of interest is matched, and the three-dimensional boundary is obtained by substituting the function relationship between the pixel point and the three-dimensional coordinate point. Finally, the three-dimensional size of the waste is calculated.

[0058] Step 10: Finally, copy the completed script and the best model file into the Docker image and package and deploy it.

[0059] In the description of this specification, specific features, structures, materials, or characteristics may be combined in any suitable manner in one or more embodiments or examples.

[0060] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. An intelligent measurement method for the flying shearing of hot-rolled bars, characterized in that, The method includes the following steps: Construct an object detection network model; Train the constructed object detection network model; Images of waste materials are acquired using binocular stereo vision. The trained object detection network model is used to extract the region of interest from the acquired image. Feature point information of waste in two regions of interest is detected and matched to obtain matching points; Using the matching points as prior knowledge, the waste boundary in the region of interest is matched, and the three-dimensional boundary is obtained by substituting the function relationship between the pixel point and the three-dimensional coordinate point. Finally, the three-dimensional size of the waste is calculated. The steps for training the constructed object detection network model are as follows: 21) Collect samples using a camera; 22) Sort samples and label all waste images; 23) Use ShuffleNet to build a backbone for object detection; 24) Use ShuffleNet to extract features from waste images; 25) Use spatial pyramid pooling to transform feature maps of arbitrary size into feature vectors of fixed size; 26) After transforming the feature map size using spatial pyramid pooling, it enters the FPN+PAN structure; the FPN structure performs upsampling from top to bottom, so that the bottom feature map contains stronger target semantic information; the PAN structure performs downsampling from bottom to top, so that the top feature contains strong target location information, and the two features are finally fused.

2. The method of claim 1, wherein the method is characterized by, The steps for collecting samples include: 31) Manually sort samples, retaining only image files containing waste materials; 32) Perform horizontal and vertical flipping, image brightness and contrast adjustments on the collected data to expand the dataset by more than four times.

Images