A bar end target tracking method based on adaptive difference

Abstract

The present application belongs to the technical field of multi-target tracking, and particularly relates to a bar end target tracking method based on adaptive difference. The method comprises the following steps: adopting a differential algorithm mixed with an adaptive threshold strategy, obtaining a bar feature map and coordinate information based on support yolov3 detection and hough circle detection algorithm; adopting a method of circle center extraction and motion trajectory calculation to track the target; according to the coordinates of all tracking targets in the first frame and the last frame and the coordinates in the tracking process, a logical algorithm is written to obtain initial sorting and final sorting, and tracking is realized. The method provided by the present application can accurately analyze the motion trend of each steel pipe, and even in the case of steel turning and steel disorder, the steel pipe can be accurately detected and tracked.

Description

Technical Field

[0001] This invention belongs to the field of multi-target tracking technology, and particularly relates to a method for tracking rod end targets based on adaptive difference, used for the detection and tracking of rod ends with similar target features. Background Technology

[0002] Traditional bar production companies primarily track data by furnace batch and batch. However, as downstream users' demands increase, they require production data to correspond to each bar within a batch. This necessitates implementing bar-by-bar tracking throughout the production process, ensuring that production data is linked to each individual bar.

[0003] After the bars are rolled out of the rolling mill, they are first lifted onto the cooling bed, then rolled onto the cooling bed via a downhill slope. During this rolling process, the bars may jump and become disorganized, meaning the front and back order of the bars changes. The same issue occurs during the stepping and lateral movement of the bars on the cooling bed. To achieve automatic tracking, cameras can be placed in designated areas to capture images of the bar ends. However, since each bar is almost identical in size and color, the aforementioned disorganization will cause target tracking failure, severely impacting subsequent production processes. Summary of the Invention

[0004] To address the aforementioned issues, this invention proposes a rod end target tracking method based on adaptive differential. The method uses support vector regression algorithm to identify the relative position of the rod end in each frame of video image, thereby achieving trajectory tracking of each rod during the conveying process.

[0005] The technical solution adopted in this invention is as follows:

[0006] A method for tracking a target at the end of a bar based on adaptive differential gradation, the method comprising:

[0007] (1) Use a high-speed camera to capture images of the entire process of beveling the bar, and perform grayscale processing frame by frame to obtain grayscale images;

[0008] (2) The bar end face region in the grayscale image is identified and extracted by deep learning methods; then the RHT circle detection algorithm is used to determine the center coordinates of the bar end face.

[0009] (3) The grayscale image described in step (1) is calculated using an adaptive thresholding algorithm, and the differential image is binarized to achieve adaptive thresholding.

[0010] (4) The center extraction and motion trajectory calculation methods are used to associate and track the target, and multi-target tracking is realized according to the logical algorithm.

[0011] Further, step (1) specifically involves setting up a high-speed camera in the rolling area of ​​the bar, acquiring video data, extracting the feature map of each frame, and performing image grayscale processing through the feature map to obtain a grayscale image.

[0012] Further, in step (2), the step of identifying and extracting the rod end face region in the grayscale image using deep learning methods specifically involves:

[0013] 1) Train the YOLO V3 network using pre-collected video images of the bar end faces;

[0014] 2) The trained YOLO V3 neural network is used to identify the grayscale image of the end face of the bar to be detected. Each identified target is marked with a prediction box.

[0015] 3) The edge image EDGE is calculated using the Canny operator on the gray image of the end face of the bar to be inspected;

[0016] 4) On the edge image, a corresponding sub-image is obtained based on the prediction box.

[0017] Further, in step (2), the determination of the center coordinates of the bar end face using the RHT circle detection algorithm specifically involves:

[0018] For the unique circle in the predicted bounding box given by the YOLO V3 network, a random Hough transform is applied, and three points are randomly selected from the edge image. The center coordinates and radius of the corresponding circle are calculated. The accumulators corresponding to the calculated center and radius in the RHT space are accumulated, and this process is repeated until a certain accumulator reaches a threshold, thus identifying a circle and obtaining the predicted center coordinates O(x). o ,y o );

[0019] Furthermore, the threshold of RHT is T1 = N; N is a constant, an integer representing the number of random numbers generated.

[0020] Furthermore, step (3) specifically involves:

[0021] S1: Before detecting dynamic targets in the video, learn the static background. For the grayscale image described in step (1), obtain the background mean μ for each frame. i and the standard deviation of the background for each frame σ i ;

[0022] The background mean μ and background standard deviation σ are calculated using the following formula:

[0023]

[0024]

[0025] Based on actual tracking experience, the value of n is 60.

[0026] S2: Achieve adaptive threshold by setting a threshold range:

[0027] Threshold T a For an interval value: T a =[μ-σ,μ=σ];

[0028] Grayscale image processing converts it to a binary image. The result of the binary image calculation is as follows:

[0029]

[0030] Here, the difference result A is a binary image, where 1 indicates that the gray-level gradient of the corresponding point has changed significantly during the difference process; 0 indicates that the gray-level gradient of the corresponding point has not changed or has changed very little during the difference process, thereby achieving threshold adaptation; time t is the time corresponding to the current frame, and time t' is the time corresponding to the next frame.

[0031] Furthermore, in step (4), the method of using the center extraction and motion trajectory calculation to associate and track the target specifically includes:

[0032] S1: Based on the position of the center of the circle at time m (x m y m Predict the center position (x) of the circle at time m+1 in the next frame. m+1 y m+1 ):

[0033] x m+1 =x m +v 0x t, y m+1 =y m +v 0y t;

[0034] In the formula, t is the time difference between time m+1 and time m; v 0x, v 0y This represents the velocity of the bar in the x and y directions; where m is the position of the center of the circle (x, y). m y m The result is obtained according to the method provided in step (2) (using YOLO V3 in conjunction with RHT).

[0035] S2: To achieve simultaneous tracking of multiple bars and ensure smooth operation, the trajectories of related bars are correlated:

[0036] |(x m+1 ,y m+1 )-(x m ,y m )|≤η

[0037] η is 80% of the radius of the end face of the bar.

[0038] Furthermore, in step (4), multi-target tracking is implemented according to the logical algorithm, specifically as follows:

[0039] Write a logical algorithm based on the coordinates of all tracked targets and the coordinates of the tracking process in the first and last frames;

[0040] The underlying logic is as follows: For the same bar, the distance the bar moves between two adjacent moments is less than the bar radius; the horizontal coordinate of the bar at the next moment is greater than the horizontal coordinate of the bar at the previous moment, and the difference between the horizontal coordinates of the bar at two adjacent moments is less than the bar radius.

[0041] By connecting the bar position information at all times through a logical algorithm, the movement trajectories of all bars are obtained; the initial sorting and final sorting are obtained, and multi-target tracking is completed.

[0042] The beneficial effects of this invention are:

[0043] The present invention provides a rod end target tracking method based on adaptive difference, which accurately and effectively extracts the moving rod through adaptive difference method, and the adaptive threshold can better separate the moving object.

[0044] Based on the underlying logic provided by this invention, the movement trend of the bar can be accurately determined to achieve bar-by-bar tracking. This method does not require the identification of moving target features, is simple in principle, has small error, and is easy to implement.

[0045] The method provided by this invention can accurately analyze the movement trend of each steel pipe, and can accurately detect and track the steel pipe even in the event of steel overturning or disorder. Attached Figure Description

[0046] The present invention will be further described below with reference to the accompanying drawings and specific embodiments. The scope of protection of the present invention is not limited to the following description.

[0047] Figure 1 This is a schematic diagram of a rod end target tracking method based on adaptive differential in an embodiment of the present invention;

[0048] Figure 2 This is a schematic diagram of the YOLOV3 and RHT circle detection methods in an embodiment of the present invention;

[0049] Figure 3 This is a schematic diagram of the bar tracking process at the cooling bed in an embodiment of the present invention. Specific implementation methods

[0050] To further illustrate the technical means and effects of the present invention in achieving its intended purpose, the following detailed description of the specific implementation methods, structures, features, and effects of the present invention, in conjunction with the accompanying drawings and preferred embodiments, is provided below.

[0051] This invention provides a method for tracking end targets of bar stock based on adaptive differential. The method detects the bar stock based on video signals and a detection model; performs image processing and establishes underlying logic by operating frame by frame based on the detected images; solves the multi-target tracking problem by using a differential algorithm for center extraction and motion trajectory calculation; and implements tracking according to the logical algorithm.

[0052] like Figure 1 As shown, the method specifically includes:

[0053] (1) Use a high-speed camera to capture images of the entire process of beveling the bar, and perform grayscale processing frame by frame to obtain grayscale images;

[0054] (2) The bar end face region in the grayscale image is identified and extracted by deep learning methods; then the RHT circle detection algorithm is used to determine the center coordinates of the bar end face.

[0055] (3) The grayscale image described in step (1) is calculated using an adaptive thresholding algorithm, and the differential image is binarized to achieve adaptive thresholding.

[0056] (4) The center extraction and motion trajectory calculation methods are used to associate and track the target, and multi-target tracking is realized according to the logical algorithm.

[0057] Step (1) specifically involves setting up a high-speed camera in the rolling area of ​​the bar to collect video data, extracting the feature map of each frame, and performing image grayscale processing through the feature map to obtain a grayscale image.

[0058] In this embodiment, step (2) identifies the end face of the rod in the image by introducing the YOLOV3 deep learning method; then, RHT is used to improve the result of YOLOV3 target detection. It includes the following two parts.

[0059] First, the trained YOLOv3 is used to perform target detection on the video images, and all the rods in the image are initially selected.

[0060] Then, based on the characteristic that the end face profile of the bar is generally approximately circular, the RHT circle detection method is used to locate the bar located by YOLOV3, and the center coordinates of the bar end face are accurately obtained.

[0061] like Figure 2 As shown, in step (2), the step of identifying and extracting the rod end face region in the grayscale image using deep learning methods specifically involves:

[0062] 1) Train the YOLO V3 network using pre-collected video images of the bar end faces;

[0063] 2) The trained YOLO V3 neural network is used to identify the grayscale image of the input bar end face to be detected. Each identified target is represented by a prediction box (b x b y b w b h ) identifier; b x b y b w b h The bounding box represents the position and size relative to the feature map;

[0064] 3) The edge image (EDGE) is calculated using the Canny operator on the gray image of the end face of the bar to be inspected;

[0065] 4) On the edge image, based on the predicted bounding box (t) x t y t w t h ), thus obtaining the corresponding subgraph; t x t y t w t h These represent the model's coordinate offset and scale, respectively.

[0066] Based on this subgraph, an improved RHT circle detection method is used, with the RHT threshold T1 = N (N is a constant, an integer representing the number of random numbers generated), to obtain the predicted circle center O(x). o ,y o ).

[0067] The process of extracting the end-face region of a grayscale image based on YOLOv3 is as follows:

[0068] The input image is first scaled to 416*416 pixels using YOLOv3, and then multi-scale images of 13*13, 26*26, and 52*52 are obtained through a deep neural network. The number of prior boxes is nine: (10*13), (16*30), (33*23), (30*61), (62*45), (59*119), (116*90), (156*198), and (373*326). These nine prior boxes are then divided into three groups and assigned to feature maps at the 13*13, 26*26, and 52*52 scales.

[0069] After calculating the multi-scale features of each cell in the image, logistic regression was used to score the content enclosed by the nine prior boxes. Only the prior box with the highest target score was predicted by the algorithm, and the prediction result was output. The multi-scale image feature information obtained by the neural network includes the center coordinates of the network output (t). x t y Width t w and height t h Four relevant parameters. If each unit is offset from the top left corner of the image (c x ,c y ), and the width and height of the prior box are p respectively. w p h Then the predicted box (b) x b y b w b h ) is represented as:

[0070] b x =σ(t) x )+c x ;

[0071] b y =σ(t) y )+c y ;

[0072]

[0073]

[0074] In the formula, b x b y b represents the coordinates of the top-left corner of the prediction box. w b h σ represents the width and height of the prediction box, respectively; σ is the sigmoid function.

[0075] In step (2) of this embodiment, determining the center coordinates of the bar end face using the RHT circle detection algorithm specifically involves:

[0076] For the unique circle in the predicted bounding box given by the YOLO V3 network, a random Hough transform is used to randomly select 3 points in the edge image, calculate the center coordinates and radius of the corresponding circle, and accumulate the accumulators corresponding to the calculated center and radius in the RHT space. This process is repeated until a certain accumulator reaches a threshold, thus identifying a circle.

[0077] In this embodiment, step (3) specifically includes:

[0078] S1: Before detecting dynamic targets in the video, learn the static background. For the grayscale image described in step (1), obtain the background mean μ for each frame. i and the standard deviation of the background for each frame σ i ;

[0079] The background mean μ and background standard deviation σ are calculated using the following formula:

[0080]

[0081]

[0082] Based on actual tracking experience, the value of n is 60.

[0083] S2: Achieve adaptive threshold by setting a threshold range:

[0084] Threshold T a For an interval value: T a =[μ-σ,μ=σ];

[0085] Grayscale image processing converts it to a binary image. The result of the binary image calculation is as follows:

[0086]

[0087] Here, the difference result A is a binary image, where 1 indicates a significant change in the gray-level gradient at the corresponding point during the difference process; 0 indicates no change or a very small change in the gray-level gradient at the corresponding point during the difference process, thus achieving adaptive thresholding. f(j,k,t) is the gray-level value at coordinate (j,k) at time t (in the current frame), and f(j,k,t′) is the gray-level value at coordinate (j,k) at time t' (in the next frame). Time t corresponds to the current frame, and time t' corresponds to the next frame.

[0088] In step (4) of this embodiment, the method of using the center extraction and motion trajectory calculation to associate and track the target is specifically as follows:

[0089] S1: Based on the position of the center of the circle at time m (x m y m Predict the center position (x) of the circle at time m+1 in the next frame. m+1 y m+1 ):

[0090] x m+1 =x m +v 0x t, y m+1 =y m +v 0y t;

[0091] In the formula, t is the time difference between time m+1 and time m; v0x, v 0y This indicates the velocity of the bar in the x and y directions;

[0092] S2: To achieve simultaneous tracking of multiple bars and ensure smooth operation, the trajectories of related bars are correlated:

[0093] |(x m+1 ,y m+1 )-(x m ,y m )|≤η

[0094] η is 80% of the radius of the end face of the bar.

[0095] In step (4) of this embodiment, multi-target tracking is implemented according to the logical algorithm, specifically as follows:

[0096] Write a logical algorithm based on the coordinates of all tracked targets and the coordinates of the tracking process in the first and last frames;

[0097] The underlying logic is as follows: For the same bar, the distance the bar moves between two adjacent moments is less than the bar radius; the horizontal coordinate of the bar at the next moment is greater than the horizontal coordinate of the bar at the previous moment, and the difference between the horizontal coordinates of the bar at two adjacent moments is less than the bar radius.

[0098] By connecting the bar position information at all times through a logical algorithm, the movement trajectories of all bars are obtained; the initial sorting and final sorting are obtained, and multi-target tracking is completed.

[0099] like Figure 3 As shown, since there are multiple bars (this figure uses 6 bars as an example), trajectory association of the relevant center points is necessary for smooth tracking. The motion trajectory can be divided into two types: those without errors and those with errors. Accurate tracking can be achieved through logical algorithms.

[0100] When no errors occur, the tracing process is as follows:

[0101] The image captured by the high-speed camera at time T can detect the positions of bars 1-6 and return six coordinates: 123456.

[0102] The image captured by the high-speed camera at time T1 can detect six coordinate positions at that time. The images from time T2, T3 to TN can also obtain six coordinate positions, just like the images from the previous two times.

[0103] In a typical video stream at 30 frames per second, during the fastest rolling phase, the distance the same rod moves between adjacent moments (two adjacent frames) is much smaller than the rod's radius. The x-coordinate of a later moment should be greater than the x-coordinate of the previous moment, and the difference in x-coordinates should be less than the radius. Based on this principle, the position information of the rods at all moments can be chained together using a logical algorithm to obtain the movement trajectories of the six rods, thereby achieving rod tracking.

[0104] When bars No. 4 and No. 5 are misaligned: the principle of time T and T1 is the same as described above.

[0105] When bar #5 tumbles at time T2, the six bars can be correlated with the previous time point based on the returned x-coordinate. Bar #5's x-coordinate should be larger than its x-coordinate at time T1, and its y-coordinate should show a significant increase and be smaller than its diameter.

[0106] At time T3, the vertical coordinate of bar #5 is closest to that at time T2, but the horizontal coordinate will undergo a small displacement (less than the radius).

[0107] At time T4, bar #5 moved before bar #4, thus enabling the tracking of all six bars in the event of a misalignment.

[0108] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any way. Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make some modifications or alterations to the above-disclosed technical content to create equivalent embodiments without departing from the scope of the present invention. Any simple modifications, equivalent changes and alterations made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the scope of the present invention.

Claims

1. A bar end target tracking method based on adaptive difference, characterized by, The method comprises: (1) using a high-speed camera to collect all process images of the bar chamfering, and performing gray processing on each frame to obtain a gray image; (2) recognizing and extracting a bar end face region in the gray image through a deep learning method; and then determining a bar end face center coordinate by using an RHT circle detection algorithm; (3) calculating the gray image in step (1) by using an adaptive threshold algorithm, binarizing a difference image, and realizing adaptive threshold; (4) correlating and tracking a target by using a center extraction and motion trajectory calculation method, and realizing multi-target tracking according to a logical algorithm; In step (4), the center extraction and motion trajectory calculation method are used to correlate and track the target, specifically as follows: S1: predict the center position of the circle at m+1 time point in the next frame according to the center position of the circle at m time point (Xm, Ym) , ) (Xm+1, Ym+1) , ): ； In the formula, t is the time difference between m+1 moment and m moment; denotes the movement speed of the bar in the x and y directions; S2: In order to realize the tracking of multiple bars at the same time, the trajectories of relevant bars are correlated: 80% of the radius of the end face of the bar; In step (4), the multi-target tracking is realized according to the logical algorithm, specifically as follows: A logical algorithm is written according to the coordinates of all tracking targets in the first frame and the last frame and the coordinates in the tracking process; The bottom logic is that, for the same bar, the moving distance of the bar between two adjacent time points is less than the radius of the bar; the horizontal coordinate of the bar in the next time point is greater than that in the previous time point, and the difference between the horizontal coordinates of the bar in the two adjacent time points is less than the radius of the bar; The position information of the bar at all time points is connected by the logical algorithm, and the moving trajectories of all bars are obtained; the initial sorting and the final sorting are obtained, and the multi-target tracking is completed.

2. The bar end object tracking method based on adaptive difference according to claim 1, wherein, Step (1) is specifically as follows: a high-speed camera is arranged in the bar lower rolling area to collect video data, extract a feature map of each frame, perform image gray processing on the feature map, and obtain a gray image.

3. The bar end object tracking method based on adaptive difference according to claim 1, wherein, In step (2), the bar end face region in the gray image is recognized and extracted by the deep learning method, specifically as follows: 1) A YOLO V3 network is trained by using pre-collected bar end face video images; 2) The trained YOLO V3 neural network is used to identify the input gray image of the bar end face to be detected, and each identified target is marked with a prediction box; 3) An edge image is obtained by using a canny operator on the gray image of the bar end face to be detected; 4) According to the prediction box, a corresponding sub-image is obtained on the edge image.

4. The bar end object tracking method based on adaptive difference according to claim 3, characterized in that, In step (2), the bar end face center coordinate is determined by using the RHT circle detection algorithm, specifically as follows: For the unique circle in the prediction frame given by the YOLO V3 network, a random Hough transform is adopted, 3 points in the edge image are randomly selected, the center coordinates and radius of the corresponding circle are calculated, the accumulators corresponding to the center and radius calculated in the RHT space are accumulated, and the process is repeated until a certain accumulator reaches a threshold, that is, a circle is identified, and the center of the circle is obtained.

5. The bar end object tracking method based on adaptive difference according to claim 4, characterized in that, Threshold of RHT N is a constant, which is an integer, representing the number of random numbers generated.

6. The bar end object tracking method based on adaptive difference according to claim 1, wherein, Step (3) is specifically as follows: S1: Learning the static background before video dynamic target detection, for the gray image in step (1), obtaining the mean value of each frame background and the standard deviation of each frame background ; The background mean is calculated by the formula and the background standard deviation : ； S2: Adaptive threshold is realized by setting a threshold interval: Threshold value is an interval value ; The gray image is processed into a binary image, and the calculation result of the binary image is: ； Wherein, the difference result A is a binary image, 1 represents that the corresponding point gray scale gradient has changed greatly when differentiating; 0 represents that the corresponding point gray scale gradient has not changed or has changed a little when differentiating, thereby realizing the self-adaptation of the threshold value; is the gray value of the coordinate (j, k) at t time, is the gray value of the coordinate (j, k) at t' time; t time is the time corresponding to the current frame, and t' time is the time corresponding to the next frame.

Images