A method for automatic counting of downhole drill rods based on machine vision

Abstract

A method for automatic counting of underground drill rods based on machine vision is characterized by comprising the following steps: (1) reading an underground drilling scene video and de-framing it, pre-processing the de-framing image to solve the problem of low image contrast; (2) using a trained target detection model to perform binary classification target detection tracking on the pre-processed video image, wherein the tracking objects include drill rods and drilling rigs; (3) drawing a periodic motion waveform diagram in the same figure according to the horizontal position change of the tracked drill rods and drilling rigs; for the rod withdrawal counting method, if the double waveforms of the drill rod and the drilling rig have a falling edge at the same time, then count once; for the rod advance counting method, if the double waveforms of the drill rod and the drilling rig have a rising edge at the same time, then count once. The counting accuracy is improved.

Description

Technical Field

[0001] The present application relates to a method for automatic counting of downhole drill rods based on machine vision.

[0002] In the process of underground coal mining, in order to prevent gas leakage and spread in the coal seam, it is usually necessary to drill in the coal seam in advance to conduct gas extraction inspection to ensure the safety of the underground environment. The underground operation specifications have clear requirements for the depth of drill rod insertion. Since the length of a drill rod is usually standardized and fixed, especially far less than the required drill rod insertion depth, multiple drill rods must be driven in practice to ensure sufficient depth. In this way, there is a problem of counting the number of drill rods driven in gas extraction, so as to judge whether the intensity of gas extraction is qualified.

[0003] At present, the traditional drill rod counting method adopted underground is mostly manual counting. With the increase of working hours, miners are prone to fatigue, resulting in counting errors. The travel switch counting method is also often used for drill rod counting, but due to the vibration of the mechanical switch contacts at the moment of contact, the closure is unstable and the counting accuracy is poor. The proximity switch is better than the travel switch and can be combined with the programmable logic controller (PLC) to realize drill rod counting, but this method cannot record time information to determine whether the drilling is inserted or removed, which is easy to cause misjudgment of drill rod counting and reduce the recognition accuracy.

[0004] Perfect drill rod counting should meet the following conditions: it can automatically determine the mode of extraction or insertion, can count correctly, the number of drillings cannot be manually modified, and can be verified afterwards. Therefore, automatic counting of downhole drill rods based on machine vision should be a direction. Background Art

[0005] The process of automatic downhole drill rod counting based on machine vision is as follows: collect downhole drilling scene videos, preprocess, track moving targets based on the trained model, draw a function curve based on the tracked moving target coordinates, and obtain the number of drill rods based on the drawn curve. Video preprocessing mainly includes deframing the original video, removing similar color interference and noise reduction, and other image preprocessing operations; to remove similar color interference and noise reduction, mean filtering, median filtering, Gaussian filtering and other methods can be used. Target tracking, filter the target area image in the monitoring video, and establish the coordinate set of the target external rectangular frame. In the learning stage, it is necessary to manually annotate the drilling scene video, train the moving target tracking model based on it, and summarize the drill rod counting rules.

[0006] An invention with application number 202110103397.7 and name “A method for counting drill rods in coal mines” is characterized in that a set of coordinates of a rectangular frame circumscribing the drill rod outline in the drill rod area image is established, a quasi-inverse parabolic curve is drawn according to the coordinates, and the troughs are counted to obtain the number of drill rods.

[0007] An invention with application number 202010448311.X and name "A visual statistical method for counting downhole drill rods" is characterized in that the trained network detects the video and outputs the confidence percentage, the video output confidence is filtered using the integral method, and finally the number of unloaded rods in the video is calculated through the falling edge.

[0008] An invention with application number 202110422894.3 and name “A method for counting the number of underground drilling in coal mines based on machine vision” is characterized in that it identifies and tracks the position of the drilling rig power head and counts based on the periodicity of its movement.

[0009] An invention with application number 202110647114.5 and name "Underground Drill Rig Drill Rod Counting Method Based on Computer Vision" is characterized in that classification and regression tasks are performed on image candidate boxes and their center points, width, and height, and the inspection box with the largest confidence in the inspection box is used as the input box. The IOU values ​​of the inspection boxes that overlap with the input box and the candidate box are calculated in turn. When the calculated IOU value is greater than the set threshold, the drill rod target images that meet the requirements are filtered and screened.

[0010] An invention with application number 202110755483.6 and name "A method for counting drill rod withdrawal in drilling videos based on human posture recognition" is characterized in that it uses whether the coordinates of key points of the human skeleton coincide with the drill rod frame selection area to determine whether there is an action of moving the drill rod for counting.

[0011] The application number is 202110755589.6 and the name is "An intelligent management method for mine drilling based on video recognition". It is characterized in that the back-end server analyzes the video, and the algorithm recognizes and records the relevant information in the drilling site signboard to complete the drill rod counting.

[0012] The paper "Downhole Drill Rod Counting Method Based on Improved ResNet Network" is characterized by combining the preheating and attenuation learning rate update strategy to effectively improve the classification accuracy of the image classification model, filtering the video classification confidence through the integration method, and counting the number of falling edges of the confidence curve to achieve drill rod counting.

[0013] Many of the methods mentioned above are common, but for the target tracking part, the above methods only use drill rods or human bodies as targets for research and judgment, and the counting accuracy needs to be improved. There is still a lot of room for research on automatic downhole drill rod counting based on machine vision. Summary of the invention

[0014] Purpose of the Invention

[0015] A method for automatic counting of downhole drill rods based on machine vision is proposed, which can automatically detect the number of drill rods stably and robustly with high accuracy.

[0016] Technical Solution

[0017] A method for automatically counting downhole drill rods based on machine vision is characterized in that it includes the following steps: (1) reading downhole drilling scene video and de-framing it, and pre-processing the de-framing image to solve the problem of low image contrast; (2) using a trained target detection model to perform binary classification target detection and tracking on the pre-processed video image, and the tracking objects include drill rods and drilling rigs; (3) according to the horizontal position changes of the tracked drill rods and drilling rigs, drawing their periodic motion waveforms in the same figure, and for the rod retreat counting method, if the double waveforms of the drill rod and the drilling rig have a falling edge at the same time, then count once; for the rod advance counting method, if the double waveforms of the drill rod and the drilling rig have a rising edge at the same time, then count once.

[0018] According to the above-mentioned method for automatic counting of underground drill rods based on machine vision, due to the dim light and flying coal dust in the underground, it is easy to cause a large recognition error in the tracking target. In step (2), an improved kernel correlation filter (KCF) tracking algorithm is used to introduce a circulant matrix to generate samples, and a correlation filter is used for tracking. The grayscale features that could only be used previously were improved to multi-channel HOG features or other features, and the training data was expanded; using the circulant matrix, the image is moved up and down by different pixels to obtain a new sample image, that is, the sample after the cycle, which directly increases the number of samples, and then the new sample image is used to train the classifier to improve the training classifier effect.

[0019] According to the above-mentioned method for automatic counting of downhole drill rods based on machine vision, step (2) performs detection and tracking based on the trained model. When training the model, the center position coordinates of the drill rod and drilling rig target frame are calculated for the preprocessed video image samples, and the prediction results are converted to the original image scale as input. The data batch is traversed, and the overlap between the prediction results and the target frame is calculated, and the target is constructed using the center coordinates of the target frame.

[0020] In order to improve the detection speed, kernel functions and dual correlation filters are used to calculate the target position for tracking. New samples are directly mapped to the kernel space, and the trained function is used to calculate the function value. All positions are calculated to quickly detect the target position.

[0021] According to the method for automatic counting of downhole drill rods based on machine vision described above, it is characterized in that when analyzing the target motion in step (3), a motion waveform diagram is drawn with the starting point of the target motion of the drill rod and the drilling rig as the anchor point, the horizontal axis is the time axis (in milliseconds), and the vertical axis is the horizontal pixel. In the periodic motion waveform diagram, different identification curves are used to record the drilling rig and drill rod motion waveforms respectively. For the rod withdrawal counting mode, if the double waveforms of the drill rod and the drilling rig have a falling edge at the same time, it is counted once. For the rod advance counting mode, if the double waveforms of the drill rod and the drilling rig have a rising edge at the same time, it is counted once.

[0022] Beneficial Effects

[0023] At present, there is no literature report on counting based on the dual waveform motion model of the drill pipe and the drilling rig power head.

[0024] The method of the present application reads downhole video, and in step (2) utilizes a tracking algorithm based on an improved kernel correlation filter algorithm (KCF) for the frame selection identification part of the drill pipe target, uses a Kalman filter to solve the problem of partial personnel occlusion, and combines motion features with color features to solve the problem of similar color interference in downhole video. Compared with the traditional Camshift algorithm in other literature, it has the advantages of accurately predicting the position of the moving target even when the moving target is accelerating, narrowing the target search range, and thus improving the target tracking speed. Therefore, when downhole videos of different scenes or angles are input for learning, the target can be tracked in real time and accurately, thereby improving the counting accuracy.

[0025] The dual waveform motion model of the drill rod and the drill rig power head solves the problem of complicated underground conditions. For the situation where the drill rod image is blocked by dust or human body at a certain moment, the number of rods withdrawn is counted only when the motion waveforms of both are on the falling edge of the trough, which solves the problem of reduced accuracy due to inaccurate recognition of a single target detection frame. According to patent CN 113111805 A "A method for counting the number of drillings in coal mines based on machine vision", the recognition accuracy rate is 92.4% when the position change of the drill rig power head is recorded alone; according to patent CN 111598042 A "A visual statistical method for counting underground drill rods", the recognition accuracy rate is 97% when the position change of the drill rod is recorded alone; the method of this application combines the dual waveform motion waveforms of the drill rod and the drill rig power head, and the accuracy rate is 99.3%.

[0026] After several experiments, the method of this application not only realizes the tracking of the drill rod of the drilling rig but also can accurately calculate the number of drillings. The system displays the current number of drillings on the system interface in real time and saves the records to the database. It can stably, reliably and quickly count the drill rods in the well. It is feasible and can be applied to the drill rod counting of underground gas extraction drilling rigs. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 , block diagram of the drill rod counting method based on machine vision in this application.

[0028] Figure 2 , schematic diagram of the drill rod counting software interface based on machine vision.

[0029] Figure 3 , the moving target tracking situation is displayed after the surveillance video is transmitted.

[0030] Figure 4 , the motion waveform diagram of the drill rod and the drilling rig in the embodiment, the solid line is the drilling rig motion waveform, the dotted line is the drill rod motion waveform, and the vertical line segment is the counting mark under the rod withdrawal counting mode. DETAILED DESCRIPTION

[0031] Example. A drill rod counting software system based on machine vision was built, such as Figure 2 As shown, this application relates to the software part of the system. According to the above, the implementation steps are as follows:

[0032] (1) Read the underground video monitoring file, or directly transmit the underground drilling video from the camera. For the convenience of the experiment, the video file is directly opened. By clicking the start button, the target tracking situation can be viewed. The user can select the result display resolution and the number of frame skips, such as Figure 2 ; Filter the images of drill pipes and drilling rigs in the monitoring area, and remove similar color interference and noise from the images through histogram equalization; After the sample images are read in, they are converted into a 416*416 grid, and grayscale bars are added to prevent distortion. After that, the images are divided into three grid images (13*13, 26*26, 52*52).

[0033] (2) The trained target detection model is used to perform binary target detection and tracking on the preprocessed video images. The tracking objects include drill rods and drilling rigs. In the implementation, the image data is divided into two types of data, which are used as the input of the network for training the monitor and the network for training the re-identification. The training set and the test set are divided at a ratio of 8:2 and processed using a tracking algorithm based on the improved kernel correlation filter algorithm (KCF); the recognition rectangle of the moving target is established, and after training with the minimum loss function, the non-maximum suppression algorithm is used to filter the output multiple anchor frames, and the anchor frames with a threshold below 0.9 are filtered as wrong predictions.

[0034] (3) Extract the center point coordinate set of all targets, use the least squares method to fit the drill rod coordinate line as the horizontal motion axis, and mark it on the video image, such as Figure 3 , use different labels to draw the motion curves of the drill rod and the drilling rig respectively, the horizontal axis is the time axis (in milliseconds), and the vertical axis is the horizontal pixel, such as Figure 4; When both signals are falling edges, the count is increased by one and displayed in real time in the software display box; and the user can pause and adjust the video at any time as needed.

Claims

1. A method for automatic counting of downhole drill pipes based on machine vision, characterized in that: The method comprises the following steps: (1) reading a video of an underground drilling scene and de-framing it, and pre-processing the de-fragmented image to solve the problem of low image contrast; (2) using a trained target detection model to perform binary classification target detection and tracking on the pre-processed video image, wherein the tracking objects include a drill rod and a drill rig; (3) drawing a periodic motion waveform diagram in the same figure according to the horizontal position change of the tracked drill rod and the drill rig, and counting the rod withdrawal method once if the double waveforms of the drill rod and the drill rig have a falling edge at the same time; and counting the rod advance method once if the double waveforms of the drill rod and the drill rig have a rising edge at the same time.

2. The method for automatic counting of downhole drill pipes based on machine vision according to claim 1, characterized in that: In step (2), an improved kernel correlation filter algorithm KCF tracking algorithm is used for tracking, and the conventional single-channel grayscale features are improved into multi-channel HOG features or other features. A circulant matrix is ​​introduced to expand the samples, and the image is moved up and down by different pixels to obtain a new sample image, so as to increase the number of samples and improve the classifier training effect.

3. According to the method for automatic counting of downhole drill rods based on machine vision in claim 1, step (2) performs detection and tracking based on the trained model, characterized in that: When training the model, the center position coordinates of the drill rod and rig target box are calculated for the preprocessed video image samples, the prediction results are converted to the original image scale as input, the data batches are traversed, the overlap between the prediction results and the target box is calculated, and the target is constructed using the center coordinates of the target box.

4. The method for automatic counting of downhole drill rods based on machine vision according to claim 1, characterized in that: When analyzing the target motion in step (3), the motion waveform diagram is drawn with the starting point of the target motion of the drill rod and the drill rig as the anchor point. The horizontal axis is the time axis in milliseconds, and the vertical axis is the horizontal pixel. In the periodic motion waveform diagram, different identification curves are used to record the drilling rig and drill rod motion waveforms. For the rod withdrawal counting mode, if the double waveforms of the drill rod and the drill rig have a falling edge at the same time, it is counted once. For the rod advance counting mode, if the double waveforms of the drill rod and the drill rig have a rising edge at the same time, it is counted once.

Images