Drill rod counting method for underground drilling rigs based on computer vision

Abstract

The invention discloses a method for counting drill rods of underground drilling rigs based on computer vision, comprising the following steps: S1, obtaining drill rod image data, and dividing the drill rod image data into two types of data; S2, performing image enhancement processing on the two types of data; S3, adding Gaussian white noise of different proportions to each cleaned image; S4, locking a sample image into a 416*416 image after being read in, and each image area passes through Darknet; S5, performing classification and regression tasks on the image candidate frame and the center point, width and height of the candidate frame, and modifying the output setting of the classification task to 1; S6, minimizing the loss function through training, and outputting a plurality of detection candidate frames through the monitoring network; S7, taking the inspection frame with the largest confidence in the inspection frame as the input frame, calculating the IOU of the overlapping inspection frames in turn, filtering when the calculated IOU value is greater than the set threshold, and repeating step S7; S8, tracking and calculating the monitored target by using image classification, Kalman filtering algorithm and Hungarian algorithm.

Description

Technical Field

[0001] The invention belongs to the technical field of underground drill rod counting, and in particular relates to an underground drill rod counting method based on computer vision. Background Art

[0002] As the first step in gas extraction, drilling needs to take into account the drilling depth. Since the path of the drill bit in the coal seam is difficult to obtain, the drilling depth is generally calculated indirectly by calculating the number of drill rods to meet the drilling depth design requirements. The drilling depth is directly related to the number of drill rods. If the drilling depth is higher than the actual depth, the gas cannot be extracted. If the drilling depth is lower than the actual depth, gas leakage or even gas explosion is very likely to occur. The measurement of the drilling depth for gas extraction is basically carried out using the measurement while drilling method. The traditional method of manually counting the number of drill rods has the disadvantages of low automation, being affected by human factors, and large errors.

[0003] Therefore, coal mines will conduct drill inspections after drilling, but this method is susceptible to human influence and often results in omissions, over-recording, and wrong recording. With the development of surveillance video, it has been applied and promoted in coal mine safety production, and counting is performed by manually monitoring the video. The biggest problem with this method is that the drilling depth cannot be reported in real time, and the supervisors on the ground cannot issue a stop-work warning in time when problems occur. In addition, human energy is limited. Staring at one place for a long time will cause eye fatigue, making it impossible to achieve real-time and complete monitoring. In addition, human negligence is inevitable in a high-intensity working environment, resulting in counting errors. In order to reduce the error rate, multiple people are required to count at the same time, and a dedicated person is arranged to supervise the entire process, which wastes a lot of human resources, increases management costs, and cannot eliminate the phenomenon of false reporting of drilling depth.

[0004] In order to reduce the labor intensity and improve the accuracy, there are proximity switch counting methods, programmable logic control counting methods and computer vision-based counting methods. The first two use proximity switches, sensors and other devices to receive distance parameters and send signals, and the technical devices and controllers automatically accumulate the counts. The latter counts by intelligent computer analysis of video images.

[0005] Although the above methods can achieve the counting effect, the underground environment is harsh, humid, dusty, and the proximity switch is easily damaged, making the drill rod counting invalid. The drilling process of the drilling rig is complicated, and the drilling and drilling processes are difficult to distinguish. A large number of control and detection equipment need to be installed additionally, and the price of hardware equipment is usually expensive, which requires a large investment and is not conducive to promotion. When the drilling rig is abnormal or interrupted, there is a strong light source or occlusion on the screen, or the target moves too fast, the recognition effect of machine vision technology will be greatly reduced, which is easy to cause statistical deviations.

[0006] An identification device is set at the drill bit of the drilling rig, and an underground camera in the coal mine is used to capture the operating image of the gas extraction drilling rig. The image is transmitted to the image processing system of the industrial computer through the Ethernet ring network, so as to perform image preprocessing, feature extraction, target detection, and target tracking, and finally realize automatic counting of drill rods.

[0007] The counting effect of the existing technology is experimentally verified by videos collected in reality. It has achieved certain results in counting the drilling of the drilling rig, but has not yet fully achieved the real-time monitoring effect. When the drilling rig is abnormal or interrupted, it will cause erroneous statistics. There is no better solution for the problem of strong light source or occlusion. When the moving target is too fast, the real-time tracking effect cannot be achieved, resulting in errors in the peak value of the line graph, and then the accuracy of drilling rod statistics is reduced, and it cannot be applied to actual industrial production. Summary of the invention

[0008] The purpose of the present invention is to provide a method for counting drill rods of underground drilling rigs based on computer vision to solve or improve the above-mentioned problems in view of the above-mentioned deficiencies in the prior art.

[0009] In order to achieve the above object, the technical solution adopted by the present invention is:

[0010] A method for counting drill rods of underground drilling rigs based on computer vision comprises the following specific steps:

[0011] S1, obtaining drill pipe image data, dividing the drill pipe image data into two types of data, and using the two types of data as inputs for training a monitor network and a re-identification network respectively;

[0012] S2, perform image enhancement processing on the two types of data;

[0013] S3, adding different proportions of Gaussian white noise to each cleaned image to simulate the effects of silica and dust;

[0014] S4. After the sample image is read in, it is locked into a 416*416 image. The image includes three scales. The image is cut into 13*13, 26*26, and 52*52 areas respectively, and each area passes through Darknet;

[0015] S5, classify and regress the candidate box of whether the image contains the monitored object, as well as the center point, width, and height of the candidate box, and modify the output of the classification task to 1;

[0016] S6. By training to minimize the loss function, the monitoring network outputs multiple detection candidate boxes, and uses the non-maximum suppression algorithm to filter the candidate boxes once, and filter out the parts with probability values ​​of the target object below 0.5 as wrong predictions;

[0017] S7, taking the inspection box with the largest confidence in the inspection box as the input box, and calculating the IOU of the inspection boxes that overlap with the input box and the candidate inspection boxes in sequence. When the calculated IOU value is greater than the set threshold, filter and repeat step S7 until the remaining candidate bounding boxes are zero;

[0018] S8. Use image classification, Kalman filter algorithm and Hungarian algorithm to track and calculate the monitored target.

[0019] Furthermore, the two types of data in step S1 include: the first type corresponds to the annotation file (c, x, y, w, h), and the second type is (c, id, x, y, w, h), where c is the type, x and y are the center position coordinates of the drill rod candidate box, w and h are the width and height of the drill rod center candidate box respectively, and id is the number of the drill rod.

[0020] Furthermore, the image enhancement processing in step S2 includes: cropping, rotation, symmetry transformation, brightness transformation and saturation transformation.

[0021] Furthermore, the loss of position information in the candidate box is:

[0022]

[0023] Among them, λ coor is the weight coefficient of coordinate error, s 2 is the total number of search grids, B is the total number of detection boxes, is the existence index, which is 1 when the j-th detection box detects the target in the detector's search grid i, and 0 when it does not contain the monitored target; w i ,h i are the width and height of the detection target box, w l ,h l are the width and height of the labeled target box, x i ,y i are the center coordinates of the detection target box, x l ,y l are the center coordinates of the labeled target box respectively.

[0024] Furthermore, the loss of the category of the object in the candidate box in each region is:

[0025]

[0026] Among them, λ cls is the weight coefficient of classification error, s 2 represents the total number of search grids, B is the total number of detection boxes, is the existence index, which is 1 when the j-th detection box detects the target in the detector's search grid i, and 0 when it does not contain the monitored target; i (c) is the probability that the target is a type c object, The detected target is the true value of the c-type object.

[0027] Furthermore, the probability loss of containing the target to be detected and not containing the target to be detected in any area is:

[0028]

[0029] Among them, λ obj ,λ noobj are the weight coefficients with and without object confidence errors, respectively, s 2 is the total number of search grids, B is the total number of detection boxes, is the existence index, which is 1 when the j-th detection box detects the target in the detector's search grid i, and 0 when it does not contain the monitored target. On the contrary; i and are the predicted value and true value of the confidence level respectively.

[0030] Furthermore, in step S8, an angle classifier is used to classify the image, including:

[0031] When the detector detects a drill rod, the detection target area image is input into the classifier, and the drill rod is classified by 180 degrees to obtain the drill rod angle value θ in the scene;

[0032] When the target leaves the monitoring area of ​​the drilling rig and the drill rod angle changes by more than the threshold, it is regarded as the drilling withdrawal process, the drilling withdrawal counter increases by one, and the drilling advance counter decreases by one;

[0033] When the target is transformed from a changing angle to a relatively fixed angle value and is relatively stable in the detection area, it is regarded as a drilling process, the drilling back counter is reduced by one, and the drilling forward counter is increased by one.

[0034] Furthermore, step S8 uses Kalman filtering to predict the position and update the state of the monitored target. When the monitored target moves, the position, speed, and angle parameters of the next frame are predicted based on the target position and velocity vector monitored in the previous frame; and the predicted values ​​and observed values ​​required for the new state parameters are obtained, and the two normally distributed states are linearly weighted to obtain the state predicted by the current system.

[0035] Further, in step S8, when multiple targets are detected in the picture, the Hungarian algorithm is used to determine whether the detected target in the next frame matches the detected target in the previous frame, including:

[0036] Calculate the similarity of each detection target and construct a similarity matrix;

[0037] Add angle similarity matrix;

[0038] Calculate the IoU of the similarity matrix and the angle similarity matrix, and use the weighted sum of the two similarities as the similarity of the overall detection target:

[0039]

[0040] Similarity=λ1SM1+λ2SM2

[0041] Among them, SM1 and SM2 are the IoU similarity matrix and the angle similarity matrix respectively, λ1 and λ2 are the weighting coefficients of the IoU similarity matrix and the angle similarity matrix respectively; any element in the matrix represents the IoU similarity of object 1 and object 2 in two consecutive frames (f1, f2) Similarity with angle

[0042] Furthermore, the calculation of IoU is:

[0043]

[0044] Among them, A and B are two candidate box matrices for calculating similarity.

[0045] The computer vision-based method for counting drill rods of underground drill rigs provided by the present invention has the following beneficial effects:

[0046] 1. The drill rod counting method based on computer vision of the present invention uses the mine's own camera as the data source, and realizes the discovery, tracking and counting of drill rods through the detection algorithm, filtering algorithm, sorting algorithm, trajectory algorithm, etc. in deep learning. The on-site coal mine only needs to deploy a server containing a GPU as an edge computing device to complete the entire drill rod counting process.

[0047] 2. To avoid occupational hazards to workers, this algorithm can promptly report the number of drill rods drilled in the hole after the drilling withdrawal process in the extraction operation process is completed, thereby calculating the depth of the hole without the need for management personnel to go underground to confirm, thus avoiding occupational diseases such as pneumoconiosis and noise-induced deafness that workers often suffer from when working underground.

[0048] 3. Reduce human resource costs, reduce underground workers, reduce work processes, and improve safety. It can realize automatic calculation and analysis of the entire construction process without manual intervention, reduce labor intensity, and realize intelligent monitoring.

[0049] 4. To avoid potential safety hazards caused by fake drilling, use intelligent and unmanned means to monitor the drill rod withdrawal process, count the drill rods in real time, and ensure that the time of each drill rod withdrawal can be checked.

[0050] 5. The present invention is based on the original camera equipment in the mine, and there is no need to invest in new equipment, which reduces the cost of the mine and improves the utilization rate of equipment. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 The flowchart of the method for counting drill rods of underground drilling rigs based on computer vision is shown in the figure. DETAILED DESCRIPTION

[0052] The specific implementation modes of the present invention are described below so that those skilled in the art can understand the present invention. However, it should be clear that the present invention is not limited to the scope of the specific implementation modes. For those of ordinary skill in the art, as long as various changes are within the spirit and scope of the present invention as defined and determined by the attached claims, these changes are obvious, and all inventions and creations utilizing the concept of the present invention are protected.

[0053] Example 1, reference Figure 1 The computer vision-based method for counting drill rods of underground drilling rigs in this scheme specifically includes the following steps:

[0054] Step S1, obtaining drill pipe image data, dividing the drill pipe image data into two types of data, and using the two types of data as inputs for training a monitor network and a re-identification network respectively;

[0055] Step S2, performing image enhancement processing on the two types of data;

[0056] Step S3, adding different proportions of Gaussian white noise to each cleaned image to simulate the impact of silica and dust;

[0057] Step S4: After the sample image is read in, it is locked into a 416*416 image. The image includes three scales. The image is cut into 13*13, 26*26, and 52*52 areas respectively, and each area passes through Darknet;

[0058] Step S5, classify and regress the candidate box of whether the image contains the monitored object, as well as the center point, width, and height of the candidate box, and modify the output setting of the classification task to 1;

[0059] Step S6: By training to minimize the loss function, the monitoring network outputs multiple detection candidate boxes, and the non-maximum suppression algorithm is used to filter the candidate boxes once, and the parts with probability values ​​of containing target objects below 0.5 are filtered out as wrong predictions;

[0060] Step S7, taking the inspection box with the largest confidence in the inspection box as the input box, and calculating the IOU of the inspection boxes that overlap with the input box and the candidate inspection boxes in sequence. When the calculated IOU value is greater than the set threshold, filter and repeat step S7 until the remaining candidate bounding boxes are zero;

[0061] Step S8: Use image classification, Kalman filter algorithm and Hungarian algorithm to track and calculate the monitored target.

[0062] The measurement of gas extraction borehole depth is an important part of the safety production regulations in coal mines. However, due to some human factors, underground workers sometimes fail to operate correctly according to the safety production regulations, which poses a huge threat to their own life safety. At the same time, the existing technology mainly relies on manual counting and manual video monitoring for test counting, which is prone to counting errors and has a high probability of error.

[0063] The present invention adopts innovative computer vision technology and the calculation method of steps S1 to S8, which can accurately identify the drilling process of directional drilling rigs and ordinary drilling rigs in the complex environment of the construction site, and accurately count without repeated calculations or omissions, thereby avoiding illegal operations and poor supervision caused by human factors, and providing basic protection for the life safety of underground workers.

[0064] Example 2, reference Figure 1 , this embodiment will describe the first embodiment in detail;

[0065] This implementation uses cameras covering underground coal mines as image data sources and runs the entire calculation process in the server room. It can realize a parallel processing structure of multiple algorithms for one camera and has the following functional features:

[0066] First, through effective data enhancement methods, the reliability of low-quality image data caused by the dark underground working environment is greatly improved.

[0067] Second, by transferring the idea of ​​pedestrian re-identification, the drill rods are re-identified and each drill rod is assigned an independent and unique mark to facilitate subsequent counting.

[0068] Due to the complex and changeable working environment in the mine, the picture is not clear in many cases, and a large amount of flying catkins appear, which seriously affects the detector effect. Flying catkins are mainly composed of free silica and dust. The silica concentration range is concentrated between 2.3% and 6.6%, which is particularly obvious when the camera is in night mode. The concentration of dust varies greatly according to the particle size. The concentration of small particles is between 50% and 70%, and that of large particles is around 50%. In extreme cases, these particles are like sandstorms, which can cause extremely severe damage to the camera image data.

[0069] This patent uses Python as the development language and is based on the deep learning framework Pytorch. It effectively realizes the tracking and counting of drill rods when the picture is unclear. The specific process is divided into the steps of identification, filtering, sorting and counting of drill rods, which specifically includes the following steps:

[0070] Step S1, obtaining drill pipe image data, dividing the drill pipe image data into two types of data, and using the two types of data as inputs for training a monitor network and a re-identification network, respectively.

[0071] The first type corresponds to the annotation file (c, x, y, w, h), and the second type corresponds to (c, id, x, y, w, h). c is the type, x, y are the center coordinates of the drill rod candidate box, w, h are the width and height of the drill rod center candidate box, and id is the drill rod number.

[0072] Step S2: For the two types of sorted data, image data enhancement processing such as cropping, rotation, symmetry transformation, brightness transformation, and saturation transformation is performed.

[0073] Step S3: Add different proportions of Gaussian white noise to each cleaned image to simulate the impact of silica and dust to ensure the robustness of the model after complex scene transformation.

[0074] Step S4: After the sample image is read in, it is locked as a 416*416 image. The image is divided into three scales, and the image is cut into 13*13, 26*26, and 52*52 areas respectively, and each area is passed through Darknet.

[0075] Step S5: classify and regress the candidate box of whether the image contains the monitored object, as well as the center point, width, and height of the candidate box, and modify the output setting of the classification task to 1.

[0076] Among them, the loss of position information in the candidate box is:

[0077]

[0078] The loss of the category of the object in the candidate box in each region is:

[0079]

[0080] The probability loss of any area containing the target to be detected and not containing the target to be detected is:

[0081]

[0082] Step S6: By training to minimize the loss function, the monitoring network outputs multiple detection candidate boxes, and the non-maximum suppression algorithm is used to filter the candidate boxes once, and the parts with probability values ​​containing target objects below 0.5 are filtered out as wrong predictions.

[0083] Step S7: Take the inspection box with the largest confidence (probability of the existence of the target object) in the inspection box as the input box, and calculate the IOU with the inspection boxes that overlap with it in other candidate inspection boxes in turn. When the IOU value is greater than the set threshold (such as 0.5), filter it. Repeat this step until the other candidate bounding boxes are zero.

[0084] Step S8: Use image classification, Kalman filter algorithm and Hungarian algorithm to track and calculate the monitored target.

[0085] Among them, image classification is a special application method of the classification algorithm added for drill rod counting. The existing drill rod counting process has the following difficulties after the detector works:

[0086] First, miners often withdraw several drill rods and then drive in several more during operation.

[0087] Second, the drill rig may experience an abnormal drill jam during the drill withdrawal process. During this period, the drill rod will move back and forth in the screen, which will have a great impact on the counting judgment.

[0088] Third, visibility in the mine is not high. The spotlights on the miners' helmets often swing towards the camera. The resulting halo will affect the detector, which may cause problems such as loss of identification tags, false identification, and no identification. If the number of frames when the detection target disappears is too high, the tracking target will be lost directly.

[0089] In response to these three problems, this solution designs an angle classifier, and its specific methods include:

[0090] 1. When the detector detects a drill rod, the detection target area image is input into the classifier, the drill rod is classified into 180 angles, and the drill rod angle value θ in this scene is obtained.

[0091] 2. When the target leaves the monitoring area of ​​the drilling rig and the drill rod angle changes significantly, it is regarded as the drilling withdrawal process, the drilling withdrawal counter increases by one, and the drilling advance counter decreases by one.

[0092] 3. When the target changes from a changing angle to a relatively fixed angle value and is relatively stable in the detection area, it is regarded as a drilling process, the drilling back counter decreases by one, and the drilling forward counter increases by one.

[0093] The Kalman filter algorithm is used to predict the position and update the state of the monitored target. When the monitored target is moving, the target position and velocity vector monitored in the previous frame are used to predict its possible position, velocity, angle and other parameters in the next frame. The predicted value and observed value required for the new state parameters are obtained, and the two normally distributed states are linearly weighted to obtain the state predicted by the current system.

[0094] The Hungarian algorithm is used to solve the problem of matching the detected targets in the next frame with the detected targets in the previous frame when multiple targets are detected in the picture.

[0095] 1. Establish a similarity matrix by calculating the similarity of each detection target.

[0096] 2. Add the angle similarity matrix.

[0097] 3. Calculate the IoU (intersection over union) of the two matrices, and use the weighted similarities of the two as the similarity of the overall detection target.

[0098]

[0099] Similarity=λ1SM1+λ2SM2

[0100] Among them, the calculation of IoU is:

[0101]

[0102] Among them, A and B are two candidate box matrices whose similarity is to be calculated.

[0103] Compared with traditional tracking and matching, the matching result of this scheme is improved from the original 70% to 87%, which greatly improves the tracking efficiency.

[0104] Embodiment 3: This embodiment is described using a specific case.

[0105] 1. Collect 5,000 images of drill rods in the mine, and the data enhancement methods are symmetric inversion, center interception, rotation, and grayscale transformation. The number of enhanced images reaches 50,000. 70% of them are used for training sets and 15% for validation sets. The models that pass the validation will be tested on the remaining 15% test set.

[0106] 2. Each drill rod image in the data set is intercepted to generate a classification data set. According to the drill rod angle, the data is divided into 180 categories representing the drill rod status from 0° to 179°. ResNet18 is used to train the classification data set, 80% of which is used as a training set and 20% as a test set. The classification model is saved as best_classify.pth.

[0107] 3. All images are transformed into 416*416 pixels before being input into the model, and the model training is accelerated by opening the CUDA Benchmark. The image is passed through the deep neural network Darknet to extract the characteristic information of the drill pipe.

[0108] 4. The obtained drill pipe characteristic information is used to obtain the angle characteristic value θ through the angle classifier.

[0109] 5. The feature information between the drill rods in two adjacent frames of images will be used to predict their moving speed and direction through filters. At the same time, the IoU similarity and angle transformation similarity are calculated to determine their matching status and to track the drill rods in real time.

[0110] 6. A total of 99 cycles are trained, each batch is 4 samples, and the trained model is saved as best_model.pth. The model can be used for inference process.

[0111] 7. Set the judgment box. When the drill rod moves out of the judgment box, it is counted as drilling back and the count is increased by 1. When the drill rod enters the judgment box from outside the judgment area, it is counted as drilling forward and the count is decreased by 1.

[0112] Embodiment 4: This embodiment uses a specific case to illustrate, and the model training process is the same as that of Embodiment 3;

[0113] When drill rod re-identification problems occur, the angle classifier and angle similarity calculation matrix can largely solve the matching problem, thereby reducing the number of duplicate counts. However, in actual applications, there is a certain probability that the miner's headlight will directly shine on the camera, causing different degrees of halo, resulting in a high probability of loss of the tracking target or detector failure.

[0114] In order to solve this problem, a halo detector is set up through Opencv, and a halo filter is performed before each frame of image is input into the model. The principle of filtering is that when the halo is detected, the lost pixels caused by the halo are supplemented by the previous frame, which can greatly reduce the image quality damage caused by the halo.

[0115] Although the specific implementation of the invention is described in detail in conjunction with the drawings, it should not be understood as limiting the scope of protection of this patent. Within the scope described in the claims, various modifications and variations that can be made by those skilled in the art without creative work still fall within the scope of protection of this patent.

Claims

1. A method for counting drill rods of underground drilling rigs based on computer vision, characterized in that: The following steps are involved: S1, obtaining drill pipe image data, dividing the drill pipe image data into two types of data, and using the two types of data as inputs for training a monitor network and a re-identification network respectively; S2, perform image enhancement processing on the two types of data; S3, adding different proportions of Gaussian white noise to each cleaned image to simulate the effects of silica and dust; S4. After the sample image is read in, it is locked into a 416*416 image. The image includes three scales. The image is cut into 13*13, 26*26, and 52*52 areas respectively, and each area passes through Darknet; S5, classify and regress the candidate box of whether the image contains the monitored object, as well as the center point, width, and height of the candidate box, and modify the output of the classification task to 1; S6. By training to minimize the loss function, the monitoring network outputs multiple detection candidate boxes, and uses the non-maximum suppression algorithm to filter the candidate boxes once, and filter out the parts with probability values ​​of the target object below 0.5 as wrong predictions; S7, taking the inspection box with the largest confidence in the inspection box as the input box, and calculating the IOU of the inspection boxes that overlap with the input box and the candidate inspection boxes in sequence. When the calculated IOU value is greater than the set threshold, filter and repeat step S7 until the remaining candidate bounding boxes are zero; S8, using image classification, Kalman filter algorithm and Hungarian algorithm to track and calculate the monitored target; In step S8, an angle classifier is used to classify the image, including: When the detector detects a drill rod, the detection target area image is input into the classifier, and the drill rod is classified by 180 degrees to obtain the drill rod angle value θ in the scene; When the target leaves the monitoring area of ​​the drilling rig and the drill rod angle changes by more than the threshold, it is regarded as the drilling withdrawal process, the drilling withdrawal counter increases by one, and the drilling advance counter decreases by one; When the target changes from a changing angle to a relatively fixed angle value and is relatively stable in the detection area, it is considered as a drilling process, the drilling back counter decreases by one, and the drilling forward counter increases by one; Step S8 uses Kalman filtering to predict the position and update the state of the monitored target. When the monitored target moves, the position, speed, and angle parameters of the next frame are predicted based on the target position and speed vector monitored in the previous frame; and the predicted values ​​and observed values ​​required for the new state parameters are obtained, and the states of the two normal distributions are linearly weighted to obtain the state predicted by the current system; In step S8, when multiple targets are detected in the picture, the Hungarian algorithm is used to determine whether the detected target in the next frame matches the detected target in the previous frame, including: Calculate the similarity of each detection target and construct a similarity matrix; Add angle similarity matrix; Calculate the IoU of the similarity matrix and the angle similarity matrix, and use the weighted sum of the two similarities as the similarity of the overall detection target: Similarity=λ1SM1+λ2SM2 Among them, SM1 and SM2 are the IoU similarity matrix and the angle similarity matrix respectively, λ1 and λ2 are the weighting coefficients of the IoU similarity matrix and the angle similarity matrix respectively; any element in the matrix represents the IoU similarity of object 1 and object 2 in two consecutive frames (f1, f2) Similarity with angle 2. The method for counting drill rods of underground drilling rigs based on computer vision according to claim 1, characterized in that: The two types of data in step S1 include: the first type corresponds to the annotation file (c, x, y, w, h), and the second type is (c, id, x, y, w, h), where c is the type, x and y are the center position coordinates of the drill rod candidate box, w and h are the width and height of the drill rod center candidate box respectively, and id is the number of the drill rod.

3. The method for counting drill rods of underground drilling rigs based on computer vision according to claim 1, characterized in that: The image enhancement process in step S2 includes: interception, rotation, symmetry transformation, brightness transformation and saturation transformation.

4. The method for counting drill rods of underground drilling rigs based on computer vision according to claim 1, characterized in that: The loss of position information in the candidate frame is: Among them, λ coor is the weight coefficient of coordinate error, s 2 is the total number of search grids, B is the total number of detection boxes, is the existence index, which is 1 when the j-th detection box detects the target in the detector's search grid i, and 0 when it does not contain the monitored target; w i ,h i are the width and height of the detection target box, w l ,h l are the width and height of the labeled target box, x i ,y i are the center coordinates of the detection target box, x l ,y l are the center coordinates of the labeled target box respectively.

5. The method for counting drill rods of underground drilling rigs based on computer vision according to claim 1, characterized in that: The loss of the category of the object in the candidate box in each region is: Among them, λ cls is the weight coefficient of classification error, s 2 represents the total number of search grids, B is the total number of detection boxes, is the existence index, which is 1 when the j-th detection box detects the target in the detector's search grid i, and 0 when it does not contain the monitored target; i (c) is the probability that the target is a type c object, The detected target is the true value of the c-type object.

6. The method for counting drill rods of underground drilling rigs based on computer vision according to claim 1, characterized in that: The probability loss of any area containing the target to be detected and not containing the target to be detected is: Among them, λ obj ,λ noobj are the weight coefficients with and without object confidence errors, respectively, s 2 is the total number of search grids, B is the total number of detection boxes, is the existence index, which is 1 when the j-th detection box detects the target in the detector's search grid i, and 0 when it does not contain the monitored target. On the contrary; i and are the predicted value and true value of the confidence level respectively.

7. The method for counting drill rods of underground drilling rigs based on computer vision according to claim 1, characterized in that: The calculation of IoU is: Among them, A and B are two candidate box matrices for calculating similarity.

Images