Coal mine drilling machine drill rod target detection and movement analysis counting method and system and medium

Abstract

The invention provides a coal mine drilling machine drill rod target detection and movement analysis counting method and system and a medium, and the method comprises the steps: collecting an underground drilling video, segmenting frames, and marking the whole drilling machine and a tail boundary frame; a multi-level feature extraction module MCFN is added in front of a YOLO model C2F module, and a detection model is trained and optimized; the whole drilling machine and a tail frame are detected frame by frame, four areas are divided based on the center of the whole frame, the overlapping area of the tail frame and two diagonal areas is calculated, and the high-frequency direction is counted through a sliding window to determine the working direction; dynamically adjusting a forward / backward area ratio according to a tail / overall frame area ratio, and diagonally dividing a dynamic area along a working direction; the tail is initialized to be in a backward state, a forward / backward counter is triggered through the center point position of the tail, and when a certain counter reaches a threshold value, the moving state is switched; when the state is switched from advancing to retreating, the number of the drill rods is plus 1, and the video is circularly processed until the operation is finished. According to the method, multi-stage features and dynamic region division are fused, and accurate counting of the drill rods in the underground complex environment is achieved.

Description

Technical Field

[0001] The present invention relates to the technical fields of computer vision and industrial automation, and particularly to a method, system and medium for target detection, movement analysis and counting of drill pipes of a coal mine drill rig. Background Art

[0002] Coal mine gas accidents are a very serious disaster threatening the safe production of coal mines, accounting for a relatively high proportion in major coal mine safety accidents. Gas is mainly stored in coal seams and is released during the coal seam mining process. Due to the narrow underground working environment, once the gas concentration reaches the explosion critical value, it is extremely easy to trigger an explosion accident, causing great harm to the lives of miners and the production of coal mines. In order to effectively prevent and control gas disasters, coal mines usually adopt the method of pre-drilling and pumping, and extract coal seam gas through drilling before coal mining to reduce the gas content and pressure and reduce the risk of gas outburst. The depth of the drill hole has an important impact on the gas extraction effect. In actual drilling operations, the drill rig drills by repeatedly inserting and pulling out drill pipes. As the drilling work continues, the number of drill pipes gradually increases. Since the length of each drill pipe is the same, the drill hole depth can be indirectly calculated by counting the number of drill pipes. To ensure that the drill hole depth meets the requirements during the extraction process, the number of drill pipes is often used as the main indicator for counting the drill hole depth. Accurately counting the number of drill pipes is of great significance for coal mine gas control, production management and safety guarantee.

[0003] Traditional methods for counting drill pipes of coal mine underground drill rigs mainly include manual counting method, instrument counting method, stress wave method, electric standing wave method, low-voltage pulse reflection method, etc. The manual counting method is for staff to observe and record the number of drill pipes on site. However, the underground working environment has insufficient light, serious dust and high noise. Long-term continuous operation is likely to cause fatigue of the staff, thus leading to misdetection or missed detection. Traditional instrument counting methods and signal detection methods also have limitations. Although the instrument counting method based on laser sensors has a high degree of automation, laser devices are easily damaged in the coal mine dust and humid environment and have large errors; the signal detection methods based on the stress wave or standing wave principle rely on complex signal processing and have potential safety hazards in the explosive gas environment of coal mine underground, making it difficult to meet the actual application requirements.

[0004] In recent years, with the advancement of coal mine intelligent construction, computer vision and deep learning technologies have been gradually applied to coal mine production. The drill pipe counting method based on computer vision realizes automatic counting by analyzing the video data of the drilling rig, which not only reduces manual intervention but also significantly improves the counting accuracy and efficiency. Gao Rui et al. proposed a downhole drill pipe counting method based on an improved ResNet network, which identifies the drill pipe unloading action of the drilling rig and counts by statistically analyzing the number of times the confidence curve drops. However, this method lacks robustness in practical applications and cannot identify changes in the number of drill pipes during the drilling process. Du Jingyi et al. designed a counting method based on an improved spatio-temporal graph convolutional neural network, which extracts human key points through the Alphapose algorithm, identifies action categories, and counts the number of actions. However, this method is prone to missing counts, resulting in deviation of the counting results. The existing methods have poor adaptability to the complex environment of coal mine underground and cannot meet the requirements of high precision, high robustness, and real-time performance, making it difficult to adapt to the needs of coal mine underground operation scenarios.

[0005] On the one hand, due to the harsh working environment in coal mine underground, factors such as low illuminance and high dust significantly reduce the visual quality, and the target features seriously degenerate, resulting in poor performance of traditional object detection models. How to extract target features at different levels with high quality is crucial for improving detection performance. At the same time, the underground video may experience phenomena such as jitter and blurring, leading to short-term drift in object detection, that is, the position of the detection box of the same target mutates in consecutive frames. This jumping change is easily misjudged as an actual change in the target position, resulting in counting errors. On the other hand, the working environment in coal mine underground is complex and changeable. There may be significant differences in the installation angle of cameras and the placement direction of drilling rigs in different scenarios. Traditional methods usually rely on fixed detection directions and detection area divisions and cannot be adjusted according to the dynamic changes of the actual scenario, limiting their applicability in complex mine environments. Summary of the Invention

[0006] To solve the above problems, the present invention aims to propose a method, system, and medium for target detection and movement analysis counting of drill pipes of a coal mine drilling rig, which collect and label the working video frames of the underground drilling rig, construct an improved object detection model integrating a multi-level feature extraction module (MCFN); detect the overall and tail detection frames of the drilling rig frame by frame based on the video, determine the working direction of the drilling rig through overlapping area analysis and sliding window statistics; dynamically adjust the region division ratio in combination with the detection frame area ratio, design a finite state machine to track the movement state of the tail, and trigger the counting logic when the movement state of the tail switches from forward to backward, ultimately achieving high-precision drill pipe counting in a complex underground environment.

[0007] To achieve the above object, the technical solution of the present invention is realized as follows: A method for target detection and movement analysis counting of drill pipes of a coal mine drilling rig, comprising the following steps: Step 1: Collect the actual downhole drilling operation video stream, split the video into individual image frames, and use a professional annotation tool to accurately annotate the bounding boxes of the entire drill rig and the tail of the drill rig; Step 2: Based on the pre-trained object detection model, add a multi-level feature extraction module MCFN before the C2F residual module in its feature extraction module; use the annotation data obtained in Step 1 to train the model and optimize the model's recognition ability for the entire drill rig and the tail; Step 3: Use the best object detection model trained in Step 2 to detect each frame of the video and extract the object detection bounding boxes of the entire drill rig and the tail; divide the object detection bounding box of the entire drill rig from the center into four regions: upper left, upper right, lower left, and lower right, and generate two diagonal directions: the direction from the lower right to the upper left and the direction from the lower left to the upper right; calculate the overlapping area between the object detection bounding box of the drill rig tail and the two direction regions, and select the direction with the larger overlapping area as the reference direction; perform a sliding window statistics on the directions of the first few frames of the video, and select the direction with the highest frequency as the working direction of the drill rig; Step 4: Calculate the area ratio between the detection bounding box of the drill rig tail and the detection bounding box of the entire drill rig, and dynamically adjust the ratio of the forward and backward regions according to the area ratio; based on the working direction determined in Step 3, divide the forward and backward regions along the corresponding diagonal of the object detection bounding box of the entire drill rig; Step 5: Initialize the moving state of the drill rig tail to the backward state, and set the forward and backward counters, and initialize them to 0; according to the relationship between the center point position of the object detection bounding box of the drill rig tail and the divided regions, if the center point hits a certain region, increase the corresponding forward or backward counter, and reset the other counter to zero; if a certain counter reaches the set threshold, update the moving state and reset the two counters; Step 6: Continuously process the drilling video of the drill rig in the coal mine. When the moving state of the drill rig tail switches from forward to backward, increment the drill pipe count by one; continuously accumulate the drill pipe count according to the above rules until the drilling operation is completed, and finally obtain the drill pipe count result of the drill rig.

[0008] Further, the professional annotation tool in Step 1 is LabelImage.

[0009] Further, the selected object detection model in Step 2 is the Yolov8 network that has been trained on the large public dataset ImageNet; introduce the multi-level feature extraction module MCFN, which is used to aggregate local and global features through multi-scale representation learning and extract local information from different receptive fields through multi-core convolution operations.

[0010] Further, the specific working method of the multi-level feature extraction module MCFN is as follows: Before the C2F module in the feature extraction framework of the basic object detection model, assume that the current input feature is , where C is the number of channels, and H and W are the height and width of the feature map; through point convolution, the number of channels is expanded from C to 2C to generate the expanded feature , and the calculation formula is as follows: (1) where is the point convolution with parameter , that is, 1×1 convolution; The expanded feature tensor can prepare a richer representation space for subsequent multi-scale convolution and enhance the representation ability of the model; in the multi-core convolution stage, the expanded feature is divided into multiple parts, and the features of each part are respectively passed through convolution kernels of different sizes to capture multi-scale local information; assuming channels are divided into n parts, and the calculation formula is as follows: (2) where represents the i-th feature part of the division, ; Apply depthwise separable convolution kernels of different sizes to each feature part, including at least 1×1, 3×3, 5×5..., and the calculation formula is as follows: (3) where represents the depthwise separable convolution with a convolution kernel size of , represents the size of the i-th convolution kernel, represents the feature of the corresponding scale obtained after depthwise separable convolution; Aggregate the outputs of all branches through channel concatenation, and the calculation formula is as follows: (4) where Concat represents the channel concatenation operation, represents the aggregated feature; Use point convolution to reduce the dimension of the aggregated feature , restore the number of channels from 2C to C, and introduce non-linearity using the GELU activation function. The calculation formula is as follows: (5) where is the feature after being processed by the multi-level feature extraction module, is the point convolution with parameter .

[0011] Further, the specific process of step 3 is as follows: Use the improved object detection model trained in step 2 to process the video frame to obtain the detection frames of the overall drill rig and the tail of the drill rig; Assume that the video frame at time t is , and the obtained detection frame of the overall drill rig is , and the detection frame of the tail of the drill rig is , and the calculation formula is as follows: (6) where MCFN-Yolov8 is the improved object detection model trained in step 2; Divide the detection frame of the overall drill rig starting from the center into four regions: upper left, upper right, lower left, and lower right, which are respectively , , and ; Generate two diagonal directions, which are respectively and , where represents the direction from lower left to upper right, and represents the direction from lower right to upper left; Calculate the overlapping areas between the detection frame of the tail of the drill rig and the four regions , , and , and the specific calculation formula is as follows: (7) (8) (9) (10) where Area represents the area of the region, and represents taking the intersection; Calculate the total overlapping areas and in the two directions, and the calculation formula is as follows: (11) (12) where + represents adding the areas; Select the direction with the larger overlapping area as the reference direction of the current frame , and the calculation formula is as follows: (13) where if represents if, and otherwise represents otherwise; Perform a sliding window statistics on the directions within the first n video frames, and select the direction with the highest frequency as the final working direction of the drill rig. , and the calculation formula is as follows: (14) Where, Count represents the counting of the directions that appear in the first n video frames, and argmax represents selecting the direction with the highest frequency.

[0012] Furthermore, the specific process of step 4 is as follows: Calculate the area ratio of the detection frame at the tail of the drill rig and the overall detection frame of the drill rig , and the calculation formula is as follows: (15) Where, and represent the area of the detection frame at the tail of the drill rig and the area of the overall detection frame of the drill rig; Dynamically adjust the proportion of the forward and backward areas according to the area ratio, and the calculation formula is as follows: (16) (17) (18) Where, is the proportion of the backward area, is the proportion of the forward area, is the proportion of the buffer area, and are the weights for dynamic adjustment.

[0013] Based on the final direction determined in step 3, divide the forward area and the backward area .

[0014] Furthermore, the specific process of step 5 is as follows: Initialize the moving state of the drill rig tail to the backward state, and the calculation formula is as follows: (19) Where, is the moving state of the drill rig tail, and Backward represents the backward state; And initialize the forward counter and the backward counter to 0; According to the relationship between the center point position of the detection frame at the drill rig tail and the areas divided in step 4, if the center point hits a certain area, then increase the forward or backward counter accordingly, and the calculation formula is as follows: (20) (21) Among them, indicates that the center point position belongs to the specified area; If a certain counter reaches the set threshold, update the moving state and clear the counter. The calculation formula is as follows: (22) (23) Among them, Forward represents the forward state, and threshold represents the set threshold.

[0015] Further, the specific process of step 6 is as follows: Continuously process the drilling video of the coal mine underground drill. Initially, set the counting result to 0; when the moving state of the drill tail switches from the forward state to the backward state, increment the drill pipe count. The calculation formula is as follows: (24) Among them, and means that both conditions are satisfied simultaneously.

[0016] To achieve the above object, the present invention also provides a coal mine drill pipe target detection and movement analysis counting system, including: Data acquisition and annotation module: used to collect the actual underground drilling operation video stream, split the video into individual image frames, and use professional annotation tools to accurately annotate the bounding boxes of the entire drill and the drill tail; Target detection model module: Based on the pre-trained target detection model, add a multi-level feature extraction module MCFN before the C2F residual module of its feature extraction module; use the annotation data obtained in the data acquisition and annotation module to train the model to optimize the model's recognition ability of the entire drill and the tail; Working direction determination module: Use the best target detection model trained by the target detection model module to perform frame-by-frame detection on the video frames, extract the target detection bounding boxes of the entire drill and the tail; divide the target detection bounding box of the entire drill from the center into four regions: upper left, upper right, lower left, and lower right, and generate two diagonal directions: the direction from the lower right to the upper left and the direction from the lower left to the upper right; calculate the overlapping area between the target detection bounding box of the drill tail and the two direction regions, and select the direction with the larger overlapping area as the reference direction; perform a sliding window statistics on the directions of the first several frames of the video, and select the direction with the highest frequency as the working direction of the drill; Dynamic area division module: used to calculate the area ratio between the detection bounding box of the drill tail and the detection bounding box of the entire drill, and dynamically adjust the ratio of the forward and backward areas according to the area ratio; based on the working direction determined by the working direction determination module, divide the forward and backward areas along the corresponding diagonal of the detection bounding box of the entire drill; State machine module: used to initialize the moving state of the drill rig tail to the backward state, set the forward and backward counters, and initialize them to 0; according to the relationship between the center point position of the detection frame at the drill rig tail and the divided areas, if the center point hits a certain area, the corresponding forward or backward counter is incremented, and the other counter is reset to zero; if a certain counter reaches the set threshold, the moving state is updated, and the two counters are reset. Drill pipe counting module: used to continuously process the drilling video of the coal mine underground drill rig. When the moving state of the drill rig tail switches from forward to backward, the drill pipe count is incremented by one; the drill pipe count is continuously accumulated according to the above rules until the drilling operation is completed, and finally the drill pipe count result of the drill rig is obtained.

[0017] To achieve the above object, the present invention also provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, the processor is caused to execute the coal mine drill rig drill pipe target detection and movement analysis counting method as described above.

[0018] Beneficial effects: The present invention collects and annotates the working video frames of the underground drill rig, constructs an improved target detection model integrating a multi-level feature extraction module (MCFN); based on frame-by-frame detection of the video, the overall and tail detection frames of the drill rig are obtained, and the working direction of the drill rig is determined through overlapping area analysis and sliding window statistics; the region division ratio is dynamically adjusted in combination with the detection frame area ratio, and a finite state machine is designed to track the moving state of the tail. When the moving state of the tail switches from forward to backward, the counting logic is triggered, and finally high-precision drill pipe counting in a complex underground environment is achieved. Description of the Drawings

[0019] The drawings constituting a part of the present invention are used to provide a further understanding of the present invention. The schematic embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation to the present invention. In the drawings: Figure 1 is the main flowchart of the coal mine drill rig drill pipe target detection and movement analysis counting method described in the embodiments of the present invention; Figure 2 is the algorithm flowchart of the coal mine drill rig drill pipe target detection and movement analysis counting method described in the embodiments of the present invention; Figure 3 is the schematic diagram of the detection screen of the coal mine drill rig drill pipe target detection and movement analysis counting method described in the embodiments of the present invention; Figure 4 is the structural schematic diagram of the coal mine drill rig drill pipe target detection and movement analysis counting system described in the embodiments of the present invention. Detailed Embodiments

[0020] It should be noted that, without conflict, the embodiments in the present invention and the features in the embodiments can be combined with each other.

[0021] The present invention will be described in detail below with reference to the accompanying drawings and in conjunction with embodiments.

[0022] Embodiment 1 See Figures 1-3 : A method for target detection, movement analysis and counting of drill pipes of a coal mine drill rig, comprising the following steps: Step 1: Collect the actual downhole drilling operation video stream, segment the video into individual image frames, and use a professional annotation tool to accurately annotate the bounding boxes of the entire drill rig and the tail of the drill rig. Step 2: Based on a pre-trained target detection model, add a multi-level feature extraction module MCFN before the C2F residual module (Coarse-to-Fine Residual Module) in its feature extraction module; use the annotation data obtained in Step 1 to train the model to optimize the model's recognition ability for the entire drill rig and the tail. Step 3: Use the best target detection model trained in Step 2 to detect each frame of the video, and extract the target detection bounding boxes of the entire drill rig and the tail; divide the target detection bounding box of the entire drill rig from the center into four regions: upper left, upper right, lower left, and lower right, and generate two diagonal directions: the direction from lower right to upper left and the direction from lower left to upper right; calculate the overlapping area between the target detection bounding box of the drill rig tail and the two direction regions, and select the direction with the larger overlapping area as the reference direction; perform a sliding window statistics on the directions of several initial frames of the video, and select the direction with the highest frequency as the working direction of the drill rig. Step 4: Calculate the area ratio between the detection bounding box of the drill rig tail and the detection bounding box of the entire drill rig, and dynamically adjust the forward and backward region ratios according to the area ratio; based on the working direction determined in Step 3, divide the forward and backward regions along the corresponding diagonal of the detection bounding box of the entire drill rig. Step 5: Initialize the movement state of the drill rig tail as the backward state, and set the forward and backward counters, and initialize them to 0; according to the relationship between the center point position of the detection bounding box of the drill rig tail and the divided regions, if the center point hits a certain region, increase the corresponding forward or backward counter, and reset the other counter to zero; if a certain counter reaches the set threshold, update the movement state and reset the two counters. Step 6: Continuously process the drill hole video of the coal mine downhole drill rig. When the movement state of the drill rig tail switches from forward to backward, increment the drill pipe count by one; continuously accumulate the drill pipe count according to the above rules until the drill hole operation is completed, and finally obtain the drill pipe count result of the drill rig.

[0023] In this embodiment, the multi-level feature extraction module (MCFN) is used to fuse shallow texture and deep semantic features, significantly improving the detection accuracy of the drill rig tail in low-light and high-dust scenarios (the mAP@0.5 reaches 90.6%), and solving the problem of missed detection caused by uneven underground light in traditional models and the anti-interference detection ability; the dynamic region division mechanism in this embodiment adjusts the region ratio in real time according to the area ratio of the tail to the overall detection frame, adapting to the changes in the posture of the drill rig (such as tilting and jittering), and avoiding misjudgment caused by fixed thresholds; the sliding window statistical strategy in this embodiment performs probability screening on the initial frame direction, eliminating the influence of instantaneous interference (such as drill pipe jamming) on direction determination, and greatly improving the direction recognition accuracy; the direction selection based on the intersection over union (IoU) rather than a single coordinate enhances the tolerance to target deformation (such as tail occlusion); this embodiment uses a forward / backward counter threshold mechanism, requiring multiple consecutive frames (such as 5 frames) to confirm the state switch, effectively suppressing the instantaneous state jump caused by vibration noise, and the error rate is greatly reduced in the measured vibration scenario; the state reset rule (counter reset to zero) prevents the accumulation of accidental interference, ensuring that the state switch is triggered only by valid actions.

[0024] In a specific example, the professional annotation tool in step 1 is LabelImage.

[0025] It should be noted that LabelImage, as a lightweight image annotation tool, demonstrates the following core advantages in object detection tasks; for example: it supports batch loading of images and automatically saves the annotation results to XML files, reducing manual frequent operations and improving the annotation efficiency; it focuses on rectangular box annotation, meeting the annotation requirements of the regular geometric features of drill rig components (such as the drill pipe tail), and is more efficient than polygon tools (such as LabelMe); LabelImage, with features such as low-threshold operation, efficient batch processing, and lightweight deployment, has become an ideal choice for industrial detection scenarios such as the annotation of drill rig components in coal mines; its optimized design focusing on object detection tasks significantly improves the annotation efficiency while ensuring accuracy, meeting the actual needs of intelligent mine construction.

[0026] In a specific example, the selected object detection model in step 2 is the Yolov8 network that has been trained on the large public dataset ImageNet; to improve the object detection performance of the model in the complex coal mine underground environment, a multi-level feature extraction module MCFN is introduced. By introducing multi-scale representation learning, it is used to aggregate local and global features, and through multi-core convolution operations, local information is extracted from different receptive fields, improving the model's ability to understand details and context information.

[0027] In a specific example, the specific working method of the multi-level feature extraction module MCFN is as follows: Before the C2F module in the feature extraction framework of the basic object detection model, assume that the current input feature is , where C is the number of channels, and H and W are the height and width of the feature map; through point convolution, the number of channels is expanded from C to 2C to generate an expanded feature , and the calculation formula is as follows: (1) where is a point convolution with parameter , that is, 1×1 convolution; The expanded feature tensor can prepare a richer representation space for subsequent multi-scale convolutions and enhance the representation ability of the model; in the multi-core convolution stage, the expanded feature is divided into multiple parts, and each part of the feature passes through convolution kernels of different sizes to capture multi-scale local information; assume The channels of are divided into n parts, and the calculation formula is as follows: (2) where represents the i-th feature part of the division, ; Apply depthwise separable convolution kernels of different sizes to each feature part, including at least 1×1, 3×3, 5×5..., and the calculation formula is as follows: (3) where represents a depthwise separable convolution with a convolution kernel size of , represents the size of the i-th convolution kernel, represents the feature of the corresponding scale obtained after depthwise separable convolution; Aggregate the outputs of all branches through channel concatenation, and the calculation formula is as follows: (4) where Concat represents the channel concatenation operation, represents the aggregated feature; Use point convolution to reduce the dimension of the aggregated feature , restore the number of channels from 2C to C, and introduce non-linearity using the GELU activation function. The calculation formula is as follows: (5) where is the feature processed by the multi-level feature extraction module, is a point convolution with parameter .

[0028] The multi-level feature extraction module of this embodiment can enhance the model's perception of details and context by expanding the input feature channels, applying multi-scale depthwise separable convolutions, and performing feature aggregation and dimensionality reduction, effectively improving the recognition ability of the target detection model in the complex coal mine environment.

[0029] In a specific example, the specific process of step 3 is as follows: Use the improved target detection model trained in step 2 to process the video frame to obtain the detection frames of the overall drilling rig and the tail of the drilling rig; Assume that the video frame at time t is , and the obtained detection frame of the overall drilling rig is , and the detection frame of the tail of the drilling rig is , and the calculation formula is as follows: (6) where MCFN-Yolov8 is the improved target detection model trained in step 2; Divide the detection frame of the overall drilling rig from the center into four regions: upper left, upper right, lower left, and lower right, which are respectively , , , and ; Generate two diagonal directions, which are respectively and , where represents the direction from lower left to upper right, and represents the direction from lower right to upper left; Calculate the overlapping areas between the detection frame of the tail of the drilling rig and the four regions , , , and (7) (8) (9) (10) where Area represents the area of the region, and represents taking the intersection; Calculate the total overlapping areas and of the two directions, and the calculation formula is as follows: (11) (12) where + represents adding areas; Select the direction with the larger overlapping area as the reference direction of the current frame , the calculation formula is as follows: (13) Among them, if means if, and otherwise means otherwise; Perform a sliding window statistics on the directions within the first n video frames, and select the direction with the highest frequency as the final working direction of the drill , the calculation formula is as follows: (14) Among them, Count represents the counting of the directions that appear in the first n video frames, and argmax represents selecting the direction with the highest frequency.

[0030] In this embodiment, through dynamic direction determination and robustness optimization; dynamic direction calibration: by sliding window statistics of the historical frame direction frequency, suppressing single-frame detection jitter (such as the drill temporarily stopping or jittering), ensuring the stability of direction determination; spatial overlap optimization: based on the overlapping area between the detection frame and the diagonal area, select the reference direction to avoid misjudgment caused by target deformation or occlusion, and improve the accuracy of direction determination.

[0031] In a specific example, the specific process of step 4 is as follows: Calculate the area ratio of the detection frame at the tail of the drill and the overall detection frame of the drill , the calculation formula is as follows: (15) Among them, and represent the area of the detection frame at the tail of the drill and the area of the overall detection frame of the drill; Dynamically adjust the proportion of the forward and backward areas according to the area ratio, and the calculation formula is as follows: (16) (17) (18) Among them, is the proportion of the backward area, is the proportion of the forward area, is the proportion of the buffer area, and are the dynamically adjusted weights.

[0032] Based on the final direction determined in step 3, divide the forward area and the backward area .

[0033] This embodiment realizes dynamic area division and adaptive adjustment; ratio adaptability: dynamically adjusts the forward / backward area ratio according to the area ratio between the tail and the overall detection frame, adapting to the size changes during the drill pipe telescoping process (such as the tail frame shrinking due to the elongation of the drill pipe), reducing the risk of misclassification; geometric constraint enhancement: divides the area along the diagonal and utilizes the geometric characteristics of the rigid structure of the drilling rig to avoid the problem of area drift caused by rotation or perspective change.

[0034] In a specific example, the specific process of step 5 is as follows: Initialize the moving state of the drill rig tail to the backward state, and the calculation formula is as follows: (19) Where, is the moving state of the drill rig tail, and Backward represents the backward state; And initialize the forward counter and the backward counter to 0; according to the relationship between the center point position of the drill rig tail detection frame and the area divided in step 4, if the center point hits a certain area, the corresponding forward or backward counter is incremented, and the calculation formula is as follows: (20) (21)Where, indicates that the center point position belongs to the specified area; If a certain counter reaches the set threshold, update the moving state and clear the counter, and the calculation formula is as follows: (22) (23) Where, Forward represents the forward state, and threshold represents the set threshold.

[0035] Anti-interference design of the state machine and counter in this embodiment; the state machine is a mathematical model that describes the behavior of the system and is used to represent the transition logic between different states of the system; it ensures that the system switches states orderly according to inputs or events and executes corresponding operations by defining the state set, trigger conditions, and transition rules; anti-instantaneous jitter: filters temporary movement interferences (such as mechanical vibrations or camera shakes) through the counter threshold mechanism (such as setting that 5 consecutive frames hitting the area are required to trigger state switching), reducing the false trigger rate; state self-locking protection: when a certain counter reaches the threshold, reset the other counter to prevent frequent state switching (such as the drill rig tail jittering repeatedly at the boundary), ensuring the continuity of counting.

[0036] In a specific example, the specific process of step 6 is as follows: Continuously process the drilling video of the coal mine underground drill rig. Initially, set the counting result to 0; when the moving state of the drill rig tail switches from the forward state to the backward state, increment the drill pipe count by one. The calculation formula is as follows: (24) where and means both conditions are satisfied simultaneously.

[0037] In this embodiment, the drill pipe count is continuously accumulated according to the above rules until the operation is completed, and the region division ratio is dynamically adjusted during the processing; continuous counting and operation integrity guarantee are achieved; process integrity verification: counting only when the state switches from forward to backward conforms to the actual operation logic of the drill pipe "extending - retracting", avoiding one - way mis - counting (such as only one - way movement caused by drill pipe jamming); real - time and scalability: support continuous processing of video streams, without caching historical frames, meeting the real - time monitoring requirements underground; the counting rules can be extended to multi - drill rig collaborative operation scenarios.

[0038] In summary, based on the object detection model pre - trained on a large - scale public dataset, this embodiment is improved by introducing a multi - level feature extraction module MCFN, enabling it to better aggregate local and global features, enhancing its understanding ability of details and context information, improving the object detection effect in the complex coal mine operation environment, and training it with the annotation data of the drill pipe video of the coal mine underground drill rig, so as to accurately identify the whole and the tail of the drill rig in the video. By dividing the overall detection frame of the drill rig, the working direction of the drill rig is quickly and accurately determined. On this basis, according to the area ratio between the drill rig tail and the overall detection frame of the drill rig, the ratio of the forward and backward regions composed of the drill rig as a whole is dynamically adjusted, enhancing the adaptability of the model to different working scenarios in the complex underground coal mine environment. In addition, by introducing a state delay update buffer mechanism, the influence of interference factors such as detection result fluctuations on the counting accuracy is effectively reduced, significantly improving the robustness and reliability of the drill pipe counting model of the drill rig.

[0039] Embodiment 2 See Figure 4 : A coal mine drill pipe object detection and movement analysis counting system, including: Data acquisition and annotation module: used to collect the actual drilling operation video stream underground, split the video into individual image frames, and use professional annotation tools to accurately label the bounding boxes of the whole drill rig and the drill rig tail; Target detection model module: Based on a pre-trained target detection model, a multi-level feature extraction module MCFN is added before the C2F residual module (Coarse-to-Fine Residual Module) in its feature extraction module; the model is trained using the labeled data obtained in the data acquisition and annotation module to optimize the model's recognition ability for the overall drill rig and the tail; Working direction determination module: Use the best target detection model trained by the target detection model module to detect each frame of the video, and extract the overall drill rig and tail target detection frames; divide the overall drill rig target detection frame from the center into four regions: upper left, upper right, lower left, and lower right, and generate two diagonal directions: the direction from the lower right to the upper left and the direction from the lower left to the upper right; calculate the overlapping area between the drill rig tail target detection frame and the two direction regions, and select the direction with the larger overlapping area as the reference direction; perform a sliding window statistics on the directions of the first few frames of the video, and select the direction with the highest frequency as the working direction of the drill rig; Dynamic area division module: Used to calculate the area ratio of the drill rig tail detection frame to the overall drill rig detection frame, and dynamically adjust the forward and backward area ratios according to the area ratio; based on the working direction determined by the working direction determination module, divide the forward and backward areas along the corresponding diagonal of the overall drill rig detection frame; State machine module: Used to initialize the movement state of the drill rig tail to the backward state, and set the forward and backward counters, and initialize them to 0; according to the relationship between the center point position of the drill rig tail detection frame and the divided area, if the center point hits a certain area, the corresponding forward or backward counter is incremented, and the other counter is reset to zero; if a certain counter reaches the set threshold, the movement state is updated, and the two counters are reset; Drill pipe counting module: Used to continuously process the video of the drill rig drilling in the coal mine. When the movement state of the drill rig tail switches from forward to backward, the drill pipe count is incremented by one; the drill pipe count is continuously accumulated according to the above rules until the drilling operation is completed, and finally the drill pipe count result of the drill rig is obtained.

[0040] The coal mine drill rig drill pipe target detection and movement analysis counting system of this embodiment has the same advantages as the above-mentioned coal mine drill rig drill pipe target detection and movement analysis counting method compared with the prior art, and will not be elaborated here.

[0041] Embodiment 3 To achieve the above object, this embodiment also provides a computer-readable storage medium storing a computer program, which when executed by a processor causes the processor to execute the coal mine drill rig drill pipe target detection and movement analysis counting method as described above.

[0042] The above are only the preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention shall be included within the protection scope of the present invention.

Claims

1. A method for target detection, movement analysis and counting of drill pipes of a coal mine drilling rig, characterized in that It includes the following steps: Step 1: Collect the actual downhole drilling operation video stream, split the video into individual image frames, and use a professional annotation tool to accurately label the bounding boxes of the entire drill rig and the tail of the drill rig; Step 2: Based on the pre-trained object detection model, add a multi-level feature extraction module MCFN before the C2F residual module in its feature extraction module; use the annotation data obtained in Step 1 to train the model and optimize the model's recognition ability for the entire drill rig and the tail; Step 3: Use the best object detection model trained in Step 2 to detect each video frame frame by frame, and extract the object detection bounding boxes of the entire drill rig and the tail; divide the object detection bounding box of the entire drill rig from the center into four regions: upper left, upper right, lower left, and lower right, and generate two diagonal directions: the direction from the lower right to the upper left and the direction from the lower left to the upper right; calculate the overlapping area between the object detection bounding box of the drill rig tail and the two direction regions, and select the direction with the larger overlapping area as the reference direction; perform a sliding window statistics on the directions of the first few frames of the video, and select the direction with the highest frequency as the working direction of the drill rig; Step 4: Calculate the area ratio between the detection bounding box of the drill rig tail and the detection bounding box of the entire drill rig, and dynamically adjust the proportion of the forward and backward regions according to the area ratio; based on the working direction determined in Step 3, divide the forward and backward regions along the corresponding diagonal of the object detection bounding box of the entire drill rig; Step 5: Initialize the moving state of the drill rig tail to the backward state, and set the forward and backward counters, and initialize them to 0; according to the relationship between the center point position of the detection bounding box of the drill rig tail and the divided regions, if the center point hits a certain region, increase the corresponding forward or backward counter, and reset the other counter to zero; if a certain counter reaches the set threshold, update the moving state and reset the two counters; Step 6: Continuously process the drilling video of the drill rig in the coal mine underground. When the moving state of the drill rig tail switches from forward to backward, increment the drill pipe count by one; continuously accumulate the drill pipe count according to the above rules until the drilling operation is completed, and finally obtain the drill pipe count result of the drill rig.

2. The method for target detection, movement analysis and counting of drill pipes of a coal mine drill rig according to claim 1, characterized in that, The professional annotation tool in Step 1 is LabelImage.

3. The method for target detection, movement analysis and counting of drill pipes of a coal mine drill rig according to claim 1, characterized in that, The selected object detection model in Step 2 is the Yolov8 network that has been trained on the large public dataset ImageNet; introduce the multi-level feature extraction module MCFN, and through introducing multi-scale representation learning, it is used to aggregate local and global features, and through multi-core convolution operations, local information is extracted from different receptive fields.

4. The method for target detection, movement analysis and counting of drill pipes of a coal mine drill rig according to claim 2, characterized in that, The specific working method of the multi-level feature extraction module MCFN is as follows: Before the C2F module in the feature extraction framework of the basic object detection model, assume that the current input feature is , where C is the number of channels, and H and W are the height and width of the feature map; through point convolution, the number of channels is expanded from C to 2C to generate an extended feature , and the calculation formula is as follows: （1） Among them, is a point convolution with parameter , that is, a 1×1 convolution; The expanded feature tensor can prepare a richer representation space for subsequent multi-scale convolutions, enhancing the model's representation ability; in the multi-core convolution stage, the expanded features are divided into multiple parts, and the features of each part are respectively passed through convolution kernels of different sizes to capture local information at multiple scales; assuming has its channels divided into n parts, the calculation formula is as follows: （2） Among them, represents the i-th feature part of the division, ; Apply depthwise separable convolution kernels of different sizes to each feature part, including at least 1×1, 3×3, 5×5..., and the calculation formula is as follows: （3） Among them, represents the depthwise separable convolution with a convolution kernel size of , represents the size of the i-th convolution kernel, represents the feature of the corresponding scale obtained after depthwise separable convolution; Aggregate the outputs of all branches by means of channel concatenation, and the calculation formula is as follows: （4） Among them, Concat represents the channel concatenation operation, represents the aggregated feature; Aggregate features using point convolution Reduce the dimension, restore the number of channels from 2C to C, and introduce non-linearity using the GELU activation function. The calculation formula is as follows: （5） Among them, is the feature processed by the multi-level feature extraction module, is the point convolution with parameter .

5. The method for target detection, movement analysis and counting of drill pipes of a coal mine drill rig according to claim 4, characterized in that, The specific process of step 3 is as follows: Use the improved object detection model trained in step 2 to process the video frames to obtain the detection frames of the overall drill rig and the tail of the drill rig; Assume that the video frame at time t is , and the obtained detection frame of the overall drill rig is , and the detection frame of the tail of the drill rig is , and the calculation formula is as follows: （6） Among them, MCFN-Yolov8 is the improved object detection model trained in Step 2; Divide the overall detection frame of the drilling rig starting from the center into four regions: upper left, upper right, lower left, and lower right, which are respectively , , and ; generate two diagonal directions, which are respectively and , where represents the direction from lower left to upper right, and represents the direction from lower right to upper left; calculate the overlapping areas between the detection frame at the tail of the drilling rig and the four regions , , and respectively. The specific calculation formula is as follows: （7） （8） （9） （10） where Area represents the area of the region, represents taking the intersection; Calculate the total overlapping area in two directions and , and the calculation formula is as follows: （11） （12） Among them, + means area addition; Select the direction with a larger overlapping area as the reference direction for the current frame , and the calculation formula is as follows: （13） Among them, if means if, otherwise means otherwise; Perform a sliding window statistics on the directions within the first n video frames, and select the direction with the highest frequency as the final working direction of the drill rig , and the calculation formula is as follows: （14） Among them, Count means counting the directions that appear in the first n video frames, and argmax means selecting the direction with the highest frequency.

6. The method for target detection, movement analysis and counting of drill pipes of a coal mine drill rig according to claim 5, characterized in that, The specific process of step 4 is as follows: Calculate the area ratio of the detection frame at the tail of the drilling rig to the overall detection frame of the drilling rig , and the calculation formula is as follows: （15） Among them, and represent the area of the detection frame at the tail of the drilling rig and the area of the overall detection frame of the drilling rig; Dynamically adjust the ratio of the forward and backward regions according to the area ratio, and the calculation formula is as follows: （16） （17） （18） Among them, is the proportion of the backward area, is the proportion of the forward area, is the proportion of the buffer area, and are the dynamically adjusted weights. Based on the final direction determined in step 3, a forward area and a backward area are divided along the diagonal corresponding to the overall detection frame of the drilling rig.

7. The method for target detection, movement analysis and counting of drill pipes of a coal mine drill rig according to claim 6, characterized in that, The specific process of step 5 is as follows: Initialize the moving state of the drill rig tail to the backward state, and the calculation formula is as follows: （19） Among them, is the moving state of the drill rig tail, and Backward indicates the backward state; And initialize the forward counter and the backward counter to 0; according to the relationship between the center point position of the detection frame at the rear of the drill rig and the area divided in step 4, if the center point hits a certain area, the forward or backward counter is incremented accordingly. The calculation formula is as follows: （20） (21) Among them, indicates that the center point position belongs to the specified area; If a certain counter reaches the set threshold, update the moving state and clear the counter, and the calculation formula is as follows: （22） （23） Among them, Forward represents the forward state, and threshold represents the set threshold.

8. The method for target detection, movement analysis and counting of drill pipes of a coal mine drill rig according to claim 7, characterized in that, The specific process of step 6 is as follows: Continuously process the drilling video of the underground coal mine drill. Initially, set the counting result to 0; when the moving state of the drill tail switches from the forward state to the backward state, the drill pipe count is incremented by one, and the calculation formula is as follows: （24） Among them, and means that both conditions are satisfied simultaneously.

9. A target detection, movement analysis and counting system for drill pipes of a coal mine drill rig, characterized in that, It includes: Data acquisition and annotation module: used to acquire the actual downhole drilling operation video stream, split the video into individual image frames, and use professional annotation tools to accurately label the bounding boxes of the entire drill rig and the drill rig tail; Target detection model module: Based on a pre-trained target detection model, add a multi-level feature extraction module MCFN before the C2F residual module in its feature extraction module; use the annotation data obtained in the data acquisition and annotation module to train the model and optimize the model's recognition ability for the entire drill rig and the tail; Working direction determination module: Use the best target detection model trained by the target detection model module to detect each frame of the video, and extract the target detection bounding boxes of the entire drill rig and the tail; divide the target detection bounding box of the entire drill rig from the center into four regions: upper left, upper right, lower left, and lower right, and generate two diagonal directions: the direction from the lower right to the upper left and the direction from the lower left to the upper right; calculate the overlapping area between the target detection bounding box of the drill rig tail and the two direction regions, and select the direction with the larger overlapping area as the reference direction; perform a sliding window statistics on the directions of the first few frames of the video, and select the direction with the highest frequency as the working direction of the drill rig; Dynamic region division module: used to calculate the area ratio between the detection bounding box of the drill rig tail and the detection bounding box of the entire drill rig, and dynamically adjust the ratio of the forward and backward regions according to the area ratio; based on the working direction determined by the working direction determination module, divide the forward and backward regions along the corresponding diagonal of the detection bounding box of the entire drill rig; State machine module: used to initialize the moving state of the drill rig tail to the backward state, set the forward and backward counters, and initialize them to 0; according to the relationship between the center point position of the detection bounding box of the drill rig tail and the divided regions, if the center point hits a certain region, increase the corresponding forward or backward counter, and reset the other counter to zero; if a certain counter reaches the set threshold, update the moving state and reset the two counters; Drill pipe counting module: used to continuously process the video of the drill rig drilling in the coal mine. When the moving state of the drill rig tail switches from forward to backward, the drill pipe count is incremented by one; continuously accumulate the drill pipe count according to the above rules until the drilling operation is completed, and finally obtain the drill pipe count result of the drill rig.

10. A computer-readable storage medium stores a computer program. When the computer program is executed by a processor, the processor is caused to execute the coal mine drill rig drill pipe target detection and movement analysis and counting method according to any one of claims 1 to 8.

Images