Drilling rod withdrawal counting method and system based on AI vision

Abstract

The application relates to a drilling rod withdrawal counting method and system based on AI vision, and belongs to the field of industrial intelligent perception and mine safety monitoring. In view of the problem that the drilling rod withdrawal counting in underground drilling operation is easily interfered by a water injector, short-time shaking and human operation, and false counting and missing counting are caused, automatic and reliable rod withdrawal counting is realized by combining visual perception and time sequence logic. The method only takes the composite structure containing a drill rod male head and a rod body as a detection object, cooperates with frame-level target detection and multi-target online tracking, and obtains a stable and continuous target trajectory; direction determination is performed based on the normalized average vertical displacement of the trajectory center point, the influence of resolution and installation difference is eliminated, and mechanisms such as time interval constraint, abnormality determination, timeout reset and counted trajectory management are combined, so that repeated counting and abnormal interference are effectively avoided. The application significantly improves the accuracy, robustness and auditability of the rod withdrawal counting under complex underground working conditions, and has good engineering applicability.

Description

Technical Field

[0001] This invention belongs to the field of industrial intelligent sensing and mine safety monitoring, and relates to a drilling rod retraction counting method and system based on AI vision, and more particularly to a method and system for automatic identification and counting of the drill rod retraction process based on machine vision and time-series information analysis. Background Technology

[0002] In underground drilling operations such as water exploration and gas extraction in coal mines, the number of drill rod advances and retreats directly affects the verification of the actual borehole depth, the determination of the authenticity of the operation process, and the effectiveness of safety supervision. Therefore, accurate and reliable counting of the drill rod advances and retreats is of great significance. In existing technologies, drill rod counting mainly relies on manual statistics or uses contact or non-contact sensing methods such as photoelectric sensors, radio frequency identification (RFID), and Hall effect sensors to achieve automatic counting. However, under complex working conditions in mines, such as high dust, high humidity, strong vibration, and electromagnetic interference, the stability of these sensor solutions is poor, prone to false alarms or missed alarms, and the equipment maintenance costs are high, making long-term stable operation difficult.

[0003] In recent years, with the development of machine vision technology, some studies have attempted to achieve non-contact automatic recognition of the drill rod's advance and retreat process through visual means. However, existing vision solutions typically rely on a large amount of training data with consistent visual features and a relatively ideal camera installation angle. They also tend to focus on the relative motion relationships between the main components of the drilling rig (such as the gripper, power head, and water injector), lacking a deep coupling judgment mechanism with the actual drilling rig operation process, target geometric features, and the temporal relationship of the motion trajectory. Therefore, when the drilling rig model changes, the installation angle deviates, or complex situations such as obstruction or multiple rods sticking together exist on site, their recognition performance can easily degrade significantly. Furthermore, because the water injector and drill rod have a certain similarity in appearance, coupled with factors such as partial obstruction and human interference, existing vision solutions are prone to miscalculation.

[0004] In summary, the existing technology still has the following shortcomings: First, visual inspection results are easily affected by downhole lighting conditions, dust environment, and changes in camera angle, resulting in limited single-frame detection accuracy and model generalization ability; second, judging drill pipe advance and retreat solely based on detection confidence or single-frame position information lacks temporal analysis and operational logic verification based on motion trajectory, making it difficult to effectively distinguish between retracting and advancing drill pipes, drill pipes and water injectors, and false movements caused by multiple rod adhesion; third, there is a lack of anti-misjudgment mechanisms for short-term disturbances (such as equipment vibration and transient lighting) and human intervention, leading to insufficient reliability and credibility of the counting results. Summary of the Invention

[0005] In response to the problems of existing drill pipe retraction counting being susceptible to environmental interference, having a high rate of miscounting and undercounting, and having insufficient reliability of counting results under complex downhole conditions, there is an urgent need in industrial sites for a technical solution that can stably and accurately count the drill pipe retraction process under harsh operating conditions, so as to meet the high reliability requirements of downhole safety supervision for counting results.

[0006] Based on this, the technical problem to be solved by the present invention is to provide a drill pipe retraction counting method and system for harsh downhole working conditions. By combining structured visual detection with multi-target trajectory tracking, and introducing a discrimination mechanism based on motion direction and temporal logic verification, reliable identification of the retraction action is achieved. At the same time, by formulating data annotation specifications and corresponding training strategies for the drill pipe "male end + shaft" structure, the recognition stability of the model in complex scenarios is improved. Furthermore, at the system level, mechanisms such as deduplication control, timeout reset, counting anti-shake, and abnormal scenario self-recovery are introduced to achieve automatic drill pipe retraction counting with high accuracy, strong robustness, and traceability.

[0007] In view of this, the purpose of this invention is to provide a drilling rod counting method based on AI vision, comprising the following steps: S1. Video Acquisition and Preprocessing: Video data of the drill rod advancing and retreating process is acquired by a camera device installed above the drill rig's power head. The acquired video frames are preprocessed at the frame level. The frame-level preprocessing includes at least noise reduction, brightness normalization, and / or frame sampling, so that the preprocessed video frames can be used as input data for subsequent analysis. S2. Drill pipe target detection: Based on the preprocessed video frames, drill pipe target detection is performed in the preset region of interest. Only composite regions containing both the drill pipe male head and the drill pipe body are detected. The drill pipe male head and the drill pipe body have distinguishable structural features. The corresponding detection box and detection confidence are output. S3. Multi-target tracking and trajectory generation: Perform online multi-target tracking processing on the drill pipe target, assign a unique trajectory identifier to different drill pipe targets, and maintain the motion trajectory of the corresponding drill pipe target in continuous video frames. The motion trajectory includes at least a sequence of the position of the detection box center point changing over time. S4. Trajectory Displacement Calculation: For each valid drill pipe target trajectory, extract its center point position in consecutive video frames, and normalize the vertical coordinates of the center point based on the video frame height. Calculate the displacement change of the trajectory in the vertical direction to obtain the average normalized vertical displacement within the trajectory segment. S5. Direction Determination and Drill Pipe Retraction Count: Based on the continuous motion trend of the trajectory, and combined with the average normalized vertical displacement obtained in S4 and the preset direction parameters, the motion direction of the drill pipe target is determined: when the average normalized vertical displacement meets the preset positive threshold condition, and the time interval since the last valid counting event is not less than the preset minimum time interval, the corresponding trajectory is determined to be a valid drill pipe retraction action, and a drill pipe retraction count is triggered; when the average normalized vertical displacement meets the reverse threshold condition, the corresponding trajectory is determined to be a drill pipe advance or abnormal operation. S6. Anomaly detection and status update: When the average normalized vertical displacement satisfies the detection condition that is opposite to the direction of the rod retraction, the corresponding trajectory is determined to be rod advance or abnormal operation, and the corresponding status update process is performed on the current counting state.

[0008] Furthermore, in S2: The following labeling constraints are applied to the sample data used for training the detection model: composite structural regions containing both the drill pipe male end and the drill pipe shaft are labeled as positive samples and assigned a unified target category label; wherein, the male end has at least half of the visible area of ​​a complete male end, and the shaft has at least clearly identifiable coarse and sparse thread texture features; when the visible part of the male end is less than half of the full area or the thread texture features of the shaft are not identifiable, the corresponding target region is not labeled as a positive sample.

[0009] Furthermore, in the process of constructing sample data for training the detection model, water injectors containing only the male end and without coarse and sparse thread texture features on the shaft, drill rod images with severe blur or motion blur, and other tools or components with similar shapes to drill rods but without drill rod structural features are uniformly included as negative samples in the training data; and when the positive samples are labeled with bounding boxes, the bounding boxes are closely attached to the minimum bounding rectangle of the male end and shaft combination structure and the background area is reduced to reduce the interference of irrelevant background information on the training of the detection model.

[0010] Furthermore, in S2: a single-stage or two-stage target detection network is used as the detection model to detect the male end and shaft structure of the drill rod in each frame of the video sequence; during the detection process, target detection processing is only performed on a preset region of interest in the input image, the region of interest is set along a preset horizontal band in the image to reduce computation and reduce environmental interference in non-working areas; the output result of the detection model for each frame includes at least the target detection box, the corresponding detection confidence score, and the target category probability; and non-maximum suppression processing and filtering processing based on the detection confidence score threshold are sequentially performed on the output result of the detection model to obtain a set of candidate detection boxes in the current frame, wherein the detection confidence score threshold is a configurable parameter.

[0011] Furthermore, after obtaining the candidate detection box set, the position information of the detection boxes is normalized: if the height of the current image frame is... H If the pixel coordinates of a point in the image in the vertical direction are y, then its normalized coordinates y′ are: y'=y / H Based on the detection output, and for subsequent trajectory motion analysis and displacement calculation, the normalized coordinates of the center point of the target single-frame detection box are extracted as follows:

[0012] in, For the first i The x-coordinate of the center pixel of the target detection box in the frame; For the first i The normalized coordinates of the target's center point in the horizontal direction within the frame; For the first i The ordinate of the center pixel of the target detection box in the frame; For the first i The normalized coordinates of the target's center point in the vertical direction within the frame; The pixel width of the current image frame.

[0013] Furthermore, S3 specifically involves: based on the drill rod target detection results, performing cross-frame target association and trajectory consistency maintenance on the drill rod targets, assigning a unique trajectory identifier to the same drill rod target, and maintaining the corresponding motion trajectory in consecutive video frames; the cross-frame target association is completed based on the geometric position information of the detection box and combined with the motion prediction results.

[0014] 7. The drilling rod counting method based on AI vision according to claim 6, characterized in that: In the cross-frame target association process, the target center point position, detection box area, and optional appearance features are used as association costs, and a matching cost is constructed by weighted fusion of the detection box intersection-union ratio and the center point distance; a trajectory state set is maintained for each drill pipe target trajectory, and the trajectory state set includes at least a trajectory identifier, a center point position sequence, an undetected frame count, and a count flag.

[0015] Furthermore, in S4: for each drill pipe target trajectory that passes the trajectory validity filter, extract the trajectory in continuous... nThe location of the center point of the detection box in the video frame is determined, and the pixel coordinates of the center point in the vertical direction are normalized based on the image height of the current video frame. The vertical displacement change of the center point between adjacent video frames is calculated, and the vertical displacement change is accumulated and averaged to obtain the average normalized vertical displacement of the drill rod target trajectory within the corresponding time window, which is used as a feature quantity for subsequent rod retraction direction determination.

[0016] Furthermore, in S5: the movement direction of the drill pipe target trajectory is determined based on the average normalized vertical displacement and in combination with preset direction parameters; when the product of the average normalized vertical displacement and the direction parameters is greater than or equal to a preset displacement determination threshold, and the time interval since the last valid drill pipe retraction count event or trajectory state change is not less than a preset minimum time interval threshold, the corresponding drill pipe target trajectory is determined to be a valid drill pipe retraction action, and a drill pipe retraction count is triggered; and after triggering the drill pipe retraction count, the corresponding drill pipe target trajectory is marked as counted.

[0017] Furthermore, in S6: when it is detected that the average normalized vertical displacement of the drill pipe target trajectory is opposite to the direction of pipe retraction in terms of direction determination, and this reverse movement continues to exist in multiple consecutive video frames, the current operation process is determined to be an abnormal operation scenario of pipe advance or mixed advance and retraction; and in the abnormal operation scenario, the cumulative pipe retraction count result is subjected to count reset processing, and the time of the most recent state change is updated.

[0018] Furthermore, if it is detected that the duration since the last valid rod retraction count event exceeds the preset timeout threshold and no new rod retraction count result is generated, the count reset operation is automatically executed to clear the cumulative rod retraction count value to zero and record the corresponding count reset log information; at the same time, the system clears the target trajectory markers of drill rods that have triggered counting and are confirmed to have left the camera's field of view.

[0019] Furthermore, a multi-visual and temporal logic constraint strategy is adopted to enhance counting robustness, wherein the constraint strategy includes at least: Spatial constraints: Drill pipe target detection and multi-target tracking are performed only within a pre-defined area of ​​interest to reduce interference from personnel, equipment, or background changes outside the work area on the detection results; Temporal constraints: By setting a minimum counting time interval parameter, the repeated triggering of the drill pipe retraction counting operation is limited within a preset time window; Trajectory integrity constraints: Drill pipe target trajectory retraction counting is only performed on drill pipe target trajectories that meet the trajectory stability conditions. The trajectory stability conditions include at least that the target trajectory has left the camera's field of view, or that it exhibits a stable motion trend and small speed fluctuations in continuous video frames; Trajectory identifier management: The identifier information of drill pipe target trajectories that have been triggered for counting is maintained, and the corresponding trajectory identifier is deleted after the corresponding trajectory is confirmed to have left the field of view.

[0020] Furthermore, when determining the retraction of the drill pipe target trajectory, a comprehensive determination mechanism based on multi-index fusion is adopted to perform threshold determination or adaptive threshold adjustment on candidate trajectory segments. The comprehensive judgment mechanism includes at least a weighted combination of the following indicators: the average normalized vertical displacement within the trajectory segment; the amplitude of the drill pipe target's motion velocity; the mean of the trajectory segment for target detection confidence; and a trajectory direction consistency index, which is calculated based on the standard deviation of trajectory direction changes. When the comprehensive judgment score obtained based on the weighted multi-indicator meets the preset judgment threshold condition, the corresponding drill pipe target trajectory retraction counting operation is triggered.

[0021] Furthermore, the drill pipe retraction count results are processed using both real-time visualization and historical data recording methods. The processing includes: Real-time display: Outputs the current cumulative lever retraction count result through a visual interface, and simultaneously displays the identifier (track_id) of each target trajectory and the corresponding vertical displacement characteristic value (dy). It also overlays the trajectory motion of the most recent N frames and provides the original video playback link associated with the counting event so that operators can intuitively check the counting process. Alarm Trigger: When abnormal operation behavior is detected, alarm processing is triggered. The abnormal operation includes the abnormal situation where the average vertical displacement (avg_dy) of the target trajectory is reversed and the amplitude exceeds the preset threshold, or the counting result within a unit time fluctuates rapidly and the change amplitude exceeds the preset threshold. Data storage and backtracking: The system automatically saves video clips and trajectory data when the alarm is triggered for subsequent manual review and security audit; Event metadata: The system saves metadata for each valid lever retraction count event. The metadata includes at least the count timestamp, track identifier (track_id), average vertical displacement (avg_dy), detection confidence mean (conf_mean), and associated frame screenshot path to support the traceability and auditing requirements of the count results. Model training and iteration: A two-stage training strategy is adopted. In the first stage, the detection model is trained on offline datasets in multiple scenarios. In the second stage, the model is fine-tuned on labeled samples collected in the target mine environment based on camera viewpoint and lighting conditions.

[0022] A drilling rod withdrawal counting system based on AI vision includes a front-end data acquisition unit, a mine ring network, a ground analysis server, and a visual alarm client, wherein: The front-end acquisition unit includes an industrial camera device installed above the drill rig's power head, used for continuous video acquisition of the drill rod retraction process; the industrial camera device is installed in a fixed manner, and its installation position, viewing angle, and imaging parameters remain stable after debugging; the front-end acquisition unit is used to perform local preprocessing operations on the acquired video frames; The mine ring network is used to transmit video data, which has been pre-processed locally at the front-end acquisition terminal, to the ground analysis server. A ground analysis server, connected to the mine ring network, is used to analyze and process received video data. The ground analysis server includes: The video preprocessing module is used to perform frame-level preprocessing on the input video frames to reduce noise interference and achieve brightness normalization. The target detection module, connected to the video preprocessing module, is used to perform target detection based on the preprocessed video frames and identify the drill pipe composite region that simultaneously contains the drill pipe male head and the drill pipe body. A multi-target tracking module, connected to the target detection module, is used to perform multi-target tracking on the detected drill pipe targets, assign a unique identifier to different drill pipe targets and maintain their motion trajectories; A counting and logic verification module, connected to the multi-target tracking module, is used to perform rod retraction counting determination based on the motion trajectory of the drill rod target. The counting and logic verification module includes: The trajectory direction determination unit is used to analyze the motion direction of the drill rod target in the image based on the trajectory information obtained from multi-target tracking, calculate the normalized displacement of the drill rod target in the vertical direction, and determine whether the corresponding trajectory meets the determination condition of the rod retraction direction based on the displacement direction. The counting logic and state management unit is used to combine the trajectory direction determination result and the time constraint condition to execute the lever retraction counting logic and state management operation. The state management operation includes at least time window control, deduplication judgment and timeout reset. The data storage module, connected to the counting and logic verification module, is used to store and manage the final lever count result, abnormal information and related log data. The visual alarm client communicates with the ground analysis server to display the drill pipe ejection count in real time and issue alarm prompts when abnormal counting behavior is detected.

[0023] Furthermore, the ground analysis server is configured to execute the AI ​​vision-based drilling rod counting method according to any one of claims 1–14.

[0024] Furthermore, it also includes: a timeout reset and trajectory management module: used to automatically clear the cumulative count result when the preset timeout period is exceeded and no new valid rod retraction count event is detected, and to clean up the target trajectory markers of the drill rod that have been counted and left the camera's field of view, so as to avoid duplicate counting caused by the reuse of trajectory markers.

[0025] The beneficial effects of this invention are as follows: Addressing the problem of miscounting and undercounting in drill pipe retraction counting during downhole drilling operations due to interference from water injection devices, short-term vibrations, and human operation, this invention proposes an automatic drill pipe retraction counting method and system that integrates AI visual perception and temporal logic judgment. By establishing strict rules for "male end + pipe body" data labeling and target definition, the uniqueness and stability of drill pipe target identification are improved from the source. Combined with frame-level target detection and multi-target online tracking, continuous and stable trajectory information is obtained, providing a reliable temporal basis for determining the retraction direction. Furthermore, this invention achieves drill pipe retraction direction identification based on the normalized average vertical displacement of the trajectory center point, effectively eliminating the influence of resolution and installation differences. Combined with multiple anti-misoperation mechanisms such as time gating, timeout reset, and counted trajectory management, the system's robustness against abnormal operations, short-term disturbances, and human intervention is significantly improved. This solution also supports the visualization and auditing of counting events, balancing accuracy, stability, and traceability. It can operate stably under complex working conditions such as low light, high noise, and diverse equipment downhole, demonstrating good engineering practicality and promotional value. Attached Figure Description

[0026] To make the objectives, technical solutions, and advantages of the present invention clearer, the preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings, wherein: Figure 1 This is a schematic diagram of the system architecture according to an embodiment of the present invention; Figure 2 This is a schematic diagram of the drill retraction counting logic according to an embodiment of the present invention; Figure 3 This is a schematic diagram of the state machine of a single tracking target according to an embodiment of the present invention. Detailed Implementation

[0027] The following specific examples illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of the present invention. Unless otherwise specified, the following embodiments and features can be combined with each other.

[0028] The accompanying drawings are for illustrative purposes only and are schematic diagrams, not actual pictures. They should not be construed as limiting the invention. To better illustrate the embodiments of the invention, some parts in the drawings may be omitted, enlarged, or reduced, and do not represent the actual product dimensions. It is understandable to those skilled in the art that some well-known structures and their descriptions may be omitted in the drawings.

[0029] In the accompanying drawings of the embodiments of the present invention, the same or similar reference numerals correspond to the same or similar components. In the description of the present invention, it should be understood that if terms such as "upper," "lower," "left," "right," "front," and "rear" indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, they are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, the terms used to describe positional relationships in the drawings are only for illustrative purposes and should not be construed as limiting the present invention. For those skilled in the art, the specific meaning of the above terms can be understood according to the specific circumstances.

[0030] The detailed technical solution of this invention: This section provides complete technical implementation steps, key formulas and visual flowcharts for implementation and engineering reference, covering aspects such as system configuration, data annotation and training, detection and tracking modules, trajectory motion discrimination, counting logic and state management, exception handling and alternative solutions.

[0031] I. System Overall Architecture Please see Figure 1 The diagram below illustrates the system architecture of an embodiment of the present invention. The drill pipe ejection counting system of the present invention adopts a layered architecture of "front-end acquisition, network transmission, back-end analysis and processing, and visualization display," and mainly includes a front-end acquisition terminal (such as a video acquisition module), a mine ring network, a ground analysis server, and a visualization alarm client.

[0032] The front-end acquisition unit is equipped with an industrial camera mounted above the drill rig's power head for continuous video capture of the drill rod retraction process. The camera is fixed in place, and its installation position, viewing angle, and imaging parameters (including resolution, frame rate, and exposure parameters) are stabilized after debugging to avoid significant viewing angle drift caused by drill rig vibration or movement, which could affect the stability of subsequent identification and counting. The front-end acquisition unit can perform necessary local preprocessing operations on the acquired video frames, such as noise reduction, brightness correction, or gamma correction.

[0033] After necessary local preprocessing of the video frames, they are transmitted to the ground analysis server via network transmission methods (such as transmission methods based on streaming media protocols RTSP / RTMP or dedicated protocols). The mine ring network can adopt wired Ethernet or PoE power supply network. The ground analysis server, as the core processing unit of the system, is used to perform drill pipe target detection, target tracking, drill pipe retraction action judgment, and counting logic processing on the received video data, and to store and manage the counting results, trajectory information, and related log data. The visual alarm client communicates with the ground analysis server to display the drill pipe retraction counting results in real time and to issue alarm prompts when abnormal counting behavior is detected.

[0034] like Figure 1 As shown, the data analysis and processing flow of each module in the ground analysis server includes the following steps in sequence: (1) Video preprocessing module: performs frame-level preprocessing on the video frames input by the video acquisition module to reduce noise interference and achieve brightness normalization; this function can also be completed by the video acquisition module set at the front-end acquisition end.

[0035] (2) Target detection module: Performs target detection based on the preprocessed image to identify the composite area of ​​the drill rod that simultaneously contains the "male end" and the "rod body"; (3) Multi-target tracking module: performs multi-target tracking on the detected drill pipe targets, assigns a unique identifier (ID) to different drill pipes and maintains their movement trajectory; (4) Counting and logic verification module: a) Trajectory direction determination unit: Based on the trajectory information obtained from multi-target tracking, analyze the motion direction of the drill rod target in the image, calculate the normalized displacement of the drill rod target in the vertical direction, and determine whether the trajectory meets the determination condition of the drill rod retraction direction based on the displacement direction. b) Counting logic and state management unit: Combining the trajectory direction determination result and time constraints, it executes the lever retraction counting logic and state management operations, including time window control, deduplication judgment and timeout reset; (5) Data storage module: Outputs the final counting results, abnormal information and related log data to the visual alarm client and stores them for subsequent auditing and backtracking.

[0036] The data storage module may also include an exception handling and status management unit: connected to the direction determination and counting module, used to process the current counting status of the system when an advance or abnormal movement state is detected, the processing includes at least one of count reset, exception state recording and / or alarm triggering.

[0037] The ground analysis server may also include: Timeout Reset and Trajectory Management Module: When the preset timeout period is exceeded and no new valid drill rod retraction count event is detected, the cumulative count result is automatically cleared, and the target trajectory markers of drill rods that have been counted and left the camera's field of view are cleared to avoid duplicate counting caused by trajectory marker reuse; The results storage module is used to visualize the rod retraction count results, abnormal status information, and drill pipe target trajectory information, and to store the relevant data for each rod retraction count event. The relevant data includes at least the counting time, trajectory identifier, average normalized vertical displacement, and detection confidence information to support subsequent auditing and traceability.

[0038] II. Data Collection and Labeling Standards Drilling operations typically include an advance and a retreat process. During the advance, a water injector is usually installed and removed from the tail of each drill pipe, causing the water injector to frequently enter and exit the field of vision, interfering with identification. Furthermore, the advance process is lengthy, with significant variations in the operational rhythm and numerous uncontrollable factors, easily interfering with vision-based automatic identification and counting. In contrast, during the retreat, the water injector is typically removed only once at the beginning of the retreat for the first drill pipe, followed by the sequential retreat of each drill pipe. This retreat process is relatively stable and has strong continuity.

[0039] Based on the aforementioned operational characteristics, in drill pipe counting applications, it is typically only necessary to count the number of drill pipes retracted during the retraction process. In this scenario, the water injector, due to its structural similarity to the drill pipe, becomes a major interfering factor affecting the accuracy of visual recognition.

[0040] To address the problem of interference recognition from water injectors and improve the generalization ability of the detection model under complex working conditions, this invention establishes the following strict data acquisition and annotation specifications.

[0041] Drill pipes typically consist of a male head, a body, and a female head. The male head connects adjacent drill pipes and resembles a bolt in shape, its main visual feature being a surface with fine, dense threads. The body is the main part of the drill pipe, and to enhance friction during drilling, its surface usually has a coarser, sparser thread feature. In contrast, water injectors are also metal components with male heads, but their bodies are usually smooth or lack the aforementioned coarser, sparser thread feature, thus exhibiting a significant difference in texture compared to drill pipes.

[0042] Based on the aforementioned structural differences, this invention, during data annotation, only annotates drill pipe structures that simultaneously contain both a male end and a shaft as positive samples, and uniformly labels them as the same target category. Specifically, the visibility requirement for the male end is that at least half of the male end structure can be identified, and the visibility requirement for the shaft is that its surface thread features are clearly discernible; targets that do not meet these conditions are not annotated. Simultaneously, image data containing water injectors and images of drill pipe targets that are blurry or have indistinguishable features are uniformly included as negative samples in the training data to enhance the model's ability to distinguish interfering targets.

[0043] The data annotation standards are summarized as follows: (1) Target label: Only the drill pipe composite structure area that contains both male head and body is labeled as a positive sample, and the target category label is uniformly assigned as "DrillPipe".

[0044] (2) Male head visibility requirement: For drill pipe targets labeled as positive samples, the male head part must have a visible area of ​​at least half of the complete male head; if the visible part of the male head is less than the above proportion, it will not be labeled.

[0045] (3) Requirements for the texture of the drill rod: The drill rod target used as a positive sample should have a thick and sparse thread feature that can be clearly observed on its rod body; if the thread feature of the rod body cannot be identified due to obstruction, blurring or insufficient resolution, it shall not be marked.

[0046] (4) Definition of negative samples: The following situations are uniformly labeled as negative samples and included in the training data: including but not limited to water injectors that only contain male heads and whose shafts do not have coarse and sparse thread characteristics, drill rod targets with severe blur or motion blur in the image, and other tools or components that are similar in shape to drill rods but do not have the structural characteristics of drill rods.

[0047] (5) Bounding box strategy: The bounding box should be close to the smallest outer rectangle of the male head and the shaft combination structure, and should avoid including irrelevant background areas as much as possible to reduce the interference of background information on model training.

[0048] III. Target Detection Module A single-stage or two-stage target detection network (such as the YOLO series models or similar lightweight target detectors, hereinafter collectively referred to as the "detection model") is employed to perform high-confidence localization of the "male head + shaft" structure of a drill rod within each frame of a video sequence. In its implementation, the detection model detects fixed regions of interest (ROIs) in the input image, where each ROI is set along a preset horizontal band in the image to reduce computational load and minimize environmental interference. The output of the detection model includes a target detection box, detection confidence (conf), and target class probability.

[0049] During the model training phase, it is recommended to use focal loss or a combination of BCE and CIoU loss functions for optimization, and to control the ratio of negative samples to positive samples within the range of 1:3 to 1:5 to prevent model overfitting. At the same time, a training process of first freezing the backbone network parameters and then fine-tuning them with a small learning rate should be adopted to improve the detection stability of the detection model for drill pipe "male head + shaft" structures in complex scenarios, especially for small-scale targets.

[0050] The detection model's output for each frame is first processed by Non-Maximum Suppression (NMS) and then filtered in conjunction with a detection confidence threshold, which defaults to conf_thresh=0.5. Furthermore, this threshold can be adjusted according to the actual application scenario to obtain the set of candidate detection boxes (Box set) in the current frame.

[0051] To improve the algorithm's versatility under different image resolutions, normalized coordinates are used to represent the position information of the detection boxes. Specifically, if the height of the current image frame is H, and the pixel coordinates of a point in the image in the vertical direction are y, then its normalized coordinates y′ are defined as: y'=y / H Based on the detection output, and for subsequent trajectory motion analysis and displacement calculation, the normalized coordinates of the center point of the target single-frame detection box are extracted and defined as:

[0052] in, For the first i The x-coordinate of the center pixel of the target detection box in the frame; For the first i The normalized coordinates of the target's center point in the horizontal direction within the frame; For the first i The ordinate of the center pixel of the target detection box in the frame; For the first i The normalized coordinates of the target's center point in the vertical direction within the frame; The pixel width of the current image frame.

[0053] The normalized height serves as the basis for subsequent trajectory displacement calculations, where H represents the height of the corresponding image frame.

[0054] IV. Multi-target tracking module To achieve continuous assignment of target identities (IDs) across frames for the same drill pipe, this invention employs a multi-target tracking method based on a combination of detection box information and motion prediction. Specifically, it can use an association matching mechanism consisting of detection boxes, Kalman filters, and the Hungarian algorithm. On this basis, existing online multi-target trackers (such as SORT, DeepSORT, etc.) can also be introduced according to application requirements, and appearance features can be extended as needed to enhance the target re-identification capability.

[0055] The key implementation points are: using fused location (center point), detection box area, and appearance features (optionally: lightweight ReID features) as association costs, and combining IoU and center distance weighted to form a matching cost matrix. The tracker maintains a state set for each track, including track_id, the center point sequence of the most recent N frames (trajectory), the count of undetected frames (miss_frames), and a counted flag (counted_flag).

[0056] The key implementation points are as follows: During the target association process, based on the geometric position information of the detection box, the association cost is comprehensively adopted using target position features (center point), detection box area, and appearance features, where lightweight re-identification (ReID) features can be optionally used for appearance features; and an association cost matrix for target matching is constructed by weighted fusion of the intersection-over-union ratio (IoU) and the center point distance. Simultaneously, the tracker maintains a corresponding state set for each target trajectory, which includes at least: a trajectory identifier (track_id), a sequence of center point positions within the most recent N frames (trajectory), a count of consecutive undetected frames (miss_frames), and a counting flag (counted_flag) to indicate whether the trajectory has triggered a counting operation, supporting subsequent trajectory analysis and counting logic processing.

[0057] V. Trajectory Motion Judgment and Counting Decision Please see Figure 2 This is a schematic diagram of the drill retraction counting logic according to an embodiment of the present invention.

[0058] After obtaining the time center point sequence corresponding to each target trajectory, determine whether the trajectory corresponds to a valid lever retraction action according to the following steps, and trigger counting when the judgment condition is met.

[0059] Step 1: Track Validity Filtering Judge the validity of the length of the target trajectory. If the number of frames included in a certain trajectory is n, when n < N_min, it is considered that the length of this trajectory is insufficient, and the back-off rod judgment and counting processing are not performed on it temporarily to avoid false triggering caused by noises such as short-term jitter and misdetection. Among them, N_min is the preset minimum trajectory length threshold, for example, it can be set as N_min = 3.

[0060] Step 2: Cumulative displacement and average displacement calculation Let the normalized coordinate sequence of the center points of a certain target trajectory in n consecutive frames be:

[0061] Among them, is the normalized abscissa of the center point of the target in the first frame of the trajectory; is the normalized ordinate of the center point of the target in the first frame of the trajectory.

[0062] Define the vertical displacement between two adjacent frames, which is defined as:

[0063] On this basis, calculate the cumulative vertical displacement of this trajectory within the entire time window and the average vertical displacement

[0064] By normalizing the center point coordinates, the displacement discrimination threshold has good consistency under different image resolution conditions, thereby improving the generality of the algorithm.

[0065] It should be noted that the sign of the average vertical displacement avg_dy is related to the installation direction of the camera. In the default installation method of the present invention, it is assumed that the target shows a downward movement trend in the image during the back-off process of the drill pipe, that is, the value of the vertical coordinate y increases with time; if the actual installation direction of the camera is opposite to the above assumption, the sign of the judgment threshold can be flipped by deploying the parameter Direction Sign to adapt to different installation conditions.

[0066] Please refer to Figure 3 , which is the state machine schematic diagram of a single tracking target in an embodiment of the present invention.

[0067] Step 3: Direction judgment and time gating processing After the trajectory validity filtering and displacement calculation are completed, perform direction judgment and time gating processing on the target trajectory to judge whether a valid back-off rod count is triggered.

[0068] Specifically, when the average vertical displacement of the target trajectory satisfies the following condition: avg_dy×DirectionSign>=DyThreshold Wherein, DyThreshold is a preset displacement judgment threshold, for example, it can be set to DyThreshold=0.001; at the same time, if the time interval since the last counting event or trajectory state change is not less than the preset minimum time interval threshold MinIntervalSecond, for example, it can be set to MinIntervalSeconds=3 s, then the target trajectory is determined to correspond to a valid lever retraction action.

[0069] When a valid lever retraction action is determined, a counting operation is triggered, and the counted_flag of the corresponding trajectory is set to true. At the same time, the timestamp m_lastCountTimem of this count is recorded.

[0070] The average vertical displacement of the target trajectory satisfies the following condition: avg_dy×DirectionSign<=-DyThreshold When the movement trend is opposite to the direction of the lever return, the trajectory can be regarded as the lever advance action or abnormal operation, and the counting reset and the status set to abnormal processing logic will be triggered. For details of the relevant abnormal handling process, please refer to step four.

[0071] To prevent the same drill pipe from being counted repeatedly in consecutive frames, when the counted_flag of a certain trajectory has been set to true, that trajectory will no longer trigger new counting events in subsequent frames until the target corresponding to that trajectory leaves the field of view and is cleared from mapCounted. The system maintains an independent Track state machine for each detected drill pipe target to achieve reliable control over the drill pipe retraction counting process.

[0072] Step 4: Handling Abnormal Scenarios (Pole Advancement or Mixed Actions) During the direction determination and time gating process, if the average vertical displacement avg_dy of the target trajectory is detected to be significantly reversed in the direction determination, and this reverse motion continues to exist in consecutive frames, then the current operation process is determined to be inconsistent with the standard lever retraction process, and can be regarded as an abnormal operation scenario of lever advance or a combination of advance and retraction.

[0073] In the above abnormal scenario, the system performs a count reset process, sets the cumulative count value m_iTotalCountm to 0, and synchronously updates the time of the most recent state change m_lastChangeTime to prevent the counting logic from being circumvented or the count from being "brushed" by short-term human intervention (such as throwing the drill rod up first and then pulling it down).

[0074] The count reset behavior can be set as a configurable strategy item, specifically including: enabling count reset processing, triggering only an abnormal alarm without executing count reset, or disabling abnormal processing logic, to adapt to the needs of different mine operation scenarios and safety supervision strategies.

[0075] Step 5: Timeout Reset and ID Cleanup If the system detects that the duration since the last valid counting event exceeds the preset timeout threshold ResetSeconds and no new counting result has been generated, for example, ResetSeconds=30, then it will automatically perform a count reset operation, clear the cumulative count value m_iTotalCountm to zero, and record the corresponding reset log information.

[0076] In addition, to avoid the problem of duplicate counting caused by the reassignment of target identifiers after the drill pipe target leaves the field of view, the system also periodically cleans up the trajectory IDs in mapCounted that have been confirmed to have left the field of view, so as to ensure that drill pipe targets that subsequently enter the field of view will not inherit the historical count status, thereby improving the accuracy and reliability of the counting results.

[0077] The calculation formula related to the lever release determination in this invention has been given in step two. Based on the determination formula and processing flow, the overall logic can be further explained in pseudocode form to facilitate engineering implementation. The pseudocode is as follows: for each frame: / / Process each frame of the image detect boxes ->boxes track.propagate_and_update(boxes) For each track in tracks: traj = track.get_recent_traj() if traj.length <N_min: continue total_dy = sum((traj[i].y - traj[i-1].y) / H for i in 1..n-1) avg_dy = total_dy / (n-1) if track.counted: if avg_dy DirectionSign>= DyThreshold: if now - lastChangeTime>= MinIntervalSeconds: totalCount += 1 track.counted = true lastCountTime = now lastChangeTime = now elif avg_dy DirectionSign<= -DyThreshold: # Abnormality: Rod advance or mixed operation track.counted = true totalCount = 0 lastCountTime = now lastChangeTime = now VI. Visual and Logical Robustness Enhancement Mechanisms (Multiple Error Prevention Strategies) To improve the robustness of the system in detection and counting under complex downhole conditions, this invention introduces multiple error prevention strategies at the levels of visual detection and temporal logic determination. These strategies can be combined and executed in parallel or serial manner depending on the specific implementation, and specifically include: (1) Spatial constraint strategy: Target detection and multi-target tracking are only performed within the pre-defined region of interest (ROI) to limit the processing range of the algorithm and reduce the interference of personnel, equipment or background changes in non-operation areas on the detection results.

[0078] (2) Time constraint strategy: By setting the minimum counting time interval parameter MinIntervalSeconds, the counting operation is restricted from being repeatedly triggered in a short period of time, thereby suppressing false counting caused by jitter, short-term round-trip motion, etc.

[0079] (3) Trajectory integrity constraint strategy: Only target trajectories that meet the trajectory stability conditions are counted and judged in the final count. The stability conditions include the target trajectory having left the field of view, or the trajectory showing small speed fluctuations and stable motion trend in continuous frames, so as to avoid misjudging incomplete or abnormal trajectories.

[0080] (4) ID management strategy: The system maintains a counted target identifier table mapCounted to record the trajectory IDs that have been triggered for counting; when the corresponding trajectory is confirmed to have left the field of view, the corresponding ID is deleted from mapCounted in order to support ID reuse while avoiding the same target being counted repeatedly.

[0081] (5) Integrated judgment strategy: In the process of retraction judgment, instead of relying on a single judgment index, multiple indicators such as average vertical displacement avg_dy, target motion speed amplitude, detection confidence and trajectory direction consistency (e.g. trajectory direction standard deviation) are weighted and combined to make judgment, thereby reducing the risk of misjudgment caused by abnormal fluctuations of a single index.

[0082] Through the synergistic effect of the above-mentioned multiple visual and logical constraint mechanisms, the accuracy and stability of the lever counting under dust obstruction, lighting changes, personnel interference, and complex operational behaviors are significantly improved.

[0083] To further improve the robustness of the stick withdrawal action judgment, a comprehensive judgment score based on multi-index fusion can be introduced for thresholding or adaptive threshold adjustment of candidate trajectory segments. For example, the comprehensive judgment score can be expressed as:

[0084] Wherein, conf_mean is the mean of target detection confidence within the trajectory segment, std_dir is the standard deviation of trajectory direction change, and the smaller the standard deviation, the higher the consistency of trajectory direction; each weight coefficient w_i can be set according to field experience and fine-tuned online after system deployment using small batch data. When the comprehensive judgment score satisfies Score>=ScoreThreshold, the counting operation is triggered.

[0085] VII. Visualized Alarms and Training Iteration Strategies To facilitate on-site monitoring and post-event auditing, the system manages the drill pipe ejection count results using both real-time visual display and historical data recording.

[0086] In terms of real-time display, the system outputs the current cumulative count result through a visual interface, and simultaneously displays the identifier (track_id) of each target trajectory and the corresponding vertical displacement characteristic value (dy). It also overlays the trajectory motion of the most recent N frames and provides the original video playback link associated with the counting event so that operators can intuitively check the counting process.

[0087] Regarding the alarm strategy, an alarm is triggered when abnormal operation behavior is detected. These abnormal operations include, but are not limited to: an abnormal pole movement where the average vertical displacement (avg_dy) of the target trajectory reverses and exceeds a preset threshold, or a rapid fluctuation in the counting result within a unit of time with a change exceeding a preset threshold. After an alarm is triggered, the system automatically saves the corresponding video clip and trajectory data for subsequent manual review and security auditing.

[0088] In addition, the system saves complete event metadata for each valid counting event. The metadata includes at least the counting timestamp, the corresponding trajectory identifier (track_id), the average vertical displacement avg_dy, the detection confidence mean conf_mean, and the associated frame screenshot path, thereby supporting the traceability and auditing requirements of the counting results.

[0089] In terms of model training and iteration, a two-stage training strategy is preferred. The first stage trains the target detection model on an offline dataset containing multiple scenarios and working conditions to obtain a basic model with general detection capabilities. The second stage fine-tunes the model on a small number of labeled samples collected in the target mine environment to adapt to specific camera installation angles, lighting conditions, and working backgrounds.

[0090] Meanwhile, a "confusion sample pool" is constructed to centrally label and train easily confused scenario samples such as suspected water injector samples, fuzzy drill pipe samples, and multi-rod adhesion samples, so as to continuously improve the model's generalization ability and stability in complex downhole environments.

[0091] VIII. Alternative Solutions and Optional Modules Without departing from the core technical concept of this invention, the system structure and functional modules can be replaced or expanded according to actual application requirements.

[0092] Required modules: The basic system of this invention includes at least the following essential modules: a camera device for acquiring drilling operation video, a frame-level target detector for analyzing video frames, an online multi-target tracker for maintaining target identity across frames, a counting and status management logic module for performing drill pipe retraction determination and counting, and a visualization interface module for storing and displaying results. These modules work together to achieve automatic identification and counting of the drill pipe retraction process.

[0093] Optional enhancement modules: Based on the above basic system architecture, the following optional enhancement modules can be introduced as needed to further improve system performance and robustness: (1) Appearance features / ReID module: A re-identification (ReID) module based on appearance features is introduced to enhance the re-identification capability of the same drill pipe target when the target is temporarily occluded or the time interval is large, thereby reducing the trajectory ID breakage problem caused by the failure of cross-union ratio (IoU) matching.

[0094] (2) Optical flow detection module: An optical flow analysis module is introduced to detect the motion presence of targets in scenarios where computing resources are limited or the target texture is weak. The optical flow module can run in parallel with the target detector, and is preferably enabled only when the detector confidence is insufficient, so as to improve the robustness of detection of weak texture targets while reducing the overall computational load.

[0095] (3) Secondary discrimination module: A secondary discriminator based on key point detection or template matching is introduced to assist in the verification or cross-verification of the detection results of the main detector, so as to further reduce the false detection probability and improve the overall judgment reliability.

[0096] By flexibly combining and configuring the above optional enhancement modules, the system performance can be optimized in a targeted manner under different computing power conditions, operating environments and regulatory requirements, without affecting the basic counting principle and technical effect of the present invention.

[0097] IX. Precautions for Project Implementation To meet the requirements of real-time performance and system stability in the mine, the system architecture can be reasonably deployed and optimized during the engineering implementation process, taking into account the actual computing power and network environment.

[0098] At the system architecture level, it is preferable to perform basic preprocessing and frame sampling operations on the acquired video data at the edge, such as noise reduction, cropping, and frame rate control, while deploying computationally intensive target detection and multi-target tracking tasks on ground analysis servers or edge computing boxes with GPU acceleration capabilities, so as to reduce the load on front-end devices while ensuring processing performance.

[0099] In scenarios with limited network conditions, video data can be cropped by region of interest (ROI) before encoding and transmission, thereby effectively reducing network bandwidth consumption and ensuring image quality in critical work areas.

[0100] Regarding system operation and maintenance, to reduce the false alarm rate and improve the long-term reliability of the system, configurable online threshold parameters are preferentially set in the production environment, and an A / B testing mechanism is introduced to compare and evaluate different parameter configurations or model versions. The system configuration is iterated step by step through the above methods, and typical false detection samples are continuously collected as a data source for subsequent model updates and training optimization.

[0101] This invention addresses the problems of drill pipe retraction counting being susceptible to interference, miscounting, and human intervention during downhole drilling operations. It proposes an automatic counting method combining visual perception and temporal logic. Its key innovations are mainly reflected in the following aspects: (1) Strict data labeling and detection target definition mechanism: By formulating strict data labeling standards, only composite areas containing both "male head" and "shaft" are detected, thus avoiding interference from visual recognition by similar-looking components such as water injectors and improving the uniqueness and stability of detection targets.

[0102] (2) Deep fusion mechanism of detection and multi-target tracking: Frame-level target detection is combined with online multi-target tracking. Stable temporal motion information is obtained through continuous trajectory maintenance, providing a reliable data foundation for subsequent direction determination and counting logic.

[0103] (3) Direction determination and counting strategy based on normalized average vertical displacement: A method for determining the direction of the lever retraction based on the normalized average vertical displacement (avg_dy) of the trajectory center point is proposed. By eliminating the influence of resolution difference, the accurate identification and counting of the lever retraction action can be achieved.

[0104] (4) Comprehensive design of multiple robustness and anti-cheating mechanisms: introduce time window constraints, timeout reset mechanism and mapCounted-based counted trajectory management strategy to improve the robustness of the system to abnormal operation, short-term disturbance and human intervention from multiple dimensions such as time, state and trajectory management, and realize reliable automatic counting.

[0105] This invention deeply integrates structured target detection with online multi-target tracking, trajectory-based motion direction discrimination, and temporal logic verification, significantly improving the robustness and accuracy of drill pipe retraction counting under complex downhole conditions. Specific technical advantages are reflected in the following aspects: (1) Improve counting accuracy: Only candidate frames that meet the "male head + rod body" combination are counted, which significantly reduces the misjudgment of the water injector; the trajectory direction determination further reduces the miscounting caused by confusion between advancing and retreating.

[0106] (2) Enhance robustness and generalization ability: The time-based trajectory judgment does not rely on single-frame features, enabling the system to work stably under different drilling rig models, slight angle changes, or occlusion conditions; the data labeling standard helps the training set to be more consistent with the actual deployment scenario, thus improving the model's generalization.

[0107] (3) Anti-interference and anti-cheating: The introduction of time window, timeout reset and mapCounted management table can effectively avoid repeated counting in a short period of time, repeated counting caused by ID reuse, and false counting caused by artificial jitter interference.

[0108] (4) Strong auditability: Each counting event is accompanied by track_id, short video segment and key frame screenshot, which facilitates post-event review and evidence collection.

[0109] (5) Easy to deploy and configurable: The thresholds (ResetSeconds, T_minInterval, etc.) can be flexibly adjusted in the visualization interface to adapt to different downhole scenarios; the algorithm can run on edge servers or low computing power platforms (with lightweight models).

[0110] Overall, this invention offers significant improvements in accuracy, robustness, and auditability compared to single-frame detection or pure sensor solutions, and is particularly suitable for industrial scenarios with high noise, low light, and diverse equipment models in underground mines.

[0111] This invention proposes an AI-based vision-based method and system for counting drill rod withdrawals under harsh downhole conditions. Through strict "male end + rod body" labeling specifications, frame-level detection and multi-target tracking, normalized trajectory displacement calculation, direction and timing gating, and multiple error prevention strategies (time window, timeout reset, mapCounted management, and anomaly reset), a high-precision, robust, and auditable automatic drill rod withdrawal counting solution is achieved. This invention is easy to deploy in engineering projects and can significantly reduce the miscounting and undercounting rates in actual mine operations, improving the reliability of production supervision.

[0112] This invention integrates target detection, multi-target tracking, and trajectory time sequence analysis to determine the direction of motion and state changes during the drill pipe retraction process, thereby achieving automatic identification and accurate counting of the retraction action.

[0113] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A drilling rod counting method based on AI vision, characterized in that, Includes the following steps: S1. Video Acquisition and Preprocessing: Video data of the drill rod advancing and retreating process is acquired by a camera device installed above the drill rig's power head. The acquired video frames are preprocessed at the frame level. The frame-level preprocessing includes at least noise reduction, brightness normalization, and / or frame sampling, so that the preprocessed video frames can be used as input data for subsequent analysis. S2. Drill pipe target detection: Based on the preprocessed video frames, drill pipe target detection is performed in the preset region of interest. Only composite regions containing both the drill pipe male head and the drill pipe body are detected. The drill pipe male head and the drill pipe body have distinguishable structural features. The corresponding detection box and detection confidence are output. S3. Multi-target tracking and trajectory generation: Perform online multi-target tracking processing on the drill pipe target, assign a unique trajectory identifier to different drill pipe targets, and maintain the motion trajectory of the corresponding drill pipe target in continuous video frames. The motion trajectory includes at least a sequence of the position of the detection box center point changing over time. S4. Trajectory Displacement Calculation: For each valid drill pipe target trajectory, extract its center point position in consecutive video frames, and normalize the vertical coordinates of the center point based on the video frame height. Calculate the displacement change of the trajectory in the vertical direction to obtain the average normalized vertical displacement within the trajectory segment. S5. Direction Determination and Drill Pipe Retraction Count: Based on the continuous motion trend of the trajectory, and combined with the average normalized vertical displacement obtained in S4 and the preset direction parameters, the motion direction of the drill pipe target is determined: when the average normalized vertical displacement meets the preset positive threshold condition, and the time interval since the last valid counting event is not less than the preset minimum time interval, the corresponding trajectory is determined to be a valid drill pipe retraction action, and a drill pipe retraction count is triggered; when the average normalized vertical displacement meets the reverse threshold condition, the corresponding trajectory is determined to be a drill pipe advance or abnormal operation. S6. Anomaly detection and status update: When the average normalized vertical displacement satisfies the detection condition that is opposite to the direction of the rod retraction, the corresponding trajectory is determined to be rod advance or abnormal operation, and the corresponding status update process is performed on the current counting state.

2. The drilling rod counting method based on AI vision according to claim 1, characterized in that, In S2: the following labeling constraints are applied to the sample data used for training the detection model: the composite structure region containing both the drill pipe male end and the drill pipe body is labeled as a positive sample and assigned a unified target category label; wherein, the male end has at least half of the visible area of ​​the complete male end, and the shaft has at least clearly identifiable coarse and sparse thread texture features; when the visible part of the male end is less than half of the full male end or the thread texture features of the shaft body are not identifiable, the corresponding target region is not labeled as a positive sample.

3. The drilling rod counting method based on AI vision according to claim 2, characterized in that: In the process of constructing sample data for training the detection model, water injectors containing only the male end and without coarse and sparse thread texture features on the shaft, drill rod images with severe blur or motion blur, and other tools or components with similar shapes to drill rods but without drill rod structural features are uniformly included as negative samples in the training data. When labeling the positive samples with bounding boxes, the bounding boxes are closely attached to the minimum bounding rectangle of the male end and shaft combination structure and the background area is reduced to reduce the interference of irrelevant background information on the training of the detection model.

4. The drilling rod counting method based on AI vision according to claim 1, characterized in that: In step S2: a single-stage or two-stage target detection network is used as the detection model to detect the male end and shaft structure of the drill rod in each frame of the video sequence; during the detection process, target detection processing is only performed on a preset region of interest in the input image, which is set along a preset horizontal band in the image to reduce computation and reduce environmental interference in non-working areas; the output of the detection model for each frame includes at least the target detection box, the corresponding detection confidence score, and the target category probability; and non-maximum suppression processing and filtering processing based on the detection confidence score threshold are sequentially performed on the output of the detection model to obtain a set of candidate detection boxes in the current frame, wherein the detection confidence score threshold is a configurable parameter.

5. The drilling rod counting method based on AI vision according to claim 4, characterized in that: After obtaining the candidate detection box set, the position information of the detection boxes is normalized: if the height of the current image frame is... H If the pixel coordinates of a point in the image in the vertical direction are y, then its normalized coordinates y′ are: y'=y / H Based on the detection output, and for subsequent trajectory motion analysis and displacement calculation, the normalized coordinates of the center point of the target single-frame detection box are extracted as follows: in, For the first i The x-coordinate of the center pixel of the target detection box in the frame; For the first i The normalized coordinates of the target's center point in the horizontal direction within the frame; For the first i The ordinate of the center pixel of the target detection box in the frame; For the first i The normalized coordinates of the target's center point in the vertical direction within the frame; The pixel width of the current image frame.

6. The drilling rod counting method based on AI vision according to claim 1, characterized in that: Specifically, S3 involves: based on the drill rod target detection results, performing cross-frame target association and trajectory consistency maintenance on the drill rod targets, assigning a unique trajectory identifier to the same drill rod target, and maintaining the corresponding motion trajectory in consecutive video frames; the cross-frame target association is completed based on the geometric position information of the detection box and combined with the motion prediction results.

7. The drilling rod counting method based on AI vision according to claim 6, characterized in that: In the cross-frame target association process, the target center point position, detection box area, and optional appearance features are used as association costs, and a matching cost is constructed by weighted fusion of the detection box intersection-union ratio and the center point distance; a trajectory state set is maintained for each drill pipe target trajectory, and the trajectory state set includes at least a trajectory identifier, a center point position sequence, an undetected frame count, and a count flag.

8. The drilling rod counting method based on AI vision according to claim 1, characterized in that: In step S4: For each drill pipe target trajectory that passes the trajectory validity filter, extract the trajectory in continuous... n The location of the center point of the detection box in the video frame is determined, and the pixel coordinates of the center point in the vertical direction are normalized based on the image height of the current video frame. The vertical displacement change of the center point between adjacent video frames is calculated, and the vertical displacement change is accumulated and averaged to obtain the average normalized vertical displacement of the drill rod target trajectory within the corresponding time window, which is used as a feature quantity for subsequent rod retraction direction determination.

9. The drilling rod counting method based on AI vision according to claim 8, characterized in that: In step S5: the movement direction of the drill pipe target trajectory is determined based on the average normalized vertical displacement and in combination with preset direction parameters; when the product of the average normalized vertical displacement and the direction parameters is greater than or equal to a preset displacement determination threshold, and the time interval since the last valid drill pipe retraction count event or trajectory state change is not less than a preset minimum time interval threshold, the corresponding drill pipe target trajectory is determined to be a valid drill pipe retraction action, and a drill pipe retraction count is triggered; and after triggering the drill pipe retraction count, the corresponding drill pipe target trajectory is marked as counted.

10. The drilling rod counting method based on AI vision according to claim 1, characterized in that: In S6: when the average normalized vertical displacement of the drill pipe target trajectory is detected to be opposite to the direction of pipe retraction in terms of direction determination, and this reverse movement continues to exist in multiple consecutive video frames, the current operation process is determined to be an abnormal operation scenario of pipe advance or mixed advance and retraction; and in the abnormal operation scenario, the cumulative pipe retraction count result is reset, and the time of the most recent state change is updated.

11. The drilling rod counting method based on AI vision according to claim 1, characterized in that: If it is detected that the duration since the last valid rod retraction count event exceeds the preset timeout threshold and no new rod retraction count result is generated, the count reset operation is automatically executed to clear the cumulative rod retraction count value to zero and record the corresponding count reset log information; at the same time, the system clears the target trajectory markers of drill rods that have triggered the count and are confirmed to have left the camera's field of view.

12. The drilling rod counting method based on AI vision according to claim 1, characterized in that: To enhance counting robustness, a multi-visual and temporal logic constraint strategy is employed, wherein the constraint strategy includes at least the following: Spatial constraints: Drill pipe target detection and multi-target tracking are performed only within a pre-defined area of ​​interest to reduce interference from personnel, equipment, or background changes outside the work area on the detection results; Temporal constraints: By setting a minimum counting time interval parameter, the repeated triggering of the drill pipe retraction counting operation is limited within a preset time window; Trajectory integrity constraints: Drill pipe target trajectory retraction counting is only performed on drill pipe target trajectories that meet the trajectory stability conditions. The trajectory stability conditions include at least that the target trajectory has left the camera's field of view, or that it exhibits a stable motion trend and small speed fluctuations in continuous video frames; Trajectory identifier management: The identifier information of drill pipe target trajectories that have been triggered for counting is maintained, and the corresponding trajectory identifier is deleted after the corresponding trajectory is confirmed to have left the field of view.

13. The drilling rod counting method based on AI vision according to claim 12, characterized in that: When performing rod retraction determination on the target trajectory of the drill pipe, a comprehensive determination mechanism based on multi-index fusion is adopted to perform threshold determination or adaptive threshold adjustment on candidate trajectory segments; The comprehensive judgment mechanism includes at least a weighted combination of the following indicators: the average normalized vertical displacement within the trajectory segment; The amplitude of the drill pipe target's motion velocity; the mean value of the trajectory segment of the target detection confidence score; the trajectory direction consistency index, which is calculated based on the standard deviation of trajectory direction changes; when the comprehensive judgment score obtained by weighting the multiple indicators meets the preset judgment threshold condition, the corresponding drill pipe target trajectory retraction counting operation is triggered.

14. The drilling rod counting method based on AI vision according to claim 1, characterized in that: The drill pipe ejection count results are processed using both real-time visualization and historical data recording methods. The processing includes: Real-time display: Outputs the current cumulative lever retraction count result through a visual interface, and simultaneously displays the identifier (track_id) of each target trajectory and the corresponding vertical displacement characteristic value (dy). It also overlays the trajectory motion of the most recent N frames and provides the original video playback link associated with the counting event so that operators can intuitively check the counting process. Alarm Trigger: When abnormal operation behavior is detected, alarm processing is triggered. The abnormal operation includes the abnormal situation where the average vertical displacement (avg_dy) of the target trajectory is reversed and the amplitude exceeds the preset threshold, or the counting result within a unit time fluctuates rapidly and the change amplitude exceeds the preset threshold. Data storage and backtracking: The system automatically saves video clips and trajectory data when the alarm is triggered for subsequent manual review and security audit; Event metadata: The system saves metadata for each valid lever retraction count event. The metadata includes at least the count timestamp, track identifier (track_id), average vertical displacement (avg_dy), detection confidence mean (conf_mean), and associated frame screenshot path to support the traceability and auditing requirements of the count results. Model training and iteration: A two-stage training strategy is adopted. In the first stage, the detection model is trained on offline datasets in multiple scenarios. In the second stage, the model is fine-tuned on labeled samples collected in the target mine environment based on camera viewpoint and lighting conditions.

15. A drilling rod counting system based on AI vision, characterized in that, This includes a front-end data acquisition unit, a mine ring network, a ground analysis server, and a visual alarm client, among which: The front-end acquisition unit includes an industrial camera device installed above the drill rig's power head, used for continuous video acquisition of the drill rod retraction process; the industrial camera device is installed in a fixed manner, and its installation position, viewing angle, and imaging parameters remain stable after debugging; the front-end acquisition unit is used to perform local preprocessing operations on the acquired video frames; The mine ring network is used to transmit video data, which has been pre-processed locally at the front-end acquisition terminal, to the ground analysis server. A ground analysis server, connected to the mine ring network, is used to analyze and process received video data. The ground analysis server includes: The video preprocessing module is used to perform frame-level preprocessing on the input video frames to reduce noise interference and achieve brightness normalization. The target detection module, connected to the video preprocessing module, is used to perform target detection based on the preprocessed video frames and identify the drill pipe composite region that simultaneously contains the drill pipe male head and the drill pipe body. A multi-target tracking module, connected to the target detection module, is used to perform multi-target tracking on the detected drill pipe targets, assign a unique identifier to different drill pipe targets and maintain their motion trajectories; A counting and logic verification module, connected to the multi-target tracking module, is used to perform rod retraction counting determination based on the motion trajectory of the drill rod target. The counting and logic verification module includes: The trajectory direction determination unit is used to analyze the motion direction of the drill rod target in the image based on the trajectory information obtained from multi-target tracking, calculate the normalized displacement of the drill rod target in the vertical direction, and determine whether the corresponding trajectory meets the determination condition of the rod retraction direction based on the displacement direction. The counting logic and state management unit is used to combine the trajectory direction determination result and the time constraint condition to execute the lever retraction counting logic and state management operation. The state management operation includes at least time window control, deduplication judgment and timeout reset. The data storage module, connected to the counting and logic verification module, is used to store and manage the final lever count result, abnormal information and related log data. The visual alarm client communicates with the ground analysis server to display the drill pipe ejection count in real time and issue alarm prompts when abnormal counting behavior is detected.

16. The drilling rod counting system based on AI vision according to claim 15, characterized in that: The ground analysis server is configured to execute the AI ​​vision-based drilling rod counting method according to any one of claims 1–14.

17. A drilling rod counting system based on AI vision according to claim 16, characterized in that, Also includes: Timeout Reset and Trajectory Management Module: When the preset timeout period is exceeded and no new valid drill rod retraction count event is detected, the module automatically clears the cumulative count result and cleans up the target trajectory markers of drill rods that have been counted and have left the camera's field of view, so as to avoid duplicate counting caused by the reuse of trajectory markers.

Images