A Multimodal Fusion Identification Method for Intelligent Drill Pipe Recognition and Counting in Drilling Site Construction

Abstract

This invention relates to the fields of intelligent mining and industrial artificial intelligence, and discloses a multimodal fusion-based intelligent identification and counting method for drill rods in drilling operations. By synchronously acquiring and time-aligning video data and operating condition signals during drilling operations, a deep learning-based visual detection model of drill rod operating status is constructed to identify drill rod connection, disconnection, and power head movement status. Based on this, a multimodal status determination mechanism is established by integrating power head propulsion signals, chuck rotation signals, and clamping signals to jointly identify key operational processes such as rod retraction, wire retraction, and drill rod disconnection. The drill rod disconnection event is used as the counting trigger condition, and the counting is corrected by incorporating drill rod loading behavior, thereby achieving automatic counting of drill rods. Finally, the drilling depth is calculated based on the counting results, and the process data and results are uploaded and stored. This invention has advantages such as high counting accuracy, strong anti-interference ability, good traceability, and high degree of automation.

Description

Technical Field

[0001] This invention belongs to the field of intelligent mining and industrial artificial intelligence technology, specifically relating to an intelligent identification and counting method for drill rods in drilling operations based on multimodal fusion identification, and more particularly to an intelligent identification and counting method for drill rods that integrates video visual information and drilling rig operating signals. Background Technology

[0002] In coal mine production, drilling rigs are typically used to drill into coal seams or rock formations to achieve purposes such as gas extraction and water exploration, thereby ensuring safe mine production. Whether the drilling depth meets the design requirements directly affects the gas extraction effect and the safety of water exploration. Therefore, accurate detection of the drilling depth is a crucial aspect of drilling construction quality control.

[0003] In existing technologies, the drilling depth is typically determined manually. Specifically, this involves deploying cameras at the drilling site to record the entire drilling process. After completion, the video is manually reviewed to count the number of drill rods that entered the borehole, thus indirectly estimating the drilling depth. However, this method has the following drawbacks: firstly, relying on manual video review for counting is time-consuming, labor-intensive, and inefficient; secondly, the counting results are easily affected by factors such as operator attention, fatigue, and differences in judgment standards, leading to poor accuracy and consistency.

[0004] With the development of artificial intelligence and machine vision technologies, existing technologies have proposed drill pipe counting methods based on visual inspection. These methods use image recognition algorithms to detect and automatically count the drill pipe's entry and exit processes. While these methods improve the level of automation to some extent, they still have significant shortcomings in actual downhole operating environments. For example, downhole environments typically experience drastic changes in lighting, severe dust interference, frequent equipment obstruction, and easy contamination of cameras, leading to poor stability of visual inspection results. Furthermore, existing pure visual methods struggle to effectively distinguish between "effective drilling / retraction behavior" and "non-counting repetitive actions" such as probing, hole washing, and short-stroke reciprocating movements, which can easily result in miscounting or undercounting, affecting the accuracy of the final borehole depth determination.

[0005] Therefore, there is a need to provide an intelligent drill pipe identification and counting method that can operate stably under complex drilling conditions, effectively integrate multi-source information, and accurately distinguish between effective drill pipe operation behaviors and non-counting interference behaviors, so as to improve the accuracy and reliability of borehole construction depth detection.

[0006] The present invention aims to overcome the problems of low accuracy, poor stability and susceptibility to interference from complex working conditions in the existing technology of drill pipe counting which relies on manual or single visual means, and provides an intelligent identification and counting method for drill pipe. Summary of the Invention

[0007] The technical problem to be solved by this invention is: how to construct a drill rod status determination mechanism based on multimodal data fusion, which can accurately identify the drill rod operation status under complex working conditions by coordinating the analysis of video visual information and drilling rig operating condition signals, and automatically count the number of drill rods based on the identification results, so as to improve the accuracy and stability of the counting results.

[0008] To achieve the above objectives, the present invention provides the following technical solution: A multimodal fusion identification method for intelligent identification and counting of drill pipes in drilling operations includes the following steps: S1: Synchronously acquire and preprocess multimodal data during drilling operations. The multimodal data includes video data and drilling rig operating condition signal data. The operating condition signals include at least the output signal OUTdt of the power head propulsion power source, the output signal OUTdkx of the power head chuck rotation power source, and the output signal OUTdkj of the power head chuck clamping power source. The video data and operating condition signal data are then timestamped and time-aligned. S2: Construct a visual detection model of drill pipe operation status based on the collected video data, and use the visual detection model to analyze the real-time video stream to output the drill pipe connection status, drill pipe disengagement status and power head movement status; S3: Receive the drill pipe retraction count start command, initialize the drill pipe counter NUM=0, and synchronously acquire the current status of the working condition signal to enter real-time status monitoring; S4: Based on the output of the visual detection model and the working condition signal, perform multimodal fusion judgment and execute drill pipe counting and correction, including: When no visual feature of the power head retracting is detected and the operating condition signal does not meet the continuous output condition of OUTdt=-1, the status detection is continuously performed in a loop. When the power head is detected to be retracted or OUTdt=-1 meets the duration threshold condition, it is determined that the lever retraction state is entered. When OUTdkx=1 and OUTdkj=1, it is determined to be a valid lever retraction operation; When OUTdkx=-1 and OUTdkj=1, it is determined to be in the unwinding state; When OUTdt=0 and OUTdkx=0 are satisfied simultaneously, and the visual detection model detects the separation of the drill pipe from the power head, a drill pipe detachment event is determined to have occurred, and NUM=NUM+1 is executed. When the visual inspection model detects that the drill pipe is reconnected to the power head and the power head is in the forward state, and OUTdt=1, OUTdkx=1, and OUTdkj=1, it determines that it is a drill pipe loading operation and executes NUM=NUM-1. S5: Determine whether the drill rod counting end command has been received. If not, return to step S4 to continue execution. If received, output the drill rod counting result NUM and calculate the drilling depth based on the drill rod length.

[0009] Furthermore, in S1, when aligning the video data and the operating condition signal data in time, a time window Δt is used for synchronization buffering, where Δt is 50ms to 500ms.

[0010] Furthermore, the preprocessing of video data in S1 includes brightness normalization, contrast enhancement, and noise filtering, as well as jitter removal and outlier filtering of the operating signal.

[0011] Furthermore, the visual detection model in S2 is a deep learning-based target detection model or behavior recognition model. Its input is a video frame or a continuous frame sequence, and its output is the spatial position of the drill pipe and the power head and the corresponding status label.

[0012] Furthermore, in S4, the condition signal for determining the backward movement of the power head is OUTdt=-1 and the duration is greater than or equal to the preset threshold T1, so as to filter out instantaneous interference signals.

[0013] Furthermore, in step S5, the drilling depth H is calculated based on the drill rod count result NUM and the length L of a single drill rod, satisfying: H = NUM×L.

[0014] A smart identification and counting device for drill pipes used in drilling operations includes: The data acquisition module is used to acquire video data and drilling rig operating signal data; The data synchronization and preprocessing module is used to perform time synchronization and preprocessing on the video data and operating condition signal data. The visual recognition module is used to analyze video data based on a visual detection model and output the drill pipe connection status, drill pipe disengagement status, and power head movement status. The status determination module is used to fuse and determine the drilling rod operation status based on visual recognition results and working condition signals; The counting control module is used to perform drill pipe counting and correction operations based on the state determination result, including executing NUM=NUM+1 when a drill pipe detachment event is detected, and executing NUM=NUM-1 when an up drill pipe operation is detected; The counting management and output module is used to determine the counting termination condition and output the counting result.

[0015] An electronic device, comprising: processor; Memory; Communication interface; The memory stores a computer program, which, when executed by a processor, causes the processor to perform a multimodal fusion judgment method for intelligent identification and counting of drill rods in drilling sites.

[0016] A computer-readable storage medium having a computer program stored thereon: When the computer program is executed by the processor, it implements the multimodal fusion identification method for intelligent identification and counting of drill pipes in drilling operations. The beneficial effects of this invention are: By integrating video visual information with drilling rig operating signals, a multimodal collaborative identification mechanism is constructed to achieve joint perception and intelligent judgment of the drill pipe operation process. Compared with traditional methods that rely on manual playback or single visual detection, this invention significantly improves the automation level and operational efficiency of drill pipe counting, and effectively reduces subjective errors caused by human intervention.

[0017] By introducing multi-dimensional operating condition signals such as the power head's motion state, chuck rotation state, and clamping state, and cross-validating them with visual inspection results, the system can maintain stable recognition performance under complex downhole environments (such as changes in lighting, dust interference, and equipment obstruction), significantly improving system robustness and reliability. Simultaneously, this invention establishes a drill pipe detachment event-triggered counting mechanism and combines it with drill pipe behavior recognition to achieve counting correction, effectively avoiding miscounting and undercounting problems, further improving counting accuracy.

[0018] Furthermore, this invention associates and records key event images and corresponding operating condition signals during the counting process, enabling traceability and auditability of the counting process and enhancing the reliability of the results. The overall solution possesses advantages such as strong real-time performance, good adaptability, and high engineering practicality, providing reliable data support for borehole construction quality assessment and production decision-making.

[0019] Other advantages, objectives, and features of the invention will be set forth in part in the description which follows, and in part will be apparent to those skilled in the art from the following examination, or may be learned from practice of the invention. The objectives and other advantages of the invention can be realized and obtained through the following description. Attached Figure Description

[0020] To make the objectives, technical solutions, and advantages of the present invention clearer, the preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings, wherein: Figure 1This is a schematic diagram of the overall process of a multimodal fusion identification method for intelligent identification and counting of drill pipes in drilling field construction, provided in an embodiment of the present invention. Figure 2 This is a schematic diagram of the sub-process for identifying the forward and backward states of the drilling rig power head in an embodiment of the present invention; Figure 3 This is a schematic diagram of a sub-process for identifying the process of raising and lowering the drill rod in an embodiment of the present invention. Detailed Implementation

[0021] The following specific examples illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of the present invention. Unless otherwise specified, the following embodiments and features can be combined with each other.

[0022] The accompanying drawings are for illustrative purposes only and are schematic diagrams, not actual pictures. They should not be construed as limiting the invention. To better illustrate the embodiments of the invention, some parts in the drawings may be omitted, enlarged, or reduced, and do not represent the actual product dimensions. It is understandable to those skilled in the art that some well-known structures and their descriptions may be omitted in the drawings.

[0023] In the accompanying drawings of the embodiments of the present invention, the same or similar reference numerals correspond to the same or similar components. In the description of the present invention, it should be understood that if terms such as "upper," "lower," "left," "right," "front," and "rear" indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, they are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, the terms used to describe positional relationships in the drawings are only for illustrative purposes and should not be construed as limiting the present invention. For those skilled in the art, the specific meaning of the above terms can be understood according to the specific circumstances.

[0024] To address the problems of low accuracy in drill pipe counting, susceptibility to interference from complex working conditions, and difficulty in distinguishing between effective operational behaviors and non-counting repetitive actions in existing technologies, this invention provides a multimodal fusion-based intelligent identification and counting method for drill pipes in drilling operations.

[0025] This invention constructs a drill rod operation status determination mechanism based on multi-source information by synchronously acquiring and fusing video visual information and drilling rig drilling condition signals. It jointly identifies drill rod advance, retraction and related operation actions, and establishes drill rod counting trigger rules based on the status determination results, thereby realizing automatic counting of drill rod quantity and correction of miscounts.

[0026] In one embodiment, the present invention utilizes the characteristic of drill rods being withdrawn one by one during the drill rod withdrawal process. By detecting and counting drill rod detachment events during the withdrawal process, and combining the state recognition results during the drill rod loading process to correct the count, the accurate determination of the drilling depth can be achieved.

[0027] Please see Figure 1 This is a schematic diagram of the overall process of a multimodal fusion identification method for intelligent identification and counting of drill pipes in drilling field construction, provided by an embodiment of the present invention. The process includes steps such as multimodal data acquisition and preprocessing, visual detection model construction, drill pipe retraction counting initialization, drill pipe state determination and counting based on multimodal fusion, and counting end and result output.

[0028] Figure 2 This is a schematic diagram of a sub-process for identifying the forward and backward states of the drilling rig's power head in an embodiment of the present invention. This sub-process is used to determine the motion state of the power head based on visual detection results and the power head propulsion signal, as one of the trigger conditions for retracting the rod.

[0029] Figure 3 This is a schematic diagram of a sub-process for identifying the process of drilling rods moving up and down in an embodiment of the present invention. This sub-process is used to combine visual detection results with chuck rotation signals and clamping signals to determine the connection, unwinding, and disengagement status of the drill rods, and trigger drill rod counting and count correction operations accordingly.

[0030] S1: Synchronous acquisition and preprocessing of multimodal data In this embodiment, multi-source data during drilling operations are first synchronously acquired and preprocessed, specifically including the following: (1) Video data acquisition At least one camera device is deployed in the drilling rig operating area to continuously capture video of the drilling operation process. In a preferred embodiment, the camera device is positioned to cover the area where the drilling rig power head and drill rod are connected, so as to ensure that key action images of the drill rod entering and exiting the hole and the connection / disconnection process can be obtained. In some embodiments, a multi-view camera device can be used to reduce the impact of obstruction.

[0031] (2) Drilling rig operating condition signal acquisition Synchronously acquire operating condition signals during the drilling rig's operation, wherein the operating condition signals include at least: The power head propels the power source output signal OUTdt, which takes the value {-1, 0, 1}, representing backward, stop, and forward respectively; The power source output signal OUTdkx for the rotation of the power head chuck has a value of {-1, 0, 1}, which represents reverse rotation, stop, and forward rotation, respectively. The power head chuck clamps the power source output signal OUTdkj, which takes the value {-1, 1}, representing loosening and clamping respectively; In practice, the operating condition signal can be a switch signal, an analog signal, or a digital signal converted by the acquisition module.

[0032] (3) Time synchronization and data alignment The video data and the operating condition signal data are timestamped and time-aligned based on a unified clock source. In a preferred embodiment, a time window Δt is used to synchronously buffer the multi-source data, where Δt ranges from 50ms to 500ms, to ensure that data with different sampling frequencies have time consistency during fusion processing.

[0033] (4) Preprocessing operations Frame extraction and image enhancement processing are performed on video data, including but not limited to brightness normalization, contrast enhancement, and noise filtering, to improve the stability of subsequent visual recognition; jitter removal and outlier filtering are performed on operating condition signals to ensure the reliability of signal judgment.

[0034] S2: Construction of Visual Inspection Model for Drill Pipe Operation Status In this embodiment, a visual detection model for identifying the operating status of drill pipes is constructed based on the collected drilling site video data, specifically including: (1) Sample data construction Collect video data of drill pipe operation under different working conditions, including but not limited to changes in lighting, dust interference, changes in viewing angle, and partial obstruction; extract image frames from the video data and annotate the drill pipe and drill rig power head areas, with the annotations including at least: Drill pipe location and quantity; Power head location; Drill pipe connection and disconnection status; Drill pipe in / out direction (inward / outward); (2) Model structure and training Based on the labeled data, a drill pipe state recognition model is constructed; in one embodiment, the model is a deep learning-based target detection model or behavior recognition model, whose input is a video frame or a continuous frame sequence, and whose output is the spatial position of the drill pipe and the power head and the corresponding state label. In a preferred embodiment, the status tag includes: Drill pipe connection status Drill pipe detached state Direction of motion of the power head (forward / backward / stop) By training the model, it can output the status recognition results of the drill pipe and power head in a continuous video stream.

[0035] (3) Online reasoning In actual operation, the real-time video stream is input into the visual detection model to obtain the visual recognition results of the corresponding time series, which are then used as input for subsequent multimodal fusion judgment.

[0036] S3: Initialize lever counter In this embodiment, after drilling is completed, the rod retraction counting process is initiated, which specifically includes: (1) Count initialization Receive the drill pipe retraction count start command and initialize the drill pipe counter NUM=0; (2) Signal state initialization Synchronously acquire the current operating status signal of the drilling rig and define: The output signal OUTdt of the power source for the propulsion head is ∈ {-1,0,1}. The power source output signal OUTdkx ∈ {-1,0,1} represents the rotation of the power head chuck. The power head chuck clamps the power source output signal OUTdkj∈ {-1,1}; (3) Status monitoring started The visual inspection model and operating condition signal acquisition module are activated, and the real-time status monitoring phase begins.

[0037] S4: Drill pipe state determination and counting based on multimodal fusion In this embodiment, the drilling rod operating status is fused and determined based on the output results of the visual detection model and the drilling rig operating condition signal. Drill rod counting and correction operations are then performed based on the determination results, specifically including the following steps: (1) State loop detection mechanism During the lever retraction counting process, video data and operating condition signals are continuously collected and judged in real time; When no visual signs of the power head reversing are detected, and the operating condition signal does not meet the continuous output condition of the power head propulsion signal OUTdt = -1, Maintain the current counting state and continue to collect data and determine the state at the next moment until the trigger condition for retracting the lever is met.

[0038] (2) Triggering conditions for lever release The system is considered to be in the retraction state when any of the following conditions are met: Visual inspection results indicate that the power head is in a backward motion state; Or the operating condition signal satisfies OUTdt = -1 and the duration is ≥ T1; Where T1 is the state duration threshold, used to filter out instantaneous jitter signals.

[0039] (3) Confirmation of lever retraction operation status After entering the lever retraction state, further determination is made based on the working condition signals to determine whether the current lever retraction operation is valid: A valid lever release operation is determined when the following conditions are met: OUTdkx = 1 (Chuck rotates forward); OUTdkj = 1 ((chuck clamping); (4) Determination of unwinding status When the following conditions are met: OUTdkx = -1; OUTdkj = 1; It was determined to be the drill pipe unwinding process.

[0040] (5) Drill pipe disengagement event determination and counting triggering A drill pipe detachment event is determined to have occurred when the following conditions are met simultaneously: OUTdt = 0; OUTdkx = 0; The visual inspection model detected the separation of the drill pipe from the power head; Then perform a counting operation: NUM = NUM ​​+ 1 (6) Count correction mechanism The visual inspection model detected that the drill pipe had been reconnected to the power head; Furthermore, visual inspection results indicate that the power head is in a forward motion state; Simultaneously, the operating condition signals meet the following requirements: OUTdt = 1; OUTdkx = 1; OUTdkj = 1; The current drill pipe raising operation is determined to be a non-counting interference behavior during the pipe retraction process. Correct the drill pipe counter: NUM = NUM ​​- 1.

[0041] (7) Loop check Repeat the above state determination and counting process until a counting end command is received.

[0042] S5: Counting End Determination, Result Output, and Data Upload In this embodiment, after executing the lever retraction status determination and counting process, it is further determined whether a lever retraction counting operation end command has been received.

[0043] If no command to end the drill rod counting operation is received, return to step S4 and continue to perform drill rod status determination, counting and correction processing; When the command to end the drill pipe counting operation is received, the current drill pipe counting program ends and the current drill pipe counter count result NUM is output.

[0044] In one embodiment, the drill rod count result NUM is calculated based on the preset length L of a single drill rod to obtain the corresponding drilling depth H; the drilling depth H can be expressed as: H = NUM ​​×L.

[0045] In this embodiment, the real-time video signal, drilling rig operating condition signal, process analysis feature image, corresponding operating condition data and counting results generated during the counting process can be uploaded to the host computer and / or local monitoring terminal in real time via wired or wireless communication for result display, operation management, historical record query and data archiving.

[0046] In a preferred embodiment, the host computer or local monitoring terminal is further used to associate and store the key event images, operating condition signal status and counting change results corresponding to each counting process, so as to realize the traceability query of the counting process.

[0047] Compared with the prior art, the present invention has at least the following beneficial effects: (1) This invention integrates video visual information and drilling rig operating condition signals in a time-synchronous manner to construct a joint judgment mechanism for drill rod status based on multimodal information, thereby realizing multi-source collaborative perception of the drill rod operation process, effectively reducing the recognition error caused by factors such as occlusion of single visual information, changes in lighting or equipment contamination, and improving the accuracy of counting results.

[0048] (2) This invention establishes a multi-condition judgment rule based on the motion state of the power head, the rotation state of the chuck, and the clamping state to accurately identify the working states of the drill rod, such as rod retraction, wire retraction, and detachment. It also triggers a counting operation based on the drill rod detachment event, thereby improving the stability and reliability of the drill rod counting process.

[0049] (3) By introducing a counting correction mechanism based on multimodal state combination, this invention identifies and eliminates non-counting behaviors caused by drilling pipe operation during the rod retraction process, thereby effectively avoiding miscounting problems and further improving counting accuracy.

[0050] (4) This invention records the key event images and corresponding working condition signal states during the drill pipe counting process, thereby enabling traceable storage of the counting process and making the counting results verifiable and auditable, thus enhancing the credibility of the results.

[0051] (5) This invention realizes the automation and real-time processing of drill rod counting, avoids the manual playback of video for counting, improves work efficiency, and provides timely and reliable data support for drilling construction quality assessment and production decision-making.

[0052] (6) This invention enables the system to maintain the stability of basic state determination by relying on working condition signals in complex downhole environments through a multimodal fusion identification mechanism. At the same time, it combines visual models to perform fine identification of specific actions, thereby achieving complementarity between the two types of information and improving the overall robustness of the system.

[0053] This invention proposes a multimodal fusion-based intelligent identification and counting method for drill rods in drilling operations. By synchronously acquiring and fusing video visual information with drilling rig operating condition signals, a joint judgment mechanism for drill rod operation status based on multi-source information is constructed. This mechanism collaboratively identifies the drill rod's advance, retreat, and related operation processes, and establishes counting triggering and correction rules based on state combinations. This enables automatic counting of drill rods and elimination of miscounts, reducing the impact of complex operating conditions and misjudgments from a single information source on the counting results, and improving the accuracy and stability of the counting results.

[0054] In one embodiment, an electronic device is provided, comprising: Processor; memory; communication interface connected to the processor; The memory stores a computer program, which, when executed by a processor, causes the processor to perform the steps of the drill pipe intelligent identification and counting method as described in any of the foregoing embodiments.

[0055] In one embodiment, the electronic device is: Industrial control computers; edge computing devices; or mine monitoring servers; In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps of the drill pipe intelligent identification and counting method as described in any of the foregoing embodiments.

[0056] In one embodiment, a smart identification and counting device for drill pipes used in drilling operations is also provided, comprising: (1) Data acquisition module This data is used to collect multimodal data during drilling operations, and the multimodal data includes: Video data; drilling rig operating signal data.

[0057] The operating condition signals include at least the power head propulsion signal, the chuck rotation signal, and the chuck clamping signal.

[0058] (2) Data synchronization and preprocessing module For time synchronization and preprocessing of the video data and operating condition signal data, including: Timestamp marking and data alignment; video image enhancement processing; jitter removal and outlier handling of operating condition signals.

[0059] (3) Visual recognition module This is used to analyze video data based on a pre-trained visual detection model and output operational status information for the drill pipe and power head, including: Drill pipe connected state; drill pipe disconnected state; power head moving state.

[0060] (4) Status determination module Used to fuse and determine the operating status of drill pipe based on visual recognition results and operating condition signals, including: Rod retraction trigger determination; rod retraction operation confirmation; thread retraction status identification; drill pipe disengagement event identification.

[0061] (5) Counting control module (core invention point) Used to perform drill pipe counting and correction operations based on the state determination result, including: When a drill pipe detachment event is detected, increment the counter: NUM = NUM ​​+ 1; Perform a count correction when drill pipe movement is detected: NUM = NUM ​​- 1; (6) Counting Management and Output Module Used for: determining the end condition of the count; outputting the drill rod count result NUM; and calculating the drilling depth based on the drill rod length.

[0062] (7) Data upload and storage module Video data; operating condition signals; images of key events; counting results.

[0063] Upload to the host computer or local terminal and manage its storage.

[0064] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A multimodal fusion identification method for intelligent identification and counting of drill pipes in drilling operations, characterized in that, Includes the following steps: S1: Synchronously acquire and preprocess multimodal data during drilling operations. The multimodal data includes video data and drilling rig operating condition signal data. The operating condition signals include at least the output signal OUTdt of the power head propulsion power source, the output signal OUTdkx of the power head chuck rotation power source, and the output signal OUTdkj of the power head chuck clamping power source. The video data and operating condition signal data are then timestamped and time-aligned. S2: Construct a visual detection model of drill pipe operation status based on the collected video data, and use the visual detection model to analyze the real-time video stream to output the drill pipe connection status, drill pipe disengagement status and power head movement status; S3: Receive the drill pipe retraction count start command, initialize the drill pipe counter NUM=0, and synchronously acquire the current status of the working condition signal to enter real-time status monitoring; S4: Based on the output of the visual detection model and the working condition signal, perform multimodal fusion judgment and execute drill pipe counting and correction, including: When no visual feature of the power head retracting is detected and the operating condition signal does not meet the continuous output condition of OUTdt=-1, the status detection is continuously performed in a loop. When the power head is detected to be retracted or OUTdt=-1 meets the duration threshold condition, it is determined that the lever retraction state is entered. When OUTdkx=1 and OUTdkj=1, it is determined to be a valid lever retraction operation; When OUTdkx=-1 and OUTdkj=1, it is determined to be in the unwinding state; When OUTdt=0 and OUTdkx=0 are satisfied simultaneously, and the visual detection model detects the separation of the drill pipe from the power head, a drill pipe detachment event is determined to have occurred, and NUM=NUM+1 is executed. When the visual inspection model detects that the drill pipe is reconnected to the power head and the power head is in the forward state, and OUTdt=1, OUTdkx=1, and OUTdkj=1, it determines that it is a drill pipe loading operation and executes NUM=NUM-1. S5: Determine whether the drill rod counting end command has been received. If not, return to step S4 to continue execution. If received, output the drill rod counting result NUM and calculate the drilling depth based on the drill rod length.

2. The intelligent identification and counting method for drill pipes in drilling operations based on multimodal fusion identification according to claim 1, characterized in that: In step S1, when aligning video data and operating condition signal data in time, a time window Δt is used for synchronization buffering, where Δt is 50ms to 500ms.

3. The intelligent identification and counting method for drill pipes in drilling operations based on multimodal fusion identification according to claim 1, characterized in that: The preprocessing of video data in S1 includes brightness normalization, contrast enhancement and noise filtering, and jitter removal and outlier filtering of the operating signal.

4. The intelligent identification and counting method for drill pipes in drilling operations based on multimodal fusion identification according to claim 1, characterized in that: The visual detection model in S2 is a deep learning-based target detection model or behavior recognition model. Its input is a video frame or a continuous frame sequence, and its output is the spatial position of the drill rod and the power head and the corresponding status label.

5. The intelligent identification and counting method for drill pipes in drilling operations based on multimodal fusion identification according to claim 1, characterized in that: The condition signal for determining the backward movement of the power head in S4 is OUTdt=-1 and the duration is greater than or equal to the preset threshold T1, so as to filter out instantaneous interference signals.

6. The intelligent identification and counting method for drill pipes in drilling operations based on multimodal fusion identification according to claim 1, characterized in that: In step S5, the drilling depth H is calculated based on the drill rod count result NUM and the length L of a single drill rod, satisfying the following: H = NUM×L.

7. A smart identification and counting device for drill pipes used in drilling operations, characterized in that, include: The data acquisition module is used to acquire video data and drilling rig operating signal data; The data synchronization and preprocessing module is used to perform time synchronization and preprocessing on the video data and operating condition signal data. The visual recognition module is used to analyze video data based on a visual detection model and output the drill pipe connection status, drill pipe disengagement status, and power head movement status. The status determination module is used to fuse and determine the drilling rod operation status based on visual recognition results and working condition signals; The counting control module is used to perform drill pipe counting and correction operations based on the state determination result, including executing NUM=NUM+1 when a drill pipe detachment event is detected, and executing NUM=NUM-1 when an up drill pipe operation is detected; The counting management and output module is used to determine the counting termination condition and output the counting result.

8. An electronic device, characterized in that, include: processor; Memory; Communication interface; The memory stores a computer program that, when executed by a processor, causes the processor to perform the method as described in any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, it implements the method as described in any one of claims 1 to 6.

Images