Intelligent drilling machine orifice backwater identification system and method based on machine vision

Abstract

The invention provides an intelligent drilling machine orifice backwater recognition system and method based on machine vision. The system comprises an orifice backwater recognition device, a camera in the orifice backwater recognition device transmits video images to an AI image processor, the AI image processor communicates with a drilling machine controller, and the drilling machine controller controls a drilling machine. The system provided by the invention can automatically detect and identify the orifice backwater in the whole process of drilling and retreating, realizes the automatic identification of the orifice backwater, effectively reduces the labor intensity of workers in the drilling process of a drilling machine, and reduces the errors caused by manual setting of water waiting time in the whole process of drilling and retreating and the influence on the construction safety. The system can effectively achieve automatic identification of orifice backwater of the intelligent drilling machine, and intelligent drilling of the drilling machine is achieved.

Description

Technical Field

[0001] The invention belongs to the technical field of underground drilling construction in coal mines, relates to water return from a borehole, and specifically relates to an intelligent drilling rig water return identification system and method based on machine vision. Background Art

[0002] At present, in the construction process of intelligent drilling rigs, before drilling, they rely on human eyes to observe the water return situation at the hole mouth to determine whether the drilling rig will drill normally in the next step. Some intelligent drilling rigs on the market currently rely on program-set water waiting time. After this time point, regardless of whether the water return in the hole is normal, the drilling rig will continue to drill. The water waiting time is set to be very uncertain as the hole depth increases and the formation changes, resulting in serious inaccurate judgment of the water return situation at the hole mouth. Observing the water return situation at the hole mouth with human eyes and setting the water waiting time by the automated drilling rig program leads to low construction efficiency of intelligent drilling rigs and the occurrence of unsafe accidents for people and equipment.

[0003] At present, automatic detection of water return at the orifice of an intelligent drilling rig has not been realized in the construction process. Therefore, it is urgently necessary to develop a set of intelligent drilling rig orifice water return detection devices and methods based on machine vision to improve the intelligence level of the drilling rig, improve on-site construction efficiency, and further improve on-site construction safety. Summary of the invention

[0004] In view of the shortcomings of the prior art, the purpose of the present invention is to provide an intelligent drilling rig orifice water return identification system and method based on machine vision, so as to solve the technical problem that the construction efficiency of the intelligent drilling rig in the prior art needs to be further improved.

[0005] In order to solve the above technical problems, the present invention adopts the following technical solutions to achieve the above problems:

[0006] An intelligent drilling rig orifice water return identification system based on machine vision, the system includes an orifice water return identification device, a camera in the orifice water return identification device transmits video images to an AI image processor, the AI ​​image processor communicates with a drilling rig controller, and the drilling rig controller controls the drilling rig.

[0007] The present invention also has the following technical features:

[0008] Specifically, the orifice backflow identification device includes a bracket base installed next to the orifice, a vertically arranged external support rod is fixed on the bracket base, the external support rod is hollow and has an open top, an internal support rod that can be locked by a locking bolt is installed in the external support rod, the internal support rod can be raised and lowered and fed in the external support rod, a camera adjustment pan-tilt is installed on the top of the internal support rod, and a camera is installed on the camera adjustment pan-tilt.

[0009] Specifically, the AI ​​image processor adopts an orifice rebate recognition model to recognize the orifice rebate image; the orifice rebate recognition model is constructed using an improved YOLOv5s network.

[0010] Preferably, the method for constructing the orifice return water identification model comprises the following steps:

[0011] Step 1: The orifice return water image captured by the camera is subjected to image clarity enhancement processing using a three-component color difference algorithm in RGB color space to obtain an enhanced orifice return water image.

[0012] Step 2: Construct a water return identification model:

[0013] An improved YOLOv5s network is used to construct an orifice return water recognition model; the improved YOLOv5s network is an algorithm obtained by respectively improving the input end, the backbone network (Backbone) and the neck network (Neck) in the network structure of the YOLOv5s network.

[0014] Step 3: Create a data set of orifice return water:

[0015] The enhanced orifice return water image obtained in step 1 is annotated to obtain the orifice return water dataset.

[0016] Step 4: Training the model:

[0017] Based on the orifice return water dataset obtained in step three, the number of iterations and the learning rate are set, and the orifice return water recognition model obtained in step two is trained to obtain a trained orifice return water recognition model.

[0018] Preferably, in step 2, the method for improving the input end is: using a Mosaic-SOD data enhancement method to replace a Mosaic data enhancement method.

[0019] Preferably, in step 2, the method for improving the backbone network is: adding a TRANS module at the end of the backbone network.

[0020] Preferably, in step 2, the method for improving the neck network is: using a GhostBottleneck module instead of a CSP module, and using a depthwise separable convolution instead of a normal convolution.

[0021] Preferably, the specific process of step three is: use LabelImg software to annotate the enhanced orifice return water image, a large number of annotated images will generate an xml file, create a new folder named kongkoufanshui, and create four new folders named Annotations, imges, ImageSets and JPEGImages under the kongkoufanshui folder; the Annotations folder is used to place the annotated xml files, the imges folder is used to place the train, test and val original images; the ImageSets folder is used to place the annotated data; the JPEGImages folder is used to place all the original images.

[0022] Specifically, the drilling rig includes a drilling rig base, on which a drilling rig power head, a drill rod magazine and a drill rod grabbing mechanical arm are installed. The drill rod grabbing mechanical arm can grab the drill rod in the drill rod magazine. A drill bit is installed on the drill rod, and the drill bit can enter the hole in the drilling formation.

[0023] The present invention also protects an intelligent drilling rig orifice water return identification method based on machine vision, characterized in that the method adopts the intelligent drilling rig orifice water return identification system based on machine vision as described above.

[0024] The process of the identification method is: fill light is performed in the drilling site, the camera in the hole return water identification device is used to collect video images of hole return water in the underground drilling site of the coal mine, the video image is transmitted to the AI ​​image processor for image recognition, and the recognition result is transmitted to the drilling rig controller.

[0025] When the AI ​​image processor recognizes the image of water returning from the orifice, the drilling rig controller controls the drilling rig to start drilling; when the AI ​​image processor does not recognize the image of water returning from the orifice, the drilling rig controller controls the drilling rig to stop drilling.

[0026] Compared with the prior art, the present invention has the following technical effects:

[0027] (I) The system of the present invention can automatically detect and identify water return from the orifice during the entire process of drilling and retracting the drill, realize automatic identification of water return from the orifice, effectively reduce the labor intensity of workers during the drilling process of the drilling rig, and reduce the errors caused by manual setting of the waiting time for water during the entire process of drilling and retracting the drill, which affects the construction safety.

[0028] (II) The system of the present invention can effectively assist the management party to check the water return from the hole in real time through the automatic water return identification of the drilling rig, determine whether there is any abnormality in the hole, and further improve the construction safety.

[0029] (III) The system of the present invention can effectively solve the problem of automatic identification of water return from the orifice of an intelligent drilling rig, thereby realizing intelligent drilling of the drilling rig.

[0030] (IV) The orifice return water image is greatly affected by the lighting background of the coal mine, and it is difficult to extract features from the conventional YOLOv5s network. The Transformer-based encoder TRANS module is introduced into the YOLOv5s backbone network to fully utilize the detection information of the global target, improve the orifice return water image feature extraction capability, and improve the detection accuracy of the return water image. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 It is a structural schematic diagram of the orifice backflow identification device.

[0032] Figure 2 Schematic diagram of the structure of the improved YOLOv5s network.

[0033] Figure 3 This is a schematic diagram of the structure of the TRANS module.

[0034] Figure 4 This is a schematic diagram for comparing the image clarity enhancement processing results. Figure 4 (a) is the original image. Figure 4 (b) in the figure is the enhanced orifice return water image.

[0035] Figure 5 This is the loss value change curve of the orifice return water identification model.

[0036] Figure 6 Orifice return water recognition image identified by the orifice return water recognition model.

[0037] The meanings of the numbers in the figure are: 1-drill rig base, 2-drill rig power head, 3-drill rod magazine, 4-drill rod grabbing mechanical arm, 5-drill rod, 6-support base, 7-drill bit, 8-external support rod, 9-locking bolt, 10-internal support rod, 11-camera adjustment pan / tilt, 12-camera, 13-drilling formation, 14-hole opening.

[0038] The specific contents of the present invention are further explained in detail below in conjunction with embodiments. DETAILED DESCRIPTION

[0039] It should be noted that, unless otherwise specified, all algorithms, modules and methods in the present invention adopt algorithms, modules and methods known in the prior art.

[0040] Specific embodiments of the present invention are given below. It should be noted that the present invention is not limited to the following specific embodiments, and all equivalent changes made on the basis of the technical solution of this application fall within the protection scope of the present invention.

[0041] Embodiment 1:

[0042] This embodiment provides an intelligent drilling rig orifice water return identification system based on machine vision, which includes an orifice water return identification device. The camera 12 in the orifice water return identification device transmits video images to the AI ​​image processor. The AI ​​image processor communicates with the drilling rig controller, and the drilling rig controller controls the drilling rig.

[0043] As a preferred solution of this embodiment, Figure 1 As shown, the orifice backflow identification device includes a bracket base 6 installed next to the orifice 14, a vertically arranged external support rod 8 is fixed on the bracket base 6, the external support rod 8 is hollow and has an open top, an internal support rod 10 that can be locked by a locking bolt 9 is installed in the external support rod 8, the internal support rod 10 can be raised and lowered in the external support rod 8, a camera adjustment platform 11 is installed on the top of the internal support rod 10, and a camera 12 is installed on the camera adjustment platform 11.

[0044] In this embodiment, the bracket base 6 and the external support rod 8 are connected by threaded connection, and the bracket position and height can be adjusted by forward and reverse rotation; the internal support rod 10 and the external support rod 8 are connected by threaded connection to achieve the bracket height and position adjustment changes, and the threaded connection is prevented from slipping or rotating. The locking bolt 9 is used for locking operation. The camera 12 and the internal support rod 10 are connected by a camera adjustment pan-tilt head 11 to achieve the up and down rotation and fixed operation of the camera. In response to changes in the wellhead position, the position of the camera 12 is adjusted by the camera adjustment pan-tilt head 11, so that the camera 12 can effectively collect the orifice return water video.

[0045] As a specific solution of this embodiment, Figure 1 As shown, the drilling rig includes a drilling rig base 1, on which a drilling rig power head 2, a drill rod magazine 3 and a drill rod grabbing mechanical arm 4 are installed. The drill rod grabbing mechanical arm 4 can grab the drill rod 5 in the drill rod magazine 3. A drill bit 7 is installed on the drill rod 5, and the drill bit 7 can enter the hole 14 in the drilling formation 13.

[0046] In this embodiment, the drilling rig must observe the water return of the orifice 14 before drilling, and confirm the water return of the orifice before the next drilling construction can be carried out; considering that the previous use of manual observation of the orifice water return or setting the water waiting time on the drilling rig is not accurate and seriously affects the construction efficiency. Therefore, machine vision perception technology is used to identify the water return of the orifice.

[0047] In this embodiment, the camera 12 is an intrinsically safe explosion-proof camera; the AI ​​image processor is an intrinsically safe AI image processor.

[0048] As a preferred solution of this embodiment, an orifice rebate recognition model is used in the AI ​​image processor to recognize the orifice rebate image; the orifice rebate recognition model is constructed using an improved YOLOv5s network.

[0049] Further, such as Figure 2 As shown, the method for constructing the orifice return water identification model includes the following steps:

[0050] Step 1: The orifice return water image captured by the camera is subjected to image clarity enhancement processing using a three-component color difference algorithm in RGB color space to obtain an enhanced orifice return water image.

[0051] In this embodiment, Figure 4 As shown, the original image is Figure 4 As shown in (a), the RGB color space three-component color difference algorithm is used to enhance the image clarity, and the enhanced orifice return water image is obtained as shown in Figure 4 As shown in (b) in .

[0052] In this embodiment, the water return from the borehole of the coal mine underground drilling site is identified. Since the coal mine underground drilling site is greatly affected by light, the RGB color space three-component color difference algorithm is used to enhance the image clarity. First, the color grayscale image R (red), G (green), and B (blue) of the collected drilling site borehole water return color image is extracted.

[0053] The RGB color space three-component color difference algorithm separates the orifice return water image from the drilling field background. This method uses the difference between the gray values ​​of the three components R, G, and B to select the feature operator that meets the threshold segmentation requirements, and finally obtains a segmented and clear orifice return water image. The three-component pixel value (0-255) of the color orifice return water image is equalized by image histogram, and the appropriate feature operator is selected according to the color information of the return water image and the background image. The three-component calculation method in the RGB color space three-component color difference algorithm is as follows:

[0054]

[0055] Where: G A Represents the grayscale value, and the three-component grayscale image is obtained by calculating the three-component difference.

[0056] Step 2: Construct a water return identification model:

[0057] An improved YOLOv5s network is used to construct an orifice backflow recognition model; the improved YOLOv5s network is an algorithm obtained by improving the input end, backbone network (Backbone) and neck network (Neck) in the network structure of the YOLOv5s network.

[0058] In this embodiment, in view of the complex background of coal mines, the size of the YOLOv5s network scaling factor is considered, which is divided into four sizes: s, m, l, and x, and the YOLOv5s network is selected for orifice return water identification.

[0059] In the present embodiment, the YOLOv5s network is a known YOLOv5s network, and the network structure of the YOLOv5s network includes an input end, a backbone network (Backbone), a neck network (Neck), a head network (Head) and a content security policy network (Communication Sequential Process, CSP). The input end performs Mosaic data enhancement, adaptive anchor frame and image cropping and scaling on the collected orifice return water image, completes image enhancement and normalization, and inputs it into the backbone network (Backbone). The backbone network (Backbone) performs feature extraction, obtains effective feature information to reduce the amount of calculation, and improves network calculation efficiency. The neck network (Neck) receives the feature data introduced by the Backbone part, and performs orifice return water data fusion processing by upsampling. The fusion structure of the neck network (Neck) is a PANet network model. The head network (Head) performs multi-scale prediction, and predicts the effective information of orifice return water respectively.

[0060] In step 2, if Figure 2 As shown, the improved method of the input end is: using the Mosaic-SOD data enhancement method to replace the Mosaic data enhancement method. In this embodiment, the original Mosaic data enhancement method has a large randomness, and an improved Mosaic data enhancement method Mosaic-SOD (Mosaic for Small Object Detection, i.e., Mosaic for small target detection) is proposed. The basic idea is as follows: First, the orifice return water image, the water texture information and the background image are separated. Since the morphological size of the water in the orifice return water image is small, the training sample size of the return water image can be greatly increased, thereby improving the generalization ability of the network.

[0061] In step 2, if Figure 2 As shown, the method for improving the backbone network is to add a TRANS module at the end of the backbone network.

[0062] In this embodiment, based on the improved YOLOv5s network model for orifice return water recognition, the original YOLOv5s structure is changed, such as Figure 2 As shown in the figure, adding a TRANS module (i.e., a Transformer-based encoder) to the backbone network can improve the model representation capability and reduce the false detection rate and missed detection rate.

[0063] In this embodiment, Figure 3As shown in the figure, the TRANS module stacks the feature map processed by normalization and multi-head attention (MHA) with the original feature map and outputs it, and then stacks the output with the result of normalization and multilayer perceptron (MLP) operation.

[0064] In this embodiment, Figure 3 As shown in the figure, the TRANS module can better capture the global information of the defect image and provide rich defect context features. In the feature extraction stage, the TRANS module collects feature information of a larger neighborhood, which helps to learn more features and improve detection accuracy. However, when the TRANS module is in a shallow position in the network layer, it is easy to trigger boundary regression, resulting in the loss of context feature information. Therefore, the TRANS module is added to the end of the YOLOv5s backbone network, where the feature map resolution is low, which effectively suppresses information loss, while reducing the model storage space and calculation amount, making it more suitable for rapid detection of cable surface defects.

[0065] In this embodiment, Figure 3 As shown in the figure, the TRANS module introduces MHA, which divides the input feature map into multiple regions and performs linear operations and attention mechanism operations in each region to obtain key feature information. MHA learns different defect features through the same attention mechanism, and then stacks and transforms different features to obtain the final output. Compared with the traditional attention mechanism, MHA effectively prevents false detection caused by excessive focus on one's own position, and extracts more feature information to reduce the missed detection rate.

[0066] In step 2, if Figure 2 As shown in the figure, the improvement method of the neck network (Neck) is to use the GhostBottleneck module instead of the CSP module, and use the depthwise separable convolution instead of the ordinary convolution. While ensuring the detection accuracy, the detection speed is greatly improved. At the same time, the parameters of the model are reduced, and the amount of calculation is reduced.

[0067] Step 3: Create a data set of orifice return water:

[0068] The enhanced orifice return water image obtained in step 1 is annotated to obtain the orifice return water dataset.

[0069] Specifically, the enhanced orifice return water image is annotated using LabelImg software. A large number of annotated images will generate an xml file, and a new folder named kongkoufanshui is created. In addition, four new folders named Annotations, imges, ImageSets and JPEGImages are created under the kongkoufanshui folder. The Annotations folder is used to place the annotated xml files, the imges folder is used to place the train, test and val original images; the ImageSets folder is used to place the annotated data; and the JPEGImages folder is used to place all the original images.

[0070] Step 4: Training the model:

[0071] Based on the orifice return water dataset obtained in step three, the number of iterations and the learning rate are set, and the orifice return water recognition model obtained in step two is trained to obtain a trained orifice return water recognition model.

[0072] In this step, click to run train.py to modify the number of iterations and learning rate of the orifice return water recognition model obtained in step 2. During the learning process, adjust the number of iterations and learning rate to optimize the orifice return water recognition model network to prevent overfitting.

[0073] In this step, the operating system is Ubuntu 12.04, and the processor is Intel i9 13900K CPU.

[0074] @2.50GHz, graphics card: RTX4090, model training framework Anoconda+Pytorch, image size 640*640, learning rate 0.0001, Step: 100k

[0075] In this step, evaluation indicators: The experiment of the present invention uses 4 indicators, including precision (Precision, P), mean average precision (mean average precision, mAP), frame rate (frames per second, FPS) and recall (recall, R). Calculation formula:

[0076]

[0077] Where:

[0078] TP is the number of correct predictions for positive samples;

[0079] FN is the number of negative sample prediction errors;

[0080] FP is the number of positive sample prediction errors;

[0081] TN is the number of correct predictions for negative samples;

[0082] AP i is the detection accuracy of the i-th category;

[0083] N is the number of categories.

[0084] In this step, the model is trained for 250 rounds and tested on the validation set to observe the changes in bounding box regression loss (box_los), object confidence loss (obj_los), and category loss (cls_los) between the improved module and the original model. Figure 5 As shown in the figure, in the first 100 iterations, the loss value of the training set and the loss value of the validation set both decreased rapidly, and gradually stabilized after 200 iterations. Finally, the loss value of the training set converged to about 0.016, and the loss value of the validation set converged to about 0.022, indicating that the network has achieved good training results.

[0085] In this step, the trained orifice return water recognition model is loaded into the AI ​​image processor, and the recognition interface is debugged to ensure that the loaded orifice return water recognition module can operate normally.

[0086] Embodiment 2:

[0087] This embodiment provides an intelligent drilling rig orifice water return identification method based on machine vision. The method adopts the intelligent drilling rig orifice water return identification system based on machine vision provided in Example 1.

[0088] The process of the identification method is: fill light is performed in the drilling field, and the camera 12 in the hole return water identification device is used to collect video images of hole return water with qualified clarity in the underground drilling field of the coal mine, and the video image is transmitted to the AI ​​image processor for image recognition, and the recognition result is transmitted to the drilling rig controller.

[0089] When the AI ​​image processor recognizes Figure 6 When the orifice water return image is shown, the drilling rig controller controls the drilling rig to start drilling operation; when the AI ​​image processor does not recognize the orifice water return image, the drilling rig controller controls the drilling rig to stop drilling operation.

[0090] In this embodiment, when it is identified that no water is returning from the orifice, the camera can be set to have a voice prompting function as needed to give a reminder, and the drilling rig power head stops without performing drilling operations.

Claims

1. An intelligent drilling rig orifice return water identification system based on machine vision, characterized in that: The system comprises a hole mouth water return identification device, wherein a camera (12) in the hole mouth water return identification device transmits video images to an AI image processor, the AI ​​image processor communicates with a drilling rig controller, and the drilling rig controller controls the drilling rig.

2. The intelligent drilling rig orifice return water identification system based on machine vision as claimed in claim 1, characterized in that: The orifice return water identification device comprises a support base (6) installed beside the orifice (14), a vertically arranged external support rod (8) is fixed on the support base (6), the external support rod (8) is hollow and has an open top, an internal support rod (10) that can be locked by a locking bolt (9) is installed in the external support rod (8), the internal support rod (10) can be raised and lowered in the external support rod (8), a camera adjustment platform (11) is installed on the top of the internal support rod (10), and a camera (12) is installed on the camera adjustment platform (11).

3. The intelligent drilling rig orifice return water identification system based on machine vision as claimed in claim 1, characterized in that: The AI ​​image processor adopts an orifice rebate recognition model to recognize the orifice rebate image; the orifice rebate recognition model is constructed using an improved YOLOv5s network.

4. The intelligent drilling rig orifice return water identification system based on machine vision as claimed in claim 3, characterized in that: The method for constructing the orifice return water identification model comprises the following steps: Step 1: The orifice return water image captured by the camera is subjected to image definition enhancement processing using a three-component color difference algorithm in RGB color space to obtain an enhanced orifice return water image; Step 2: Construct a water return identification model: An improved YOLOv5s network is used to construct an orifice return water recognition model; the improved YOLOv5s network is an algorithm obtained by respectively improving the input end, the backbone network and the neck network in the network structure of the YOLOv5s network; Step 3: Create a data set of orifice return water: Annotate the enhanced orifice return water image obtained in step 1 to obtain an orifice return water dataset; Step 4: Training the model: Based on the orifice return water dataset obtained in step three, the number of iterations and the learning rate are set, and the orifice return water recognition model obtained in step two is trained to obtain a trained orifice return water recognition model.

5. The intelligent drilling rig orifice return water identification system based on machine vision as claimed in claim 4, characterized in that: In step 2, the method for improving the input end is: using the Mosaic-SOD data enhancement method to replace the Mosaic data enhancement method.

6. The intelligent drilling rig orifice return water identification system based on machine vision as claimed in claim 4, characterized in that: In step 2, the method for improving the backbone network is: adding a TRANS module at the end of the backbone network.

7. The intelligent drilling rig orifice return water identification system based on machine vision as claimed in claim 4, characterized in that: In step 2, the neck network is improved by using a GhostBottleneck module instead of a CSP module, and using a depthwise separable convolution instead of a common convolution.

8. The intelligent drilling rig orifice return water identification system based on machine vision as claimed in claim 4, characterized in that: The specific process of step three is: use LabelImg software to annotate the enhanced orifice return water image, a large number of annotated images will generate an xml file, create a new folder named kongkoufanshui, and create four new folders named Annotations, imges, ImageSets and JPEGImages under the kongkoufanshui folder; the Annotations folder is used to place the annotated xml files, the imges folder is used to place the train, test and val original images; the ImageSets folder is used to place the annotated data; the JPEGImages folder is used to place all the original images.

9. The intelligent drilling rig orifice return water identification system based on machine vision as claimed in claim 1, characterized in that: The drilling rig comprises a drilling rig base (1), on which a drilling rig power head (2), a drill rod bin (3) and a drill rod grabbing mechanical arm (4) are mounted, the drill rod grabbing mechanical arm (4) being capable of grabbing a drill rod (5) in the drill rod bin (3), a drill bit (7) being mounted on the drill rod (5), and the drill bit (7) being capable of entering a hole (14) in a drilling formation (13).

10. An intelligent drilling rig orifice return water identification method based on machine vision, characterized in that: The method adopts the intelligent drilling rig orifice return water identification system based on machine vision as described in any one of claims 1 to 9; The process of the identification method is as follows: supplementary lighting is performed in the drilling site, a camera (12) in the hole return water identification device is used to collect a video image of the hole return water in the underground coal mine drilling site, the video image is transmitted to the AI ​​image processor for image recognition, and the recognition result is transmitted to the drilling rig controller; When the AI ​​image processor recognizes the image of water returning from the orifice, the drilling rig controller controls the drilling rig to start drilling; when the AI ​​image processor does not recognize the image of water returning from the orifice, the drilling rig controller controls the drilling rig to stop drilling.

Images