Visual determination method of whether there is a drilling tool in a clamp

Abstract

The application discloses a visual judgment method for whether there is a drilling tool in a clamp, which comprises image acquisition, image preprocessing and ROI region calibration, data set labeling and model training, real-time image model preliminary detection, feature fine extraction, feature comparison and threshold judgment, state output and feedback, installs a visual acquisition device on a tube arranging mechanical arm clamp, filters background interference from the source by calibrating the region of interest of the clamp working area, adopts a deep learning model to preliminarily identify the real-time image of the clamp, then fine extracts the features of the region of interest image and compares them with a standard feature library, and finally accurately judges whether there is a drilling tool in the clamp through the double judgment mechanism of model preliminary identification and feature fine verification. The application avoids the reliability defects of the contact type sensor and solves the misjudgment problem under the complex background and light interference.

Description

Technical Field

[0001] This invention relates to the field of intelligent visual inspection technology for oil drilling platforms, and in particular to a visual determination method for whether there are drill bits in the clamps. Background Technology

[0002] During oil drilling platform operations, tripping in and out of the drill string and laying out the drill string are core, high-frequency operations. Currently, intelligent drilling platforms generally use robotic arms to replace manual labor for gripping, handling, and arranging drill strings. The end gripper of the robotic arm, as the drill string clamping actuator, determines whether it is holding drill strings, which is the core basis for the robotic arm's automatic control, operation logic switching, and equipment safety protection.

[0003] In existing technologies, the status determination of chuck drill bits mainly relies on contact sensors or manual visual inspection. Among them, contact sensors are exposed to the harsh working environment of drilling platforms, which are humid, dusty, and vibrating, making them prone to problems such as sensor failure, contact corrosion, and false triggering. This results in poor detection stability and high maintenance costs. On the other hand, manual inspection is inefficient, has a strong lag, and the high-altitude and high-risk working environment of drilling platforms can easily lead to safety accidents, making it unsuitable for the needs of automated continuous operation.

[0004] Existing visual inspection technologies for clamp status detection generally suffer from weak anti-interference capabilities. Drilling platforms are subject to various interference factors, including densely packed drilling tools, complex equipment backgrounds, alternating lighting conditions, and mechanical obstructions. Traditional visual algorithms are prone to misidentifying surrounding drilling tools as clamped tools, or experiencing missed or incorrect detections due to light and shadow interference. Consequently, the detection accuracy and stability fail to meet industrial operation standards. Therefore, there is an urgent need for a high-precision, high-stability visual method for clamped drilling tools that is adaptable to complex drilling conditions. Summary of the Invention

[0005] The purpose of this invention is to provide a visual determination method for whether there is a drill string in the clamp, which solves the technical problems of high sensor detection failure rate, low efficiency of manual detection, poor anti-interference of traditional visual detection, and high misjudgment rate in the prior art. It realizes automated, high-precision, and real-time determination of the state of the clamp drill string under complex drilling conditions, and is adapted to the fully automated operation requirements of drilling platforms.

[0006] To achieve the above technical objective, a visual determination method for whether a drill bit is in a clamp includes the following steps: Step 1: Image acquisition. Using an industrial camera mounted on the gripper of the pipe-laying robotic arm, acquire raw image datasets of the gripper area under different lighting conditions, working postures, and background conditions. Step 2: Image preprocessing and region labeling. The acquired raw image is preprocessed to label the fixed region of interest in the clamp working area of ​​the image and remove invalid background interference outside the region. Step 3: Dataset labeling and model training. The preprocessed images are classified and labeled into two categories: samples with and without drill bits in the clamp. A training dataset is constructed, and a deep learning model for clamp and drill bit status recognition is trained. Step 4: Real-time image detection. Using the trained recognition model, perform preliminary state recognition on the real-time images of the clamps collected during the operation and output preliminary judgment results. Step 5: Refined feature extraction. The image of the region of interest in the clamp is processed by grayscale conversion, binarization, and noise reduction to extract contour features, pixel features, and texture features within the region. Step Six: Feature Comparison and Threshold Determination. The features extracted in real time are compared with the preset standard feature library for the presence or absence of drilling tools. Combined with the preset determination threshold, the presence or absence of drilling tools inside the chuck is accurately determined.

[0007] As a preferred implementation, the original images collected in step one cover complex lighting conditions such as strong light, weak light, and backlight, and include complex backgrounds such as multiple nearby interfering drilling tools, robotic arm structures, and drilling equipment. At the same time, they cover various working postures such as clamps open, half-closed, and fully closed, ensuring the comprehensiveness of the dataset and improving the model's generalization ability.

[0008] As a preferred implementation, step two involves preprocessing the image by denoising, equalizing illumination, and enhancing contrast to eliminate the effects of dust noise and uneven illumination on site. By fixing pixel coordinates and locking the core clamping position of the clamp as the region of interest, interference from drilling tools and equipment background outside the region is thoroughly filtered out, reducing the impact of interference factors from the source.

[0009] As a preferred implementation scheme, step three adopts a binary classification labeling mode to screen effective samples and remove invalid and ambiguous samples. The training set and validation set are divided in an 8:2 ratio. Transfer learning training is carried out based on a lightweight deep learning model, which effectively improves the model's detection speed and recognition accuracy, and adapts to the real-time detection needs of industry.

[0010] As a preferred implementation, a high confidence threshold of 0.9 is set in step four to perform a secondary fine-grained verification of the low confidence recognition results of the model, thereby avoiding misjudgment caused by a single model and improving detection stability.

[0011] As a preferred implementation, step five uses Gaussian filtering to remove image noise and adaptive threshold binarization to fully preserve the edge contours of the clamp structure and drill bit, accurately extracting target features and providing reliable data for subsequent feature comparison.

[0012] As a preferred implementation scheme, step six involves constructing a dedicated working condition feature library. Matching and judgment are performed based on the circular outline features of the drill string and the pixel ratio features of the region. Through dual threshold constraints, the structure of the clamp itself and the drill string target are effectively distinguished, thus completely solving the problem of misjudgment caused by interference from neighboring drill strings.

[0013] As a preferred embodiment, the method of the present invention further includes step seven: status output and feedback, outputting the final result of whether the cleat drill has a status and feeding the result back to the drilling automation control system to assist the robotic arm in completing the gripping, pipe laying and resetting operations.

[0014] Compared with the prior art, the present invention has the following advantages: 1. This invention adopts a dual judgment mechanism of "model initial identification + feature fine verification", abandoning the traditional single detection method, effectively solving the problems of misjudgment and missed judgment caused by complex drilling background, light interference, and obstruction by adjacent drilling tools, and significantly improving the detection accuracy. 2. This invention uses fixed ROI region calibration to filter out invalid backgrounds and interfering targets at the source, reducing image computation, improving detection efficiency, and meeting the response requirements of real-time control of robotic arms; 3. This invention eliminates the need for contact sensors, enabling pure vision-based non-contact detection. This avoids the problems of sensor damage and malfunction under harsh working conditions, reduces equipment maintenance costs, and improves equipment operational stability. 4. This invention is adaptable to drilling site conditions with multiple lighting, postures, and interferences. It has strong generalization ability and can be directly adapted to various specifications of pipe-laying robotic arms and drilling tool models, making it highly versatile. Attached Figure Description

[0015] Figure 1 This is a schematic diagram of the method of the present invention. Detailed Implementation

[0016] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0017] like Figure 1 As shown, a visual method for determining whether a drill bit is in a clamp includes the following steps: Step 1: Image acquisition. Using an industrial camera mounted on the gripper of the pipe-laying robotic arm, acquire raw image datasets of the gripper area under different lighting conditions, working postures, and background conditions. Step 2: Image preprocessing and region labeling. The acquired raw image is preprocessed to label the fixed region of interest in the clamp working area of ​​the image and remove invalid background interference outside the region. Step 3: Dataset labeling and model training. The preprocessed images are classified and labeled into two categories: samples with and without drill bits in the clamp. A training dataset is constructed, and a deep learning model for clamp and drill bit status recognition is trained. Step 4: Real-time image detection. Using the trained recognition model, perform preliminary state recognition on the real-time images of the clamps collected during the operation and output preliminary judgment results. Step 5: Refined feature extraction. The image of the region of interest in the clamp is processed by grayscale conversion, binarization, and noise reduction to extract contour features, pixel features, and texture features within the region. Step Six: Feature Comparison and Threshold Determination. The features extracted in real time are compared with the preset standard feature library for the presence or absence of drilling tools. Combined with the preset determination threshold, the presence or absence of drilling tools inside the chuck is accurately determined.

[0018] As a preferred implementation, the original images collected in step one cover complex lighting conditions such as strong light, weak light, and backlight, and include complex backgrounds such as multiple nearby interfering drilling tools, robotic arm structures, and drilling equipment. At the same time, they cover various working postures such as clamps open, half-closed, and fully closed, ensuring the comprehensiveness of the dataset and improving the model's generalization ability.

[0019] As a preferred implementation, step two involves preprocessing the image by denoising, equalizing illumination, and enhancing contrast to eliminate the effects of dust noise and uneven illumination on site. By fixing pixel coordinates and locking the core clamping position of the clamp as the region of interest, interference from drilling tools and equipment background outside the region is thoroughly filtered out, reducing the impact of interference factors from the source.

[0020] As a preferred implementation scheme, step three adopts a binary classification labeling mode to screen effective samples and remove invalid and ambiguous samples. The training set and validation set are divided in an 8:2 ratio. Transfer learning training is carried out based on a lightweight deep learning model, which effectively improves the model's detection speed and recognition accuracy, and adapts to the real-time detection needs of industry.

[0021] As a preferred implementation, a high confidence threshold of 0.9 is set in step four to perform a secondary fine-grained verification of the low confidence recognition results of the model, thereby avoiding misjudgment caused by a single model and improving detection stability.

[0022] As a preferred implementation, step five uses Gaussian filtering to remove image noise and adaptive threshold binarization to fully preserve the edge contours of the clamp structure and drill bit, accurately extracting target features and providing reliable data for subsequent feature comparison.

[0023] As a preferred implementation scheme, step six involves constructing a standard feature library (a dedicated working condition feature library). Matching and judgment are performed based on the circular contour features of the drill string and the regional pixel ratio features. Through dual threshold constraints, the structure of the clamp itself and the drill string target are effectively distinguished, thus completely solving the problem of misjudgment caused by interference from neighboring drill strings.

[0024] In step six, the standard feature library includes fixed features of the clamp contour, average pixel value of blank areas, and texture grayscale features in the absence of drill bits, as well as circular contour features of the drill bit and pixel proportion features of the area in the presence of drill bits. The state is determined by matching the similarity between real-time features and standard features, combined with a pixel proportion threshold. When the effective contour pixel proportion of the drill bit within the region of interest is greater than a first preset threshold, and the similarity of the circular contour feature is greater than a second preset threshold, the clamp is determined to have drill bits; otherwise, the clamp is determined to have no drill bits. The first preset threshold is 15%, and the second preset threshold is 85%.

[0025] As a preferred embodiment, the method of the present invention further includes step seven: status output and feedback, outputting the final result of whether the cleat drill has a status and feeding the result back to the drilling automation control system to assist the robotic arm in completing the gripping, pipe laying and resetting operations.

[0026] Compared with the prior art, the present invention has the following advantages: 1. This invention adopts a dual judgment mechanism of "model initial identification + feature fine verification", abandoning the traditional single detection method, effectively solving the problems of misjudgment and missed judgment caused by complex drilling background, light interference, and obstruction by adjacent drilling tools, and significantly improving the detection accuracy. 2. This invention uses fixed ROI region calibration to filter out invalid backgrounds and interfering targets at the source, reducing image computation, improving detection efficiency, and meeting the response requirements of real-time control of robotic arms; 3. This invention eliminates the need for contact sensors, enabling pure vision-based non-contact detection. This avoids the problems of sensor damage and malfunction under harsh working conditions, reduces equipment maintenance costs, and improves equipment operational stability. 4. This invention is adaptable to drilling site conditions with multiple lighting, postures, and interferences. It has strong generalization ability and can be directly adapted to various specifications of pipe-laying robotic arms and drilling tool models, making it highly versatile.

[0027] This invention provides a visual method for determining whether a drill string is inside a clamp. A visual acquisition device is installed on the clamp of a pipe-laying robotic arm. Background interference is filtered out at the source by calibrating the region of interest (ROI) of the clamp's working area. A deep learning model is used to perform preliminary state recognition on real-time images of the clamp. Then, features are refined and extracted from the ROI image and compared with a standard feature library. Through a dual determination mechanism of "initial model recognition + fine feature verification," the presence of a drill string inside the clamp is accurately determined. This method solves the problem of misjudgment under complex backgrounds and lighting interference, and avoids the reliability defects of contact sensors.

[0028] like Figure 1 As shown, the method of the present invention includes image acquisition, image preprocessing and ROI region labeling, dataset annotation and model training, real-time image model detection, refined feature extraction, feature comparison and threshold determination, state output and feedback, etc., and each step is connected in sequence to form a complete detection process.

[0029] As a preferred embodiment, this embodiment provides a visual determination method for whether there is a drill bit in the clamp, and the specific implementation steps are as follows: S1. Image Dataset Acquisition: At the actual drilling platform operation site, an industrial camera was fixedly installed on the side of the pipe-laying robotic arm clamp. The camera's focal length and shooting angle were adjusted to ensure that the captured image completely covered the clamping position. Images of the clamp were acquired under different time periods, different light intensities, and different operating postures, totaling 5,000 original images. The image samples included various lighting conditions such as bright sunlight, weak light on cloudy days, supplemental lighting at night, and backlighting. They also included complex backgrounds such as dense drilling tools, robotic arm supports, and drilling operation platforms, covering three operating states: fully open, partially closed (partial clamping), and fully closed (fully clamping).

[0030] S2. Image Preprocessing and ROI Region Calibration: 5000 original images were batch preprocessed. First, a Gaussian filtering algorithm was used to remove dust and light noise. Then, an illumination equalization algorithm was used to correct uneven brightness. Contrast enhancement was applied to highlight the edge features of the clamp and drill bit. Based on the fixed camera installation position, a fixed pixel area was calibrated as the core working ROI region of the clamp. This region only covers the clamping station, and all surrounding invalid background areas were removed.

[0031] S3. Dataset Labeling and Model Training: Preprocessed images were manually and precisely labeled, categorized into "with drill bit in clamp" and "without drill bit in clamp." Invalid samples (blurred, overexposed, or severely occluded) were removed, resulting in 4800 valid images. These valid samples were then divided into a training set of 3840 images and a validation set of 960 images in an 8:2 ratio. A lightweight YOLO model was used as the base network, and iterative training was conducted using transfer learning. The initial learning rate was set to 0.001, with 100 iterations. After training, the model achieved a precision of 0.996 and a recall of 0.998, meeting the accuracy requirements for industrial detection.

[0032] S4. Real-time Image Model Initial Detection: During drilling operations, the camera captures real-time images of the clamping position and inputs them into the trained deep learning model for state recognition. The model confidence threshold is set to 0.9. When the model outputs a confidence score ≥ 0.9, a preliminary judgment result is output; when the confidence score < 0.9, it is marked as pending review and enters the subsequent refined feature detection process.

[0033] S5. Refined Image Feature Extraction: The ROI regions of the images to be reviewed and the real-time detection images are converted to grayscale, transforming the color images into grayscale images. Then, adaptive threshold binarization is applied to segment the foreground target and background, resulting in high-resolution binary images. Contour features are retrieved from the binary images, and core features such as the number of contours, contour area, pixel ratio, and average grayscale value within the ROI regions are extracted.

[0034] S6. Feature Comparison and Precise Threshold Determination: A standard feature library for on-site working conditions is pre-built, storing the clamp's fixed contour features and the average pixel value of the blank area when there is no drill bit, as well as the drill bit's circular contour features and standard pixel ratio features when there is a drill bit. The features extracted in real time are matched with the standard feature library for similarity: when the effective drill bit contour pixel ratio in the ROI area is greater than 15% and the similarity of the contour circular feature matching is greater than 85%, it is determined that there is a drill bit in the clamp; otherwise, it is determined that there is no drill bit in the clamp.

[0035] S7. Status Output and Feedback: The system outputs the status signal of whether the clamped drill string is present to the drilling platform's automated control system in real time. If the presence of the drill string is determined, the system locks the clamping state, prohibits the robotic arm from resetting or misoperating the release mechanism, and executes subsequent pipe laying and lowering operations. If the absence of the drill string is determined, the system allows the robotic arm to execute operations such as gripping, resetting, and moving, achieving intelligent linkage of operational logic.

[0036] This embodiment has been verified through field testing. Under complex drilling conditions, the detection time for a single image is ≤30ms, which can meet the real-time control response requirements of the robotic arm. The accuracy of state determination can reach over 99.5%, effectively solving the problems of misjudgment, missed judgment, and high failure rate of traditional detection methods. It can be fully adapted to the automated continuous operation scenario of drilling platforms.

[0037] It should be understood that the scope of protection sought by this invention is not limited to the non-limiting embodiments, which are merely illustrative examples. The substantive scope of protection claimed in this application is further embodied in the scope provided by the independent claims and their dependent claims.

Claims

1. A visual method for determining whether a drill bit is in a clamp, characterized in that, Includes the following steps: Step 1: Image acquisition. Using an industrial camera mounted on the gripper of the pipe-laying robotic arm, acquire raw image datasets of the gripper area under different lighting conditions, working postures, and background conditions. Step 2: Image preprocessing and region labeling. The acquired raw image is preprocessed to label the fixed region of interest in the clamp working area of ​​the image and remove invalid background interference outside the region. Step 3: Dataset labeling and model training. The preprocessed images are classified and labeled into two categories: samples with and without drill bits in the clamp. A training dataset is constructed, and a deep learning model for clamp and drill bit status recognition is trained. Step 4: Real-time image detection. Using the trained recognition model, perform preliminary state recognition on the real-time images of the clamps collected during the operation and output preliminary judgment results. Step 5: Refined feature extraction. The image of the region of interest in the clamp is processed by grayscale conversion, binarization, and noise reduction to extract the contour features, pixel features, and texture features within the region. Step Six: Feature Comparison and Threshold Determination. The features extracted in real time are compared with the preset standard feature library for the presence or absence of drilling tools. Combined with the preset determination threshold, the presence or absence of drilling tools inside the chuck is accurately determined.

2. The visual determination method for whether a drill bit is present in a clamp according to claim 1, characterized in that: In step one, the acquired raw images include complex lighting conditions such as strong light, weak light, and backlight, as well as complex backgrounds such as multiple nearby interfering drilling tools, robotic arm structures, and drilling equipment. They also cover various operating postures such as open, semi-closed, and fully closed clamps.

3. The visual determination method for whether a drill bit is present in a clamp according to claim 1, characterized in that: The specific preprocessing methods in step two include image noise reduction, illumination equalization, and contrast enhancement. The core working area of ​​the clamp is locked by fixing pixel coordinates. This area completely covers the clamping station and avoids fixed obstruction structures such as robotic arm brackets and equipment shells.

4. The visual determination method for whether a drill bit is present in a clamp according to claim 1, characterized in that: In step three, the dataset is labeled using a binary classification method. The clamping area in the image is labeled with its state, and invalid samples that are blurry, overexposed, or severely occluded are removed. The dataset is divided into a training set and a validation set in an 8:2 ratio. A lightweight detection model is trained using transfer learning and iteratively trained until the model's precision and recall reach the preset standards.

5. The visual determination method for whether a drill bit is present in a clamp according to claim 1, characterized in that: In step four, during the initial detection process of the model, a confidence threshold of 0.9 is set. Detection results below the confidence threshold are marked as pending review and then proceed to the subsequent refined feature extraction process for secondary judgment to avoid model misjudgment.

6. The visual determination method for whether a drill bit is present in a clamp according to claim 1, characterized in that: In step five, the image denoising uses a Gaussian filtering algorithm to remove noise caused by dust and light and shadow jitter on site; the binarization uses an adaptive threshold binarization method to adapt to uneven lighting conditions on site and fully preserve the edge contour features of the clamp and drill bit.

7. The visual determination method for whether a drill bit is present in a clamp according to claim 1, characterized in that: In step six, the standard feature library includes fixed features of the clamp contour, average pixel value of blank areas, and texture grayscale features in the absence of drill bit, as well as circular contour features of drill bit and pixel ratio features of the area in the presence of drill bit. The state determination is completed by matching the similarity between real-time features and standard features, combined with the pixel ratio threshold.

8. The visual determination method for whether a drill bit is present in a clamp according to claim 7, characterized in that: When the proportion of effective contour pixels of the drill bit in the region of interest is greater than the first preset threshold and the similarity of the contour circular feature is greater than the second preset threshold, it is determined that the clamp has a drill bit; otherwise, it is determined that the clamp does not have a drill bit.

9. The visual determination method for whether a drill bit is present in a clamp according to claim 8, characterized in that: The first preset threshold is 15%, and the second preset threshold is 85%.

10. The visual determination method for whether a drill bit is present in a clamp according to claim 1, characterized in that: It also includes step seven: status output and feedback, which outputs the final status result of the cleat drill bit and feeds the result back to the drilling automation control system to assist the robotic arm in completing the gripping, pipe laying and resetting operations.

Images