A monocular vision-based drill tool joint height detection method and system

Abstract

The application provides a drilling tool joint height detection method and device based on monocular vision, wherein the drilling tool joint height detection method comprises the following steps: S1, after the monocular camera is installed, the pixel height and the physical space height in the image collected by the camera are initially calibrated to obtain a calibration relationship; S2, a drilling tool image collected by the camera is acquired, and a region of interest of the drilling tool image is divided to obtain a region of interest containing only a wellhead drilling tool or a mousehole drilling tool; S3, a drilling tool female joint existing in the region of interest is identified to obtain a female joint identification frame; S4, the pixel height information at the upper edge of the female joint identification frame and the calibration relationship are used to obtain drilling tool joint height information; and S5, after a preset time interval from the last calibration, the pixel height and the physical space height in the image collected by the camera are adaptively calibrated, and S2 to S5 are repeated according to the updated calibration relationship. The problem that the existing drilling tool joint height is mostly manually adjusted is solved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of drilling engineering, and particularly relates to a drilling tool joint height detection method and system based on monocular vision. BACKGROUND

[0002] During the tripping process of the oil drilling platform, the iron roughneck is a key device for performing the make-up and break-out of the drilling tool. During the make-up process, the jaw of the iron roughneck clamps the connecting head on the drill pipe, and through accurate rotation operation, the tight connection of the two drilling tools is realized. When the break-out is needed, the iron roughneck jaw is reversely rotated to easily remove the connecting head. At present, the iron roughneck device needs to be manually adjusted in height during use, and the drill pipe diameter is inputted to determine the jaw moving position, the jaw clamping force and the make-up torque, which reduces the work efficiency. Sometimes, the manual operation deviates, for example, the height of the iron roughneck jaw is adjusted with a large error, and when the jaw clamping part contains the wear-resistant band of the drilling tool, the front teeth of the jaw are broken, which causes the abnormal loss of the device and increases the probability of the drilling operation failure.

[0003] Therefore, an automatic drilling tool joint height detection method is needed to reduce human intervention, improve the automation degree of the iron roughneck, reduce the operation failure risk and improve the drilling operation efficiency. SUMMARY

[0004] (I) Technical problems to be solved

[0005] In view of the above technical problems, the present application at least solves the problems to some extent. Therefore, the purpose of the present application is to provide a drilling tool joint height detection method and device based on monocular vision, which solves the technical problem that the height of the iron roughneck jaw needs to be manually adjusted during the use of the iron roughneck device.

[0006] (II) Technical solutions

[0007] In order to achieve the above purpose, the main technical solutions of the present application include:

[0008] In the first aspect, the present application provides a drilling tool joint height detection method based on monocular vision, which detects the drilling tool joint height based on a monocular camera installed on a drilling platform, and includes the following steps:

[0009] Step S1, after the monocular camera is installed on the drilling platform, the initial calibration of the correspondence between the pixel height and the physical space height in the drilling tool image collected by the monocular camera is performed to obtain a calibration relationship;

[0010] Step S2, obtain the drilling tool image collected by the monocular camera, and divide the drilling tool image into a region of interest (ROI) containing only the wellhead drilling tool or the mousehole drilling tool; wherein the field of view of the monocular camera contains the iron roughneck area, the wellhead rotary table area and the mousehole area;

[0011] Step S3, identify that the drilling tool box exists in the ROI region of interest, and obtain a box recognition of the box; the upper edge of the box recognition of the box coincides with the upper edge of the drilling tool box in the ROI region of interest;

[0012] Step S4, according to the pixel height information at the upper edge of the box recognition of the box and the calibration relationship, the drilling tool joint height information is obtained;

[0013] Step S5, after a preset time interval from the last calibration, the correspondence between the pixel height and the physical space height in the drilling tool image collected by the monocular camera is adaptively calibrated, and the calibration relationship is updated; according to the updated calibration relationship, steps S2 to S5 are repeated.

[0014] Optionally, the initial calibration of the correspondence between the pixel height and the physical space height in the drilling tool image collected by the monocular camera comprises:

[0015] Step A1, obtain three drilling tool images collected by the monocular camera, the three drilling tool images are respectively taken when the physical height of the iron roughneck hand grab pincers from the drilling platform surface is W1, W2 and W3;

[0016] Step A2, identify the iron roughneck hand grab pincers in the three drilling tool images, obtain the pincers recognition box, and determine the pixel height of the iron roughneck hand grab pincers in the three drilling tool images according to the pixel coordinates of the center position of the pincers recognition box, which are P1, P2 and P3 respectively;

[0017] Step A3, according to W1, W2, W3 and P1, P2, P3, initial calibration is carried out to obtain the calibration relationship.

[0018] Optionally, according to W1, W2, W3 and P1, P2, P3, initial calibration is carried out, and the formula is represented as:

[0019] H x =(P x -P2)×Δh3+W2

[0020]

[0021]

[0022] In the formula, P x is the pixel height of pixel x in the drilling tool image; H x is the corresponding physical space height of pixel x in the drilling tool image; K is a calibration quality evaluation index;

[0023] If |K-1|<0.0005, the calibration accuracy meets the use requirement, and if |K-1|≥0.0005, steps A1 to A3 are repeated.

[0024] Optionally, step S3 comprises: identifying that the female drill tool joint and the male drill tool joint exist in the ROI region of interest, and obtaining a female joint identification frame and a male joint identification frame; the upper edge of the female joint identification frame coincides with the upper edge of the female drill tool joint in the ROI region of interest, and the lower edge of the male joint identification frame coincides with the lower edge of the male drill tool joint in the ROI region of interest.

[0025] Step S4 further comprises: obtaining the drill tool state information according to the pixel height information at the upper edge of the female joint identification frame and the pixel height information at the lower edge of the male joint identification frame.

[0026] Optionally, the drill tool state information is obtained according to the pixel height information at the upper edge of the female joint identification frame and the pixel height information at the lower edge of the male joint identification frame, comprising: obtaining a height difference value according to the pixel height information at the upper edge of the female joint identification frame and the pixel height information at the lower edge of the male joint identification frame; if the height difference value is greater than a first preset threshold, obtaining that the drill tool state is a male joint female joint non-connection state; if the height difference value is between the first preset threshold and a second preset threshold, obtaining that the drill tool state is a male joint female joint connection state; and if the height difference value is less than the second preset threshold, obtaining that the drill tool state is a male joint female joint tightening state; wherein the second preset threshold is less than the first preset threshold.

[0027] Optionally, step S3 further comprises: identifying that only the female drill tool joint exists in the ROI region of interest, and obtaining that the drill tool state is a female joint only state; and identifying that only the male drill tool joint exists in the ROI region of interest, and obtaining that the drill tool state is a male joint only state.

[0028] Optionally, the correspondence between the pixel height and the physical space height in the drill tool image collected by the monocular camera is adaptively calibrated, and the calibration relationship is updated, comprising:

[0029] Step B1: taking the drill tool image collected in the last calibration as a reference image, performing SIFT feature extraction on the reference image to obtain a reference key point descriptor set and a reference key point pixel coordinate set;

[0030] Step B2: taking the drill tool image collected in real time by the monocular camera as a correction image, performing SIFT feature extraction on the correction image to obtain a correction key point descriptor set and a correction key point pixel coordinate set;

[0031] Step B3, matching the reference key points and the rectification key points by using a brute force matching method with pixel neighborhood added according to the reference key point descriptor set, the reference key point pixel coordinate set, the rectification key point descriptor set and the rectification key point pixel coordinate set, to obtain matched key points;

[0032] Step B4, performing adaptive calibration according to three groups of matched key points selected from the matched key points, and updating the calibration relationship.

[0033] Optionally, the reference key points and the rectification key points are matched by using a brute force matching method with pixel neighborhood added according to the reference key point descriptor set, the reference key point pixel coordinate set, the rectification key point descriptor set and the rectification key point pixel coordinate set, and a formula is represented as:

[0034]

[0035] If Δp<PI and Δd<DI, the am and the bm are matched successfully.

[0036] In the formula, am is the reference key point, bm is the rectification key point, d am is a descriptor of the reference key point am, is the n-dimensional vector in the descriptor of the reference key point am, bm is a descriptor of the reference key point bm, is the n-dimensional vector in the descriptor of the reference key point bm, Δd is the Euclidean distance between the key point am and the key point bm, p amx is the horizontal pixel coordinate of the reference key point am, amy is the vertical pixel coordinate of the reference key point am, bmx is the horizontal pixel coordinate of the reference key point bm, bmy is the vertical pixel coordinate of the reference key point bm, PI is a third preset threshold value, and DI is a fourth preset threshold value.

[0037] Optionally, the reference key points and the rectification key points are matched by using a brute force matching method with pixel neighborhood added according to the reference key point descriptor set, the reference key point pixel coordinate set, the rectification key point descriptor set and the rectification key point pixel coordinate set, and a formula is represented as:

[0038] H x = (P x -P b2y ) × Δh + a2h

[0039]

[0040] In the formula, P x is the pixel height of the pixel x in the drilling tool image, H xis the physical space height corresponding to the pixel x in the drilling tool image; the reference key point a1 is matched with the correction key point b1, the reference key point a2 is matched with the correction key point b2, and the reference key point a3 is matched with the correction key point b3; p b1y is the pixel longitudinal coordinate of the reference key point b1, p b2y is the pixel longitudinal coordinate of the reference key point b2, p b3y is the pixel longitudinal coordinate of the reference key point b3; a1h is the physical space height corresponding to the reference key point a1, a2h is the physical space height corresponding to the reference key point a2, and a3h is the physical space height corresponding to the reference key point a3; a1h is calculated according to the calibration relationship obtained in the last calibration and the pixel longitudinal coordinate of the reference key point a1, a2h is calculated according to the calibration relationship obtained in the last calibration and the pixel longitudinal coordinate of the reference key point a2, and a3h is calculated according to the calibration relationship obtained in the last calibration and the pixel longitudinal coordinate of the reference key point a3.

[0041] In a second aspect, the present application provides a drilling tool joint height detection system based on monocular vision, comprising a monocular camera and a detection device; the monocular camera is used for collecting a drilling tool image;

[0042] The detection device comprises a memory and a processor, the memory stores a computer program, and the processor realizes the steps of the drilling tool joint height detection method based on monocular vision as described above when executing the computer program stored in the memory.

[0043] (Three) beneficial effects

[0044] The beneficial effects of the present application are:

[0045] The drilling tool joint height detection method based on monocular vision provided by the present application can realize drilling tool height detection and drilling tool state detection without additional calibration devices and without comparing sample data. At the same time, in the drilling tool detection method, the correction image is collected at a fixed time, the sift features are extracted for matching and comparison, the adaptive correction of the height detection system is realized, and the calibration workload of the staff is reduced. BRIEF DESCRIPTION OF DRAWINGS

[0046] The present application is described with the help of the following drawings:

[0047] Figure 1 is a flowchart of the drilling tool joint height detection method based on monocular vision according to the specific embodiment of the present application;

[0048] Figure 2 is a schematic diagram of identifying that the drilling tool female joint and the male joint exist in the ROI region of interest and obtaining the female joint identification frame and the male joint identification frame according to the specific embodiment of the present application. DETAILED DESCRIPTION

[0049] In order to better explain the present application, in order to facilitate understanding, the present application is described in detail below through specific embodiments combined with the drawings.

[0050] Figure 1 The flowchart of the single-vision-based drilling tool joint height detection method provided by the present application.

[0051] As shown in Figure 1 The single-vision-based drilling tool joint height detection method includes the following steps:

[0052] Step S1, after the monocular camera is installed on the drilling platform, the correspondence between the pixel height and the physical space height in the drilling tool image collected by the monocular camera is initially calibrated to obtain a calibration relationship.

[0053] Specifically, after the monocular camera is installed on the drilling platform, the monocular camera position is fixed and no longer moves.

[0054] Among them, the field of view area of the monocular camera contains the iron driller area, the wellhead rotary table area and the mouse hole area. The iron driller performs the make-up and break-out operation on the wellhead drill pipe and the mouse hole drill pipe. In order to realize the drilling tool joint height detection to guide the operation of the iron driller, the field of view area of the monocular camera needs to contain the iron driller area, the wellhead rotary table area and the mouse hole area.

[0055] Preferably, the correspondence between the pixel height and the physical space height in the drilling tool image collected by the monocular camera is initially calibrated, including:

[0056] Step A1, three drilling tool images collected by the monocular camera are obtained, and the three drilling tool images are respectively taken when the physical height of the iron driller hand grab pincers from the drilling platform surface is W1, W2 and W3.

[0057] Step A2, the iron driller hand grab pincers in the three drilling tool images are identified to obtain a pincers identification frame, and the pixel height of the iron driller hand grab pincers in the three drilling tool images is determined to be P1, P2 and P3 respectively according to the pixel coordinates of the center position of the pincers identification frame.

[0058] Step A3, according to W1, W2, W3 and P1, P2, P3, initial calibration is performed to obtain a calibration relationship.

[0059] Specifically, the acquisition process of the three drilling tool images is as follows: first, control the iron driller to move to the wellhead position, then control the iron driller hand grab pincers to move to any three different heights respectively, and control the monocular camera to take pictures respectively to obtain three drilling tool images. The physical height W1, W2 and W3 of the iron driller hand grab pincers from the drilling platform surface can be read from the iron driller control system.

[0060] Further preferably, according to W1, W2, W3 and P1, P2, P3, the initial calibration is performed, which is expressed by the formula:

[0061] H x = (P x -P2) x Ah3 + W2

[0062]

[0063] In the formula, P x is the pixel height of pixel x in the drill tool image; H x is the physical space height corresponding to pixel x in the drill tool image; and K is a calibration quality evaluation index.

[0064] If |K-1|<0.0005, the calibration accuracy meets the use requirement, and if |K-1|≥0.0005, steps A1 to A3 are repeated.

[0065] In step S2, the drill tool image collected by the monocular camera is obtained, and the region of interest (ROI) of the drill tool image is divided to obtain the ROI region of interest containing only the wellhead drill tool or the mousehole drill tool.

[0066] In step S3, it is identified that the drill tool female joint and the drill tool male joint exist in the ROI region of interest, and the female joint recognition frame and the male joint recognition frame (as shown in Figure 2 ) are obtained; the upper edge of the female joint recognition frame coincides with the upper edge of the drill tool female joint in the ROI region of interest, and the lower edge of the male joint recognition frame coincides with the lower edge of the drill tool male joint in the ROI region of interest.

[0067] Preferably, step S3 further comprises: identifying that only the drill tool female joint exists in the ROI region of interest to obtain a drill tool state of only the female joint state; and identifying that only the drill tool male joint exists in the ROI region of interest to obtain a drill tool state of only the male joint state. In this way, the drill tool state information, i.e., the only female joint state information and the only male joint state information, can be provided to the staff.

[0068] In step S4, according to the pixel height information at the upper edge of the female joint recognition frame and the calibration relationship, the drill tool joint height information is obtained; and according to the pixel height information at the upper edge of the female joint recognition frame and the pixel height information at the lower edge of the male joint recognition frame, the drill tool state information is obtained.

[0069] The drill tool joint height is the physical space height corresponding to the pixel at the upper edge of the female joint recognition frame. After the current drill tool joint height is calculated, the iron roughneck can read the data to automatically perform the make-up and break-out operation of the drill tool.

[0070] Preferably, the drilling tool state information is obtained according to the pixel height information at the upper edge of the female joint recognition frame and the pixel height information at the lower edge of the male joint recognition frame, including: obtaining a height difference value according to the pixel height information at the upper edge of the female joint recognition frame and the pixel height information at the lower edge of the male joint recognition frame; if the height difference value is greater than a first preset threshold, obtaining that the drilling tool state is a female joint male joint non-connection state; if the height difference value is between the first preset threshold and a second preset threshold, obtaining that the drilling tool state is a female joint male joint connection state; and if the height difference value is less than the second preset threshold, obtaining that the drilling tool state is a female joint male joint tightening state; wherein the second preset threshold is less than the first preset threshold. In this way, the drilling tool joint height detection method provided by the present application can provide comprehensive drilling tool state information for the staff, including only female joint, only male joint, female joint male joint non-connection, female joint male joint connection, and female joint male joint tightening; and provide a reference for the staff controlling the iron driller (for example, when the drilling tool state is only the female joint state and only the male joint state, the iron driller is prohibited from performing the iron driller make-up and breakout operation).

[0071] Step S5, after a preset time interval from the last calibration, adaptively calibrating the correspondence between the pixel height and the physical space height in the drilling tool image collected by the monocular camera, updating the calibration relationship; and repeating steps S2 to S5 according to the updated calibration relationship.

[0072] With long-time drilling work on the drilling platform, the monocular camera structure gradually changes slightly with the vibration of the platform, and the visual range of the monocular camera also changes slightly, resulting in errors in the calibrated pixel height and physical space height in the drilling tool image, and further affecting the automatic iron driller make-up and breakout operation. Therefore, it is necessary to adaptively calibrate the correspondence between the pixel height and the physical space height in the drilling tool image collected by the monocular camera after a preset time interval from the last calibration.

[0073] Preferably, the adaptive calibration of the correspondence between the pixel height and the physical space height in the drilling tool image collected by the monocular camera, and the updating of the calibration relationship, include:

[0074] Step B1, taking the drilling tool image collected in the last calibration as a reference image, performing SIFT feature extraction on the reference image to obtain a reference key point descriptor set and a reference key point pixel coordinate set.

[0075] Step B2, taking the drilling tool image collected by the monocular camera in real time as a correction image, performing SIFT feature extraction on the correction image to obtain a correction key point descriptor set and a correction key point pixel coordinate set.

[0076] Step B3, according to the reference key point descriptor set, the reference key point pixel coordinate set, the corrected key point descriptor set and the corrected key point pixel coordinate set, the reference key points and the corrected key points are matched by using a brute force matching method with pixel neighborhood added, and matched successful key points are obtained.

[0077] Step B4, according to three groups of matched key points selected from the matched successful key points, self-adaptive calibration is performed, and the calibration relationship is updated.

[0078] The reference key point descriptor set is represented as αd(d a1 ,d a2 ,...,d am ,...,d ao ), each key point descriptor is composed of a 128-dimensional vector, for example The reference key point pixel coordinate set is represented as αp(p a1 ,p a2 ,...,p am ,...,p ao ), the coordinate of p am is represented as (p amx ,p amy ). The corrected key point descriptor set is represented as βd(d b1 ,d b2 ,...,d bm ,...,d bo ), each key point descriptor is composed of a 128-dimensional vector, for example The corrected key point pixel coordinate set is represented as βp(p b1 ,p b2 ,...,p bm ,...,p bo ), the coordinate of p bm is represented as (p bmx ,p bmy ).

[0079] Further preferably, according to the reference key point descriptor set, the reference key point pixel coordinate set, the corrected key point descriptor set and the corrected key point pixel coordinate set, the reference key points and the corrected key points are matched by using a brute force matching method with pixel neighborhood added, and the formula is represented as:

[0080]

[0081] If Δp<PI and Δd<DI, am and bm are matched successfully.

[0082] In the formula, am is a reference key point, bm is a corrected key point; d am is a descriptor of the reference key point am, is the n-dimensional vector in the descriptor of the reference key point am; d bm is the descriptor of the reference key point bm, is the n-dimensional vector in the descriptor of the reference key point bm; Δd is the Euclidean distance between the key point am and the key point bm; p amx is the pixel horizontal coordinate of the reference key point am, p amy is the pixel vertical coordinate of the reference key point am; p bmx is the pixel horizontal coordinate of the reference key point bm, p bmy is the pixel vertical coordinate of the reference key point bm; PI is a third preset threshold value, and DI is a fourth preset threshold value.

[0083] The traditional key point brute force matching mode is to directly calculate the Euclidean distance of two descriptors, and it is easy to have a matching error key point. According to the actual working condition of the drilling platform, the camera structure changes little, and the image pixel position change is also limited, so on the basis of the traditional key point brute force matching, the pixel neighborhood calculation is increased, and the matching accuracy is improved.

[0084] Further preferably, according to three groups of matching key points selected from the matching successful key points, adaptive calibration is performed, and the calibration relationship is updated, and the updated calibration relationship formula is expressed as:

[0085] H x = (P x -P b2y ) × Δh + a2h

[0086]

[0087] In the formula, P x is the pixel height of the pixel x in the drilling tool image; H x is the physical space height corresponding to the pixel x in the drilling tool image; the reference key point a1 is matched with the corrected key point b1, the reference key point a2 is matched with the corrected key point b2, and the reference key point a3 is matched with the corrected key point b3; p b1y is the pixel vertical coordinate of the reference key point b1, p b2y is the pixel vertical coordinate of the reference key point b2, p b3y is the pixel vertical coordinate of the reference key point b3; a1h is the physical space height corresponding to the reference key point a1, a2h is the physical space height corresponding to the reference key point a2, and a3h is the physical space height corresponding to the reference key point a3; a1h is calculated according to the calibration relationship obtained in the last calibration and the pixel vertical coordinate of the reference key point a1, a2h is calculated according to the calibration relationship obtained in the last calibration and the pixel vertical coordinate of the reference key point a2, and a3h is calculated according to the calibration relationship obtained in the last calibration and the pixel vertical coordinate of the reference key point a3.

[0088] If the calibration relationship of the last calibration is the calibration relationship of the first calibration, the calculation of a1h, a2h and a3h is expressed by the following formula:

[0089] a1h=b1h=(P a1y -P2)Δh3+W2

[0090] a2h=b2h=(P a2y -P2)Δh3+W2

[0091] a3h=b3h=(P a3y -P2)Δh3+W2

[0092] In the formula, b1h is the physical space height corresponding to the corrected key point b1, b2h is the physical space height corresponding to the corrected key point b2, and b3h is the physical space height corresponding to the corrected key point b3.

[0093] In this way, the adaptive ability of the drill tool joint height detection method is improved.

[0094] Preferably, three groups of matched key points are selected from the matched key points, including: averaging the pixel horizontal coordinates and the pixel vertical coordinates of all the matched reference key points to obtain an average pixel horizontal coordinate and an average pixel vertical coordinate, the pixel point represented by the average pixel horizontal coordinate and the average pixel vertical coordinate is an average pixel point, and the key point matching group corresponding to the reference key point with the largest pixel vertical coordinate, the reference key point with the smallest pixel vertical coordinate and the reference key point closest to the average pixel point is selected from the matched key points.

[0095] The drill tool joint height detection method based on monocular vision provided by the application can realize drill tool height detection and drill tool state detection without additional calibration devices and sample data comparison. Meanwhile, the adaptive correction of the height detection system is realized by collecting correction images at regular time intervals, extracting sift features for matching and comparison in the drill tool detection method, and the calibration workload of the staff is reduced.

[0096] The application further provides a drill tool joint height detection system based on monocular vision, which comprises a monocular camera and a detection device.

[0097] Those skilled in the art will appreciate that embodiments of the present application can be readily used as a method, a system or a computer program product. Accordingly, the present application can take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application can take the form of a computer program product on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROMs, optical storage devices, etc.) embodying computer readable program code thereon for use by or in connection with an instruction execution system. For the purposes of this description, a computer-usable or computer readable storage medium can be any apparatus that can contain, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device.

[0098] The present application is described in reference to the flowchart and / or block diagram illustrations of the method, apparatus (system) and computer program product according to embodiments of the present application. It will be understood that each block of the flowchart and / or block diagrams, and combinations of blocks in the flowchart and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions of the flowchart and / or block diagrams.

[0099] It should be noted that any references made herein to elements or components should not be construed as limiting the scope of the claims to having only those specific elements. Rather, the language is intended to encompass all possible combinations of elements, even if not explicitly listed. It is intended that each element recited in a claim can be substituted with alternative elements or combinations of elements, as appropriate. It is further noted that the use of "a", "an" or "the" to describe a singular entity should not be construed as limiting the scope of the claims to having only that specific singular entity. Rather, the language is intended to encompass all possible combinations of elements, even if not explicitly listed. It is intended that each element recited in a claim can be substituted with alternative elements or combinations of elements, as appropriate. It is further noted that the use of the terms "first", "second", "third", etc. to describe a particular element or step should not be construed as limiting the scope of the claims to having only those specific elements. Rather, the language is intended to encompass all possible combinations of elements, even if not explicitly listed. It is intended that each element recited in a claim can be substituted with alternative elements or combinations of elements, as appropriate.

[0100] Furthermore, it is to be understood that the use of certain terms to describe the present application is neither meant to limit the scope of the present application nor to imply that the terms are meant to be synonymous. For example, the terms "comprising", "comprises" and "comprised of" as well as "including", "includes" and "included of" are used synonymously to mean that the element listed is present in the item, combination of items, article or compositions of matter which is described by the rest of the sentence even though the other elements of that item, combination of items, article or compositions of matter are not necessarily listed. As used herein, "comprising" means "including, but not limited to". It is further noted that the claims can be drafted to exclude any essential elements of the application. As such, these terms are intended to be as broad as reasonably permitted by the Federal Judges Association and any other regulatory bodies.

[0101] While the preferred embodiments of the application have been described, additional variations and modifications can be made to the embodiments by those skilled in the art once they have the benefit of the present disclosure. Therefore, the present application should not be limited to the embodiments described herein, but should be given a broad construction consistent with the scope of the appended claims, and any and all equivalents thereof.

[0102] It will be apparent to those skilled in the art that various modifications and variations can be made to the present application without departing from the spirit or scope of the application. Thus, it is intended that the present application cover modifications and variations of this application provided they come within the scope of the appended claims and their equivalents.

Claims

1. A method for detecting drill string joint height based on monocular vision, characterized in that, The height of drill string joints is detected using a monocular camera mounted on the drilling platform, including the following steps: Step S1: After the monocular camera is installed on the drilling platform, the correspondence between pixel height and physical space height in the drill string image acquired by the monocular camera is initially calibrated to obtain the calibration relationship. Step S2: Acquire the drill string image captured by the monocular camera, and divide the drill string image into regions of interest to obtain ROI regions of interest that only contain wellhead drill strings or rat hole drill strings; wherein, the field of view of the monocular camera includes the iron drill string area, the wellhead rotary table area and the rat hole area. Step S3: Identify the presence of a female drill string connector in the Region of Interest (ROI) and obtain a female drill string connector identification frame; the upper edge of the female drill string connector identification frame coincides with the upper edge of the female drill string connector in the ROI. Step S4: Obtain the drill string joint height information based on the pixel height information and calibration relationship at the upper edge of the female connector identification frame; Step S5: After the preset time interval since the last calibration start, adaptively calibrate the correspondence between pixel height and physical space height in the drill image acquired by the monocular camera, and update the calibration relationship; repeat steps S2 to S5 according to the updated calibration relationship. Adaptive calibration is performed on the correspondence between pixel height and physical spatial height in the drill string images acquired by the monocular camera, and the calibration relationship is updated, including: Step B1: Use the drill string image collected in the previous calibration as the reference image, perform SIFT feature extraction on the reference image to obtain the reference key point descriptor subset and the reference key point pixel coordinate set; Step B2: Obtain the drill string image acquired in real time by the monocular camera as the correction image, perform SIFT feature extraction on the correction image, and obtain the correction key point descriptor subset and the correction key point pixel coordinate set. Step B3: Based on the reference key point descriptor subset, the reference key point pixel coordinate set, the corrected key point descriptor subset, and the corrected key point pixel coordinate set, a brute-force matching method that increases pixel neighborhood is used to match the reference key points and the corrected key points to obtain the successfully matched key points. Step B4: Based on the three sets of matching key points selected from the successfully matched key points, perform adaptive calibration and update the calibration relationship.

2. The method for detecting drill joint height based on monocular vision according to claim 1, characterized in that, Initial calibration is performed on the correspondence between pixel height and physical spatial height in the drill string images acquired by the monocular camera, including: Step A1: Acquire three drill string images captured by a monocular camera. The three drill string images were taken when the physical height of the driller's hand gripping the pliers from the drilling platform surface was W1, W2, and W3, respectively. Step A2: Identify the iron driller's hand gripping the jaws in the three drill tool images to obtain the jaw recognition frame. Based on the pixel coordinates of the center position of the jaw recognition frame, determine the pixel heights of the iron driller's hand gripping the jaws in the three drill tool images as P1, P2, and P3, respectively. Step A3: Perform initial calibration based on W1, W2, W3 and P1, P2, P3 to obtain the calibration relationship.

3. The method for detecting drill joint height based on monocular vision according to claim 2, characterized in that, Based on W1, W2, W3 and P1, P2, P3, the initial calibration is performed, expressed by the formula: H x =(P x -P2)×Δh3+W2 In the formula, P x H represents the pixel height of pixel x in the drill string image. x is the physical space height corresponding to pixel x in the drill string image; K is the calibration quality evaluation index; If |K - 1| < 0.0005, the calibration accuracy meets the usage requirements. If |K - 1| ≥ 0.0005, repeat steps A1 to A3.

4. The method for detecting drill joint height based on monocular vision according to claim 1, characterized in that, Step S3 is as follows: Identify that there are tool joint boxes and tool joint pins in the ROI (Region of Interest), and obtain the box recognition frame for the box and the pin recognition frame for the pin; the upper edge of the box recognition frame coincides with the upper edge of the tool joint box in the ROI, and the lower edge of the pin recognition frame coincides with the lower edge of the tool joint pin in the ROI. Step S4 further includes: Obtain the tool state information based on the pixel height information at the upper edge of the box recognition frame and the pixel height information at the lower edge of the pin recognition frame.

5. The method for detecting drill joint height based on monocular vision according to claim 4, characterized in that, Obtain the tool state information based on the pixel height information at the upper edge of the box recognition frame and the pixel height information at the lower edge of the pin recognition frame, including: Obtain the height difference based on the pixel height information at the upper edge of the box recognition frame and the pixel height information at the lower edge of the pin recognition frame; if the height difference is greater than the first preset threshold, obtain that the tool state is the state where the pin and the box are not connected; if the height difference is between the first preset threshold and the second preset threshold, obtain that the tool state is the state where the pin and the box are connected; if the height difference is less than the second preset threshold, obtain that the tool state is the state where the pin and the box are tightened; where the second preset threshold is less than the first preset threshold.

6. The method for detecting drill joint height based on monocular vision according to claim 4, characterized in that, Step S3 further includes: Identify that there is only a tool joint box in the ROI, and obtain that the tool state is the state of only the box; identify that there is only a tool joint pin in the ROI, and obtain that the tool state is the state of only the pin.

7. The method for detecting drill joint height based on monocular vision according to claim 1, characterized in that, Based on the reference key point descriptor set, the reference key point pixel coordinate set, the corrected key point descriptor set, and the corrected key point pixel coordinate set, use the brute - force matching method of increasing the pixel neighborhood to match the reference key points and the corrected key points. The formula is expressed as: If Δp < PI and Δd < DI, then am and bm are successfully matched. In the formula, am is the reference key point, bm is the correction key point; d am The descriptor for the reference key point am, d is the nth dimension vector in the reference keypoint am descriptor; bm The descriptor for the baseline keypoint bm, p is the nth-dimensional vector in the descriptor of the reference keypoint bm; Δd is the Euclidean distance between keypoint am and keypoint bm; amx p is the pixel x-coordinate of the reference key point am. amy The pixel ordinate of the reference key point am; p bmx p is the pixel x-coordinate of the reference keypoint bm. bmy The pixel ordinate of the reference keypoint bm; PI is the third preset threshold, and DI is the fourth preset threshold.

8. The method for detecting drill joint height based on monocular vision according to claim 1, characterized in that, Based on three groups of matched key points selected from the successfully matched key points, perform adaptive calibration, update the calibration relationship, and the updated calibration relationship formula is expressed as: H x =(P x -P b2y )×Δh+a2h In the formula, P x H represents the pixel height of pixel x in the drill string image. x The physical height of pixel x in the drill string image; reference keypoint a1 matches correction keypoint b1, reference keypoint a2 matches correction keypoint b2, and reference keypoint a3 matches correction keypoint b3; p b1y p is the pixel ordinate of the reference keypoint b1. b2y p is the pixel ordinate of the reference keypoint b2. b3y The pixel ordinate of the reference key point b3; a1h is the physical space height corresponding to the reference key point a1, a2h is the physical space height corresponding to the reference key point a2, and a3h is the physical space height corresponding to the reference key point a3; a1h is calculated based on the calibration relationship obtained from the previous calibration and the pixel ordinate of the reference key point a1, a2h is calculated based on the calibration relationship obtained from the previous calibration and the pixel ordinate of the reference key point a2, and a3h is calculated based on the calibration relationship obtained from the previous calibration and the pixel ordinate of the reference key point a3.

9. A drill string joint height detection system based on monocular vision, characterized in that, It includes a monocular camera and a detection device; The monocular camera is used to collect tool images; The detection device includes a memory and a processor which stores a computer program. When the processor executes the computer program stored in the memory, it implements the steps of the method for detecting the seam height of a tool based on monocular vision according to any one of claims 1 to 8.

Images