Real-time three-dimensional positioning method and device for unmanned inspection of gas well and medium

Abstract

According to the gas well unmanned inspection-oriented real-time three-dimensional positioning method and device and the computer readable storage medium provided by the invention, the pixel deviation caused by lens distortion is eliminated by strictly participating in distortion calibration in a camera; calculating a depth value of a pixel point by combining a known geometric dimension, and realizing reliable mapping from a pixel coordinate to a camera coordinate; rigid transformation of the camera relative to the tail end of the mechanical arm is obtained through a hand-eye calibration method, and high-precision transformation from camera coordinates to robot base coordinates is achieved; a vision-to-motion closed-loop control algorithm is designed, the mechanical arm is subjected to iterative fine adjustment according to a real-time vision positioning result, and positioning errors are automatically converged; according to the invention, the quadruped robot is used as a platform, end-to-end automatic positioning of a borehole can be realized automatically, stably and precisely in an unmanned inspection process, reliable target coordinates are provided for subsequent methane detection or maintenance operation, and reliable guarantee is provided for safety monitoring and maintenance of a gas pipe network.

Description

Technical Field

[0001] The present invention relates to the field of three-dimensional positioning technology, and in particular to a real-time three-dimensional positioning method, device and computer-readable storage medium for unmanned inspection of gas wells. Background Art

[0002] In the operation and maintenance of modern urban gas pipeline networks, underground wells are key points for gas leak monitoring and pipeline inspection. With the acceleration of urbanization and the continuous expansion of gas pipeline coverage, traditional manual inspection methods are no longer able to meet the needs of high-frequency, high-safety, and high-efficiency inspections. Manual inspections are not only labor-intensive but also pose safety risks in complex environments such as nighttime, rainy and snowy conditions, and narrow roads. To improve inspection efficiency and reduce risks, unmanned inspection technologies have been introduced. Quadruped robots, with their excellent obstacle-crossing capabilities and stable terrain adaptability, have become a promising platform.

[0003] Existing inspection solutions based on drones or tracked ground vehicles typically use fixed or simple pan-tilt cameras to photograph manhole covers and detect methane, transmitting the images back to the backend for analysis. These solutions offer the advantages of remote monitoring and wide coverage. However, due to their fixed viewing angle or limited pan-tilt rotation, wellbores are often located at the edge of the image, and they lack robustness to extreme conditions such as ambient lighting and rain reflection. Furthermore, traditional solutions only capture two-dimensional pixel information and lack the ability to map pixels to true three-dimensional coordinates. This makes any subsequent automated operations difficult to perform reliably due to positioning errors. Summary of the Invention

[0004] The present invention aims to provide a real-time three-dimensional positioning method, device and computer-readable storage medium for unmanned inspection of gas wells that overcome the above-mentioned problems or at least partially solve the above-mentioned problems.

[0005] To achieve the above object, the technical solution of the present invention is specifically implemented as follows:

[0006] A first aspect of the present invention provides a real-time three-dimensional positioning method for unmanned inspection of gas wells, comprising:

[0007] Offline calibration is performed on the cameras on the preset inspection equipment to obtain a rigid transformation matrix from the camera coordinate system to the tool coordinate system at the end of the robotic arm. The preset inspection equipment includes: a quadruped robot, an edge computing unit is set in the back expansion dock of each of the four groups of robots, and a robotic arm is set in the back expansion dock of each of the four groups of robots. The end of the robotic arm has an integrated camera and a laser methane detector. The edge computing unit is used to perform real-time detection of manhole covers and wellbores, and convert pixel coordinates to camera coordinates and to robotic arm coordinates, and calculate closed-loop control instructions.

[0008] Performing identification and coordinate conversion of the target gas manhole cover to obtain the true three-dimensional coordinates of the wellbore in the workspace of the quadruped robot;

[0009] If the distance between the base of the quadruped robot and the target position is greater than the preset distance, the robotic arm is fine-tuned, and the process returns to the offline calibration of the camera on the preset inspection equipment and the online recognition and coordinate conversion to obtain the true three-dimensional coordinates of the wellbore in the workspace of the quadruped robot, until the distance between the base of the quadruped robot and the target position is less than or equal to the preset distance.

[0010] Optionally, the offline calibration of the camera on the preset inspection device to obtain the rigid transformation matrix from the camera coordinate system to the tool coordinate system at the end of the robot arm includes:

[0011] Calculate camera intrinsic coefficients and distortion coefficients;

[0012] Perform hand-eye extrinsic parameter calibration to obtain the rigid transformation matrix from the camera coordinate system to the tool coordinate system at the end of the robotic arm.

[0013] Optionally, the calculation of camera intrinsic parameter coefficients and distortion coefficients includes

[0014] Using a preset inner corner checkerboard, the camera at the end of the robotic arm is used to capture calibration images from a preset number of different postures;

[0015] Extract sub-pixel corner points and calculate the intrinsic parameter matrix and distortion vector;

[0016] Verify whether the pixel coordinate measurement error is less than the preset error value.

[0017] Optionally, the hand-eye extrinsic parameter calibration to obtain a rigid transformation matrix from the camera coordinate system to the tool coordinate system at the end of the manipulator includes:

[0018] Fixing a same checkerboard calibration plate in the working space of the quadruped robot;

[0019] Control the robotic arm to move in sequence to a set of different postures greater than or equal to a preset set, record the base→end homogeneous matrix of each set, and calculate the camera shooting calibration plate→camera homogeneous matrix;

[0020] Solve the hand-eye equation to obtain the rigid transformation matrix from the camera coordinate system to the tool coordinate system at the end of the robot arm

[0021] Optionally, the identifying and coordinate conversion of the target gas manhole cover to obtain the real three-dimensional coordinates of the wellbore in the workspace of the quadruped robot includes:

[0022] Acquire the original image of the target gas manhole cover by the camera, and pre-process the original image;

[0023] The preprocessed image is input into the preset detection network, which outputs the bounding boxes of the manhole cover and wellbore and the center pixel coordinates;

[0024] Normalizing the pixel coordinates to a camera coordinate system;

[0025] Using the standard geometric dimensions of the target gas manhole cover, measuring the pixel diameter of the target gas manhole cover in the image, and calculating the depth of the target gas manhole cover in the direction of the camera's line of sight;

[0026] The homogeneous coordinates are mapped to the robot base coordinate system through the rigid transformation matrix and the real-time posture matrix of the manipulator to obtain the real three-dimensional coordinates of the wellbore in the working space of the quadruped robot.

[0027] Optionally, the preprocessing includes: dedistortion and contrast enhancement.

[0028] Optionally, the preset detection network includes: a YOLOv8 detection network or a YOLOv11 detection network.

[0029] Optionally, the fine-tuning of the robotic arm includes:

[0030] The joint increments are calculated based on the Jacobian matrix of the robotic arm, and the robotic arm is controlled to perform fine-tuning according to the joint increments.

[0031] A second aspect of the present invention provides a real-time three-dimensional positioning device for unmanned inspection of gas wells, comprising: a processor, a memory;

[0032] The memory is used to store computer programs;

[0033] The processor is used to execute the above-mentioned real-time three-dimensional positioning method for unmanned inspection of gas wells by calling the computer program.

[0034] A third aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein the program, when executed by a processor, implements the above-mentioned real-time three-dimensional positioning method for unmanned inspection of gas wells.

[0035] It can be seen that the real-time three-dimensional positioning method, device and computer-readable storage medium for unmanned inspection of gas wells provided by the present invention can eliminate pixel deviation caused by lens distortion through strict in-camera distortion calibration; combine with known geometric dimensions to calculate the depth value of pixel points to achieve reliable mapping of pixel coordinates to camera coordinates; use hand-eye calibration method to obtain the rigid transformation of the camera relative to the end of the robotic arm, and achieve high-precision conversion of camera coordinates to robot base coordinates; design a vision-to-motion closed-loop control algorithm, so that the robotic arm can iteratively fine-tune according to the real-time visual positioning results and automatically converge positioning errors; the present invention uses a quadruped robot as a platform, and can automatically, stably and highly accurately achieve end-to-end automated positioning of the wellbore during unmanned inspection, providing reliable target coordinates for subsequent methane detection or maintenance operations, and providing reliable protection for the safe monitoring and maintenance of the gas pipeline network. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0037] Figure 1 A flow chart of a real-time three-dimensional positioning method for unmanned inspection of gas wells provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0038] Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. Rather, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.

[0039] Figure 1 A flowchart of a real-time three-dimensional positioning method for unmanned inspection of gas wells provided by an embodiment of the present invention is shown. Figure 1 The embodiment of the present invention provides a real-time three-dimensional positioning method for unmanned gas well inspection, including:

[0040] S1, offline calibration of the camera on the preset inspection equipment to obtain the rigid transformation matrix from the camera coordinate system to the tool coordinate system at the end of the robotic arm; wherein the preset inspection equipment includes: a quadruped robot, four sets of edge computing units are set in the robot back expansion docks, four sets of robotic arms are set in the robot back expansion docks, cameras and laser methane detectors are integrated at the end of the robotic arms, and the edge computing units are used for real-time detection of manhole covers and wellbores, and for conversion from pixel coordinates to camera coordinates and to robotic arm coordinates, as well as calculation of closed-loop control instructions.

[0041] Specifically, the present invention achieves end-to-end closed-loop positioning of manhole covers and wells, from image detection to the robot's three-dimensional coordinates, through the organic integration of a quadruped robot platform, a robotic arm, an edge computing unit, and a high-performance camera. The quadruped robot provides mobility and obstacle avoidance capabilities, and an NVIDIA Jetson Orin NX is installed in the robot's back dock as an edge computing unit. The robotic arm is fixed to the robot's back dock and has six degrees of freedom. A monocular camera and a laser methane detector are integrated at the end of the robotic arm. The camera is responsible for capturing images of the manhole cover area, and the laser methane module is responsible for measuring methane concentration. The YOLOv8 model or other models that can be run on the Jetson Orin NX complete the real-time detection of manhole covers and wells, and are also responsible for converting pixel coordinates to camera coordinates, to robotic arm coordinates, and calculating closed-loop control instructions. Of course, the present invention can also use a depth camera (such as Intel RealSense) instead of the monocular camera's depth estimation algorithm to directly obtain point cloud data. Of course, the use of other types of cameras should also fall within the scope of protection of the present invention.

[0042] As an optional implementation of an embodiment of the present invention, offline calibration is performed on the camera on the preset inspection equipment to obtain the rigid transformation matrix from the camera coordinate system to the tool coordinate system at the end of the robot arm, including: calculating the camera intrinsic parameter coefficients and distortion coefficients; performing hand-eye extrinsic parameter calibration to obtain the rigid transformation matrix from the camera coordinate system to the tool coordinate system at the end of the robot arm.

[0043] Among them, the calculation of camera internal parameter coefficients and distortion coefficients includes

[0044] Using a preset inner corner checkerboard, the camera at the end of the robotic arm takes calibration images from a preset number of different postures;

[0045] Extract sub-pixel corner points and calculate the intrinsic parameter matrix and distortion vector;

[0046] Verify whether the pixel coordinate measurement error is less than the preset error value.

[0047] Among them, the hand-eye extrinsic parameter calibration is performed to obtain the rigid transformation matrix from the camera coordinate system to the tool coordinate system at the end of the robot arm, including:

[0048] Fix the same checkerboard calibration plate in the working space of the quadruped robot;

[0049] Control the robotic arm to move to a set of postures greater than or equal to the preset group, record the base→end homogeneous matrix of each group, and calculate the camera shooting calibration plate→camera homogeneous matrix;

[0050] Solve the hand-eye equation to obtain the rigid transformation matrix from the camera coordinate system to the tool coordinate system at the end of the robot arm.

[0051] In the specific implementation, during the offline calibration phase, the camera's internal distortion coefficients are first solved. Using an 8×6 checkerboard with 25mm square edges, the end-of-arm camera captures calibration images from at least 40 different poses, and calls OpenCV's cv2.findChessboardCorners() and cv2.cornerSubPix() to extract sub-pixel corners. Then, cv2.calibrateCamera() is used to calculate the intrinsic parameter matrix.

[0052]

[0053] where f x and f y Represents the focal length in the x and y directions, which refers to the distance from the optical center to the imaging plane. x and c y are the x and y coordinates of the principal point, which is the intersection of the optical axis and the imaging plane, usually near the center of the image.

[0054] and distortion vector

[0055] d=[k1,k2,p1,p2,k3]

[0056] Where k1, k2, and k3 are radial distortion coefficients, controlling the degree to which the image expands or contracts outward from the center. k1, k2, and k3 are the coefficients for barrel distortion, pincushion distortion, and higher-order correction terms, respectively. p1 and p2 are tangential distortion coefficients, simulating the curvature caused by imperfect alignment between the lens and the image sensor. p1 represents image tilt caused by nonparallelism between the lens plane and the image plane, while p2 represents trapezoidal distortion caused by lens mounting rotation errors.

[0057] Through the intrinsic parameter matrix and distortion vector, the two-dimensional image coordinates can be mapped to the real three-dimensional camera coordinate system, eliminating the error caused by lens distortion.

[0058] After calibration, the image dedistortion effect is verified by cv2.undistort() to ensure that the pixel coordinate measurement error is less than 0.2 pixel.

[0059] Then, the camera-arm external parameter calibration is performed to obtain the rigid transformation matrix from the camera coordinate system to the tool coordinate system at the end of the robot arm. The same checkerboard calibration plate is fixed in the robot working space, and the robot arm moves to ≥10 groups of different postures in sequence, and the base→end homogeneous matrix A of each group is recorded. i , while the camera shoots the calibration plate and uses cv2.solvePnP() to obtain the plate→camera homogeneous matrix B i . i} and {B i Enter cv2.calibrateHandEye() to solve the hand-eye equation:

[0060] A i X=XB i

[0061] get The error between the predicted and actual postures is verified to be less than 5mm / 0.5° on the new posture. The specific calculation process of solving the equation is as follows:

[0062] The i-th group of calibration data is recorded as:

[0063] The pose from the robot base to the end effector of the robotic arm:

[0064]

[0065] in, Represents the rotation matrix of the robot base to the i-th group of postures at the end of the robotic arm, represents the translation vector of the robot base → arm end posture i. SE(3) represents the set of all 4×4 homogeneous transformation matrices, which are used to describe rigid motion (rotation + translation) in three-dimensional space. SO(3) represents the set of all 3×3 intersection matrices with a determinant of +1, which are used to describe the rotation relationship between different coordinate systems. Representing three-dimensional Euclidean space, the translation vector t is An element in represents the displacement in three-dimensional space. The pose from the calibration plate to the camera (obtained by cv2.solvePnP):

[0066]

[0067] The hand-eye equation is:

[0068] A i X=XB i , i=1,…,N

[0069] The unknown matrix

[0070]

[0071] R is the rotation matrix from the camera coordinate system to the end-of-arm tool coordinate system, and t is the translation vector from the camera coordinate system to the end-of-arm tool coordinate system.

[0072] Performing matrix operations yields:

[0073]

[0074] Extract the rotation part, we have:

[0075]

[0076] Each pair Calculate the rotation increment:

[0077]

[0078] Using quaternion representation, let

[0079]

[0080] Corresponding to R and ΔR respectively i , ΔR′ i .but:

[0081]

[0082] Expand to a linear equation:

[0083]

[0084] Where Q, Q′ are 4×4 multiplication matrices constructed from quaternions. Represents quaternion multiplication. is the set of all quaternions.

[0085] Superimpose all equations for i=1,…,N and solve for q corresponding to the minimum singular value R , and then normalized to the rotation matrix r.

[0086] After the rotation is solved, the translation part is extracted, and we have:

[0087]

[0088] Right now:

[0089]

[0090] Stack all equations as:

[0091] Mt=b

[0092] in

[0093]

[0094] Solving by least squares

[0095] t=(M T M) -1 M T b

[0096] S2, identifies the target gas manhole cover and performs coordinate conversion to obtain the real three-dimensional coordinates of the wellbore in the quadruped robot workspace.

[0097] As an optional implementation of the embodiment of the present invention, identifying and converting the target gas manhole cover to obtain the real three-dimensional coordinates of the wellbore in the workspace of the quadruped robot includes:

[0098] The original image of the target gas manhole cover is collected by a camera and preprocessed;

[0099] The preprocessed image is input into the preset detection network, which outputs the bounding boxes of the manhole cover and wellbore and the center pixel coordinates;

[0100] Normalize pixel coordinates to the camera coordinate system;

[0101] Using the standard geometric dimensions of the target gas manhole cover, the pixel diameter of the target gas manhole cover in the image is measured, and the depth of the target gas manhole cover in the camera's line of sight is calculated;

[0102] The homogeneous coordinates are mapped to the robot base coordinate system through the rigid transformation matrix and the real-time pose matrix of the robotic arm to obtain the real three-dimensional coordinates of the wellbore in the workspace of the quadruped robot.

[0103] Preprocessing includes distortion removal and contrast enhancement. The default detection network includes the YOLOv8 detection network or the YOLOv11 detection network.

[0104] In specific implementation, during online positioning, the quadruped robot platform drives to the target manhole cover area according to the preset path. After the robot stops at the target manhole cover, the robotic arm moves the camera to the initial observation position and loops through the following steps:

[0105] 1. The camera captures the original image, dedistorts it using cv2.undistort(), and applies CLAHE to enhance the contrast.

[0106] 2. The image is fed into the YOLOv8 detection network, which outputs the bounding boxes of the manhole cover and wellbore and the center pixel coordinates (u, v).

[0107] 3. Normalize the pixel coordinates to the camera coordinate system (x c ,y c ):

[0108] x c =(uc x ) / f x

[0109] y c =(vc y ) / f y

[0110] 4. Use the standard geometric dimensions D of the gas manhole cover real , measure its pixel diameter D in the image pixels , according to the classic monocular ranging formula

[0111]

[0112] Calculate the depth z in the direction of the line of sight (unit: the same as the camera calibration length unit). This method is simple and efficient, and has a natural advantage for standard components with known dimensions (such as gas manhole covers), and can recover the depth within an error range of ±5%.

[0113] 5. After obtaining the depth z, the homogeneous coordinate P c =[x c z,y c z,z,1] T Rigid transformation matrix obtained by offline hand-eye calibration And the real-time pose matrix of the robotic arm Mapping to the robot base coordinate system:

[0114]

[0115] Get the real three-dimensional coordinates (X r ,Y r ,Z r ).

[0116] S3: If the distance between the base of the quadruped robot and the target position is greater than the preset distance, the robot arm is fine-tuned, and the process returns to execute offline calibration of the camera on the preset inspection equipment, online recognition, and coordinate conversion to obtain the true three-dimensional coordinates of the wellbore in the workspace of the quadruped robot, until the distance between the base of the quadruped robot and the target position is less than or equal to the preset distance.

[0117] As an optional implementation of the embodiment of the present invention, fine-tuning the robotic arm includes: calculating joint increments based on the Jacobian matrix of the robotic arm, and controlling the robotic arm to perform fine-tuning according to the joint increments.

[0118] In specific implementation, in order to eliminate the mechanism error, the present invention introduces closed-loop vision-motion fine-tuning. robot -P targetWhen || (the target position is the coordinate corresponding to the center of the field of view)>1cm, the system calculates the joint increment Δθ=J based on the Jacobian matrix J of the robot arm. + ΔP is sent to the robotic arm for fine-tuning. After fine-tuning, the image is re-taken and the above positioning process is repeated until the deviation converges to within ±1cm.

[0119] It can be seen that the real-time three-dimensional positioning method for unmanned inspection of gas wells provided by the embodiment of the present invention uses offline calibration technology to accurately solve the camera intrinsic parameters, distortion coefficients and rigid transformation between the camera and the robotic arm (hand-eye calibration matrix), and constructs a closed-loop coordinate mapping path from the image pixel space to the robotic arm workspace, thereby solving the "executability" problem of visual information in terminal positioning; uses known geometric dimensions (manhole cover diameter) combined with a pinhole camera model for scale restoration to achieve centimeter-level depth recovery without the need for additional hardware; the identified wellbore pixel position is converted into three-dimensional coordinates of the camera coordinate system through a monocular depth algorithm, and the precise position of the wellbore in the robotic arm or robot base coordinate system is obtained through hand-eye coordinate system conversion; the image information fed back in real time by the camera is used to construct the inverse solution of the robotic arm Jacobian matrix, and the end position of the robotic arm is corrected through small step increments, ultimately achieving alignment between the wellbore center and the tool coordinate system.

[0120] The present invention can utilize a robotic arm, a computing platform (Jetson Orin NX), and an image acquisition module to work together to achieve portable, mobile, low-power, and highly real-time outdoor positioning detection.

[0121] Based on this, the present invention, through the aforementioned integrated design of visual perception and geometric transformation, can stably identify and locate manhole covers and wellbores in environments such as complex lighting and occlusion, relying on the precise calibration of the monocular camera and depth estimation algorithm, thereby solving the problem of non-robust recognition under traditional fixed-viewing angles. Because pixel coordinates are accurately calibrated with camera internal parameters and reliable depth is obtained through monocular ranging of known dimensions, the present invention can also restore reliable three-dimensional position without depth hardware, ensuring the accuracy of wellbore spatial coordinate conversion and meeting the key requirement of transitioning from two-dimensional detection to three-dimensional execution.

[0122] Furthermore, the present invention combines the rigid transformation obtained from hand-eye calibration with closed-loop feedback from the inverse kinematics of the robotic arm, allowing each positioning error to be eliminated through iterative vision-motion fine-tuning, ultimately converging the positioning accuracy to within ±1 cm. As a result, the present invention not only enables efficient and automatic wellbore positioning during unmanned inspections but also provides reliable target support for subsequent robotic arm alignment testing or sampling operations, significantly improving inspection efficiency and safety, and achieving the three major invention objectives of "highly robust recognition," "precise three-dimensional mapping," and "closed-loop adaptive control."

[0123] Another aspect of the present invention provides a real-time three-dimensional positioning device for unmanned inspection of gas wells, comprising: a processor, a memory;

[0124] memory for storing computer programs;

[0125] The processor is used to execute the above-mentioned real-time three-dimensional positioning method for unmanned inspection of gas wells by calling a computer program.

[0126] It can be seen that the real-time three-dimensional positioning device for unmanned inspection of gas wells provided by the embodiment of the present invention uses offline calibration technology to accurately solve the camera intrinsic parameters, distortion coefficients and rigid transformation between the camera and the robotic arm (hand-eye calibration matrix), and constructs a closed-loop coordinate mapping path from the image pixel space to the robotic arm workspace, thereby solving the "executability" problem of visual information in terminal positioning; uses known geometric dimensions (manhole cover diameter) combined with a pinhole camera model for scale restoration to achieve centimeter-level depth recovery without the need for additional hardware; converts the identified wellbore pixel position into three-dimensional coordinates of the camera coordinate system through a monocular depth algorithm, and obtains the precise position of the wellbore in the robotic arm or robot base coordinate system through hand-eye coordinate system conversion; uses the image information fed back in real time by the camera to construct the inverse solution of the robotic arm Jacobian matrix, and corrects the end position of the robotic arm through small step increments, ultimately achieving alignment between the wellbore center and the tool coordinate system; realizes portable, mobile, low-power, and highly real-time outdoor positioning detection.

[0127] In another aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the above-mentioned real-time three-dimensional positioning method for unmanned inspection of gas wells.

[0128] It can be seen that the computer-readable storage medium provided by the embodiment of the present invention uses offline calibration technology to accurately solve the camera intrinsic parameters, distortion coefficients and rigid transformation between the camera and the robotic arm (hand-eye calibration matrix), and constructs a closed-loop coordinate mapping path from the image pixel space to the robotic arm workspace, thereby solving the "executability" problem of visual information in terminal positioning; uses known geometric dimensions (manhole cover diameter) combined with a pinhole camera model for scale restoration to achieve centimeter-level depth recovery without the need for additional hardware; converts the identified wellbore pixel position into three-dimensional coordinates of the camera coordinate system through a monocular depth algorithm, and obtains the precise position of the wellbore in the robotic arm or robot base coordinate system through hand-eye coordinate system conversion; uses the image information fed back in real time by the camera to construct the inverse solution of the robotic arm Jacobian matrix, and corrects the end posture of the robotic arm through small step increments, and finally achieves alignment of the wellbore center with the tool coordinate system; realizes portable, movable, low-power, and highly real-time outdoor positioning detection.

[0129] The above are merely embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application should all be included within the scope of the claims of the present application.

Claims

1. A real-time three-dimensional positioning method for unmanned inspection of gas wells, characterized in that: include: Offline calibration is performed on the cameras on the preset inspection equipment to obtain a rigid transformation matrix from the camera coordinate system to the tool coordinate system at the end of the robotic arm. The preset inspection equipment includes: a quadruped robot, an edge computing unit is set in the back expansion dock of each of the four groups of robots, and a robotic arm is set in the back expansion dock of each of the four groups of robots. The end of the robotic arm has an integrated camera and a laser methane detector. The edge computing unit is used to perform real-time detection of manhole covers and wellbores, and convert pixel coordinates to camera coordinates and to robotic arm coordinates, and calculate closed-loop control instructions. Performing identification and coordinate conversion of the target gas manhole cover to obtain the true three-dimensional coordinates of the wellbore in the workspace of the quadruped robot; If the distance between the base of the quadruped robot and the target position is greater than the preset distance, the robotic arm is fine-tuned, and the process returns to the offline calibration of the camera on the preset inspection equipment and the online recognition and coordinate conversion to obtain the true three-dimensional coordinates of the wellbore in the workspace of the quadruped robot, until the distance between the base of the quadruped robot and the target position is less than or equal to the preset distance.

2. The method according to claim 1, characterized in that The offline calibration of the camera on the preset inspection device to obtain the rigid transformation matrix from the camera coordinate system to the tool coordinate system at the end of the robot arm includes: Calculate camera intrinsic coefficients and distortion coefficients; Perform hand-eye extrinsic parameter calibration to obtain the rigid transformation matrix from the camera coordinate system to the tool coordinate system at the end of the robotic arm.

3. The method according to claim 2, characterized in that The calculation of camera internal parameter coefficients and distortion coefficients includes Using a preset inner corner checkerboard, the camera at the end of the robotic arm is used to capture calibration images from a preset number of different postures; Extract sub-pixel corner points and calculate the intrinsic parameter matrix and distortion vector; Verify whether the pixel coordinate measurement error is less than the preset error value.

4. The method according to claim 2, characterized in that The hand-eye extrinsic parameter calibration is performed to obtain a rigid transformation matrix from the camera coordinate system to the tool coordinate system at the end of the manipulator, including: Fixing a same checkerboard calibration plate in the working space of the quadruped robot; Control the robotic arm to move in sequence to a set of different postures greater than or equal to a preset set, record the base→end homogeneous matrix of each set, and calculate the camera shooting calibration plate→camera homogeneous matrix; Solve the hand-eye equation to obtain the rigid transformation matrix from the camera coordinate system to the tool coordinate system at the end of the robot arm 5. The method according to claim 4, characterized in that The identification and coordinate conversion of the target gas manhole cover to obtain the real three-dimensional coordinates of the wellbore in the workspace of the quadruped robot includes: Acquire the original image of the target gas manhole cover by the camera, and pre-process the original image; The preprocessed image is input into the preset detection network, which outputs the bounding boxes of the manhole cover and wellbore and the center pixel coordinates; Normalizing the pixel coordinates to a camera coordinate system; Using the standard geometric dimensions of the target gas manhole cover, measuring the pixel diameter of the target gas manhole cover in the image, and calculating the depth of the target gas manhole cover in the direction of the camera's line of sight; The homogeneous coordinates are mapped to the robot base coordinate system through the rigid transformation matrix and the real-time posture matrix of the manipulator to obtain the real three-dimensional coordinates of the wellbore in the working space of the quadruped robot.

6. The method according to claim 5, characterized in that The preprocessing includes: dedistortion and contrast enhancement.

7. The method according to claim 6, characterized in that The preset detection network includes: a YOLOv8 detection network or a YOLOv11 detection network.

8. The method according to claim 7, characterized in that The fine-tuning of the robotic arm comprises: The joint increments are calculated based on the Jacobian matrix of the robotic arm, and the robotic arm is controlled to perform fine-tuning according to the joint increments.

9. A real-time three-dimensional positioning device for unmanned inspection of gas wells, characterized in that: include: Processor, memory; The memory is used to store computer programs; The processor is configured to execute the real-time three-dimensional positioning method for unmanned inspection of gas wells according to any one of claims 1 to 8 by calling the computer program.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the real-time three-dimensional positioning method for unmanned inspection of gas wells as described in any one of claims 1 to 8 is implemented.

Images