Automatic alignment method and device for oil sample collection, storage medium and computer equipment

Abstract

The invention relates to the technical field of industrial automatic detection, and discloses an oil sample collection automatic alignment method and device, a storage medium and computer equipment, the method is applied to an oil sample automatic collection system, and the method comprises the following steps: controlling an industrial camera to collect a current frame image, detecting and matching feature points by using an SIFT algorithm, and obtaining 2D-3D matching point pairs corresponding to visual marks; calculating observation pose information of the industrial camera relative to the visual mark by using a PnP algorithm; calculating absolute pose information of the sampling port under a robot coordinate system based on a spatial pose relationship between the visual mark and the sampling port; and then a deviation vector with current mechanical arm tail end pose information is calculated, a target movement track is generated according to the deviation vector, the mechanical arm is controlled to move, and automatic alignment between the tail end executor and the sampling port is achieved. The method does not depend on manual assistance, and the automation degree, the positioning precision and the field adaptability of the oil sample collection process are remarkably improved.

Description

Technical Field

[0001] This invention relates to the field of industrial automation testing technology, and in particular to an automatic alignment method, apparatus, storage medium, and computer equipment for oil sample collection. Background Technology

[0002] In industries such as petroleum, power, and chemicals, the regular collection and analysis of equipment oil samples (such as transformer oil, lubricating oil, and fuel oil) is crucial for ensuring healthy equipment operation and predicting potential failures. With the increasing level of industrial automation, traditional methods relying on manual handheld sampling tools and visual alignment of the sampling port are no longer sufficient to meet the efficiency, safety, and data consistency requirements of modern smart factories. In recent years, some automated equipment has begun to incorporate robotic arms to perform sampling tasks. However, due to limitations such as environmental complexity, insufficient target positioning accuracy, and poor stability of end effector docking, most systems still require manual calibration or are only suitable for highly standardized scenarios, and have not yet achieved truly fully automated, high-precision oil sample collection.

[0003] In existing technologies, automated oil sampling devices generally suffer from insufficient visual positioning accuracy. Most systems rely on ordinary template matching or simple edge detection, which are prone to losing targets or generating large pose errors under interference from industrial environments such as oil stains, reflections, and low light. This makes it impossible for the robotic arm to accurately align with the tiny sampling port. At the same time, there is a lack of closed-loop dynamic correction mechanisms. The system often only takes a single frame image at the initial position for positioning and does not continuously track and adjust during the movement of the robotic arm. It is difficult to cope with mechanical vibration, accumulated errors, or slight target deviations, which seriously restricts the reliable deployment and application of automated sampling devices in complex industrial environments. Summary of the Invention

[0004] In view of this, this application provides an automatic oil sample collection alignment method, device, storage medium and computer equipment, the main purpose of which is to solve the technical problem that the visual positioning accuracy of the existing automatic oil sample collection device is insufficient, which causes the robotic arm to be unable to align with the sampling port.

[0005] According to a first aspect of the present invention, an automatic alignment method for oil sample collection is provided, applied to an automatic oil sample collection system, the automatic oil sample collection system comprising a sampling robot and a target device, wherein an industrial camera is mounted on the end effector of the robotic arm of the sampling robot, and the target collection device is provided with a sampling port, wherein a visual marker is provided at the sampling port; the method includes:

[0006] The industrial camera is controlled to acquire the current frame image at the sampling port. The SIFT algorithm is used to detect the image feature points of the current frame image, and the image feature points are matched with the template feature points of the visual mark to obtain the 2D-3D matching point pair corresponding to the visual mark.

[0007] Based on the 2D-3D matching point pairs and the intrinsic parameter matrix of the industrial camera, the observation pose information of the industrial camera relative to the visual marker is calculated using the PnP algorithm.

[0008] Based on the preset spatial pose relationship between the visual marker and the sampling port, the absolute pose information of the sampling port in the robot coordinate system is calculated according to the observed pose information.

[0009] Calculate the deviation vector between the absolute pose information and the current end effector pose information of the robotic arm, generate the target motion trajectory based on the deviation vector and control the movement of the robotic arm to achieve automatic alignment between the end effector and the sampling port.

[0010] Optionally, before controlling the industrial camera to acquire the current frame image at the sampling port, the method further includes: responding to a remote control command sent by the user, starting a sampling task, and controlling the sampling robot to move to the target device; the sampling robot, based on preset target device location information, controlling the robotic arm to move to the sampling port, and controlling the robotic arm to drive the industrial camera to align with the sampling port.

[0011] Optionally, the step of detecting image feature points of the current frame image using the SIFT algorithm and matching the image feature points with the template feature points of the visual marker to obtain a 2D-3D matching point pair corresponding to the visual marker includes: performing preprocessing operations on the current frame image to obtain an enhanced image, wherein the preprocessing operations include grayscale processing and Gaussian blur processing; extracting key points and descriptors corresponding to the key points of the enhanced image using the SIFT algorithm, wherein the key points include at least the four corner points of the visual marker; performing similarity matching between the descriptors and the feature descriptor library of the template image of the visual marker to obtain an initial matching point pair; and using the RANSAC algorithm to remove mismatched point pairs from the initial matching point pairs, selecting the matching point pair with the highest consistency as the 2D-3D matching point pair corresponding to the visual marker.

[0012] Optionally, the step of calculating the absolute pose information of the sampling port in the robot coordinate system based on the observed pose information, based on the preset spatial pose relationship between the visual marker and the sampling port, includes: calculating intermediate pose information by homogeneous transformation matrix multiplication based on the preset spatial pose relationship between the visual marker and the sampling port and the observed pose information, wherein the intermediate pose information is used to represent the pose of the sampling port coordinate system in the industrial camera coordinate system; and calculating the absolute pose information of the sampling port in the robot coordinate system based on the extrinsic parameter matrix of the robotic arm end effector and the positive kinematics solution of the robotic arm, combined with the intermediate pose information.

[0013] Optionally, the method for obtaining the spatial pose relationship between the visual marker and the sampling port includes: using a calibration probe on the end effector of the robotic arm to sequentially contact multiple physical corner points of the visual marker, while recording the corresponding end effector pose of the robotic arm; based on the relative pose of the visual marker by the industrial camera, the spatial pose relationship between the visual marker and the sampling port is obtained by jointly solving the hand-eye calibration algorithm.

[0014] Optionally, after calculating the absolute pose information of the sampling port in the robot coordinate system based on the observed pose information, the method further includes: constructing a reprojection error model based on the absolute pose information, the 2D-3D matching point pair, and the intrinsic parameter matrix of the industrial camera; in the reprojection error model, performing nonlinear optimization on the absolute pose information with the goal of minimizing the reprojection error to obtain optimized absolute pose information.

[0015] Optionally, after calculating the deviation vector between the absolute pose information and the current end-effector pose information, generating the target motion trajectory based on the deviation vector, and controlling the movement of the robotic arm, the method further includes: during the movement of the robotic arm, controlling the industrial camera to continuously acquire real-time image frames at the sampling port; based on the real-time image frames, repeatedly performing feature point detection and matching, pose information estimation, and coordinate transformation to update the current absolute pose information of the sampling port in the robot coordinate system in real time; calculating the real-time spatial deviation vector between the updated current absolute pose information and the current end-effector pose information; dynamically adjusting the motion trajectory based on the real-time spatial deviation vector, and controlling the robotic arm to move along the adjusted motion trajectory until the end-effector and the sampling port complete physical docking.

[0016] According to a second aspect of the present invention, an automatic oil sample collection alignment device is provided, the device being applied to an automatic oil sample collection system, comprising:

[0017] The feature matching module is used to control the industrial camera to acquire the current frame image at the sampling port, use the SIFT algorithm to detect the image feature points of the current frame image, and match the image feature points with the template feature points of the visual mark to obtain the 2D-3D matching point pair corresponding to the visual mark;

[0018] The pose acquisition module is used to calculate the observation pose information of the industrial camera relative to the visual marker based on the 2D-3D matching point pairs and the intrinsic parameter matrix of the industrial camera using the PnP algorithm.

[0019] The pose conversion module is used to calculate the absolute pose information of the sampling port in the robot coordinate system based on the observed pose information, based on the preset spatial pose relationship between the visual marker and the sampling port.

[0020] An automatic alignment module is used to calculate the deviation vector between the absolute pose information and the current end effector pose information of the robotic arm, generate a target motion trajectory based on the deviation vector, and control the movement of the robotic arm to achieve automatic alignment between the end effector and the sampling port.

[0021] According to a third aspect of the present invention, a storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the above-described automatic alignment method for oil sample collection.

[0022] According to a fourth aspect of the present invention, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the above-described automatic alignment method for oil sample collection.

[0023] This invention provides an automatic alignment method, apparatus, storage medium, and computer equipment for oil sample collection. By setting visual markers at the sampling port and controlling an end-effector industrial camera to acquire the current frame image, stable feature points are extracted using the SIFT algorithm. This effectively overcomes the problem of easy failure of ordinary template matching or edge detection in complex industrial environments such as oil stains, reflections, and low light, ensuring feature robustness. Through 2D-3D point-pair matching and the PnP algorithm, the observed pose of the camera relative to the visual markers is accurately solved, avoiding pose jumps caused by unstable features in traditional methods. Combining the preset spatial pose relationship between the visual markers and the sampling port, the local observed pose is converted into the absolute pose of the sampling port in the robot coordinate system, providing a high-confidence target position for the robotic arm. Finally, by calculating the pose deviation and generating a motion trajectory, automatic and precise alignment of the end effector with the tiny sampling port is achieved. This method does not rely on manual assistance and does not require highly standardized environments, significantly improving the automation, positioning accuracy, and field adaptability of the oil sample collection process.

[0024] The above description is only an overview of the technical solution of this application. In order to better understand the technical means of this application and to implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of this application more obvious and understandable, specific embodiments of this application are given below. Attached Figure Description

[0025] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this application, illustrate exemplary embodiments of the invention and, together with their description, serve to explain the invention and do not constitute an undue limitation thereof. In the drawings:

[0026] Figure 1 A flowchart illustrating an automatic alignment method for oil sample collection provided by an embodiment of the present invention is shown.

[0027] Figure 2 A flowchart illustrating another automatic alignment method for oil sample collection provided by an embodiment of the present invention is shown;

[0028] Figure 3 This diagram illustrates the structure of an automatic oil sample collection alignment device provided in an embodiment of the present invention.

[0029] Figure 4 This diagram illustrates the structure of an automatic oil sample collection alignment device provided in an embodiment of the present invention.

[0030] Figure 5 A schematic diagram of the device structure of a computer device provided in an embodiment of the present invention is shown. Detailed Implementation

[0031] Exemplary embodiments of the present application will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the present application are shown in the drawings, it should be understood that the present application may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this application will be thorough and complete, and will fully convey the scope of the present application to those skilled in the art.

[0032] This application provides an automatic alignment method for oil sample collection, such as... Figure 1 As shown, the method includes the following steps:

[0033] 101. Control the industrial camera to acquire the current frame image at the sampling port, use the SIFT algorithm to detect the image feature points of the current frame image, and match the image feature points with the template feature points of the visual marker to obtain the 2D-3D matching point pairs corresponding to the visual marker.

[0034] It should be noted that the automatic alignment method for oil sample collection provided in this application is applied to an automatic oil sample collection system. The automatic oil sample collection system includes a sampling robot and a target device. An industrial camera is installed on the end effector of the robotic arm of the sampling robot. The target collection device is equipped with a sampling port, and a visual marker is installed at the sampling port. The sampling robot is usually deployed next to substations, refining plants, or large units. Its robotic arm end effector integrates a high-precision industrial camera and a sampling actuator, such as an automatic valve connector or a sampling needle, which can autonomously complete the sampling task in an unattended environment. The target device has a standard sampling port on its shell or pipeline. A high-contrast visual marker, such as an Aruco code or a customized coded pattern, is fixedly installed near the sampling port as a stable reference for visual positioning.

[0035] Among them, SIFT (Scale-Invariant Feature Transform) is a classic local feature extraction algorithm that can stably detect and describe key points in images under different scales, rotations, and lighting conditions. Image feature points refer to pixel locations in an image that have significant discriminative power and repeatability, such as corner points and edge intersections. Visual markers are structured patterns, such as Aruco codes, that are pre-set near the sampling port, and their physical dimensions and spatial coordinates are known. Template feature points refer to the set of feature points of visual markers obtained in the offline acquisition or calibration stage under ideal conditions and their corresponding 3D physical coordinates. 2D-3D matching point pairs refer to the correct correspondence between the 2D pixels detected in the current image and their corresponding 3D spatial points in the real world, which is the basic input data for subsequent pose calculation.

[0036] Specifically, the system first controls an industrial camera mounted at the end of a robotic arm to capture the current frame's RGB image at the sampling port. The image is then preprocessed with grayscale conversion and Gaussian blur to improve feature stability. Next, the SIFT algorithm is invoked to automatically detect scale- and rotation-invariant keypoints in the image and generate corresponding 128-dimensional descriptor vectors. These real-time extracted feature point descriptors are then matched against a pre-stored visual marker template feature point library to establish preliminary candidate matching relationships. Next, geometric consistency verification algorithms such as RANSAC are used to eliminate mismatches, ultimately retaining a subset of matches with stable geometric structures and correct spatial relationships. This forms a set of precise 2D (image pixel coordinates) - 3D (physical coordinates in the visual marker coordinate system) point pairs, serving as reliable input for subsequent PnP pose calculations.

[0037] In this embodiment, by introducing a matching mechanism that combines the SIFT algorithm with structured visual markers, the robustness of feature detection and matching accuracy in complex industrial environments are significantly improved, avoiding the positioning failure or error amplification problems caused by interference in traditional template matching or edge detection methods. At the same time, by establishing high-precision 2D-3D point pairs, high-quality input is provided for the subsequent PnP algorithm, ensuring the accuracy of pose calculation from the source. Ultimately, it can still stably and automatically obtain the accurate image correspondence of the visual reference of the sampling port under non-standardized and non-ideal lighting environments, providing a highly reliable and high-precision data foundation for the entire automatic alignment process.

[0038] 102. Based on 2D-3D matching point pairs and the intrinsic parameter matrix of the industrial camera, the observation pose information of the industrial camera relative to the visual marker is calculated using the PnP algorithm.

[0039] Among them, the intrinsic parameter matrix of an industrial camera is a 3x3 calibration parameter matrix used to describe the imaging characteristics of the camera itself, including focal length, principal point coordinates, etc. It reflects how the industrial camera projects three-dimensional spatial points onto a two-dimensional image plane and is the prerequisite for all visual measurements and pose calculations. The PnP algorithm (Perspective-n-Point) is a computer vision algorithm used to solve the spatial pose of the camera relative to the coordinate system of a 3D object given n sets of 2D-3D point pairs and camera intrinsic parameters. Common implementations include EPnP and solvePnP, which are widely used in robot vision localization. The observation pose information refers to the spatial transformation relationship between the industrial camera coordinate system and the visual marker coordinate system. It is usually composed of a rotation matrix R and a translation vector t, or represented as a 4x4 homogeneous transformation matrix, used to describe from which position and in what posture the camera observes the visual marker.

[0040] Specifically, after the system completes image acquisition and feature matching, it obtains a stable set of 2D-3D matching point pairs, usually no less than 4 pairs. Then, these point pairs, along with the pre-calibrated industrial camera intrinsic parameter matrix, are input into the PnP algorithm module. The algorithm establishes a perspective projection model, that is, transforms the 3D points to the camera coordinate system according to the currently estimated camera pose (R, t), and then projects them onto the image plane through the intrinsic parameter matrix. It calculates the reprojection error between the 3D points and the actual 2D observation points. By minimizing this error, it iteratively solves for the optimal R and t, and finally outputs the observation pose information of the industrial camera relative to the visual marker. This result accurately quantifies the spatial relationship between the industrial camera and the visual marker, providing key input for subsequent coordinate transformation.

[0041] In this embodiment, by introducing the PnP geometric pose solving mechanism, low-level image features are rigorously mathematically bound to high-level spatial structures, achieving high-precision and high-stability conversion from pixel observation to spatial pose. Compared with traditional methods that rely on manual calibration of offsets, image template matching displacement estimation, or edge fitting, the PnP algorithm fully utilizes multi-point geometric constraints and camera imaging models, significantly improving anti-interference capability and positioning accuracy in complex industrial environments. At the same time, the output observation pose information has clear physical meaning and coordinate system affiliation, providing a structured, traceable, and reusable mathematical foundation for subsequent coordinate chain derivation, thereby ensuring that the entire automatic alignment system has high-precision repeatability.

[0042] 103. Based on the preset spatial pose relationship between the visual markers and the sampling port, the absolute pose information of the sampling port in the robot coordinate system is calculated according to the observed pose information.

[0043] Among them, the spatial pose relationship between the visual marker and the sampling port refers to the fixed rigid body transformation relationship between the visual marker coordinate system and the sampling port coordinate system, usually represented by a homogeneous transformation matrix, which describes the position and orientation of the sampling port relative to the visual marker. This relationship is obtained through calibration before system deployment and is the key anchor point connecting visual observation and physical target. The observation pose information refers to the spatial transformation of the industrial camera coordinate system relative to the visual marker coordinate system, which is calculated by the previous PnP algorithm. The absolute pose information of the sampling port in the robot coordinate system refers to the 6-DOF spatial pose of the sampling port coordinate system relative to the robot base coordinate system, which is the final target input for the robot arm motion planning and control. The robot coordinate system, i.e. the robot base coordinate system, is the global reference coordinate system for robot motion control. All end-effector trajectories are planned and executed based on this coordinate system.

[0044] Specifically, after obtaining the observation pose of the industrial camera relative to the visual marker, the system first uses the preset, calibrated spatial relationship to calculate the pose of the sampling port in the industrial camera coordinate system through coordinate transformation; then, combining the extrinsic parameter matrix of the industrial camera relative to the end effector of the robotic arm and the current pose of the end effector of the robotic arm in the base coordinate system, the system finally calculates the absolute pose information of the sampling port in the robot's global coordinate system through chain-like coordinate transformation.

[0045] In this embodiment, by constructing a complete coordinate transformation chain, a precise mapping from local visual observation to global pose is achieved. Compared with the traditional approach of relying on manual measurement of offsets or assuming that the sampling port coincides with the marker, this application improves the geometric accuracy and system consistency of sampling port positioning through preset calibration spatial relationships and matrix operations. At the same time, the absolute pose information has clear physical coordinates, which can be directly input into the robot controller for trajectory planning, avoiding docking failures caused by coordinate system confusion or unit conversion errors, so as to realize a fully automatic, high-precision, and cross-device universal oil sample collection process.

[0046] 104. Calculate the deviation vector between the absolute pose information and the current end effector pose information of the robotic arm, generate the target motion trajectory based on the deviation vector and control the movement of the robotic arm to achieve automatic alignment between the end effector and the sampling port.

[0047] Among them, absolute pose information refers to the 6-DOF spatial pose of the sampling port in the robot base coordinate system, including three-dimensional position and three-dimensional attitude. It is the target docking position finally obtained by the system through visual positioning and coordinate transformation. Current end-effector pose information refers to the real-time 6-DOF spatial pose of the end-effector in the robot base coordinate system, which is obtained by the robot controller through real-time calculation of forward kinematics. Deviation vector refers to the spatial difference between the target pose of the sampling port and the current pose of the end-effector. It is usually decomposed into position deviation and attitude deviation, and is the direct basis for generating motion control commands. Target motion trajectory refers to the spatial path followed by the end-effector of the robot to move smoothly and safely from the current position to the target sampling port pose. It is usually generated by path planning algorithm to ensure smooth, collision-free and shock-free motion.

[0048] Specifically, after obtaining the absolute pose of the sampling port, the system synchronously reads the current pose of the end effector fed back by the robot controller. Through relative transformations between coordinate systems, it calculates the spatial deviation vector from the end effector to the sampling port. This transformation can be further decomposed into translational and rotational components. Subsequently, the robot motion planning module generates a continuous and smooth spatial motion trajectory from the current pose to the target pose based on this deviation vector and preset motion parameters. Finally, the robot controller drives the joint motors in real time according to this trajectory, enabling the end effector to gradually approach and ultimately precisely align with the sampling port, completing the physical docking preparation. The entire process requires no manual intervention, achieving fully automatic closed-loop control.

[0049] In this embodiment, the visual positioning results are directly converted into motion control commands that the robot can execute, which solves the problems of low efficiency, poor consistency and reliance on experience in traditional manual alignment; by accurately calculating spatial deviations and generating optimized trajectories, the robotic arm is ensured to complete docking efficiently, improving the safety, efficiency and repeatability of sampling operations, and enhancing environmental adaptability and deployment flexibility in complex industrial sites.

[0050] This invention provides an automatic alignment method, apparatus, storage medium, and computer equipment for oil sample collection. By setting visual markers at the sampling port and controlling an end-effector industrial camera to acquire the current frame image, stable feature points are extracted using the SIFT algorithm. This effectively overcomes the problem of easy failure of ordinary template matching or edge detection in complex industrial environments such as oil stains, reflections, and low light, ensuring feature robustness. Through 2D-3D point-pair matching and the PnP algorithm, the observed pose of the camera relative to the visual markers is accurately solved, avoiding pose jumps caused by unstable features in traditional methods. Combining the preset spatial pose relationship between the visual markers and the sampling port, the local observed pose is converted into the absolute pose of the sampling port in the robot coordinate system, providing a high-confidence target position for the robotic arm. Finally, by calculating the pose deviation and generating a motion trajectory, automatic and precise alignment of the end effector with the tiny sampling port is achieved. This method does not rely on manual assistance and does not require highly standardized environments, significantly improving the automation, positioning accuracy, and field adaptability of the oil sample collection process.

[0051] This application provides another automatic alignment method for oil sample collection, such as... Figure 2 As shown, the method includes the following steps:

[0052] 201. Start the sampling task, control the robotic arm to drive the industrial camera to the sampling port, and control the industrial camera to collect the current frame image at the sampling port.

[0053] Specifically, in response to a remote control command sent by the user, the sampling task is initiated, and the sampling robot is controlled to move to the target device. Based on the preset target device location information, the sampling robot controls the robotic arm to move to the sampling port, and controls the robotic arm to drive the industrial camera to align with the sampling port.

[0054] In this embodiment, when a user selects a target device and clicks the start sampling button via a remote monitoring terminal, the system automatically generates structured control instructions and sends them to the local controller of the sampling robot via industrial Ethernet. Upon receiving the instructions, the robot first parses the target device ID and retrieves the precise docking pose of the device in the global map coordinate system from the pre-stored device location database. Then, it activates the navigation module, plans an obstacle avoidance path based on LiDAR and wheel speed sensor data, and autonomously moves to a preset safe distance in front of the target device for precise docking. Once in position, the robotic arm controller, based on the relative installation parameters of the sampling port corresponding to the target device, calculates the... Based on the current robot base pose, the system calculates the estimated spatial position of the sampling port in the base coordinate system and drives the six-axis robotic arm to move along a Cartesian straight line trajectory, so that the industrial camera carried by the end effector is initially aligned with the sampling port area. During the movement, the camera acquires low-resolution images in real time. The system determines whether the visual marker has entered the central field of the image through fast template matching. At this time, the coarse alignment is determined to be completed, and the system automatically switches to high-precision visual servo mode to provide stable initial observation conditions for subsequent SIFT feature matching and PnP pose calculation. This realizes the automation and unmanned connection of the entire process from remote command to visual fine alignment.

[0055] 202. Use the SIFT algorithm to detect image feature points in the current frame image, and match the image feature points with the template feature points of the visual markers to obtain 2D-3D matching point pairs corresponding to the visual markers.

[0056] Specifically, the current frame image is preprocessed to obtain an enhanced image, including grayscale conversion and Gaussian blurring. Keypoints and corresponding descriptors of the enhanced image are extracted using the SIFT algorithm, where each keypoint includes at least the four corner points of the visual marker. The descriptors are then matched against a feature descriptor library of the template image of the visual marker to obtain initial matching point pairs. Finally, the RANSAC algorithm is used to remove mismatched point pairs from the initial matching point pairs, and the matching point pair with the highest consistency is selected as the 2D-3D matching point pair corresponding to the visual marker.

[0057] In this embodiment, the system first performs preprocessing on the current frame image captured by the industrial camera. Specifically, it converts the image into a single-channel grayscale image using a weighted average method to reduce computational complexity and enhance feature stability. Then, it applies a Gaussian blur filter with a standard deviation σ = 1.0 and a kernel size of 5x5 to effectively suppress oil reflections, metal surface glare, and environmental noise interference, resulting in an enhanced grayscale image. Next, it calls the SIFT feature extraction module to automatically detect keypoints in the image that are scale- and rotation-invariant, generating a 128-dimensional descriptor vector for each keypoint. The system actively filters keypoints based on preset visual marker geometric constraints, ensuring that at least four physical corner points containing visual markers are included as keypoints to guarantee the structural integrity of subsequent matching. The algorithm first extracts descriptors in real time and then performs fast approximate nearest neighbor matching with an offline visual marker template descriptor library to generate an initial set of matching point pairs. Finally, it inputs the initial matching point pairs into the RANSAC (Random Sample Consensus) algorithm module, sets the inlier threshold to 5 pixels, iterates 200 times, and eliminates mismatched point pairs caused by occlusion, oil stains, or background interference by fitting a perspective transformation model. It retains the subset of inliers with the highest geometric consistency as the final 2D-3D matching point pair output, ensuring that the matching results of each frame contain a stable and accurate correspondence between the four corner points of the visual marker. This provides high-confidence and interference-resistant input data for subsequent PnP pose calculation, significantly improving the robustness and repeatability of visual positioning in complex industrial environments.

[0058] 203. Based on 2D-3D matching point pairs and the intrinsic parameter matrix of the industrial camera, the observation pose information of the industrial camera relative to the visual marker is calculated using the PnP algorithm.

[0059] In this embodiment, after obtaining the selected stable 2D-3D matching point pairs, the system first loads the pre-stored industrial camera intrinsic parameter matrix K from the camera calibration parameter file. This matrix has been obtained and stored in the factory calibration. Then, the system uses EPnP as the default solver to quickly estimate the initial values ​​of the camera extrinsic parameters using algebraic methods. Finally, it combines Levenberg-Marquardt nonlinear optimization iteration to minimize the reprojection error, ultimately outputting the 6-DOF observation pose information of the industrial camera relative to the visual marker coordinate system, which includes a 3x3 rotation matrix R and a 3x1 translation vector t. cam T marker The structure is passed to the next coordinate transformation module, which accurately describes where the camera is located and which direction it is facing from the origin of the visual marker. This provides a high-precision and high-reliability spatial geometric reference for the subsequent derivation of the absolute pose of the sampling port in the robot base coordinate system, ensuring that the entire automatic alignment process operates stably within a high-precision error range.

[0060] 204. Based on the preset spatial pose relationship between the visual markers and the sampling port, the absolute pose information of the sampling port in the robot coordinate system is calculated according to the observed pose information.

[0061] Specifically, based on the spatial pose relationship between the preset visual marker and the sampling port and the observed pose information, intermediate pose information is calculated through homogeneous transformation matrix multiplication. The intermediate pose information is used to represent the pose of the sampling port coordinate system in the industrial camera coordinate system. Based on the extrinsic parameter matrix of the robotic arm end effector and the positive kinematics solution of the robotic arm, combined with the intermediate pose information, the absolute pose information of the sampling port in the robot coordinate system is calculated.

[0062] In this embodiment, the visual marker acts as an intermediary, and its spatial pose relationship with the sampling port is known, denoted as... sample T marker The observation pose information of the industrial camera relative to the visual marker is also obtained by solving the PnP algorithm in the previous step. cam T marker To obtain the pose of the sampling port under the industrial camera, a coordinate chain transformation is required to obtain intermediate pose information. cam T sample The specific calculation method is as follows:

[0063] cam T sample = cam T marker ·( sample T marker ) -1

[0064] However, considering that robot control requires the sampling port to be accurately positioned in the robot coordinate system, further coordinate chain transformation is needed to calculate the absolute pose information of the sampling port in the robot coordinate system. base T sample The specific calculation method is as follows:

[0065] base T sample = base T ee · ee T cam · cam T sample

[0066] in, ee T cam Specifically, this refers to the extrinsic parameter matrix at the end of the robotic arm. This matrix is ​​fixed, taking into account that the camera is mounted on the end of the robotic arm. base T eeIt is the current pose of the robotic arm end effector in the robot coordinate system, which is calculated in real time by the robot controller based on the angles of each joint through forward kinematics. Multiplying the three values ​​together completes the mapping from local visual observation to the robot coordinate system.

[0067] The method for obtaining the spatial pose relationship between the visual marker and the sampling port includes: using a calibration probe on the end effector of the robotic arm to sequentially contact multiple physical corner points of the visual marker, while recording the corresponding end pose of the robotic arm; based on the relative pose of the visual marker by the industrial camera, the spatial pose relationship between the visual marker and the sampling port is obtained by jointly solving the hand-eye calibration algorithm.

[0068] In this embodiment, a high-precision calibration probe is installed at the end of a robotic arm. The position of its tip in the robotic arm's end-effector coordinate system is known. The robotic arm is controlled to move so that the probe sequentially and precisely contacts multiple physical corner points of a visually marked object, such as the four corner points of an Aruco code. Each time a corner point is contacted, the robot controller records the current pose of the robotic arm's end-effector in the base coordinate system. base T ee Furthermore, since the relationship between the probe tip and the end-effector coordinate system is known, the 3D coordinates of each corner point in the base coordinate system can be calculated. Simultaneously with the probe contacting the corner points, an industrial camera at the end-effector captures a visual mark, and the pose of the industrial camera relative to the visual mark is calculated using the aforementioned PnP algorithm. base T cam Then, a joint optimization solution for an optimal spatial transformation is obtained using hand-eye calibration algorithms. sample T marker This refers to the spatial pose relationship calibrated between the visual marker and the sampling port.

[0069] 205. Perform nonlinear optimization on the absolute pose information.

[0070] Specifically, a reprojection error model is constructed based on absolute pose information, 2D-3D matching point pairs, and the intrinsic parameter matrix of the industrial camera. In the reprojection error model, the absolute pose information is nonlinearly optimized with the goal of minimizing the reprojection error, and the optimized absolute pose information is obtained.

[0071] In this embodiment, the system utilizes the 2D-3D matching point pairs extracted from the current frame image and camera intrinsic parameters to construct a reprojection error model. This involves reverse-engineering the current absolute pose back to the image plane, calculating the theoretical projection position of each 3D spatial point in the image, and comparing it with the actual detected 2D image point positions to obtain the projection deviation. Subsequently, with the goal of minimizing the overall projection error of all matching points, a nonlinear optimization algorithm is used to iteratively adjust the absolute pose parameters, making the optimized pose more closely match the real visual observation results. The entire process effectively compensates for the cumulative errors that may be introduced in the early stages of visual positioning, coordinate transformation, robotic arm kinematics, and installation calibration. Without increasing hardware costs, the docking accuracy is improved to the sub-millimeter level, significantly enhancing the robustness and reliability of the system in complex industrial environments.

[0072] 206. Calculate the deviation vector between the absolute pose information and the current end effector pose information of the robotic arm, generate the target motion trajectory based on the deviation vector and control the movement of the robotic arm to achieve automatic alignment between the end effector and the sampling port.

[0073] In this embodiment, the system calculates the spatial deviation between the absolute pose of the sampling port in the robot coordinate system and the real-time pose of the current end effector of the robotic arm, obtains a six-dimensional deviation vector containing position and attitude differences, and generates a smooth and safe target motion trajectory based on this. The robotic arm is driven to gradually approach the target along the optimal path, and finally the end effector is automatically aligned with the sampling port with high precision. No manual intervention is required throughout the process, ensuring stable and reliable docking. For details, please refer to step 104.

[0074] 207. During the movement of the robotic arm, the motion trajectory of the robotic arm is adjusted in real time.

[0075] Specifically, during the movement of the robotic arm, the industrial camera is controlled to continuously acquire real-time image frames at the sampling port; based on the real-time image frames, feature point detection and matching, pose information estimation and coordinate transformation are repeatedly performed to update the current absolute pose information of the sampling port in the robot coordinate system in real time; the real-time spatial deviation vector between the updated current absolute pose information and the current pose information of the robotic arm end effector is calculated; the motion trajectory is dynamically adjusted based on the real-time spatial deviation vector, and the robotic arm is controlled to follow the adjusted motion trajectory until the end effector and the sampling port complete physical docking.

[0076] In this embodiment, as the robotic arm moves toward the sampling port, the industrial camera at the end of the control system continuously acquires real-time image frames of the sampling port area. In each frame, feature point detection, SIFT matching, PnP pose calculation, and coordinate system transformation are repeatedly performed to dynamically update the current absolute pose of the sampling port in the robot coordinate system. Subsequently, the system instantly calculates the spatial deviation vector between the updated pose and the current actual pose of the robotic arm end effector, and adjusts the direction and step size of the motion trajectory in real time based on this deviation to form a closed-loop visual servo control. The robotic arm continuously adjusts during the movement, gradually converging the error until the end effector accurately and smoothly completes the physical docking with the sampling port. This process effectively copes with mechanical vibration, accumulated errors, or slight target offset, significantly improving the alignment success rate and system robustness, and realizing fully automatic closed-loop sampling.

[0077] Furthermore, as Figure 1 and Figure 2 To specifically implement the method, this application provides an automatic oil sample collection alignment device, such as... Figure 3 As shown, the device includes: a feature matching module 301, a pose acquisition module 302, a pose conversion module 303, and an automatic alignment module 304.

[0078] The feature matching module 301 is used to control the industrial camera to acquire the current frame image at the sampling port, use the SIFT algorithm to detect the image feature points of the current frame image, and match the image feature points with the template feature points of the visual marker to obtain the 2D-3D matching point pairs corresponding to the visual marker.

[0079] The pose acquisition module 302 is used to calculate the observation pose information of the industrial camera relative to the visual marker based on the 2D-3D matching point pairs and the intrinsic parameter matrix of the industrial camera using the PnP algorithm.

[0080] The pose conversion module 303 is used to calculate the absolute pose information of the sampling port in the robot coordinate system based on the observed pose information, according to the spatial pose relationship between the preset visual marker and the sampling port.

[0081] The automatic alignment module 304 is used to calculate the deviation vector between the absolute pose information and the current end-effector pose information of the robotic arm, generate the target motion trajectory based on the deviation vector and control the movement of the robotic arm to achieve automatic alignment between the end effector and the sampling port.

[0082] In specific application scenarios, such as Figure 4 As shown, the device also includes a task initiation module 305, which is specifically used to initiate a sampling task in response to a remote control command sent by the user, and control the sampling robot to move to the target device; the sampling robot controls the robotic arm to move to the sampling port based on the preset target device location information, and controls the robotic arm to drive the industrial camera to align with the sampling port.

[0083] In specific application scenarios, the feature matching module 301 is specifically used to perform preprocessing operations on the current frame image to obtain an enhanced image. The preprocessing operations include grayscale processing and Gaussian blur processing; extracting key points and corresponding descriptors of the enhanced image using the SIFT algorithm, wherein the key points include at least the four corner points of the visual marker; performing similarity matching between the descriptors and the feature descriptor library of the template image of the visual marker to obtain initial matching point pairs; and using the RANSAC algorithm to remove mismatched point pairs from the initial matching point pairs, selecting the matching point pair with the highest consistency as the 2D-3D matching point pair corresponding to the visual marker.

[0084] In specific application scenarios, the pose transformation module 303 can be used to calculate intermediate pose information based on the spatial pose relationship and observation pose information between the preset visual marker and the sampling port, through homogeneous transformation matrix multiplication. The intermediate pose information is used to represent the pose of the sampling port coordinate system in the industrial camera coordinate system. Based on the extrinsic parameter matrix of the robotic arm end and the positive kinematic solution of the robotic arm, combined with the intermediate pose information, the absolute pose information of the sampling port in the robot coordinate system is calculated.

[0085] In specific application scenarios, the pose conversion module 303 can also be used to use the calibration probe on the end effector of the robotic arm to sequentially contact multiple physical corners of the visual marker, while recording the corresponding end pose of the robotic arm; based on the relative pose of the visual marker by the industrial camera, the spatial pose relationship between the visual marker and the sampling port is obtained through joint calculation by hand-eye calibration algorithm.

[0086] In specific application scenarios, the device also includes a pose optimization module 306, which is used to construct a reprojection error model based on absolute pose information, 2D-3D matching point pairs and the intrinsic parameter matrix of the industrial camera. In the reprojection error model, the absolute pose information is nonlinearly optimized with the goal of minimizing the reprojection error, so as to obtain the optimized absolute pose information.

[0087] In specific application scenarios, the device also includes a feedback adjustment module 307, which is specifically used to control the industrial camera to continuously acquire real-time image frames at the sampling port during the movement of the robotic arm; based on the real-time image frames, repeatedly perform feature point detection and matching, pose information estimation and coordinate transformation, and update the current absolute pose information of the sampling port in the robot coordinate system in real time; calculate the real-time spatial deviation vector between the updated current absolute pose information and the current pose information of the robotic arm end effector; dynamically adjust the motion trajectory based on the real-time spatial deviation vector, and control the robotic arm to follow the adjusted motion trajectory until the end effector and the sampling port complete physical docking.

[0088] It should be noted that other corresponding descriptions of the functional units involved in the automatic oil sample collection alignment device provided in this embodiment can be found in [reference needed]. Figure 1 and Figure 2 The corresponding descriptions in [the document] will not be repeated here.

[0089] Based on the above, Figure 1 and Figure 2 Accordingly, this embodiment also provides a storage medium storing a computer program that, when executed by a processor, implements the above-described automatic alignment method for oil sample collection.

[0090] Based on this understanding, the technical solution of this application can be embodied in the form of a software product. The software product to be identified can be stored in a non-volatile storage medium (such as CD-ROM, USB flash drive, mobile hard drive, etc.) and includes several instructions to enable a computer device (such as personal computer, server, or network device, etc.) to execute the automatic alignment method for oil sample collection in various implementation scenarios of this application.

[0091] Based on the above, Figure 1 and Figure 2 The method shown, and Figure 3 and Figure 4 The illustrated embodiment of the automatic oil sample collection alignment device is designed to achieve the above objectives, such as... Figure 5 As shown, this embodiment also provides a physical device for automatic oil sample collection and alignment. This device includes a communication bus, a processor, a memory, and a communication interface. It may also include input / output interfaces and a display device. The various functional units can communicate with each other via the bus. The memory stores a computer program, and the processor executes the program stored in the memory to perform the automatic oil sample collection and alignment method described in the above embodiment.

[0092] Optionally, the physical device may also include a user interface, a network interface, a camera, radio frequency (RF) circuitry, sensors, audio circuitry, a Wi-Fi module, etc. The user interface may include a display screen, input units such as a keyboard, etc., and optional user interfaces may also include USB interfaces, card reader interfaces, etc. The network interface may optionally include standard wired interfaces, wireless interfaces (such as Wi-Fi interfaces), etc.

[0093] Those skilled in the art will understand that the structure of the automatic alignment device for oil sample collection provided in this embodiment does not constitute a limitation on the device, and may include more or fewer components, or combine certain components, or have different component arrangements.

[0094] The storage medium may also include an operating system and a network communication module. The operating system is a program that manages the hardware and software resources of the aforementioned physical device, supporting the operation of information processing programs and other software and / or programs to be identified. The network communication module is used to enable communication between the various components within the storage medium, as well as communication with other hardware and software in the information processing physical device.

[0095] Through the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented using software plus necessary general-purpose hardware platforms, or it can be implemented in hardware. By applying the technical solution of this application, visual markers are set at the sampling port, and the end-effector industrial camera is controlled to acquire the current frame image. Stable feature points are extracted using the SIFT algorithm, effectively overcoming the problem of easy failure of ordinary template matching or edge detection in complex industrial environments such as oil stains, reflections, and low light, ensuring feature robustness. Through 2D-3D point pair matching and PnP algorithm, the observation pose of the camera relative to the visual markers is accurately solved, avoiding problems such as pose jump caused by feature instability in traditional methods. Combining the preset spatial pose relationship between the visual markers and the sampling port, the local observation pose is converted into the absolute pose of the sampling port in the robot coordinate system, providing a high-confidence target position for the robotic arm. Finally, by calculating the pose deviation and generating the motion trajectory, the automatic and precise alignment of the end effector and the tiny sampling port is achieved. The above method does not rely on manual assistance and does not require highly standardized structural scenarios, significantly improving the automation level, positioning accuracy, and field adaptability of the oil sample collection process.

[0096] Those skilled in the art will understand that the accompanying drawings are merely schematic diagrams of a preferred embodiment, and the modules or processes shown in the drawings are not necessarily essential for implementing this application. Those skilled in the art will understand that the modules in the apparatus of the embodiment can be distributed within the apparatus of the embodiment as described, or can be modified to be located in one or more apparatuses different from this embodiment. The modules of the above-described embodiment can be combined into one module, or further divided into multiple sub-modules.

[0097] The serial numbers in this application are for descriptive purposes only and do not represent the superiority or inferiority of any particular implementation scenario. The above disclosures are merely a few specific implementation scenarios of this application; however, this application is not limited thereto, and any variations conceived by those skilled in the art should fall within the protection scope of this application.

Claims

1. An automatic alignment method for oil sample collection, characterized in that, An automatic oil sample collection system is applied, comprising a sampling robot and a target device. The end effector of the sampling robot's robotic arm is equipped with an industrial camera, and the target collection device has a sampling port with a visual marker at the sampling port. The method includes: The industrial camera is controlled to acquire the current frame image at the sampling port. The SIFT algorithm is used to detect the image feature points of the current frame image, and the image feature points are matched with the template feature points of the visual mark to obtain the 2D-3D matching point pair corresponding to the visual mark. Based on the 2D-3D matching point pairs and the intrinsic parameter matrix of the industrial camera, the observation pose information of the industrial camera relative to the visual marker is calculated using the PnP algorithm. Based on the preset spatial pose relationship between the visual marker and the sampling port, the absolute pose information of the sampling port in the robot coordinate system is calculated according to the observed pose information. Calculate the deviation vector between the absolute pose information and the current end effector pose information of the robotic arm, generate the target motion trajectory based on the deviation vector, and control the movement of the robotic arm to achieve automatic alignment between the end effector and the sampling port.

2. The method according to claim 1, characterized in that, Before controlling the industrial camera to acquire the current frame image at the sampling port, the method further includes: In response to a remote control command sent by the user, a sampling task is initiated, and the sampling robot is controlled to move to the target device; Based on preset target device location information, the sampling robot controls the robotic arm to move to the sampling port and controls the robotic arm to drive the industrial camera to be aligned with the sampling port.

3. The method according to claim 1, characterized in that, The step of detecting image feature points of the current frame image using the SIFT algorithm and matching the image feature points with the template feature points of the visual marker to obtain 2D-3D matching point pairs corresponding to the visual marker includes: An enhanced image is obtained by performing preprocessing operations on the current frame image, wherein the preprocessing operations include grayscale processing and Gaussian blur processing; The SIFT algorithm is used to extract key points and descriptors corresponding to the key points from the enhanced image, wherein the key points include at least the four corner points of the visual marker; The descriptors are matched with the feature descriptor library of the template images of the visual tags to obtain initial matching point pairs; After removing mismatched point pairs from the initial matching point pairs using the RANSAC algorithm, the matching point pairs with the highest consistency are selected as the 2D-3D matching point pairs corresponding to the visual markers.

4. The method according to claim 1, characterized in that, The step of calculating the absolute pose information of the sampling port in the robot coordinate system based on the observed pose information, based on the preset spatial pose relationship between the visual marker and the sampling port, includes: Based on the preset spatial pose relationship between the visual marker and the sampling port and the observation pose information, intermediate pose information is calculated by homogeneous transformation matrix multiplication, wherein the intermediate pose information is used to represent the pose of the sampling port coordinate system in the industrial camera coordinate system. Based on the extrinsic parameter matrix at the end of the robotic arm and the forward kinematics solution of the robotic arm, combined with the intermediate pose information, the absolute pose information of the sampling port in the robot coordinate system is calculated.

5. The method according to claim 4, characterized in that, The method for obtaining the spatial pose relationship between the visual marker and the sampling port includes: The calibration probe on the end effector of the robotic arm is used to sequentially contact multiple physical corner points of the visual marker, and the corresponding end effector pose of the robotic arm is recorded at the same time. Based on the relative pose of the visual marker by the industrial camera, the spatial pose relationship between the visual marker and the sampling port is obtained by jointly solving the hand-eye calibration algorithm.

6. The method according to claim 1, characterized in that, After calculating the absolute pose information of the sampling port in the robot coordinate system based on the observed pose information, the method further includes: Based on the absolute pose information, the 2D-3D matching point pairs, and the intrinsic parameter matrix of the industrial camera, a reprojection error model is constructed. In the reprojection error model, the absolute pose information is nonlinearly optimized to minimize the reprojection error, resulting in optimized absolute pose information.

7. The method according to claim 1, characterized in that, After calculating the deviation vector between the absolute pose information and the current robotic arm end-effector pose information, generating the target motion trajectory based on the deviation vector, and controlling the robotic arm to move, the method further includes: During the movement of the robotic arm, the industrial camera is controlled to continuously acquire real-time image frames at the sampling port; Based on the real-time image frame, feature point detection and matching, pose information estimation and coordinate transformation are repeatedly performed to update the current absolute pose information of the sampling port in the robot coordinate system in real time. Calculate the real-time spatial deviation vector between the updated current absolute pose information and the current robotic arm end-effector pose information; The motion trajectory is dynamically adjusted based on the real-time spatial deviation vector, and the robotic arm is controlled to move along the adjusted trajectory until the end effector and the sampling port are physically docked.

8. An automatic alignment device for oil sample collection, characterized in that, The device is used in an automatic oil sample collection system and includes: The feature matching module is used to control the industrial camera to acquire the current frame image at the sampling port, use the SIFT algorithm to detect the image feature points of the current frame image, and match the image feature points with the template feature points of the visual mark to obtain the 2D-3D matching point pair corresponding to the visual mark; The pose acquisition module is used to calculate the observation pose information of the industrial camera relative to the visual marker based on the 2D-3D matching point pairs and the intrinsic parameter matrix of the industrial camera using the PnP algorithm. The pose conversion module is used to calculate the absolute pose information of the sampling port in the robot coordinate system based on the observed pose information, based on the preset spatial pose relationship between the visual marker and the sampling port. An automatic alignment module is used to calculate the deviation vector between the absolute pose information and the current end effector pose information of the robotic arm, generate a target motion trajectory based on the deviation vector, and control the movement of the robotic arm to achieve automatic alignment between the end effector and the sampling port.

9. A storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 7.

10. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 7.

Images