Mechanical arm control method for tracking dynamic target in real time

Abstract

The invention provides a mechanical arm control method for tracking a dynamic target in real time, and belongs to the technical field of mechanical arm control, and the mechanical arm control method comprises the steps that a vision measurement system is constructed through a binocular infrared camera and a visible light camera, and a marker image on the dynamic target is captured; based on a visual detection algorithm, two-dimensional code data corresponding to the marker image are extracted for tracking identification detection, and when it is detected that the dynamic target is a to-be-tracked target, real-time position information and real-time attitude information of the to-be-tracked target are calculated through a perspective attitude estimation algorithm; performing fusion processing on low-frequency vision measurement data constructed by the real-time position information and the real-time attitude information and the high-frequency motion signal based on an inter-frame difference method to obtain real-time pose data; and the visual servo controller controls the mechanical arm to move according to a motion optimal solution obtained based on the mechanical arm kinematics model. According to the method, real-time tracking of the dynamic target is achieved, and the method has high operation grabbing efficiency and is suitable for various task scenes.

Description

Technical Field

[0001] This invention relates to the field of robotic arm control technology, and in particular to a robotic arm control method for real-time tracking of dynamic targets. Background Technology

[0002] A robotic arm is an automated device used in industrial production to replace manual labor in completing tasks. It performs tasks such as tracking, monitoring, and grasping according to a set program, trajectory, and requirements.

[0003] In scenarios such as drone capture and modular component replacement, existing robotic arm dynamic target tracking technologies mainly rely on static visual measurement or low-frame-rate sensors to collect the position information of dynamic targets, and then the robotic arm plans its motion trajectory to execute corresponding operations. However, static visual measurement has a low frequency, which is difficult to meet the tracking and recognition of high-speed dynamic targets. Furthermore, data acquisition using a single sensor is susceptible to interference from environmental factors such as lighting, leading to deviations in the pose estimation of dynamic targets. At the same time, there is a lag in the motion planning and execution of the robotic arm, making it difficult to achieve accurate tracking and capture of dynamic targets.

[0004] Therefore, this invention proposes a robotic arm control method for real-time tracking of dynamic targets. Summary of the Invention

[0005] To address the aforementioned technical problems, this invention provides a robotic arm control method for real-time tracking of dynamic targets, thereby solving the problem in traditional technologies where it is difficult to track dynamic targets in real time, leading to robotic arm grasping failures and low operational efficiency.

[0006] This invention provides a robotic arm control method for real-time tracking of dynamic targets, comprising:

[0007] A visual measurement system is constructed using a binocular infrared camera and a visible light camera to capture images of markers on dynamic targets.

[0008] Based on the visual detection algorithm, the corresponding QR code data of the marker image is extracted for tracking and recognition detection. When the dynamic target is detected as the target to be tracked, the pose information of the marker image relative to the visual measurement system is calculated by the perspective pose estimation algorithm. Then, the real-time position information and real-time pose information of the target to be tracked are obtained by the relative position of the marker image and the target to be tracked.

[0009] The instantaneous velocity and angular velocity of the target to be tracked are calculated based on the inter-frame difference method to obtain simulated high-frequency motion signals. Low-frequency visual measurement data constructed from real-time position information and real-time attitude information are fused with the high-frequency motion signals to obtain real-time pose data.

[0010] A kinematic model of the robotic arm is constructed. Based on the real-time pose data of the target to be tracked, the optimal motion solution of the joint angles of the robotic arm is solved. The vision servo controller controls the movement of the robotic arm according to the optimal motion solution.

[0011] Furthermore, this invention provides a robotic arm control method for real-time tracking of dynamic targets, comprising the following steps: based on a visual detection algorithm, extracting the corresponding QR code data of the marker image for tracking and recognition detection; when the dynamic target is detected as the target to be tracked, calculating the pose information of the marker image relative to the visual measurement system using a perspective pose estimation algorithm; and then obtaining the real-time position information and real-time pose information of the target to be tracked through the relative position of the marker image and the target to be tracked; including:

[0012] A preprocessed marker image is obtained by performing Gaussian blur and histogram equalization preprocessing on the marker image;

[0013] The preprocessed marker image is processed based on a threshold segmentation algorithm to obtain a segmented marker image;

[0014] Extract the contour of the segmentation identifier image and perform quadrilateral contour discrimination. When the contour of the segmentation identifier image is detected to be a quadrilateral contour, perform perspective transformation on the segmentation identifier image to obtain a perspective identifier image.

[0015] Segmentation of perspective identifier images based on Otsu grayscale thresholding algorithm, resulting in QR code pixel segmentation images;

[0016] Hamming code recognition is performed based on the QR code pixel segmentation map to obtain the QR code data of the dynamic target;

[0017] The dynamic target is marked as a target to be tracked when the QR code data is matched and identified with data in a predefined dictionary.

[0018] Furthermore, the present invention provides a robotic arm control method for real-time tracking of dynamic targets, wherein the steps include: performing Hamming code recognition based on the QR code pixel segmentation map to obtain the QR code data of the dynamic target; including:

[0019] Set the value of the black squares in the QR code pixel segmentation image to 0 and the value of the white squares to 1, convert the QR code pixel segmentation image into matrix form, and obtain QR code matrix data;

[0020] Remove the check bits from the QR code matrix data to obtain preprocessed QR code matrix data;

[0021] The preprocessed QR code matrix data is converted into binary form to obtain the QR code data.

[0022] Furthermore, this invention provides a robotic arm control method for real-time tracking of dynamic targets, comprising the following steps: based on a visual detection algorithm, extracting the corresponding QR code data of the marker image for tracking and recognition detection; when the dynamic target is detected as the target to be tracked, calculating the pose information of the marker image relative to the visual measurement system using a perspective pose estimation algorithm; and then obtaining the real-time position information and real-time pose information of the target to be tracked through the relative position of the marker image and the target to be tracked; further comprising:

[0023] Based on the camera coordinate system, image coordinate system, world coordinate system, and target object coordinate system, construct the target object task coordinate system;

[0024] The intrinsic parameter matrix and distortion vector of the visual measurement system are obtained by camera calibration method, and the rotation matrix and translation matrix of the identifier image relative to the visual measurement system are solved.

[0025] The pose information of the identifier image relative to the camera coordinate system is calculated using a perspective pose estimation algorithm;

[0026] Based on the relative position of the identifier image and the target to be tracked, the real-time position and attitude information of the target to be tracked are obtained.

[0027] Furthermore, this invention provides a robotic arm control method for real-time tracking of dynamic targets, comprising the steps of: obtaining the intrinsic parameter matrix and distortion vector of the vision measurement system through a camera calibration method, and solving the rotation and translation matrices of the identifier image relative to the vision measurement system; including:

[0028] Using the image coordinate system as a reference, obtain the corner coordinates of the identifier image;

[0029] ;

[0030] in, These are the corner coordinates of the four corner points of the identifier image in the world coordinate system. The side length of the identifier image;

[0031] Let the projection coordinates of the identifier image onto the horizontal plane be:

[0032] ;

[0033] in, These are the pixel coordinates corresponding to the projection of the corner coordinates onto the horizontal plane. pixel coordinate data coordinates, pixel coordinate data coordinates, pixel coordinate data coordinates, pixel coordinate data The coordinate values;

[0034] ;

[0035] in, For projection parameters, , For focal length parameters, These are the coordinate parameters of the principal point;

[0036] ;

[0037] in, For rotation matrix, For rotation factor, It is the rotation vector factor;

[0038] ;

[0039] in, It is a translation matrix. The translation factor;

[0040] Since the intrinsic parameter matrix is ​​a fixed value, let

[0041] ;

[0042] ;

[0043] in, The transformation matrix is... For transformation factor, This is a matrix of corner coordinate data;

[0044] ;

[0045] Obtain the constraints;

[0046] ;

[0047] ;

[0048] The above formula is used to calculate the rotation and translation matrices under constraints.

[0049] Furthermore, this invention provides a robotic arm control method for real-time tracking of dynamic targets, comprising the following steps: calculating the instantaneous velocity and angular velocity of the target to be tracked based on the inter-frame difference method, acquiring simulated high-frequency motion signals, and fusing low-frequency visual measurement data constructed from real-time position information and real-time attitude information with the high-frequency motion signals to obtain real-time pose data; including:

[0050] ;

[0051] in, for Real-time pose data at any given moment. These are the filter coefficients. This is low-frequency visual measurement data. for Real-time pose data at any given moment. The instantaneous velocity of the target to be tracked. For time intervals.

[0052] Furthermore, this invention provides a robotic arm control method for real-time tracking of dynamic targets, comprising the following steps: constructing a kinematic model of the robotic arm; solving for the optimal motion solution of the joint angles of the robotic arm based on the real-time pose data of the target to be tracked; and a visual servo controller controlling the movement of the robotic arm based on the optimal motion solution; including:

[0053] A six-degree-of-freedom kinematic model of a robotic arm was established based on DH parameters;

[0054] Based on the real-time pose data of the target to be tracked, the optimal motion solution of the joint angles of the robotic arm is obtained through inverse kinematics.

[0055] The vision servo controller controls the robotic arm to perform corresponding operations based on the optimal motion solution.

[0056] Compared with traditional technologies, the advantages of this invention are as follows: A robotic arm control method for real-time tracking of dynamic targets, comprising a vision measurement system constructed from binocular infrared and visible light cameras, identifies and detects marker images on dynamic targets, achieving efficient identification of the target to be tracked. The vision measurement system, based on the redundant design of binocular infrared and visible light cameras, can adapt to complex lighting environments, avoiding the problems of traditional technologies where static vision measurement has a low frequency, making it difficult to meet the tracking and identification of high-speed dynamic targets, or where data acquisition using a single sensor is easily interfered with by lighting conditions. The method constructs a low-frequency... Visual measurement data is fused with high-frequency motion signals to obtain real-time pose data, achieving the integration of visual measurement and filtering. This improves the update frequency of real-time pose data, meeting the need for rapid tracking of high-speed moving targets. Based on the real-time pose data of the target to be tracked, the optimal motion solution of the robotic arm joint angles is solved. The visual servo controller optimizes and controls the robotic arm motion. The robotic arm motion is controlled based on the inverse kinematics solution of the robotic arm end effector, achieving accurate tracking and capture of the target to be tracked. The above method achieves real-time tracking of dynamic targets and has high operational capture efficiency, applicable to various mission scenarios such as UAV capture and battery, gimbal, and ammunition replacement.

[0057] Other features and advantages of the invention will be set forth in the description which follows, and will be apparent in part from the description, or may be learned by practicing the invention. The objects and other advantages of the invention may be realized and obtained by means of the structures particularly pointed out in the written description, claims, and drawings.

[0058] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0059] Figure 1 A flowchart illustrating a robotic arm control method for real-time tracking of dynamic targets provided by the present invention;

[0060] Figure 2 A perspective identifier image for a robotic arm control method for real-time tracking of dynamic targets provided by the present invention;

[0061] Figure 3 A QR code pixel segmentation diagram for a robotic arm control method for real-time tracking of dynamic targets provided by the present invention;

[0062] Figure 4 A schematic diagram of the target object coordinate system for a robotic arm control method for real-time tracking of dynamic targets provided by the present invention.

[0063] Figure 5This is a schematic diagram of the identifier image projection for a robotic arm control method for real-time tracking of dynamic targets provided by the present invention. Detailed Implementation

[0064] The preferred embodiments of the present invention will be described below with reference to the accompanying drawings. It should be understood that the preferred embodiments described herein are for illustration and explanation only and are not intended to limit the present invention.

[0065] Example 1:

[0066] This invention provides a robotic arm control method for real-time tracking of dynamic targets, referring to... Figure 1 ,include:

[0067] A visual measurement system is constructed using a binocular infrared camera and a visible light camera to capture images of markers on dynamic targets.

[0068] Based on the visual detection algorithm, the corresponding QR code data of the marker image is extracted for tracking and recognition detection. When a dynamic target is detected as the target to be tracked, the pose information of the marker image relative to the visual measurement system is calculated by the perspective pose estimation algorithm. Then, the real-time position information and real-time pose information of the target to be tracked are obtained by the relative position of the marker image and the target to be tracked.

[0069] The instantaneous velocity and angular velocity of the target to be tracked are calculated based on the inter-frame difference method to obtain simulated high-frequency motion signals. Low-frequency visual measurement data constructed from real-time position information and real-time attitude information are fused with the high-frequency motion signals to obtain real-time pose data.

[0070] A kinematic model of the robotic arm is constructed. Based on the real-time pose data of the target to be tracked, the optimal motion solution of the joint angles of the robotic arm is solved. The vision servo controller controls the movement of the robotic arm according to the optimal motion solution.

[0071] In the above embodiments, a visual measurement system is constructed using a binocular infrared camera and a visible light camera to capture the image of a marker on a dynamic target. Based on a visual detection algorithm, the corresponding QR code data of the marker image is extracted for tracking and identification detection. When the dynamic target is detected as the target to be tracked, the pose information of the marker image relative to the visual measurement system is calculated using a perspective pose estimation algorithm. Then, the real-time position information and real-time pose information of the target to be tracked are obtained through the relative position of the marker image and the target to be tracked.

[0072] The instantaneous velocity and angular velocity of the target to be tracked are calculated based on the inter-frame difference method to obtain simulated high-frequency motion signals. Low-frequency visual measurement data constructed from real-time position information and real-time attitude information are fused with the high-frequency motion signals to obtain real-time pose data.

[0073] A kinematic model of the robotic arm is constructed. Based on the real-time pose data of the target to be tracked, the optimal motion solution of the joint angles of the robotic arm is solved. The vision servo controller controls the movement of the robotic arm according to the optimal motion solution.

[0074] In the above embodiments, the binocular infrared camera can be a D435i model camera, which can capture the marker image on the dynamic target in real time at a frame rate of 60Hz.

[0075] In the above embodiments, the marker image can be implemented as an Aruco code image.

[0076] In the above embodiments, the visual detection algorithm can be implemented as the Aruco detection algorithm.

[0077] In the above embodiments, the update frequency of the real-time pose data obtained by the above method can reach 200Hz, the error of the real-time position information of the target to be tracked is ≤0.5cm, the real-time pose information is ≤0.7°, and the average operation time of the robotic arm to perform the grasping task is ≤120s.

[0078] In the above embodiments, the robotic arm control method based on real-time tracking of dynamic targets can be extended into a multi-robotic arm collaborative system to realize target capture and operation in complex scenarios.

[0079] The beneficial effects of the above technologies are as follows: The visual measurement system, constructed using a binocular infrared camera and a visible light camera, identifies and detects marker images on dynamic targets, achieving efficient identification of the target to be tracked. Based on the redundant design of the binocular infrared and visible light cameras, the visual measurement system can adapt to complex lighting environments, avoiding the problems of traditional technologies where static visual measurement frequencies are too low to meet the tracking and identification of high-speed dynamic targets, or where data acquisition using a single sensor is easily affected by lighting conditions. Furthermore, based on the inter-frame difference method, low-frequency visual measurement data constructed from real-time position and attitude information is fused with high-frequency motion signals to obtain… By acquiring real-time pose data and integrating visual measurement with filtering, the update frequency of real-time pose data is improved, meeting the need for rapid tracking of high-speed moving targets. Based on the real-time pose data of the target to be tracked, the optimal motion solution of the joint angles of the robotic arm is solved, and the motion of the robotic arm is optimized and controlled by the visual servo controller. The motion of the robotic arm is controlled based on the inverse kinematics solution of the robotic arm end effector, realizing accurate tracking and capture of the target to be tracked. This is a robotic arm control method for real-time tracking of dynamic targets, which realizes real-time tracking of dynamic targets and has high operation and grasping efficiency. It is suitable for various mission scenarios such as UAV capture and battery, gimbal, and ammunition replacement.

[0080] Example 2:

[0081] This invention provides a robotic arm control method for real-time tracking of dynamic targets. The steps include: based on a visual detection algorithm, extracting the corresponding QR code data from a marker image for tracking and recognition; when a dynamic target is detected as the target to be tracked, calculating the pose information of the marker image relative to the visual measurement system using a perspective pose estimation algorithm; and then obtaining the real-time position and pose information of the target to be tracked through the relative position of the marker image and the target. The method includes:

[0082] A preprocessed marker image is obtained by performing Gaussian blur and histogram equalization preprocessing on the marker image;

[0083] The preprocessed marker image is processed based on a threshold segmentation algorithm to obtain a segmented marker image;

[0084] Extract the contour of the segmentation identifier image and perform quadrilateral contour discrimination. When the contour of the segmentation identifier image is detected to be a quadrilateral contour, perform perspective transformation on the segmentation identifier image to obtain the perspective identifier image.

[0085] Segmentation of perspective identifier images based on Otsu grayscale thresholding algorithm, resulting in QR code pixel segmentation images;

[0086] Hamming code recognition is performed based on the QR code pixel segmentation map to obtain the QR code data of dynamic targets;

[0087] The system matches and identifies QR code data with data in a predefined dictionary. When data matching the QR code data is found in the predefined dictionary, the dynamic target is marked as a target to be tracked.

[0088] In the above embodiments, a preprocessed marker image is obtained by performing Gaussian blur and histogram equalization preprocessing on the marker image. This preprocessed marker image is then processed using a threshold segmentation algorithm to obtain a segmented identifier image. The contour of the segmented identifier image is extracted, and quadrilateral contour discrimination is performed. When a quadrilateral contour is detected in the segmented identifier image, a perspective transformation is performed to obtain a perspective identifier image. The perspective identifier image is then segmented using the Otsu grayscale thresholding algorithm, dividing it into a black and white grid with the same number of cells as the median of the QR code, thus obtaining a QR code pixel segmentation map. Hamming code recognition is then performed based on the QR code pixel segmentation map to obtain the QR code data of the dynamic target.

[0089] In one embodiment, a perspective identifier image such as Figure 2 As shown, the QR code pixel segmentation image obtained based on the Otsu grayscale image thresholding algorithm is as follows: Figure 3 As shown.

[0090] In the above embodiments, the QR code data is matched and identified with the data in the predefined dictionary. When the data in the predefined dictionary that matches the QR code data is identified, the dynamic target is marked as the target to be tracked.

[0091] The beneficial effects of the above technology are as follows: by processing and recognizing the marker image, dynamic target QR code data is obtained, and matching and recognition are performed based on data in a predefined dictionary to achieve the recognition of the target to be tracked, which facilitates subsequent robotic arm tracking operations.

[0092] Example 3:

[0093] This invention provides a robotic arm control method for real-time tracking of dynamic targets, comprising the following steps: performing Hamming code recognition based on a QR code pixel segmentation map to obtain the QR code data of the dynamic target; including:

[0094] Set the value of the black squares in the QR code pixel segmentation image to 0 and the value of the white squares to 1, convert the QR code pixel segmentation image into matrix form, and obtain the QR code matrix data;

[0095] Remove the check digits from the QR code matrix data to obtain the preprocessed QR code matrix data;

[0096] The preprocessed QR code matrix data is converted into binary form to obtain the QR code data.

[0097] In the above embodiments, the values ​​of black squares in the QR code pixel segmentation image are set to 0 and the values ​​of white squares are set to 1 to obtain QR code matrix data. The check bits are removed, and the preprocessed QR code matrix data is converted into binary form to obtain QR code data.

[0098] In one embodiment, QR code matrix data Represented as:

[0099] ;

[0100] Remove the check digits, defining them as odd-numbered bits, and obtain the preprocessed QR code matrix data. ;

[0101] ;

[0102] Convert the preprocessed QR code matrix data into binary form to obtain the QR code data. .

[0103] ;

[0104] The beneficial effects of the above technology are as follows: by performing Hamming code recognition on the QR code pixel segmentation map, the QR code data is obtained, and then the identifier image on the dynamic target is recognized, which facilitates the identification of the target to be tracked in subsequent steps.

[0105] Example 4:

[0106] This invention provides a robotic arm control method for real-time tracking of dynamic targets, comprising the following steps: Based on a visual detection algorithm, extracting the corresponding QR code data from a marker image for tracking and recognition detection; when a dynamic target is detected as the target to be tracked, calculating the pose information of the marker image relative to the visual measurement system using a perspective pose estimation algorithm; and then obtaining the real-time position and pose information of the target to be tracked through the relative position of the marker image and the target; further comprising:

[0107] Based on the camera coordinate system, image coordinate system, world coordinate system, and target object coordinate system, construct the target object task coordinate system;

[0108] The intrinsic parameter matrix and distortion vector of the vision measurement system are obtained by camera calibration method, and the rotation and translation matrices of the identifier image relative to the vision measurement system are solved.

[0109] The pose information of the identifier image relative to the camera coordinate system is calculated using a perspective pose estimation algorithm;

[0110] Based on the relative position of the identifier image and the target to be tracked, the real-time position and attitude information of the target to be tracked are obtained.

[0111] In the above embodiments, a camera coordinate system, an image coordinate system, a world coordinate system, and a target object coordinate system are constructed. The intrinsic parameter matrix and distortion vector of the visual measurement system are obtained through the camera calibration method. The rotation matrix and translation matrix of the identifier image relative to the visual measurement system are solved. The pose information of the identifier image relative to the camera coordinate system is calculated based on the perspective pose estimation algorithm. Based on the relative position of the identifier image and the target to be tracked, the real-time position information and real-time pose information of the target to be tracked are obtained.

[0112] In the above embodiments, reference is made to Figure 4 Based on the camera coordinate system, image coordinate system, world coordinate system, and target object coordinate system, a target object task coordinate system is constructed.

[0113] In the above embodiments, the perspective pose estimation algorithm is implemented as the EFPnP (Efficient Perspective nPoint) algorithm.

[0114] The beneficial effects of the above technology are as follows: when a dynamic target is detected as the target to be tracked, the perspective pose estimation algorithm realizes the acquisition of the real-time position information and real-time pose information of the target to be tracked based on the relative positions of the marker image and the visual measurement system, and the marker image and the target to be tracked.

[0115] Example 5:

[0116] This invention provides a robotic arm control method for real-time tracking of dynamic targets, comprising the following steps: obtaining the intrinsic parameter matrix and distortion vector of the vision measurement system through a camera calibration method, and solving for the rotation and translation matrices of the identifier image relative to the vision measurement system; including:

[0117] Using the image coordinate system as a reference, obtain the corner coordinates of the identifier image;

[0118] ;

[0119] in, These are the corner coordinates of the four corner points of the identifier image in the world coordinate system. The side length of the identifier image;

[0120] Let the projection coordinates of the identifier image onto the horizontal plane be:

[0121] ;

[0122] in, These are the pixel coordinates corresponding to the projection of the corner coordinates onto the horizontal plane. pixel coordinate data coordinates, pixel coordinate data coordinates, pixel coordinate data coordinates, pixel coordinate data The coordinate values;

[0123] ;

[0124] in, For projection parameters, , For focal length parameters, These are the coordinate parameters of the principal point;

[0125] ;

[0126] in, For rotation matrix, For rotation factor, It is the rotation vector factor;

[0127] ;

[0128] in, It is a translation matrix. The translation factor;

[0129] Since the intrinsic parameter matrix is ​​a fixed value, let

[0130] ;

[0131] ;

[0132] in, The transformation matrix is... For transformation factor, This is a matrix of corner coordinate data;

[0133] ;

[0134] Obtain the constraints;

[0135] ;

[0136] ;

[0137] The above formula is used to calculate the rotation and translation matrices under constraints.

[0138] In the above embodiments, by obtaining the corner coordinate data of the four corner points of the identifier image in the world coordinate system, the projection coordinates of the identifier image projected onto the horizontal plane are obtained, and the rotation matrix and translation matrix of the identifier image relative to the visual measurement system are solved based on the constraints.

[0139] In one embodiment, reference Figure 5 Obtain the four corner points of the identifier image in the world coordinate system ( Corner coordinate data in ) The projections are respectively onto the horizontal plane ( Projected coordinates on ) .

[0140] Example 6:

[0141] This invention provides a robotic arm control method for real-time tracking of dynamic targets, comprising the following steps: calculating the instantaneous velocity and angular velocity of the target to be tracked based on the inter-frame difference method, acquiring simulated high-frequency motion signals, and fusing low-frequency visual measurement data constructed from real-time position information and real-time attitude information with the high-frequency motion signals to obtain real-time pose data; including:

[0142] ;

[0143] in, for Real-time pose data at any given moment. These are the filter coefficients. This is low-frequency visual measurement data. for Real-time pose data at any given moment. The instantaneous velocity of the target to be tracked. For time intervals.

[0144] In the above embodiments, the weights of low-frequency visual measurement data and high-frequency motion signals are controlled by the filtering coefficients, and the low-frequency visual measurement data and high-frequency motion signals are fused to obtain real-time pose data.

[0145] In the above embodiments, Kalman filtering or particle filtering can be used instead of complementary filtering.

[0146] The beneficial effects of the above technologies are as follows: by performing complementary filtering and fusion processing on low-frequency visual measurement data and high-frequency motion signals based on the filtering coefficients, real-time pose data can be obtained, which solves the problems of delay and jitter in target tracking and improves the measurement accuracy and anti-interference ability of real-time pose data acquisition.

[0147] Example 7:

[0148] This invention provides a robotic arm control method for real-time tracking of dynamic targets, comprising the following steps: constructing a kinematic model of the robotic arm; solving for the optimal motion solution of the joint angles of the robotic arm based on the real-time pose data of the target to be tracked; and using a visual servo controller to control the movement of the robotic arm based on the optimal motion solution.

[0149] A six-degree-of-freedom kinematic model of a robotic arm was established based on DH parameters;

[0150] Based on the real-time pose data of the target to be tracked, the optimal motion solution of the joint angles of the robotic arm is obtained through inverse kinematics.

[0151] The vision servo controller controls the robotic arm to perform corresponding operations based on the optimal motion solution.

[0152] In the above embodiments, a six-degree-of-freedom kinematic model of the robotic arm is established based on the DH parameters (Denavit-Hartenberg). According to the real-time pose data of the target to be tracked, the optimal motion solution of the joint angle of the robotic arm is obtained through inverse kinematics solution. The visual servo controller controls the robotic arm to perform corresponding operations according to the optimal motion solution, which effectively improves the calculation efficiency and accuracy of the joint angle of the robotic arm.

[0153] In the above embodiments, the visual servo controller adjusts the end effector trajectory of the robotic arm based on real-time pose data to achieve target tracking, positioning, and capture.

[0154] The beneficial effects of the above technologies are as follows: by constructing a kinematic model of the robotic arm, inverse kinematics is solved based on the real-time pose data of the target to be tracked, and the optimal motion solution of the joint angles of the robotic arm is obtained. The visual servo controller controls the robotic arm to perform corresponding operations, thereby realizing the end effector control of the robotic arm. This enables the robotic arm to track and locate the target to be tracked, and to perform grasping operations according to the corresponding instructions, further realizing the intelligent and agile control of the robotic arm.

[0155] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. A robotic arm control method for real-time tracking of dynamic targets, characterized in that, include: A visual measurement system is constructed using a binocular infrared camera and a visible light camera to capture images of markers on dynamic targets. Based on the visual detection algorithm, the corresponding QR code data of the marker image is extracted for tracking and recognition detection. When the dynamic target is detected as the target to be tracked, the pose information of the marker image relative to the visual measurement system is calculated by the perspective pose estimation algorithm. Then, the real-time position information and real-time pose information of the target to be tracked are obtained by the relative position of the marker image and the target to be tracked. The instantaneous velocity and angular velocity of the target to be tracked are calculated based on the inter-frame difference method to obtain simulated high-frequency motion signals. Low-frequency visual measurement data constructed from real-time position information and real-time attitude information are fused with the high-frequency motion signals to obtain real-time pose data. A kinematic model of the robotic arm is constructed. Based on the real-time pose data of the target to be tracked, the optimal motion solution of the joint angles of the robotic arm is solved. The vision servo controller controls the movement of the robotic arm according to the optimal motion solution.

2. The robotic arm control method for real-time tracking of dynamic targets according to claim 1, characterized in that, The steps are as follows: Based on a visual detection algorithm, the corresponding QR code data of the marker image is extracted for tracking and recognition detection. When the dynamic target is detected as the target to be tracked, the pose information of the marker image relative to the visual measurement system is calculated using a perspective pose estimation algorithm. Then, the real-time position information and real-time pose information of the target to be tracked are obtained through the relative position of the marker image and the target to be tracked. This includes: A preprocessed marker image is obtained by performing Gaussian blur and histogram equalization preprocessing on the marker image; The preprocessed marker image is processed based on a threshold segmentation algorithm to obtain a segmented marker image; Extract the contour of the segmentation identifier image and perform quadrilateral contour discrimination. When the contour of the segmentation identifier image is detected to be a quadrilateral contour, perform perspective transformation on the segmentation identifier image to obtain a perspective identifier image. Segmentation of perspective identifier images based on Otsu grayscale thresholding algorithm, resulting in QR code pixel segmentation images; Hamming code recognition is performed based on the QR code pixel segmentation map to obtain the QR code data of the dynamic target; The dynamic target is marked as a target to be tracked when the QR code data is matched and identified with data in a predefined dictionary.

3. The robotic arm control method for real-time tracking of dynamic targets according to claim 2, characterized in that, The step involves: performing Hamming code recognition based on the QR code pixel segmentation map to obtain the QR code data of the dynamic target; including: Set the value of the black squares in the QR code pixel segmentation image to 0 and the value of the white squares to 1, convert the QR code pixel segmentation image into matrix form, and obtain QR code matrix data; Remove the check bits from the QR code matrix data to obtain preprocessed QR code matrix data; The preprocessed QR code matrix data is converted into binary form to obtain the QR code data.

4. The robotic arm control method for real-time tracking of dynamic targets according to claim 1, characterized in that, The steps include: Based on a visual detection algorithm, extracting the corresponding QR code data from the marker image for tracking and recognition detection; when the dynamic target is detected as the target to be tracked, calculating the pose information of the marker image relative to the visual measurement system using a perspective pose estimation algorithm; and then obtaining the real-time position and pose information of the target to be tracked through the relative position of the marker image and the target to be tracked; further including: Based on the camera coordinate system, image coordinate system, world coordinate system, and target object coordinate system, construct the target object task coordinate system; The intrinsic parameter matrix and distortion vector of the visual measurement system are obtained by camera calibration method, and the rotation matrix and translation matrix of the identifier image relative to the visual measurement system are solved. The pose information of the identifier image relative to the camera coordinate system is calculated using a perspective pose estimation algorithm; Based on the relative position of the identifier image and the target to be tracked, the real-time position and attitude information of the target to be tracked are obtained.

5. The robotic arm control method for real-time tracking of dynamic targets according to claim 4, characterized in that, The steps include: obtaining the intrinsic parameter matrix and distortion vector of the visual measurement system through camera calibration, and solving for the rotation and translation matrices of the identifier image relative to the visual measurement system; including: Using the image coordinate system as a reference, obtain the corner coordinates of the identifier image; ； in, These are the corner coordinates of the four corner points of the identifier image in the world coordinate system. The side length of the identifier image; Let the projection coordinates of the identifier image onto the horizontal plane be: ； in, These are the pixel coordinates corresponding to the projection of the corner coordinates onto the horizontal plane. pixel coordinate data coordinates, pixel coordinate data coordinates, pixel coordinate data coordinates, pixel coordinate data The coordinate values; ； in, For projection parameters, , For focal length parameters, These are the coordinate parameters of the principal point; ； in, For rotation matrix, For rotation factor, It is the rotation vector factor; ； in, It is a translation matrix. The translation factor; Since the intrinsic parameter matrix is ​​a fixed value, let ； ； in, The transformation matrix is... For transformation factor, This is a matrix of corner coordinate data; ； Obtain the constraints; ； ； The above formula is used to calculate the rotation and translation matrices under constraints.

6. The robotic arm control method for real-time tracking of dynamic targets according to claim 1, characterized in that, The steps are as follows: calculate the instantaneous velocity and angular velocity of the target to be tracked based on the inter-frame difference method, obtain simulated high-frequency motion signals, and fuse low-frequency visual measurement data constructed from real-time position information and real-time attitude information with high-frequency motion signals to obtain real-time pose data. include: ； in, for Real-time pose data at any given moment. These are the filter coefficients. This is low-frequency visual measurement data. for Real-time pose data at any given moment. The instantaneous velocity of the target to be tracked. For time intervals.

7. The robotic arm control method for real-time tracking of dynamic targets according to claim 1, characterized in that, The steps include: constructing a kinematic model of the robotic arm; solving for the optimal motion solution of the joint angles of the robotic arm based on the real-time pose data of the target to be tracked; and using a visual servo controller to control the movement of the robotic arm based on the optimal motion solution. A six-degree-of-freedom kinematic model of a robotic arm was established based on DH parameters; Based on the real-time pose data of the target to be tracked, the optimal motion solution of the joint angles of the robotic arm is obtained through inverse kinematics. The vision servo controller controls the robotic arm to perform corresponding operations based on the optimal motion solution.

Images