A mechanical arm vision driving control method based on color probability model

Abstract

The application discloses a mechanical arm visual driving control method based on a color probability model, relates to the technical field of visual perception driving, and comprises the following steps: S1, calibrating a binocular camera, correcting an image by using a Bouguet method, matching an image pair based on an SGBM algorithm, calculating a left-eye parallax map, and converting the parallax into a depth value according to a binocular ranging principle to complete establishment of a binocular vision system; S2, establishing a color probability model and calculating a pixel-level target probability, and obtaining a position response by using an integral image; S3, calculating a response oscillation criterion and updating the model by using a high-confidence strategy; S4, realizing rotation and scale estimation of a target object by using a logarithmic polar domain conversion and a Fourier shift target; and S5, estimating an end working pose and driving a mechanical arm to complete fine work based on visual servoing. The method provided by the application can obviously improve the influence of a target being blocked and deformed, and realizes real-time pose adjustment of a mechanical arm end tool on a target object.

Description

Technical Field

[0001] This invention belongs to the field of visual perception driving technology, specifically relating to a visual driving control method for a robotic arm based on a color probability model. Background Technology

[0002] The development of indoor scene distance sensing technology has led to numerous practical visual applications. Collaborative robots, as a crucial human-computer interaction technology, combined with visual perception, can significantly improve production flexibility and automation, reduce the complexity of manual operations, prevent safety hazards, and increase production efficiency. Therefore, embedding intelligent vision systems into collaborative robotic arms, integrating robot vision algorithms with hardware and software, is an effective solution for achieving human-computer interaction. Autonomous grasping and manipulation in static scenes has been extensively studied for many years, solving non-human operation problems in structured workshops and other scenarios. However, robotic arm tracking and guidance for non-cooperative dynamic scenes is suitable for robots to perform more advanced and flexible engineering applications, but dynamic guidance is more challenging. Currently, there are no mature systems or solutions for this type in the industry, relying on robot systems with more powerful computing power and environmental perception capabilities.

[0003] Vision-driven systems are typically based on color / grayscale / depth images. However, if the object to be tracked only has weak texture tracking, a single color feature, or relies solely on distance perception, tracking may fail. Furthermore, they lack continuous and controllable visual feedback, making it difficult to guide robotic arms to perform refined tasks in dynamic scenes (such as dynamic pick-and-place or workpiece picking). Currently, some scholars both domestically and internationally are researching dynamic robotic arm vision-driven technologies. To address the shortcomings of visual servoing, such as the target escaping the camera's field of view during the driving process, Ma Hongxuan et al. disclosed a "Visual Servo-Based Robotic Arm Target Guidance Method" (patent number CN 110116410 A). This method pre-sets markers on the robotic arm and the target, specifying their shapes and coordinate relationships. However, it is unsuitable for tracking and guiding non-cooperative targets and struggles to ensure that the markers are simultaneously captured by the camera platform mounted on the mobile robot. Regarding the technical challenges of high-precision target tracking and position positioning for robotic arms over a wide range, Meng Hao et al. disclosed a "Method and Device for Multi-Camera and Line Laser-Assisted Robotic Arm Target Tracking" (patent number CN 114378808 A). This method is suitable for precise tracking and guidance in structured environments, but when the target's position changes (e.g., a cylindrical workpiece rolls) or the robotic arm's base coordinate system moves (e.g., when mounted on a mobile platform), the robotic arm's end-effector cannot adjust the target's pose in real time. In summary, for precise robotic arm tasks in dynamic scenarios, traditional visual-driven control struggles to achieve real-time pose adjustment of the end-effector and cannot provide continuous and accurate visual feedback. Summary of the Invention

[0004] To address the shortcomings of existing technologies, the present invention aims to provide a vision-driven control method for robotic arms based on a color probability model. This method can significantly improve the impact of target occlusion and deformation, enabling real-time attitude adjustment of the robotic arm's end-effector on the target object, and vision-driven robotic arms to perform refined tasks.

[0005] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0006] A vision-driven control method for a robotic arm based on a color probability model is disclosed. The method is implemented on a vision-driven control system for a robotic arm based on a color probability model. The system mainly includes a vision module, a robotic arm, and a worktable. The vision module includes a right-eye camera and a left-eye camera, which are fixedly mounted on a frame. An operating tool is installed at the end of the robotic arm, and the target object is placed on the worktable.

[0007] The method includes the following steps:

[0008] S1. The right and left cameras are calibrated, and then the images are corrected using the Bouguet binocular stereo correction method. Finally, based on the stereo matching correction of the image pairs using the SGBM algorithm, the disparity map of the left eye is calculated, and the disparity is converted into the depth value at each pixel according to the binocular ranging principle, thereby completing the establishment of the binocular vision system.

[0009] S2. Establish a color probability model and calculate pixel-level target probability. The position response f of the color probability model is obtained using the integral image;

[0010] S3. Calculate the response oscillation criterion and use a high-confidence strategy to update the model to avoid target loss;

[0011] S4. Utilize logarithmic polar domain transformation and Fourier shift target to achieve target rotation and scale estimation;

[0012] S5. Estimate the end-effector's working pose and drive the robotic arm to complete precision tasks based on visual servo control.

[0013] Furthermore, in the visual drive control method for the robotic arm based on the color probability model described above, the calibration of the right and left cameras in step S1 specifically involves:

[0014] The Zhang calibration method was used to calibrate the right and left cameras. The intrinsic parameter matrices of the left and right cameras were obtained by taking multiple photos of the calibration board. / Distortion parameter matrix / Using the extrinsic parameter matrix, the relative relationship between the left and right camera coordinate systems is obtained: rotation matrix. Translation matrix Eigenmatrix and fundamental matrix .

[0015] Furthermore, in the visual-driven control method for a robotic arm based on a color probability model as described above, the image correction using the Bouguet binocular stereo correction method in step S1 specifically involves:

[0016] The left and right cameras rotated respectively. Half of the image plane is coplanar, and then a row alignment and wrapping matrix is ​​established. This achieves row alignment by shifting the poles to infinity, minimizing the number of reprojections for each of the two images, and maximizing the observed area; the row alignment transposition matrix... The expression is:

[0017] (1)

[0018] Will Left multiply to The decomposition yields the correction rotation matrix for the coplanarity of the left and right camera planes:

[0019] (2)

[0020] Simultaneously acquire the perspective projection matrices of two cameras and reprojection matrix The correction lookup table is calculated based on the correction results to make the two images coplanar and row aligned, and the corrected image is cropped so that the principal points of the left and right views coincide.

[0021] Furthermore, in the visual drive control method for the robotic arm based on the color probability model described above, the method for calculating the depth value at each pixel in step S1 is as follows:

[0022] (3)

[0023] In equation (3), Indicates the camera's focal length; This represents the distance between the optical centers of the two cameras, i.e., the baseline distance; This represents the disparity value.

[0024] Furthermore, in the visual-driven control method for robotic arms based on the color probability model described above, step S2 uses a Bayesian classifier based on the color histogram to estimate the likelihood term and obtain the pixel-level target probability. The calculation formula is:

[0025] (4)

[0026] In equation (4): It represents the probability that each pixel location is the target, expressing pixel-level probability; For pixel values, Indicates assignment to Color module range , Indicates in Color-based histograms computed over regions The size of the interval.

[0027] Furthermore, in the vision-driven control method for a robotic arm based on a color probability model as described above, in step S2, the integral image... The formula for calculating the value at this location is:

[0028] (5)

[0029] (6)

[0030] In formula (5): The original image pixels, For integral images in The value at;

[0031] The formula for calculating the position response of the color probability model is:

[0032] (7)

[0033] Furthermore, in the vision-driven control method for a robotic arm based on a color probability model as described above, the calculation formula for the response oscillation criterion in step S3 is as follows:

[0034] (8)

[0035] In equation (8), , , These represent the maximum, minimum, and position of the response, respectively. The response value on; when and The model is updated only when the set proportion is greater than the historical average, in order to avoid incorrect updates polluting the model and causing tracking failure.

[0036] Furthermore, in the visual drive control method for robotic arms based on the color probability model described above, step S4 specifically includes:

[0037] Cartesian coordinate system Same logarithmic polar coordinate system The conversion relationship between them is as follows:

[0038] (9)

[0039] After performing a discrete Fourier transform on equation (9), it maps to the frequency domain and has the following form:

[0040] (10)

[0041] Based on the Fourier shift theorem, the cross power spectrum between the sensed image and the reference image is calculated:

[0042] (11)

[0043] Let the unit impulse function be... , For inverse Fourier transform, It has the following properties:

[0044] (12)

[0045] From this we can obtain The location of the peak is determined by transforming the scaling and rotation in the Cartesian domain into a translation in the logarithmic polar domain, thus achieving the estimation of the target's scale and rotation.

[0046] (13)

[0047] In equation (13), The length of the axis of rotation For the estimated scaling factor, This is the estimated scaling factor.

[0048] Furthermore, in the visual drive control method for robotic arms based on the color probability model described above, step S5 specifically includes:

[0049] S51. Complete the robotic arm hand-eye calibration and establish the coordinate relationship between the end-effector coordinate system and the robotic arm base coordinate system. ;

[0050] S52. Based on the established binocular stereo vision and hand-eye relationship, the coordinates of the center point of the initial target object in the left eye image are transformed to the robot base coordinate system, and position-based visual servoing is used to guide the end effector to the initial pose.

[0051] S53. Driven by a color probability model and scale / rotation estimation, the robotic arm uses image-based visual servoing to dynamically follow the target object's movement, achieving precise operation.

[0052] Compared with existing technologies, the vision-driven control method for robotic arms based on a color probability model provided by this invention has the following advantages:

[0053] This invention combines robotic arm control technology with color histogram-based visual perception technology. First, a binocular vision system is established, and disparity is calculated and converted into depth values ​​using SGBM stereo matching. Second, the likelihood term is estimated based on the color histogram to obtain pixel-level target probabilities, and a high-confidence model update strategy with position response oscillation constraints is designed to avoid target loss. Then, logarithmic polar domain transformation and Fourier shift target are used to estimate the target's rotation angle and scale, estimate the end-effector pose, and drive the robotic arm to complete precision operations based on visual servo control.

[0054] This method utilizes a low-cost binocular camera vision-driven robotic arm to perform fine manipulation tasks, and is robust to object occlusion, shape changes, and rotation. The method provided by this invention is effective and the process is simple and straightforward. Attached Figure Description

[0055] Figure 1 This is a flowchart of a vision-driven control method for a robotic arm based on a color probability model, provided in an embodiment of the present invention.

[0056] Figure 2 This is a schematic diagram of the structure of a robotic arm vision drive control system based on a color probability model provided in an embodiment of the present invention;

[0057] Figure 3 This is a schematic diagram illustrating the principle of integral graphs.

[0058] Figure 4 This is a set of visual localization and scale estimation results;

[0059] Figure 2 In the diagram: 1-robotic arm, 2-operating tool, 3-target object (workpiece), 4-right eye camera, 5-left eye camera, 6-frame, 7-worktable. Detailed Implementation

[0060] The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments.

[0061] For precise robotic arm operations in dynamic scenarios, traditional vision-driven control struggles to achieve real-time attitude adjustment of the end effector towards the target object and cannot provide continuous and controllable visual feedback. This invention provides a vision-driven control method for robotic arms based on a color probability model. This method is implemented on a vision-driven control system for robotic arms based on a color probability model. Figure 2The diagram shows the structure of the system, which mainly includes a robotic arm 1, a vision module, and a worktable 7. The vision module mainly includes a right eye camera 4 and a left eye camera 5, which are mounted and fixed on an aluminum profile frame 6. An operating tool 2 is installed at the end of the robotic arm 1. The target object 3 is placed on the worktable 7. The operating tool 2 is guided to the target object 3 and dynamically followed by a vision algorithm based on a color probability model.

[0062] The overall flow of the vision-driven control method for robotic arms based on a color probability model provided by this invention is attached. Figure 1 As shown, the method includes the following steps:

[0063] S1. Calibrate the right and left cameras, then correct the images using the Bouguet method. Finally, calculate the left disparity map based on the stereo matching correction image pairs using the SGBM algorithm, and convert the disparity into depth values ​​at each pixel according to the binocular ranging principle.

[0064] First, both cameras were simultaneously calibrated using Zhang's calibration method. Zhang's calibration method, proposed by Dr. Zhang Zhengyou, is a practical method for camera calibration using a planar checkerboard pattern. This method lies between photogrammetric calibration and self-calibration, overcoming the drawbacks of photogrammetric calibration (requiring high-precision 3D calibration objects, which are expensive and cumbersome) and self-calibration (lacking robustness). The calibration process only requires a printed checkerboard pattern and several sets of images taken from different directions. It is not only practical and flexible but also highly accurate and robust. Therefore, it has been widely adopted worldwide, greatly accelerating the transition of 3D computer vision from the laboratory to the real world.

[0065] This invention obtains the intrinsic parameter matrices of the left and right eyes cameras by taking multiple images of the calibration board. / Distortion parameter matrix (radial / tangential) / The extrinsic parameter matrix. Using the extrinsic parameter matrix, the relative relationship between the left and right camera coordinate systems can be obtained: rotation matrix. Translation matrix and eigenmatrix and fundamental matrix .

[0066] Secondly, due to differences in camera placement deviating from the ideal binocular system, distortion correction and line alignment need to be performed separately for the left and right views. This invention employs the Bouguet binocular stereo correction method, with each camera rotating... Half of the image plane is coplanar, and then a row alignment and wrapping matrix is ​​established. This achieves row alignment by shifting the poles to infinity, minimizing the number of reprojections for each image in the two images, and maximizing the observed area. Row alignment transposition matrix. The expression is:

[0067] (1)

[0068] Will Multiplying by the original rotation matrices of the left and right cameras, we obtain the corrected rotation matrix where the left and right camera planes are coplanar:

[0069] (2)

[0070] Simultaneously acquire the perspective projection matrices of two cameras and reprojection matrix The correction lookup table is calculated based on the correction results to make the two images coplanar and row aligned, and the corrected image is cropped so that the principal points of the left and right views coincide.

[0071] Finally, based on the principle of binocular ranging, the SGBM algorithm is used to match the corrected image pairs and calculate the disparity map relative to the left camera. To achieve ranging, the depth value at each pixel is calculated:

[0072] (3)

[0073] In the formula, Indicates the camera's focal length; This represents the distance between the optical centers of the two cameras, i.e., the baseline distance; This represents the disparity value.

[0074] S2. Establish a color probability model and calculate pixel-level target probability. The position response f of the color probability model is obtained using the integral image.

[0075] Most detection / tracking methods are limited to learning rigid templates. Image histograms discard pixel location information, and Bayesian estimation rules based on color histograms are insensitive to shape changes.

[0076] This invention employs a Bayesian classifier based on color histograms to estimate the target probability at a pixel. It assumes the initial bounding box or estimated object region is... The area surrounding the target As a background model, the area occupied by the target is determined by the mask region. This indicates that histograms of the foreground and background regions based on color are calculated separately. And the pixel-level target probability is obtained by directly estimating the likelihood term from the color histogram:

[0077] (4)

[0078] In the formula: It represents the probability that each pixel location is the target, expressing pixel-level probability; For pixel values, Indicates assignment to Color module range , Indicates in Color-based histograms computed over regions The size of the interval.

[0079] This invention employs a high-speed integral image method for calculating the sum of pixel values ​​within a rectangular region when solving for the position response. The integral image contains... The value at this point is the sum of the grayscale values ​​of all pixels within the rectangular region enclosed by the top-left corner of the original grayscale image and the current point, that is:

[0080] (5)

[0081] In the formula: The original image pixels, For integral images in The value at that location.

[0082] To fully utilize the calculated results and avoid repeated calculations, first, the upper left corner of the integral graph... The first row of the integral image With the first column Finally, the integral image of other pixel regions is recursively calculated. The integral image of the original image is obtained:

[0083] (6)

[0084] Solving the response of the color probability model using integral images:

[0085] (7)

[0086] S3. Calculate the response oscillation criterion and use a high-confidence strategy to update the model to avoid target loss.

[0087] To avoid occlusion or even target loss, and considering the severe oscillations in the response map when the target is missing, this invention employs a high-confidence model update strategy with response oscillation constraints. This strategy can avoid erroneous updates by analyzing the degree of oscillation in the position response. A criterion describing the degree of response oscillation is defined as follows:

[0088] (8)

[0089] In the formula, , , These represent the maximum, minimum, and position of the response, respectively. The response value on. When and The model is updated when the percentage is greater than the historical average to avoid incorrect updates polluting the model and causing tracking failure.

[0090] S4. Utilize logarithmic polar domain transformation and Fourier shift target to achieve target rotation and scale estimation.

[0091] Logarithmic polar transformations possess scale invariance and rotation invariance. Based on the properties of logarithmic polar mappings, the rectangular coordinate system... Same logarithmic polar coordinate system The following transformation relationship exists between them:

[0092] (9)

[0093] Grayscale images are real-valued discrete two-dimensional signals, and their Discrete Fourier Transform (DFT) can be calculated. After performing a DFT on the above equation and mapping it to the frequency domain, it takes the following form:

[0094] (10)

[0095] From the above equation, it can be seen that two images with translation amounts have the same amplitude when transformed into the frequency domain. Based on the above properties, the target region images of the two frames are logarithmically polarized, transforming the scale and rotation changes into translation relationships. Based on the Fourier displacement theorem, the cross power spectrum of the perceived image and the reference image is calculated:

[0096] (11)

[0097] Let the unit impulse function be... , For inverse Fourier transform, It has the following properties:

[0098] (12)

[0099] From this we can obtain The location of the peak is determined by transforming the scaling and rotation in the Cartesian domain into a translation in the logarithmic polar domain, thus achieving the estimation of the target's scale and rotation.

[0100] (13)

[0101] In the formula, The length of the axis of rotation For the estimated scaling factor, This is the estimated scaling factor.

[0102] S5. Estimate the end-effector's working pose and drive the robotic arm to complete precision tasks based on visual servo control.

[0103] First, complete the robotic arm hand-eye calibration and establish the coordinate relationship between the end-effector coordinate system and the robotic arm base coordinate system. Based on the established binocular stereo vision and hand-eye relationship, the coordinates of the center point of the initial target object in the left eye image are transformed to the robot's base coordinate system. Position-based visual servoing guides the end effector to the initial pose. Then, driven by the color probability model and scale / rotation estimation, the robotic arm dynamically follows the target object's movement using image-based visual servoing to achieve refined operation.

[0104] Example

[0105] Using the UR16e multi-axis serial collaborative robot and a Tensor 1 Pro binocular stereo camera, taking a polygonal columnar workpiece with hole features as an example, the implementation process of the present invention is illustrated with the help of the OpenCV vision library and Visual Studio software.

[0106] First, the corner positions of the chessboard grid are identified using the Harris corner detection function of the OpenCV vision library. The stereoCalibrate() function is called, and the intrinsic parameter matrices of the left and right cameras are calculated using the Zhang Zhengyou calibration method based on the corner points. / Distortion coefficient matrix / And the extrinsic parameter matrix. Call the `stereoRectify()` function, using the Bouguet epipolar correction method, inputting the intrinsic parameter matrix and distortion coefficients to calculate the rotation of the right camera relative to the left camera. Translation matrix and the correction transformation matrices of the left and right cameras. / Finally, the `initUndistortRectifyMap()` and `remap()` functions are called to map and correct the images captured by the left and right cameras. Since the SGBM algorithm offers moderate accuracy and speed, the `stereoSGBM::create()` function is called to match the corrected image pairs using the SGBM algorithm, calculate the disparity map relative to the left camera, and convert the disparity into depth values ​​at each pixel based on the binocular ranging principle.

[0107] The specific parameter results are as follows:

[0108]

[0109]

[0110]

[0111] After camera calibration, a color probability model is established, and histograms of the foreground and background regions based on color are calculated. The likelihood term is estimated from the color histograms to obtain the pixel-level target probability. Next, an integral image method for high-speed calculation of the sum of pixel values ​​in a rectangular region is adopted, the calculation principle of which is shown in the appendix. Figure 3 As shown, the response of the color probability model is solved using integral images. .

[0112] Secondly, to avoid occlusion or even target loss, considering the severe oscillations in the response map when the target is missing, a high-confidence model update strategy with response oscillation constraints is adopted. and The model is updated when the percentage is greater than the historical average to avoid incorrect updates polluting the model and causing tracking failure.

[0113] Then, to estimate the rotation angle and scale change of the target object, the image is converted from a Cartesian coordinate system. Transform to logarithmic polar coordinates Based on the Fourier shift theorem, the cross-power spectrum between the perceived image and the reference image is calculated. Calculate the Dirac function using inverse Fourier transform Obtain the location coordinates of the peak and calculate the scale factor. With rotation angle .

[0114] Image sequences are rapidly acquired using binocular cameras. The image sequences acquired by the left eye camera are processed using a color probability model to obtain the two-dimensional coordinates of the workpiece center and the workpiece dimensions for each frame. The hand-eye relationship matrix between the end-effector coordinate system and the robot arm base coordinate system is calibrated.

[0115]

[0116] Based on the established binocular stereo vision and hand-eye relationship, the end effector moves to its initial pose under the guidance of position-based visual servo control:

[0117]

[0118] Finally, driven by a color probability model, scale / rotation estimation, and image-based visual servo control, the robotic arm dynamically follows the target object's movement, as shown in the attached diagram. Figure 4 The system performs fine manipulation tasks, and the operation is robust to target occlusion, shape changes, and rotation.

[0119] This invention provides a vision-driven control method for robotic arms based on a color probability model. It employs a low-cost binocular camera to drive the robotic arm, exhibiting robustness against target object occlusion, shape changes, and rotation, enabling the execution of refined operational tasks. The proposed vision-driven method utilizes a color-perceptive tracker, leveraging information obtained from visual perception within a servo loop to control the moving robotic arm. While providing high precision, it is robust to challenges such as target object occlusion, shape changes (flexible bodies), and rotation in dynamic scenes. It can accurately calculate the target position, scale, and depth in real time and determine the pose of the robotic arm's end effector, making it suitable for refined robotic arm operations (such as dynamic pick-and-place and workpiece picking), thereby improving production flexibility.

[0120] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention is also intended to include these modifications and variations.

Claims

1. A vision-driven control method for a robotic arm based on a color probability model, the method being implemented on a vision-driven control system for a robotic arm based on a color probability model, the system comprising a vision module, a robotic arm, and a worktable, wherein the vision module comprises a right-eye camera and a left-eye camera, the right-eye camera and the left-eye camera being fixedly mounted on a frame, an operating tool being mounted at the end of the robotic arm, and a target object being placed on the worktable; The method includes the following steps: S1. The right and left cameras are calibrated, and then the images are corrected using the Bouguet binocular stereo correction method. Finally, based on the stereo matching correction of the image pairs using the SGBM algorithm, the disparity map of the left eye is calculated, and the disparity is converted into the depth value at each pixel according to the binocular ranging principle, thereby completing the establishment of the binocular vision system. S2, establish color probability model and calculate pixel-level target probability , obtain the position response f of the color probability model by using the integral image; S3. Calculate the response oscillation criterion and use a high-confidence strategy to update the model to avoid target loss; S4. Utilize logarithmic polar domain transformation and Fourier shift target to achieve target rotation and scale estimation; S5. Estimate the end-effector's working pose and drive the robotic arm to complete precision tasks based on visual servo control.

2. The color probability model based visual servoing control method for a robot arm according to claim 1, wherein, The calibration of the right and left cameras in step S1 is specifically as follows: The Zhang calibration method was used to calibrate the right and left cameras. The intrinsic parameter matrices of the left and right cameras were obtained by taking multiple pictures of the calibration board. / Distortion parameter matrix / Using the extrinsic parameter matrix, the relative relationship between the left and right camera coordinate systems is obtained: rotation matrix. Translation matrix Eigenmatrix and fundamental matrix .

3. The robotic arm vision-driven control method based on a color probability model according to claim 2, characterized in that, The image correction using the Bouguet binocular stereo correction method in step S1 is specifically as follows: The left and right cameras rotated respectively. Half of the image plane is coplanar, and then a row alignment and wrapping matrix is ​​established. This achieves row alignment by shifting the poles to infinity, minimizing the number of reprojections for each image in the two images, and maximizing the observed area; the row alignment transposition matrix... The expression is: (1) Will Multiplying by the original rotation matrices of the left and right cameras yields the corrected rotation matrices for the coplanarity of the left and right camera planes: （2） Simultaneously acquire the perspective projection matrices of two cameras and reprojection matrix The correction lookup table is calculated based on the correction results to make the two images coplanar and row aligned, and the corrected image is cropped so that the principal points of the left and right views coincide.

4. The vision-driven control method for a robotic arm based on a color probability model according to claim 3, characterized in that, The method for calculating the depth value at each pixel in step S1 is as follows: (3) In equation (3), Indicates the camera's focal length; This represents the distance between the optical centers of the two cameras, i.e., the baseline distance; This represents the disparity value.

5. The vision-driven control method for a robotic arm based on a color probability model according to any one of claims 1-4, characterized in that, The step S2, which involves establishing a color probability model and calculating pixel-level target probabilities, specifically involves using a Bayesian classifier based on a color histogram to estimate the likelihood term and obtain the pixel-level target probabilities. The calculation formula is: (4) In equation (4): It represents the probability that each pixel location is the target, expressing pixel-level probability; For pixel values, Indicates assignment to Color module range , Indicates in Color-based histogram computed over a region The size of the interval.

6. The vision-driven control method for a robotic arm based on a color probability model according to claim 5, characterized in that, The step S2, which involves obtaining the position response f of the color probability model using the integral image, specifically involves: In integral image The formula for calculating the value at this location is: (5) (6) In equation (5): The original image pixels, For integral images in The value at; The formula for calculating the position response of the color probability model is: (7)。 7. The vision-driven control method for a robotic arm based on a color probability model according to claim 6, characterized in that, The formula for calculating the oscillation criterion in step S3 is as follows: (8) In equation (8), , , These represent the maximum, minimum, and position of the response, respectively. The response value on; when and The model is updated only when the set proportion is greater than the historical average, in order to avoid incorrect updates polluting the model and causing tracking failure.

8. The vision-driven control method for a robotic arm based on a color probability model according to claim 7, characterized in that, Step S4 is as follows: Cartesian coordinate system Same logarithmic polar coordinate system The conversion relationship between them is as follows: (9) After performing a discrete Fourier transform on equation (9), it maps to the frequency domain and has the following form: (10) Based on the Fourier shift theorem, the cross power spectrum between the sensed image and the reference image is calculated: (11) Let the unit impulse function be... , For inverse Fourier transform, It has the following properties: (12) From this we can obtain The location of the peak is determined by transforming the scaling and rotation in the Cartesian domain into a translation in the logarithmic polar domain, thus achieving the estimation of the target's scale and rotation. (13) In equation (13), The length of the axis of rotation For the estimated scaling factor, This is the estimated scaling factor.

9. The vision-driven control method for a robotic arm based on a color probability model according to claim 7 or 8, characterized in that, Step S5 is as follows: S51. Complete the robotic arm hand-eye calibration and establish the coordinate relationship between the end-effector coordinate system and the robotic arm base coordinate system. ; S52. Based on the established binocular stereo vision and hand-eye relationship, the coordinates of the center point of the initial target object in the left eye image are transformed to the robot base coordinate system, and position-based visual servoing is used to guide the end effector to the initial pose. S53. Driven by a color probability model and scale / rotation estimation, the robotic arm uses image-based visual servoing to dynamically follow the target object's movement, achieving precise operation.

Images