A highly versatile automated machining process for a rack-type manipulator

Through CCD camera and image processing technology, the problem of low accuracy of workpiece recognition and grabbing by mount robots is solved, accurate identification and handling of workpieces is achieved, and the automation level of the processing process is improved.

CN115431284BActive Publication Date: 2024-11-22EVERITE (SUZHOU) MECHANICAL PROD CO LTD
View PDF 5 Cites 0 Cited by

Patent Information

Application Number
CN202211148149.5
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-09-20
Publication Date
2024-11-22
Estimated Expiration
2042-09-20

AI Technical Summary

Technical Problem

During the processing of column workpieces of automobile engines, existing rack-type robots cannot accurately identify the workpieces to be moved, resulting in low gripping accuracy, which increases labor intensity and poses safety risks.

Method used

The CCD camera is used for image acquisition and processing. Through coordinate transformation, grayscale, image enhancement, binarization and edge detection, the pixel coordinates of the inner circle of the workpiece are obtained and converted into three-dimensional spatial coordinates to ensure the accurate grasp of the robot.

Benefits of technology

It effectively improves the accuracy of identification and grasping of workpieces by robots, reduces labor intensity and reduces safety risks to workers.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN115431284B_ABST
    Figure CN115431284B_ABST
Patent Text Reader

Abstract

The present invention relates to the field of robot technology, and in particular to an automated processing technology for a highly versatile gantry robot; the processing technology comprises the following steps: coordinate transformation in CCD camera calibration; image acquisition and processing; image noise processing; positioning and transportation of a workpiece, wherein a CCD camera is introduced to photograph the workpiece, and the position of the workpiece is determined by the acquired image, thereby effectively improving the recognition accuracy of the workpiece. During the recognition process, the pixel coordinates of the center of the inner circle of the workpiece are obtained by graying the image, and after image enhancement and binarization processing, the pixel coordinates of the center of the inner circle of the workpiece are obtained, and after converting them into three-dimensional space coordinates in a reference coordinate system, the reference coordinate system of the workpiece is accurately recognized in cooperation with the robot, thereby effectively improving the accuracy of the robot in grasping the workpiece.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention relates to the technical field of manipulators, and in particular to an automated processing technology for a highly versatile rack-type manipulator. Background Art

[0002] When processing automobile engine column workpieces, it is often necessary to carry out transportation work before processing to move the workpiece to the processing unit. In the past, transportation was often done manually, which on the one hand would increase the labor intensity of the personnel, and on the other hand, there was a risk of harming the workers' bodies.

[0003] In the prior art, a gantry-type robot is used for automated handling to reduce the labor intensity of workers. However, it is found during use that due to the large number of workpieces to be handled, it is often impossible to accurately identify the workpieces when grabbing the workpieces on the conveyor belt, resulting in low grasping accuracy of the robot. Summary of the invention

[0004] The purpose of the present invention is to provide a highly versatile automated machining process for a gantry-type robot, so as to effectively improve the grasping accuracy of the gantry-type robot on the workpiece.

[0005] To achieve the above-mentioned object, the present invention provides a highly versatile automated processing technology for a gantry type manipulator, wherein an actuator is arranged at the end of the manipulator, and a plurality of light sources for illumination are arranged on the upper side of the manipulator, and a CCD camera for acquiring an image of a workpiece to be transported is also arranged on one side of the light source;

[0006] The processing technology includes the following steps:

[0007] Coordinate transformation in CCD camera calibration, taking several images of the calibration plate from different angles to calibrate the CCD camera, so as to obtain the rotation and translation relationship between the CCD camera's own coordinate system and the reference coordinate system;

[0008] Image acquisition and processing: grayscale the acquired image, and perform image enhancement and image binarization processing in sequence to obtain the pixel coordinates of the center of the inner circle of the workpiece, and convert the pixel coordinates of the center of the inner circle of the workpiece into three-dimensional space coordinates in the reference coordinate system;

[0009] Image noise processing, using Sobel operator for edge detection to reduce the error caused by image algorithm;

[0010] The workpiece is positioned and transported by aligning the three-dimensional spatial coordinates of the end effector of the robot with the center of the inner circle of the workpiece to grasp the workpiece. The robot holding the workpiece is then transported to the processing device through the overall moving frame to complete the processing.

[0011] On the basis of the existing technology, a CCD camera is introduced to shoot the workpiece, and the position of the workpiece is determined by the acquired image, thereby effectively improving the recognition accuracy of the workpiece. In the recognition process, the coordinates of the CCD camera are first transformed, and the shooting position of the CCD camera is calibrated by cooperating with a calibration plate, so as to obtain the relationship between the CCD camera's own coordinate system and the reference coordinate system, and then the workpiece image taken by the CCD camera is grayed, enhanced and binarized to obtain the pixel coordinates of the center of the inner circle of the workpiece, and after converting it into three-dimensional space coordinates in the reference coordinate system, the manipulator is used to realize accurate recognition of the reference coordinate system of the workpiece, thereby effectively improving the manipulator's grasping accuracy of the workpiece.

[0012] Among them, the reference coordinate system is the absolute coordinate system of the objective world, which is used to represent the three-dimensional spatial coordinates of the objective world. The CCD camera's own coordinate system is a three-dimensional rectangular coordinate system established with the CCD camera as the center. The pixel coordinate system is the coordinate system used for digital images, with the origin located in the upper left corner of the image, the X-axis pointing to the right, and the Y-axis pointing downward, respectively representing the column number and row number of a certain pixel in the image. The reference coordinate system and the CCD camera's own coordinate system, as well as the pixel coordinate system and the CCD camera's own coordinate system can be transformed into each other.

[0013] Linear transformations between coordinate systems are performed as follows:

[0014] Three-dimensional space point Xw = (Xw, Yw, Zw) T Projected onto the two-dimensional imaging plane, the imaging point m = (x i ,y i ) T Since the aforementioned coordinate systems are interconnected, the following transformation is required in the process of converting the pixel coordinate system into the reference coordinate system.

[0015] The coordinate transformation between the reference coordinate system and the CCD camera coordinate system is shown in the following formula 1:

[0016]

[0017] In Formula 1, Xc and Xw are the coordinates of the camera coordinate system, R and T are the relative position and posture between the reference coordinate system and the camera coordinate system, where R is the 3D rotation matrix and T is the 3D translation vector.

[0018] The transformation between the camera coordinate system and the pixel coordinate system is shown in the following formula 2:

[0019]

[0020] In formula 2, f is the focal length, that is, the distance between the focal plane of the camera and the imaging plane, (x i ,y i ) is the coordinate of the imaging point in the pixel coordinate system.

[0021] According to equations 1 and 2, the relationship between any pixel in the image in the two coordinate systems can be obtained as follows:

[0022]

[0023] In Formula 3, (Cx, Cy) is the pixel coordinate of the optical center, Sx and Sy are the number of pixels per unit distance on the image plane, Sx=1 / dx, Sy=1 / dy.

[0024] The grayscale processing of the image is to convert the color image into a grayscale image, wherein the conversion relationship between the RGB color model and the grayscale model is:

[0025] Gray(i,j)=0.11R(i,j)+0.59G(i,j)+0.3B(i,j)

[0026] Where Gray(i,j) represents the gray value of the converted black and white image pixel at point (i,j).

[0027] Where Gray(i,j) represents the gray value of the converted black and white image pixel at point (i,j).

[0028] In order to determine the boundary between the workpiece and the background, the image needs to be grayed. The quality of the processing result has a great influence on the subsequent processing. The purpose of graying is to transform the color image containing brightness and color into a gray image to prepare for subsequent processing. The image enhancement is to enhance the information required by the user in the image using relevant technologies, and to perform contrast enhancement on the studied image. The main purpose is to adjust the contrast of the image, improve the visual quality, and highlight important details. Before this, the image needs to be grayed. Image enhancement has two main purposes: one is to improve the clarity of the image and improve the visual effect of the image; the other is to make the image more conducive to computer processing. In the binarization of the image, the binarization is to set the gray value of the pixel on the image to 0 or 255, that is, to make the entire image present a visual effect of only white and black. The commonly used method is to set a threshold T, and use T to divide the pixel group into two parts: a pixel group greater than T and a pixel group less than T, and assign values ​​to the two pixel groups respectively. In order to obtain the information of the workpiece on the image, the gray image of the workpiece is further binarized to obtain a black and white image.

[0029] The image enhancement refers to adjusting the contrast of the grayscale processed image to improve the visual quality and display important details.

[0030] The image binarization refers to dividing a pixel group into two parts by setting a threshold, converting the grayscale value of the pixel group greater than the threshold to 255, and converting the grayscale value of the pixel group less than or equal to the threshold to 0, so that the image presents a visual effect of only white and black.

[0031] The use of the Sobel operator for edge detection refers to processing the grayscale image to obtain a gradient image, dividing the image into regions with different characteristics according to the characteristics of the image and extracting the target.

[0032] The Sobel operator extracts edges in the form of a filter operator. A template is used in the X and Y directions respectively, and the two templates are combined to form a gradient operator. The X-direction template has the greatest impact on vertical edges, and the Y-direction template has the greatest impact on horizontal edges.

[0033] The highly versatile automated processing technology of a gantry robot of the present invention improves the processing flow of the robot on the workpiece on the basis of the existing technology. By adding a CCD camera and processing the image captured by the CCD camera, a connection is established between the reference coordinate system and the image coordinate system to meet the robot's demand for workpiece handling accuracy during the processing, thereby effectively improving the robot's handling accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0035] Figure 1 The present invention provides a process diagram of a highly versatile automated machining process using a rack-type manipulator.

[0036] Figure 2 It is a grayscale schematic diagram of a highly versatile rack-type robot automated processing technology provided by the present invention.

[0037] Figure 3 It is a schematic diagram of image enhancement of a highly versatile rack-type manipulator automated processing technology provided by the present invention.

[0038] Figure 4 It is a binary schematic diagram of a highly versatile rack-type manipulator automated processing technology provided by the present invention.

[0039] Figure 5 It is a schematic diagram of circle center determination in a highly versatile automated processing technology for a rack-type manipulator provided by the present invention. DETAILED DESCRIPTION

[0040] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present invention, and should not be construed as limiting the present invention.

[0041] See also Figures 1 to 5 The present invention provides a highly versatile automated processing technology for a gantry type manipulator, wherein an actuator is arranged at the end of the manipulator, and a plurality of light sources for illumination are arranged on the upper side of the manipulator, and a CCD camera for acquiring an image of a workpiece to be transported is also arranged on one side of the light source;

[0042] The processing technology includes the following steps:

[0043] S1: Coordinate transformation in CCD camera calibration, taking several images of the calibration plate from different angles to calibrate the CCD camera to obtain the rotation and translation relationship between the CCD camera's own coordinate system and the reference coordinate system;

[0044] S2: Image acquisition and processing, graying the acquired image, and performing image enhancement and image binarization processing in sequence to obtain the pixel coordinates of the center of the inner circle of the workpiece, and converting the pixel coordinates of the center of the inner circle of the workpiece into three-dimensional space coordinates in the reference coordinate system;

[0045] S3: Image noise processing, using the Sobel operator for edge detection to reduce the error caused by the image algorithm;

[0046] S4: Positioning and transporting of the workpiece, aligning the three-dimensional spatial coordinates of the end effector of the manipulator with the center of the inner circle of the workpiece to grasp the workpiece, and then transporting the manipulator holding the workpiece to the processing device through the overall moving frame to complete the processing.

[0047] On the basis of the existing technology, a CCD camera is introduced to shoot the workpiece, and the position of the workpiece is determined by the acquired image, thereby effectively improving the recognition accuracy of the workpiece. In the recognition process, the coordinates of the CCD camera are first transformed, and the shooting position of the CCD camera is calibrated by cooperating with a calibration plate, so as to obtain the relationship between the CCD camera's own coordinate system and the reference coordinate system, and then the workpiece image taken by the CCD camera is grayed, enhanced and binarized to obtain the pixel coordinates of the center of the inner circle of the workpiece, and after converting it into three-dimensional space coordinates in the reference coordinate system, the manipulator is used to realize accurate recognition of the reference coordinate system of the workpiece, thereby effectively improving the manipulator's grasping accuracy of the workpiece.

[0048] Furthermore, the reference coordinate system is an absolute coordinate system of the objective world, which represents the three-dimensional spatial coordinates of the objective world. The CCD camera's own coordinate system is a three-dimensional rectangular coordinate system centered on the CCD camera. The pixel coordinate system is a coordinate system used for digital images, with the origin located in the upper left corner of the image, the X-axis pointing to the right, and the Y-axis pointing downward, respectively representing the column number and row number of a certain pixel in the image. The reference coordinate system and the CCD camera's own coordinate system, as well as the pixel coordinate system and the CCD camera's own coordinate system can be transformed into each other.

[0049] Linear transformations between coordinate systems are performed as follows:

[0050] Three-dimensional space point Xw = (Xw, Yw, Zw) T Projected onto the two-dimensional imaging plane, the imaging point m = (x i ,y i ) T Since the aforementioned coordinate systems are interconnected, the following transformation is required in the process of converting the pixel coordinate system into the reference coordinate system.

[0051] The coordinate transformation between the reference coordinate system and the CCD camera coordinate system is shown in the following formula 1:

[0052]

[0053] In Formula 1, Xc and Xw are the coordinates of the camera coordinate system, R and T are the relative position and posture between the reference coordinate system and the camera coordinate system, where R is the 3D rotation matrix and T is the 3D translation vector.

[0054] The transformation between the camera coordinate system and the pixel coordinate system is shown in the following formula 2:

[0055]

[0056] In formula 2, f is the focal length, that is, the distance between the focal plane of the camera and the imaging plane, (x i ,y i ) is the coordinate of the imaging point in the pixel coordinate system.

[0057] According to equations 1 and 2, the relationship between any pixel in the image in the two coordinate systems can be obtained as follows:

[0058]

[0059] In Formula 3, (Cx, Cy) is the pixel coordinate of the optical center, Sx and Sy are the number of pixels per unit distance on the image plane, Sx=1 / dx, Sy=1 / dy.

[0060] Furthermore, the grayscale processing of the image is to convert the color image into a grayscale image, wherein the conversion relationship between the RGB color model and the grayscale model is:

[0061] Gray(i,j)=0.11R(i,j)+0.59G(i,j)+0.3B(i,j)

[0062] Where Gray(i,j) represents the gray value of the converted black and white image pixel at point (i,j).

[0063] Where Gray(i,j) represents the gray value of the converted black and white image pixel at point (i,j).

[0064] In order to determine the boundary between the workpiece and the background, the image needs to be grayed. The quality of the processing result has a great influence on the subsequent processing. The purpose of graying is to transform the color image containing brightness and color into a gray image to prepare for subsequent processing. The image enhancement is to enhance the information required by the user in the image using relevant technologies, and to perform contrast enhancement on the studied image. The main purpose is to adjust the contrast of the image, improve the visual quality, and highlight important details. Before this, the image needs to be grayed. Image enhancement has two main purposes: one is to improve the clarity of the image and improve the visual effect of the image; the other is to make the image more conducive to computer processing. In the binarization of the image, the binarization is to set the gray value of the pixel on the image to 0 or 255, that is, to make the entire image present a visual effect of only white and black. The commonly used method is to set a threshold T, and use T to divide the pixel group into two parts: a pixel group greater than T and a pixel group less than T, and assign values ​​to the two pixel groups respectively. In order to obtain the information of the workpiece on the image, the gray image of the workpiece is further binarized to obtain a black and white image.

[0065] Furthermore, the image enhancement refers to adjusting the contrast of the grayscale processed image to improve the visual quality and display important details.

[0066] Furthermore, the image binarization refers to dividing a pixel group into two parts by setting a threshold, converting the grayscale value of the pixel group greater than the threshold to 255, and converting the grayscale value of the pixel group less than or equal to the threshold to 0, so that the image presents a visual effect of only white and black.

[0067] Furthermore, the use of the Sobel operator for edge detection refers to processing the grayscale image to obtain a gradient image, dividing the image into regions with different characteristics according to the characteristics of the image and extracting the target.

[0068] The Sobel operator extracts edges in the form of a filter operator. A template is used in the X and Y directions respectively, and the two templates are combined to form a gradient operator. The X-direction template has the greatest impact on vertical edges, and the Y-direction template has the greatest impact on horizontal edges.

[0069] Furthermore, the end effector includes a cylinder, a connecting plate, a gripper, a compression mechanism and a synchronization mechanism.

[0070] The highly versatile automated processing technology of a gantry robot of the present invention improves the processing flow of the robot on the workpiece on the basis of the existing technology. By adding a CCD camera and processing the image captured by the CCD camera, a connection is established between the reference coordinate system and the image coordinate system to meet the robot's demand for workpiece handling accuracy during the processing, thereby effectively improving the robot's handling accuracy.

[0071] What is disclosed above is only a preferred embodiment of the present invention, and it certainly cannot be used to limit the scope of rights of the present invention. Ordinary technicians in this field can understand that all or part of the processes of the above embodiment and equivalent changes made according to the claims of the present invention still fall within the scope of the invention.

Claims

1. A highly versatile automated processing technology for rack-type manipulators. It is characterized in that An actuator is arranged at the end of the manipulator, and a plurality of light sources for illumination are arranged on the upper side of the manipulator, and a CCD camera for acquiring an image of the workpiece to be transported is also arranged on one side of the light source; The processing technology includes the following steps: Coordinate transformation in CCD camera calibration, taking several images of the calibration plate from different angles to calibrate the CCD camera to obtain the rotation and translation relationship between the CCD camera's own coordinate system and the reference coordinate system; Image acquisition and processing: grayscale the acquired image, and perform image enhancement and image binarization processing in sequence to obtain the pixel coordinates of the center of the inner circle of the workpiece, and convert the pixel coordinates of the center of the inner circle of the workpiece into three-dimensional space coordinates in the reference coordinate system; Image noise processing, using Sobel operator for edge detection to reduce the error caused by image algorithm; Positioning and transporting the workpiece, aligning the three-dimensional spatial coordinates of the end effector of the manipulator with the center of the inner circle of the workpiece to grasp the workpiece, and then transporting the manipulator holding the workpiece to the processing device through the overall moving frame to complete the processing; The reference coordinate system is an absolute coordinate system of the objective world, which represents the three-dimensional spatial coordinates of the objective world. The CCD camera's own coordinate system is a three-dimensional rectangular coordinate system formulated with the CCD camera as the center. The pixel coordinate system is a coordinate system used by digital images, with the origin located in the upper left corner of the image, the X axis pointing to the right, and the Y axis pointing downward, respectively representing the number of columns and rows of a certain pixel in the image. The reference coordinate system and the CCD camera's own coordinate system, as well as the pixel coordinate system and the CCD camera's own coordinate system can be transformed into each other. The grayscale processing of the image is to convert the color image into a grayscale image, wherein the conversion relationship between the RGB color model and the grayscale model is: , In the formula, Gray(i,j) Indicates that the converted black and white image pixels are (i,j) Gray value at the point; The image enhancement refers to adjusting the contrast of the grayscale processed image to improve the visual quality and display important details; The image binarization refers to dividing a pixel group into two parts by setting a threshold, converting the grayscale value of the pixel group greater than the threshold to 255, and converting the grayscale value of the pixel group less than or equal to the threshold to 0, so that the image presents a visual effect of only white and black; The use of the Sobel operator for edge detection refers to processing a grayscale image to obtain a gradient image, dividing the image into regions with different characteristics according to the characteristics of the image and extracting the target.

Citation Information

Patent Citations

  • Calibration and transformation of a camera system's coordinate system

    CN103702607A

  • Halcon-based industrial robot vision system calibration method

    CN109859277A

  • Two-arm cooperative intelligent assembly system based on visual positioning

    CN111251336A

  • Visual system for four-axis industrial palletizing robot

    CN111604909A

  • Calibration and transformation of a camera system's coordinate system

    US20130010081A1