A method for grasping a workpiece through machine vision and a grasping robot

Abstract

The present invention discloses a method for achieving workpiece grasping through machine vision, which is used for automatically unloading heat exchanger cores from a brazing frame. The method comprises the following steps: Step 1: acquiring and preprocessing images using a camera; Step 2: using the Euclidean Cluster algorithm to distinguish the number of pixels between different workpieces and between workpieces and backgrounds, thereby achieving point cloud clustering; Step 3: matching the features of the projected template with the captured image to obtain the position coordinates of the grasping points. Combined with the grasping robot of the present invention, the method can achieve simultaneous multiple grasping of multiple brazing toolings with heat exchanger cores.

Description

Technical Field

[0001] The invention relates to a method for realizing workpiece grasping through machine vision and a grasping robot used in the field of intelligent brazing. Background Art

[0002] During the batch brazing process of heat exchanger cores, brazing fixtures with heat exchanger cores are transported to the brazing furnace in batches through the brazing furnace frame. The main dimensions of a single brazing furnace frame are 2620 (height) x 3960 (width) x 400 (depth). A brazing furnace frame is divided into 3 rows and 7 layers with a total of 21 storage cells. Each cell can hold 2 rows of fixtures with heat exchanger cores, and each row can hold 8 fixtures with heat exchanger cores. Therefore, a brazing furnace frame can hold 336 fixtures with heat exchanger cores, such as Figure 1 As shown. The weight of a single heat exchanger core tooling is about 5kg. Figure 2 There are two disadvantages of manual unloading: (1) The brazing furnace frame is large in size and is divided into grids of different heights. The number of brazing fixtures for the heat exchanger core stored at a time is large, and the storage time is relatively limited. This puts the operator in a high labor intensity; (2) The fixtures with the heat exchanger core that come out of the brazing furnace contain residual heat generated by the brazing process. This heat cannot be quantified and can easily cause burns to the operator.

[0003] Given the aforementioned reasons, automated unloading of the brazing furnace frame has become a priority. However, during automated unloading, precise positioning of each brazing fixture within the frame is not possible. This inevitably results in irregular movement of each fixture within the brazing frame as it is transported to the unloading station. Furthermore, the brazing frame also experiences varying degrees of deformation. Consequently, accurately grasping the fixture during automated unloading becomes a critical factor affecting the stability of the unloading system. Industrial robots offer unparalleled advantages, including precise repeatability, high reliability, flexible production, and a high degree of automation. After the brazing device completes the brazing process for all the heat exchanger cores within the brazing furnace frame, the frame, driven by its circulating conveyor chain, moves all fixtures out of the brazing area to the unloading station. During this movement, slight misalignment is inevitable, as the fixtures cannot be precisely positioned within the frame. In the planning of automatic blanking, the first problem that needs to be solved is to obtain the position information of each core tool before grabbing it. Summary of the Invention

[0004] The purpose of the present invention is to overcome the deficiencies of the prior art and to provide a method and a grasping robot for grasping workpieces through machine vision, which can realize synchronous multiple grasping of multiple brazing fixtures with heat exchanger cores.

[0005] A technical solution to achieve the above object is: a method for grasping a workpiece using machine vision, which is used to automatically unload a heat exchanger core on a brazing frame, comprising the following steps:

[0006] Step 1: Acquire an image through a camera and pre-process the image;

[0007] Step 2: Using the EuclideanCluster algorithm, the number of pixels is distinguished between different artifacts and between artifacts and backgrounds to achieve point cloud clustering.

[0008] Step 3: perform feature matching between the projection template and the captured image to obtain the position coordinates of the grasping point.

[0009] Furthermore, the specific method of step 1 is: images of the object to be measured are obtained by simultaneously shooting the left and right cameras, and a suitable camera projection matrix is ​​obtained according to the structured light encoding method, so as to calculate the depth map and the point cloud with the normal vector, and then the target area is narrowed according to the preset range of interest, and the point cloud within the 3D range of interest is extracted, and according to the pre-given reference direction, the angle between each point in the point cloud and the reference direction is calculated, and the point cloud with an angle value exceeding the limit is removed.

[0010] Furthermore, the specific method of step 2 is as follows: the algorithm uses Euclidean distance as the distance measurement method, which is defined as follows:

[0011]

[0012] Where, X i ,X j are two sample vectors respectively.

[0013] Furthermore, the specific method of step 3 is:

[0014] Perform median filtering and binarization on the collected point cloud to obtain a binarized reference image. Calculate the depth value of each pixel (x0, y0) in the image using the following method, where N is the search radius:

[0015]

[0016] Find the point that is most similar between this pixel and the reference template:

[0017] min xp,yp Dif(X p, Y p ,x 0, y0)=temp(x0,y0)–temp(X p ,Y p, )

[0018] st|X p -x0|+|Y p -y0|≤domain, where domain is the set boundary;

[0019] Solve the point (X pmin ,Y pmin ), (X pmin ,Y pmin )-(X0,Y0) to get the direction k, and (X pmin ,Y pmin ) extends in the direction k, and obtains (X q ,Y q ), and in (X q ,Y q ) and (X pmin ,Y pmin ) most similar point;

[0020] min xp2,yp2 Dif(X p2, Y p2 ,X pmin ,Y pmin )=temp(X pmin ,Y pmin )–temp(X p2 ,Y p2, )

[0021] st|X q -x p2 |+|Y q -y p2 |≤dis2, where domain is the set distance;

[0022] Solved point (X q ,Y qmin ), and thus iterate to complete the entire image. Using this method, the preset template is matched with the 2D point cloud, and the position coordinates of the grasping point are further obtained; according to the robot inverse kinematics, the angles of each joint of the robot can be obtained.

[0023] Furthermore, the process of visual positioning for the Nth grasping process is as follows:

[0024] First, the robot moves to the photographing position of the Nth grasping position and takes a photo of the Nth grasping position of the brazing frame;

[0025] Then, the actual coordinates of the feature points at the Nth grasping position are calculated, and the coordinate compensation with its theoretical value is calculated;

[0026] Then, the system calculates the position movement value of each heat exchanger core at the Nth grabbing position and determines whether the change in movement is within the allowable range;

[0027] If it is not within the allowable range, the system will alarm, manual intervention will be required to eliminate the abnormal movement of the tooling, and the abnormality handling procedure will be started;

[0028] If it is within the allowable range, the system calculates the coordinate compensation of the grasping positions of the four core toolings caused by the position movement values ​​of each core tooling, and calculates and updates the final coordinate compensation of the overall grasping position; finally, the system calculates and updates the final grasping position coordinates, and guides the robot to run to the grasping position for grasping.

[0029] A grasping robot using the above-mentioned method for grasping a workpiece through machine vision is characterized in that it includes a robot moving system (1) with a walking axis, a robot gripper (2) for grasping a heat exchanger core, and a 3D industrial camera system (3), wherein the 3D industrial camera system (3) is arranged on the top of the robot gripper (2), and the robot gripper (2) grasps a plurality of brazing toolings with heat exchanger cores at one time.

[0030] The present invention provides a method and a grasping robot for grasping workpieces using machine vision. These methods fully utilize the capabilities of existing visual system units and, in combination with algorithms, enable rapid positioning of the dimensions of multiple products simultaneously, enabling a fully automated robotic unloading system that matches the rhythm of the brazing system. Compared to conventional automated systems, the present invention precisely positions grasped objects. Furthermore, the present invention enhances the system's flexibility, simplifying on-site space. Furthermore, by utilizing the non-contact photography of a 3D industrial camera and the remote operation of a robotic arm, the system avoids harsh manual operating environments caused by residual brazing temperatures and other factors, meeting the requirements of modern intelligent production. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 It is a structural diagram of the brazing furnace frame;

[0032] Figure 2 It is a structural diagram of the tooling with a heat exchanger core;

[0033] Figure 3 This is a schematic diagram of the structure of the grasping robot;

[0034] Figure 4 for Figure 3 A detailed enlarged picture;

[0035] Figure 5 This is a schematic diagram of the visual system framework;

[0036] Figure 6 Flowchart of the visual positioning for the Nth grasping process. DETAILED DESCRIPTION

[0037] In order to better understand the technical solution of the present invention, the following is a detailed description through specific embodiments:

[0038] See also Figure 3 and Figure 4 The grasping robot of the present invention includes a robot moving system 1 with a walking axis, a robot gripper 2 for grasping a brazing tool with a heat exchanger core, and a 3D industrial camera system 3, wherein the 3D industrial camera system 3 is arranged on the top of the robot gripper 2. The robot gripper 2 grasps multiple brazing tools 5 with heat exchanger cores at one time, and the brazing tool with the heat exchanger core is set on the rotary brazing furnace frame 4.

[0039] Machine vision uses a 3D industrial camera lens mounted on a robot gripper to convert captured images of core-carrying tooling into digital image signals. These signals are then transmitted to an image processing system, which then obtains information such as the object's dimensions. Based on the image's pixel distribution, brightness, contrast, and color contrast, these signals are converted into digital signals. The imaging system then performs specialized calculations on these signals to extract the target's features. It then classifies the samples using classification rules, and then controls the actions of actuators on-site based on the identified results, completing the entire operational process.

[0040] After all on-site units are configured, the positional accuracy between the robot coordinate system and the brazing furnace frame coordinate system needs to be calibrated. The robot's sixth axis integrates a gripper, which is also integrated with a 3D industrial camera. By setting TCP, the tool coordinate system of the robot gripper and the lens coordinate system of the 3D industrial camera lens are obtained, and the conversion relationship between these two coordinate systems is determined. The 3D industrial camera lens is then used to capture the positional information of the four brazing core fixtures that need to be grasped.

[0041] Using surface structured light binocular vision to guide robot grasping, the depth map and feature point set of the workpiece are quickly extracted, and real-time depth value calculation within the range of 650mm-1200mm is successfully achieved, realizing feature-based robot posture evaluation.

[0042] In the visual servo system, when a 6-axis robot guided by a fixed eye grasps a target workpiece, it needs to match the feature point set of the photographed object. An infrared laser light source is used to project a structured light pattern, and an infrared camera is used to receive the image returned after the structured light is projected on the workpiece. After obtaining the depth information of the workpiece, feature matching is performed between the projected template and the captured image to obtain the position information of the robot's grasping point, thereby achieving the grasping of the workpiece. The visual system framework is as follows: Figure 5 .

[0043] The present invention obtains the workpiece image from the camera to generate a depth map and a point cloud with a normal vector. Then, the target area is narrowed down according to the pre-set region of interest (ROI) to improve processing efficiency. Through a pre-given reference direction, the angle between each point in the point cloud and the reference direction is calculated, and the point cloud outside the range is removed. After point cloud filtering and clustering, the 3D point cloud is converted into a 2D point cloud. Feature matching is performed based on the preset 2D point cloud template and the generated 2D projection point cloud to obtain the position and direction of the workpiece's grasping point, and further obtain the robot's posture at the grasping point. The specific implementation steps are as follows:

[0044] Step 1: Get image from camera and pre-process the image:

[0045] The relationship between points on the workpiece and corresponding points in the captured image is determined by the camera's geometric model, camera position, and orientation. Once the structured light system is calibrated, the captured depth image is independent of ambient lighting, shadows, and object color, and its pixels can clearly express the collective shape and orientation of the workpiece surface. The left and right cameras simultaneously capture images of the object being measured, and the appropriate camera projection matrix is ​​obtained based on the structured light encoding method. Further calculations can produce a depth map and a point cloud with normal vectors. The target area is then narrowed down based on the pre-set ROI, and the point cloud within the 3D ROI is extracted. Then, based on the pre-given reference direction, the angle between each point in the point cloud and the reference direction is calculated, and point clouds with angle values ​​exceeding the limit are removed.

[0046] Step 2: Using the EuclideanCluster algorithm, the number of pixels is used to distinguish between different artifacts and between artifacts and backgrounds to achieve point cloud clustering. The mean clustering algorithm usually uses Euclidean distance as the distance measurement method, which is defined as follows:

[0047]

[0048] Where, X i ,X j are two sample vectors, respectively. Using this algorithm for point cloud clustering, it is possible to distinguish between workpieces and between workpieces and backgrounds. The number of points within each clustered point cloud is then determined and compared with a preset range to obtain a point cloud with the desired features.

[0049] Step 3: Perform feature matching between the projection template and the captured image:

[0050] Perform median filtering and binarization on the collected point cloud to obtain a binarized reference image. Calculate the depth value of each pixel (x0, y0) in the image using the following method, where N is the search radius:

[0051]

[0052] Find the point that is most similar between this pixel and the reference template:

[0053] min xp,yp Dif(X p, Y p ,x 0, y0)=temp(x0,y0)–temp(X p ,Y p ,)

[0054] st|X p -x0|+|Y p -y0|≤domain, where domain is the set boundary;

[0055] Solve the point (X pmin ,Y pmin ), (X pmin ,Y pmin )-(X0,Y0) to get the direction k, and (X pmin ,Y pmin ) extends in the direction k, and obtains (X q ,Y q ), and in (X q ,Y q ) and (X pmin ,Y pmin ) most similar point;

[0056] min xp2,yp2 Dif(X p2, Y p2 ,X pmin ,Y pmin )=temp(X pmin ,Y pmin )–temp(X p2 ,Y p2 ,)

[0057] st|X q -x p2 |+|Y q -y p2 |≤dis2, where domain is the set distance;

[0058] Solved point (X q ,Y qmin ), and thus iterate to complete the entire image. Using this method, the preset template is matched with the 2D point cloud, and the position coordinates of the grasping point are further obtained; according to the robot inverse kinematics, the angles of each joint of the robot can be obtained.

[0059] The process of visual positioning for the Nth grasping process is as follows: Figure 6 As shown:

[0060] First, the robot moves to the photographing position of the Nth grasping position and takes a photo of the Nth grasping position of the brazing frame;

[0061] Then, the actual coordinates of the feature points at the Nth grasping position are calculated, and the coordinate compensation with its theoretical value is calculated;

[0062] Then, the system calculates the position movement value of each heat exchanger core at the Nth grabbing position and determines whether the change in movement is within the allowable range;

[0063] If it is not within the allowable range, the system will alarm, manual intervention will be required to eliminate the abnormal movement of the tooling, and the abnormality handling procedure will be started;

[0064] If it is within the allowable range, the system calculates the coordinate compensation of the grasping positions of the four core toolings caused by the position movement values ​​of each core tooling, and calculates and updates the final coordinate compensation of the overall grasping position; finally, the system calculates and updates the final grasping position coordinates, and guides the robot to run to the grasping position for grasping.

[0065] Those skilled in the art should recognize that the above embodiments are merely intended to illustrate the present invention and are not intended to limit the present invention. As long as they are within the spirit of the present invention, any changes or modifications to the above embodiments will fall within the scope of the claims of the present invention.

Claims

1. A method for grasping a workpiece by machine vision, used for automatically unloading a heat exchanger core on a brazing frame, characterized in that: The steps include: Step 1: Acquire an image through a camera and pre-process the image; Step 2: Using the EuclideanCluster algorithm, the number of pixels is distinguished between different artifacts and between artifacts and backgrounds to achieve point cloud clustering. Step 3: Match the projection template with the captured image to obtain the position coordinates of the grabbing point. The specific method of step 1 is as follows: the image of the object to be measured is obtained by simultaneously shooting the left and right cameras, and the appropriate camera projection matrix is ​​obtained according to the structured light encoding method, thereby calculating the depth map and the point cloud with the normal vector. Then, the target area is narrowed according to the preset range of interest, and the point cloud within the 3D range of interest is extracted. Based on the pre-given reference direction, the angle between each point in the point cloud and the reference direction is calculated, and the point cloud with the angle value exceeding the limit is removed; The specific method of step 2 is: the algorithm uses Euclidean distance as the distance measurement method, which is defined as follows: Where, X i ,X j are two sample vectors respectively; The specific method of step 3 is: Perform median filtering and binarization on the collected point cloud to obtain a binarized reference image. Calculate the depth value of each pixel (x0, y0) in the image using the following method, where N is the search radius: Find the point that is most similar between this pixel and the reference template: min xp,yp Dif(X p, Y p ,x 0, y0)=temp(x0,y0)–temp(X p ,Y p, ) st|X p -x0|+|Y p -y0|≤domain, where domain is the set boundary; Solve the point (X pmin ,Y pmin ), (X pmin ,Y pmin )-(x0,y0) to get the direction k, and (X pmin ,Y pmin ) extension direction k, get (X q ,Y q ), and in (X q ,Y q ) and (X pmin ,Y pmin ) most similar point; min xp2,yp2 Diff(X p2, AND p2 ,X pmin ,AND pmin )=temp(X pmin ,AND pmin )– temp(X p2 ,Y p2, ) st|X q -X p2 |+|Y q -Y p2 |≤dis2, where dis2 is the set distance; Solved point (X qmin ,Y qmin ), and thus iterate to complete the entire image. Using this method, the preset template is matched with the 2D point cloud, and the position coordinates of the grasping point are further obtained; according to the robot inverse kinematics, the angles of each joint of the robot can be obtained.

2. The method for grasping a workpiece by machine vision according to claim 1, characterized in that: The process of visual positioning for the Nth grasping process is as follows: First, the robot moves to the photographing position of the Nth grasping position and takes a photo of the Nth grasping position of the brazing frame; Then, the actual coordinates of the feature points at the Nth grasping position are calculated, and the coordinate compensation with its theoretical value is calculated; Then, the system calculates the positional movement value of each heat exchanger core tooling at the Nth grasping position and determines whether the movement value is within the allowable range; If it is not within the allowable range, the system will alarm and manual intervention will be required to eliminate the abnormal heat exchanger core tooling and start the abnormal handling procedure; If it is within the allowable range, the system calculates the coordinate compensation of the grasping positions of the four core toolings caused by the position movement values ​​of each heat exchanger core tooling, and calculates and updates the final coordinate compensation of the overall grasping position; finally, the system calculates and updates the final grasping position coordinates, and guides the robot to run to the grasping position for grasping.

3. A grasping robot using the method for grasping a workpiece by machine vision as claimed in claim 1 or 2, characterized in that: The invention comprises a robot moving system (1) with a walking axis, a robot gripper (2) for grabbing a heat exchanger core tooling, and a 3D industrial camera system (3), wherein the 3D industrial camera system (3) is arranged on the top of the robot gripper (2), and the robot gripper (2) grabs a plurality of heat exchanger core toolings at one time.

Images