An Automated Pearl Identification Method Based on Multi-Camera Image Processing

Abstract

This invention discloses an automated pearl recognition method based on multi-camera image processing, belonging to the field of video processing technology. By confirming the three-dimensional posture of the incoming pearl through multiple visions, a 6-axis robot is guided to grasp the pearl in one posture and place it on an automated drilling machine to complete the fully automated drilling of the pearl. Cameras are installed above and to the side of the grasping table to capture front and side images of the pearl. The images are processed to calculate the pearl's position coordinates and deviation angle. Step 1: Front and side image processing; Step 2: Posture calculation; The automatic pearl grasping based on stereo vision improves the consistency of pearl grasping. Improved drilling consistency increases the yield rate of product production, enhances assembly efficiency, and reduces the input of production personnel.

Description

Technical Field

[0001] This invention relates to the field of video processing technology. Background Technology

[0002] Pearls are an important aquatic product and occupy an important position in the jewelry industry. Pearl processing is a unique industry in China. With the improvement of people's living standards, the demand for pearl jewelry is also growing rapidly. The production of pearl jewelry mainly relies on manual labor, especially manual drilling. Because pearls are three-dimensional, it is necessary to manually adjust the pearl's position and confirm the drilling direction. The finished product quality is inconsistent, resulting in low output pearl processing yield, low efficiency, and a significant increase in enterprise production costs.

[0003] In response, Chinese invention patent CN10519438B, entitled "Embedded Automatic Pearl Sorter and Sorting Method Based on Image Recognition," proposes a technical solution for sorting pearls by calculating their area and roundness using a single visual perspective. However, this method only considers a single plane angle, while pearls are three-dimensional ellipsoids, and a single visual perspective cannot determine their posture, thus lacking practical production significance. Therefore, how to identify the posture of pearls from multiple angles and then grasp and process them has become a technical problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0004] To address the above problems, this invention proposes an automated pearl recognition method based on multi-camera image processing. By confirming the three-dimensional posture of the incoming pearl through multiple visions, a 6-axis robot is guided to grasp the pearl in one posture and place it on an automated drilling machine to complete the fully automated drilling of the pearl.

[0005] The technical solution of the present invention is as follows: cameras are installed above and to the side of the grasping platform to capture images of the front and side of the pearl, and the images are processed to obtain the position coordinates and deviation angle of the pearl.

[0006] Specifically, the following steps are included:

[0007] Step 1: Front and side image processing;

[0008] Step 1.1: Calculate the major axis of the front and side images;

[0009] Step 1.1.1, Front and side image thresholding: The front and side cameras are mounted vertically to acquire front and side images, and the images are segmented based on the threshold Thre based on the foreground and background.

[0010]

[0011] Where V′ represents the value of the segmented image, V represents the original image values ​​of the front camera and the side camera, and Thre represents the threshold;

[0012] Step 1.1.2: Front and side image circle positioning to determine the position of the pearl in the image: After image segmentation, find the smallest circumcircle (RR, CR, Radius) of the pearl, which is the area where the pearl is located.

[0013] (RR-RR a ) 2 +(CR-CR a ) 2 =Radisus 2

[0014] (RR-RR b ) 2 +(CR-CR b ) 2 =Radisus 2

[0015] (RR-RR c ) 2 +(CR-CR c ) 2 =Radisus 2 Formula (1-2)

[0016]

[0017] Among them: (RR) a ,CR a ),(RR b ,CR b ),(RR c ,CR c ) represents the coordinates of the boundary points of the region, RR represents the row coordinates of the center of the circle, CR represents the column coordinates of the center of the circle, and Radius represents the radius;

[0018] Step 1.1.3: Calculate the major axis position of the front and side images;

[0019] Step 1.1.3.1: Extract the set of pearl contour points: Use circle (RR, CR, Radius) to locate and determine the position of each pearl in the image. Use the edge gradient feature algorithm to process the segmented image and extract the set of pearl contour points {Row, Col}.

[0020]

[0021] Where S represents the feature factor for calculating the marginal gradient. Let {Row,Col} represent the gray value of the i-th point, and {Row,Col} represent the set of points of the boundary contour.

[0022] Step 1.1.3.2: Based on the parameters of the contour point set and the circle, calculate the boundary contour close to the circle boundary. Using a distance threshold, calculate the boundary contour with a distance less than Thre relative to the circle boundary. Radisus The set of points (Row′, Col′);

[0023]

[0024] Radius i -Radisus|<Thre Radisus Formula (1-6)

[0025] Among them: Thre Radisus Indicates the distance deviation threshold, Radisus i The actual radius of the contour is represented by (Row′,Col′), which represents the set of contour points close to the contour boundary.

[0026] Step 1.1.3.3: Based on the fact that the distance to the edge of the circle is less than Thre Radisus Given a set of points (Row′, Col′), find the coordinates of the major axis: Classify the points (Row′, Col′) by labeling them, grouping points that are close in distance into one class (Row′, Col′). T′ Col T′ ), calculate the row and column mean of different classes, and calculate the coordinates of the point with the largest radius, which is the position of the major axis;

[0027]

[0028] Among them: Dis i Dis represents the distance between the i-th point and the (i+1)-th point. thre Indicates the distance threshold; (row) T ,col T () represents the row and column mean of a coordinate.

[0029] Step 1.2, Calculation of deviation angles for the front and side views:

[0030] Calculate the rotation angle of the major axis relative to the horizontal direction: Knowing the coordinates of the center (RR, CR), determine the coordinates of the major axis (row). T ,col T ), calculate the rotation angle θ of the major axis relative to the positive horizontal direction, and obtain the rotation angle θ of the pearl relative to the horizontal direction in the field of view of the front and side cameras. S and θ C :

[0031]

[0032] in: and Represents the horizontal and major axis vectors, θ represents the angle between the two vectors, θ > 0 is an acute angle, θ < 0 is an obtuse angle. S and θ C These represent the rotation angles of the front camera and the side camera, respectively.

[0033] Step 1.3: Calculate the pearl grabbing relationship: (RR) s ,CR s ) represents the pixel coordinates of the front view of the pearl, RR c The pixel coordinates of the pearl's side are represented, and the physical coordinates in the robot coordinate system are obtained using the scaling factor and R|T:

[0034]

[0035] Where: R s |T s R represents the rotational and translational relationship between the frontal camera and the robot's coordinate system. c |T c Dx represents the rotational and translational relationship between the side-view camera's vision and the robot. s Dy s Dx represents the calibration coefficient for frontal camera vision. c Dy c X, Y, and Z represent the calibration coefficients for the side-view camera vision, and X, Y, and Z represent the physical equivalents of the robot coordinate system.

[0036] Step 2, Attitude Calculation;

[0037] Step 2.1: Determine the robot's grasping reference point and visual reference point;

[0038] Place the pearls within the field of vision, and calculate the standard grasping position and angle of the pearls based on visual calculations: (X) Rs0 Y Rs0 Z Rc0 θ Rs0 ,θ Rc0 The standard grasping posture of the teaching robot (X0, Y0, Z0, a0, b0, c0);

[0039] Where: X Rs0 Y Rs0 θ Rs0 This represents the standard position and angle for pearl grasping calculated by the front-facing camera. Z Rc0 θ Rc0 This indicates the standard position and angle for pearl grasping calculated by the side camera;

[0040] Step 2.2: Calculate the new grasping angle based on the grasping standard point and the deviation angle;

[0041] The robot calculates the Euler matrix R1 based on the grasping posture (a0,b0,c0) and, combined with the visual bias angle, calculates the new grasping posture of the pearl.

[0042]

[0043] Dθ Rs0 =θ Rs -θ Rs0

[0044] Dθ Rc0 =θ Rc -θ Rc0

[0045]

[0046]

[0047] resultB=atan(-r10,r00)*180 / 3.1415926

[0048] resultC = atan(r21,r22)*180 / 3.1415926 (Formula 1-15)

[0049] Wherein: the deviation angle between the visual front and side views is Dθ Rs0 and Dθ Rc0 θ Rs and θ Rc For each calculation, the major axis angle is given, and (resultA, resultB, resultC) represents the robot's new grasping posture.

[0050] Step 2.3: Based on the new grasping position and pose, the robot grasps the pearl:

[0051] The robot's new grasping position pose (X0+X) Rs -X Rs0 ,Y0+Y Rs -Y Rs0 ,Z0+Z Rc -Z Rc0 The robot performs grasping based on the new position and pose (X, resultA, resultB, resultC), where: Rs Y Rs Z Rc () refers to the location where the pearl was randomly placed, i.e., the location where the pearl was randomly delivered. For example, X Rs0 This represents the value of the pearl in the camera's field of view, +X Rs -X Rs0 , indicating the calculated deviation position of the pearl in the field of view.

[0052] This invention utilizes dual cameras in conjunction with a robot to grasp and drill pearls. It employs multi-view calculations to determine the pearl's front and side contours, calculates the pearl's major axis position based on these contour features, and then obtains the pearl's rotation angle. By calculating the rotation angles from different viewpoints, it determines the most suitable posture for three-dimensional grasping of the pearl, guiding the robot to grasp and drill with consistent posture. This significantly improves both drilling quality and efficiency.

[0053] The beneficial effects of this invention are as follows:

[0054] 1. Calculate the three-dimensional posture of the pearl using multi-vision calculation to determine the optimal grasping posture.

[0055] Second, the automatic pearl grasping based on stereo vision improves the consistency of pearl grasping.

[0056] Third, improved punching consistency increases product yield, improves assembly efficiency, and reduces the need for production personnel. Attached Figure Description

[0057] Figure 1 It is a long axis identification image obtained from a frontal photograph taken when the pearl material arrives;

[0058] Figure 2 It is a long axis identification image obtained from a side photo taken when the pearl material is received;

[0059] Figure 3 This is the flowchart of the method in this case;

[0060] Figure 4 This is a reference diagram showing the working status of this case. Detailed Implementation

[0061] To clearly illustrate the technical features of this patent, the following detailed description is provided through specific embodiments and in conjunction with the accompanying drawings.

[0062] Step 1: Image processing of the front and side views.

[0063] Step 1.1: Calculate the major axis of the front and side images.

[0064] Step 1.1.1, Thresholding processing of frontal and side images:

[0065] The front camera and the side camera are mounted vertically to capture images from the front and side. The images are segmented based on a threshold (Thre) for the foreground and background. The threshold parameter is set to 170, which means that if the grayscale value is greater than 170, the region pixel value is set to 255; otherwise, it is set to 0.

[0066] Step 1.1.2: Front and side image circle positioning to determine the position of the pearl in the image: The coordinate position of the pearl is obtained by using the minimum circumscribed circle algorithm: The circle coordinates of the front are (1285.00, 1844.948, 132.195), and the circle coordinates of the side are (1376.786564, 889.4252, 205.2649).

[0067] Step 1.1.3: Calculate the major axis of the front and side images.

[0068] Step 1.1.3.1: Extract the set of pearl contour points: Using circular positioning, determine the position of each pearl in the image. Then, process the segmented image using an edge gradient feature algorithm to extract the set of pearl contour points. The formula for the gradient feature S is:

[0069]

[0070] Step 1.1.3.2: Based on the parameters of the contour point set and the circle, calculate the boundary contour close to the circle boundary. Using a distance threshold, calculate the boundary contour with a distance less than Thre relative to the circle boundary. Radisus The set of points with a value of 3.

[0071] Step 1.1.3.3: Based on the fact that the distance to the edge of the circle is less than Thre Radisus Given a set of points with a value of 3, find the coordinates of the major axes of the front and side views. The coordinates of the major axis of the front view are (1281.00, 1730), and the coordinates of the major axis of the side view are (1377.5, 1071.5).

[0072] Step 1.2, Calculation of Frontal and Side Deviation Angles: Obtain the rotation angle θ of the pearl relative to the horizontal direction in the field of view of the frontal and side cameras. S =178 and θ C = -0.22488;

[0073] Step 1.3: Calculate the grasping relationship of the pearls;

[0074] Rotation and translation relationship between the front camera and the robot coordinate system The relationship between the side camera's vision and the robot's rotation and translation Calibration coefficient Dx for frontal camera vision s =Dy s =0.01519, calibration coefficient Dx for side-view camera vision c =Dy c =0.01795.

[0075] Step 2, Attitude Calculation

[0076] Step 2.1: Determine the robot's grasping reference point and visual reference point.

[0077] Place the pearls within the field of vision, and calculate the standard grasping position and angle of the pearls based on visual calculations: (X) Rs0 Y Rs0 Z Rc0 θ Rs0 ,θ Rc0 = (9.00, 7.045, 11.566, 170.007, 1.55), the standard grasping posture of the teaching robot (X0, Y0, Z0, a0, b0, c0) = (8.04, 18.426, -167.677, 80.069, 2.1, -147.42);

[0078] Step 2.2: Calculate the new grasping angle based on the grasping standard point and the deviation angle:

[0079] Robot grasping posture (80.069, 2.1, -147.42) and visual deviation angle (Dθ) Rs0 =θ Rs -θ Rs0 =0.44 and Dθ Rc0 =θ Rc -θ Rc0 =3.55:), calculate the new grasping posture of the pearl = (resultA, resultB, resultC) = (80.65525, 2.6810, -143.9188)

[0080] Step 2.3: Based on the new grasping position, the robot grasps the pearl:

[0081] The robot's new grasping position and pose are (10.54, 16.526, -166.7778, 0.65525, 2.6810, -143.9188). Based on this new position and pose, the robot grasps the pearl and punches a hole.

[0082] There are many specific ways to implement this invention. The above description is only a preferred embodiment of this invention. It should be noted that for those skilled in the art, several improvements can be made without departing from the principle of this invention, and these improvements should also be considered within the scope of protection of this invention.

Claims

1. An automated pearl identification method based on multi-camera image processing, characterized in that, Cameras are installed above and to the side of the gripping platform to capture images of the pearl's front and side views. The images are then processed to determine the pearl's position coordinates and deviation angle. Specifically, the following steps are included: Step 1: Front and side image processing; Step 1.1: Calculate the major axis of the front and side images, and obtain the position coordinates of the major axis; Step 1.2: Calculate the deviation angles of the front and side views to obtain the rotation angle of the pearl relative to the horizontal direction in the field of view of the front and side cameras. Step 1.3: Calculate the grasping relationship of the pearls; Step 2, Attitude Calculation; Step 2.1: Determine the robot's grasping reference point and visual reference point; Step 2.2: Calculate the new grasping angle based on the grasping standard point and the deviation angle; Step 2.3: Based on the new grasping position and pose, the robot grasps the pearl; Step 2.1 specifically involves: Place the pearls within the field of vision, and calculate the standard grasping position and angle of the pearls based on visual calculations: (X) Rs0 Y Rs0 Z Rc0 θ Rs0 ,θ Rc0 The standard grasping posture of the teaching robot (X0, Y0, Z0, a0, b0, c0); Where: X Rs0 Y Rs0 θ Rs0 Z represents the standard position and angle for pearl grasping calculated by the front-facing camera. Rc0 θ Rc0 This indicates the standard position and angle for pearl grasping calculated by the side camera; Step 2.2 specifically involves: Based on the grasping posture (a0, b0, c0), the robot calculates the Euler matrix R1 and, combined with the visual bias angle, calculates the new grasping posture (resultA, resultB, resultC) of the pearl. resultB=atan(-r10,r00)*180 / 3.1415926 resultC = atan(r21,r22)*180 / 3.1415926 (Formula 1-15) Wherein: the deviation angle between the visual front and side views is Dθ Rs0 and Dθ Rc0 θ Rs and θ Rc For each calculation, the major axis angle is given, and (resultA, resultB, resultC) represents the robot's new grasping posture.

2. The automated pearl identification method based on multi-camera image processing according to claim 1, characterized in that, Step 1.1 specifically includes: Step 1.1.1, Front and side image thresholding: The front and side cameras are mounted vertically to acquire front and side images, and the images are segmented based on the threshold Threshold based on the foreground and background. Where V′ represents the value of the segmented image, V represents the original image values ​​of the front camera and the side camera, and Thre represents the threshold; Step 1.1.2: Front and side image circle positioning to determine the position of the pearl in the image: After image segmentation, find the smallest circumcircle (RR, CR, Radius) of the pearl, which is the area where the pearl is located. (RR-RR a ) 2 +(CR-CR a ) 2 =Radisus 2 (RR-RR b ) 2 +(CR-CR b ) 2 =Radisus 2 (RR-RR c ) 2 +(CR-CR c ) 2 =Radisus 2 Formula (1-2) Among them: (RR) a ,CR a ),(RR b ,CR b ),(RR c ,CR c ) represents the coordinates of the boundary points of the region, RR represents the row coordinates of the center of the circle, CR represents the column coordinates of the center of the circle, and Radius represents the radius; Step 1.1.3: Calculate the major axis position of the front and side images; Step 1.1.3.1: Extract the set of pearl contour points: Use circle (RR, CR, Radius) to locate and determine the position of each pearl in the image. Use the edge gradient feature algorithm to process the segmented image and extract the set of pearl contour points {Row, Col}. Where S represents the feature factor for calculating the marginal gradient. Let {Row,Col} represent the gray value of the i-th point, and {Row,Col} represent the set of points of the boundary contour. Step 1.1.3.2: Based on the parameters of the contour point set and the circle, calculate the boundary contour close to the circle boundary. Using a distance threshold, calculate the boundary contour with a distance less than Thre relative to the circle boundary. Radisus The set of points (Row′, Col′); Among them: Thre Radisus Indicates the distance deviation threshold, Radisus i The actual radius of the contour is represented by (Row′,Col′), which represents the set of contour points close to the contour boundary. Step 1.1.3.3: Based on the fact that the distance to the edge of the circle is less than Thre Radisus Given a set of points (Row′, Col′), find the coordinates of the major axis: Classify the points (Row′, Col′) by labeling them, grouping points that are close in distance into one class (Row′, Col′). T′ Col T′ ), calculate the row and column mean of different classes, and calculate the coordinates of the point with the largest radius, which is the position of the major axis; Among them: Dis i Dis represents the distance between the i-th point and the (i+1)-th point. thre Indicates the distance threshold; (row) T ,col T () represents the row and column mean of a coordinate.

3. The automated pearl identification method based on multi-camera image processing according to claim 2, characterized in that, Step 1.2 specifically involves: Given the coordinates of the center of the circle (RR, CR), determine the coordinates of the major axis (row). T ,col T ), calculate the rotation angle θ of the major axis relative to the positive horizontal direction, and obtain the rotation angle θ of the pearl relative to the horizontal direction in the field of view of the front and side cameras. S and θ C : in: and Represents the horizontal and major axis vectors, θ represents the angle between the two vectors, θ > 0 is an acute angle, θ < 0 is an obtuse angle. S and θ C These represent the rotation angles of the front camera and the side camera, respectively.

4. The automated pearl identification method based on multi-camera image processing according to claim 3, characterized in that, Step 1.3 specifically involves: (RR s ,CR s ) represents the pixel coordinates of the front view of the pearl, RR c The pixel coordinates of the pearl's side are represented, and the physical coordinates in the robot coordinate system are obtained using the scaling factor and R|T: Where: R s |T s R represents the rotational and translational relationship between the frontal camera and the robot's coordinate system. c |T c Dx represents the rotational and translational relationship between the side-view camera's vision and the robot. s Dy s Dx represents the calibration coefficient for frontal camera vision. c Dy c X, Y, and Z represent the calibration coefficients for the side-view camera vision, and X, Y, and Z represent the physical equivalents of the robot coordinate system.

5. The automated pearl identification method based on multi-camera image processing according to claim 1, characterized in that, Step 2.3 specifically involves: The robot's new grasping position and pose (X0+X) Rs -X Rs0 ,Y0+Y Rs -Y Rs0 ,Z0+Z Rc -Z Rc0 The robot performs grasping based on the new position and pose (X, resultA, resultB, resultC), where: Rs Y Rs Z Rc (This is the new location where the pearl was photographed.)

Images