Optical lens circuit board edge grabbing feeding and discharging method and system

Abstract

The application discloses an optical lens circuit board edge grabbing and feeding and discharging method, fixes an RGB-D camera outside a mechanical arm working space, solves a pose transformation matrix through hand-eye calibration, establishes a circuit board model library and a parameter template, collects RGB images and depth images to generate point cloud data, inputs the RGB images into a target detection model to obtain a circuit board bounding box and a category identifier and index the parameter template, fits a circuit board plane based on the point cloud, constructs an optical forbidden contact volume and a high-piece volume, takes the union of the optical forbidden contact volume and the high-piece volume as a global forbidden contact volume, performs a difference set operation on the edge band volume and the global forbidden contact volume to obtain a graspable edge volume, samples candidate grabbing points in the graspable edge volume, selects optimal edge grabbing points through collision detection and comprehensive scoring, and plans a collision-free trajectory through coordinate transformation to drive a gripper to complete feeding and discharging operation.

Description

Technical Field

[0001] This invention relates to the fields of industrial automation and vision, specifically to a method and system for edge gripping and loading / unloading optical lens circuit boards. Background Technology

[0002] With the widespread application of optical products such as smartphone camera modules, automotive cameras, security cameras, and industrial camera modules, modules with optical lenses mounted on circuit boards need to be frequently moved and loaded / unloaded between multiple workstations during the production process. In mixed-flow production scenarios with multiple varieties and small batches, the frequent changes in circuit board and optical lens models require a high degree of flexibility in the loading / unloading system.

[0003] Currently, among common loading and unloading methods, manual handling is labor-intensive, inefficient, and prone to causing fingerprints, stains, and scratches on lenses. Fixture-based fixed-posture loading and unloading solutions rely heavily on the precision and consistency of the fixtures. When the circuit board size, optical lens layout, or product model changes, the fixtures need to be redesigned and replaced, resulting in poor flexibility.

[0004] Traditional vision-guided robotic arms rely solely on two-dimensional contours to locate circuit boards, lacking precise three-dimensional spatial modeling of the optical lenses on the board. This makes it difficult to ensure a safe distance between the gripper and the lens's optical surface during gripping. Furthermore, using fixed templates prevents adaptive gripping of multiple models, and can easily lead to the gripper approaching or interfering with the optical lens, surrounding tall components, or even directly clamping the lens barrel, posing a risk of damage. Most systems only use two-dimensional cameras and lack the ability to utilize depth information for three-dimensional collision detection and safe distance constraints. Summary of the Invention

[0005] This invention proposes a method for edge gripping and unloading optical lens circuit boards, comprising: S1. Fix the camera outside the workspace of the robotic arm, solve the pose transformation matrix between the camera coordinate system and the base coordinate system of the robotic arm through hand-eye calibration, establish a circuit board model library and parameter template, collect RGB images and depth images to generate point cloud data, input the RGB images into the target detection model to obtain the circuit board bounding box and category label, and read the parameter template accordingly. S2. Based on the bounding box, the point cloud of the circuit board area is extracted from the point cloud and the plane is fitted to obtain the plane equation and three-dimensional pose. The optical forbidden volume is constructed based on the parameter template. The points exceeding the preset threshold are clustered based on the point cloud to generate the high component volume. The optical forbidden volume and the high component volume are combined to obtain the global forbidden volume. The edge band volume is constructed at the edge of the circuit board and the difference operation is performed with the global forbidden volume to obtain the grabbable edge volume. S3. Sample candidate grab points within the grabbable edge volume, solve the joint posture based on inverse kinematics to generate the trajectory, perform collision detection on the trajectory and the global no-touch volume to remove candidate grab points with collision risk, and construct a comprehensive scoring function according to preset weights for the remaining candidate grab points to select the candidate grab point with the highest score as the target edge grab point and the target gripper posture. S4. Transform the target edge gripping point and target gripper posture from the camera coordinate system to the robot arm base coordinate system through the hand-eye calibration relationship to obtain the target gripping posture. Plan a collision-free trajectory under kinematic constraints and control the end gripper of the robot arm to complete the gripping and loading / unloading operations.

[0006] Furthermore, before constructing the optically forbidden volume in S2, an outlier removal operation based on statistical filtering is performed on the point cloud of the circuit board region. The average distance between each point and its set of neighboring points is calculated. When the average distance exceeds a preset multiple of the sum of the global mean and the standard deviation, the point is marked as an outlier and removed. The plane fitting is then re-performed on the point cloud after removing outliers.

[0007] Furthermore, before constructing the comprehensive scoring function in S3, a gripper reachability check is performed on the candidate gripping points that pass the collision detection. The gripper posture of the candidate gripping point is substituted into the inverse kinematics solver of the robotic arm. If the solver returns no solution or any joint angle exceeds the physical limit range of the joint, the candidate gripping point is marked as unreachable and is removed.

[0008] Furthermore, the generation of the optical forbidden volume includes constructing a circular, elliptical, or polygonal forbidden area on the circuit board plane with the center of the optical lens as the reference, and extending it upwards by a preset safety height margin in the height direction, stretching the two-dimensional forbidden area into a three-dimensional cylinder or polyhedron, and performing a union operation on the optical forbidden volumes corresponding to multiple optical lenses to obtain the total optical forbidden volume.

[0009] Furthermore, the generation of the high component volume includes calculating the vertical height of each point in the point cloud to the circuit board plane, marking a high component point when the height exceeds a preset component height threshold, performing connected component clustering on the high component point cloud, and calculating the three-dimensional bounding box or convex hull of each connected component as the high component volume.

[0010] Furthermore, the comprehensive scoring function includes the following normalized sub-item: effective contact length between the gripper and the edge of the circuit board. Distance between gripping point and circuit board center of gravity Minimum safe distance between the capture trajectory and the global no-touch volume Robotic arm joint posture margin The scores are then linearly weighted according to preset weights to obtain a comprehensive score.

[0011] Furthermore, after constructing the edge band volume in S2, edges that are not suitable for grabbing are ignored according to the grabbing priority in the parameter template, and then the edge band volume and the global forbidden volume are subjected to geometric difference operation.

[0012] Further, solving the pose transformation matrix in S1 includes mounting a calibration plate on the end effector of the robotic arm, setting the motion trajectory of the robotic arm, acquiring calibration images and corresponding robotic arm poses in multiple postures, and assuming the pose of the end effector of the robotic arm in the robotic arm base coordinate system is... The transformation from the camera coordinate system to the robot arm's end effector coordinate system is as follows: The transformation from the camera coordinate system to the robot arm's base coordinate system is as follows: For the first Group and No. Group calibration data meets The constraints are solved using the least squares method. Then by The pose transformation matrix between the camera coordinate system and the robot arm base coordinate system is obtained.

[0013] Furthermore, the present invention also proposes an edge gripping and loading / unloading system for optical lens circuit boards, comprising: Image and depth data acquisition module, used to acquire RGB images, depth images and point cloud data; The deep learning recognition and model matching module is used to output the circuit board bounding box and category identifier using the object detection network and to complete the parameter template indexing. The 3D modeling and touch-free volume generation module is used to construct circuit board planes, optical touch-free volumes, high-component volumes, and global touch-free volumes based on point clouds and parameter templates. The grabbable edge volume generation module is used to obtain the grabbable edge volume by performing a geometric difference operation on the edge band volume and the global untouchable volume; The candidate grab point planning and comprehensive scoring module is used to generate candidate grab points within the grabbable edge volume and complete collision detection and multi-index scoring. The robotic arm trajectory planning and control module is used to plan a collision-free trajectory based on the target grasping posture and send it to the robotic arm controller to realize the edge grasping and loading / unloading operations of the circuit board. Attached Figure Description

[0014] Figure 1 This is a flowchart illustrating a battery swapping service scheduling method proposed in this invention. Detailed Implementation

[0015] This invention proposes a method for edge gripping and unloading optical lens circuit boards, the core innovation of which lies in: By simultaneously acquiring two-dimensional images and depth information using an RGB-D camera, the union of the optically forbidden volume and the height volume is constructed in three-dimensional space as the global forbidden volume. The dangerous areas in the circuit board edge band volume are eliminated through geometric difference operations to obtain the grippable edge volume, so that the clamp only clamps in the safe edge area of ​​the circuit board, fundamentally avoiding contact and interference with the optical surface of the optical lens.

[0016] Unlike existing technologies that rely solely on two-dimensional contour positioning, this invention utilizes depth information to achieve three-dimensional collision detection and safety distance constraints, and enables adaptive switching between multiple models through a model library and parameter template mechanism.

[0017] The aforementioned method for edge gripping and unloading of optical lens circuit boards specifically includes: S1. Fix the RGB-D camera outside the workspace of the robotic arm, solve the pose transformation matrix between the camera coordinate system and the robotic arm base coordinate system through hand-eye calibration, establish a circuit board model library and parameter template, collect RGB images and depth images to generate point cloud data, input the RGB images into the target detection model to obtain the circuit board bounding box and category identifier, and read the parameter template accordingly.

[0018] Among them, the RGB-D camera is fixed outside the working space of the robotic arm to form an "eye outside the hand" structure. The "eye outside the hand" structure refers to the installation method in which the camera is fixedly installed outside the working space of the robotic arm and does not change its position with the movement of the robotic arm.

[0019] A model library is a database that stores various circuit board models equipped with optical lenses and their corresponding parameter templates.

[0020] A parameter template is a set of parameters created for each circuit board model, including shape contour parameters, optical lens center coordinates and number, planar shape and height safety margin of the optical forbidden area, gripping priority of each side of the circuit board, and minimum length of the edge segment that can be clamped.

[0021] Point cloud data refers to a set of three-dimensional points generated after filtering and distortion correction of a depth image. A circuit board bounding box is a rectangular box marking the location of a circuit board, output by the object detection model on an RGB image. A category label refers to the circuit board model category label output by the object detection model.

[0022] Specifically, an RGB-D camera is fixedly mounted outside the workspace of the robotic arm to form an eye-on-hand structure, thus obtaining the camera coordinate system. A circuit board model library is established, and parameter templates are created for each circuit board model equipped with an optical lens. The parameter templates include at least the circuit board's outline parameters in its own coordinate system, the center coordinates and number of optical lenses, the planar shape and height safety margin of the optical forbidden area, the gripping priority of each side of the circuit board, and the minimum length of the edge segment that can be gripped.

[0023] When a circuit board equipped with an optical lens enters the loading / unloading station, the RGB-D camera is triggered to acquire RGB and depth images of the circuit board. The depth image is then filtered and distortion corrected to obtain point cloud data.

[0024] An RGB image is input into a pre-trained deep learning object detection model, which is a YOLOv13 model, to perform end-to-end inference on the RGB image and output a two-dimensional bounding box and category label of a circuit board equipped with an optical lens.

[0025] The YOLOv13 model was trained under supervised conditions using a large number of RGB image samples labeled with circuit board categories and bounding boxes. The training dataset contained no fewer than 5000 RGB images covering different models of circuit boards equipped with optical lenses. Each image was labeled with a circuit board bounding box and category label. The model used random cropping, color dithering, and mosaic enhancement for data augmentation. The initial learning rate was set to 0.01 and decayed using a cosine annealing strategy. The total number of training epochs was 300, the batch size was 16, and the input resolution was uniformly scaled to [resolution value missing]. Pixel.

[0026] The bounding box and category identifier of the target circuit board are selected based on the detection confidence level, and the parameter template of the corresponding model is indexed from the circuit board model library according to the category identifier to read the shape contour parameters, optical lens position and optical forbidden area parameters.

[0027] The system hardware includes: an RGB-D camera and light source fixedly installed outside the workspace, a multi-degree-of-freedom robotic arm and a control cabinet for controlling the robotic arm, an electric parallel gripper installed at the end of the robotic arm, a workpiece stage for placing a circuit board containing optical lenses, and a processor for running visual recognition and grasping planning algorithms.

[0028] An RGB-D camera is used to acquire RGB and depth images of the circuit board. The pixel coordinates of the circuit board to be grasped in the image are transformed into the coordinate system of the robotic arm base. Under the control of the controller, the robotic arm completes the grasping and placement of the circuit board.

[0029] In one specific implementation: The system includes two types of circuit boards, Model A and Model B, each equipped with an optical lens. The Model A parameter template records the external outline as follows: A rectangle containing one optical lens with center coordinates (25, 30). The optical no-touch area is a circle with a radius of 15mm centered at the lens center. The height safety margin is 5mm. The gripping priority is longer sides first, then shorter sides. The minimum allowable clamping edge length is 20mm. The outline is recorded in the Model B parameter template. The model is a rectangle containing two optical lenses with center coordinates of (20, 25) and (60, 25). The optical no-touch areas are circles with a radius of 12mm centered on the centers of the two lenses. The height safety margin is 4mm. The gripping priority is shorter side first, and the minimum allowable edge length is 15mm. A total of 8000 RGB images were collected from both Model A and Model B circuit boards as the training set, and 2000 images as the validation set. The model achieved an average accuracy of over 0.92 on the validation set, and the processing time for a single image during the inference phase did not exceed 15ms.

[0030] Further, solving the pose transformation matrix in S1 includes mounting a calibration plate on the end effector of the robotic arm, setting the motion trajectory of the robotic arm, acquiring calibration images and corresponding robotic arm poses in multiple postures, and assuming the pose of the end effector of the robotic arm in the robotic arm base coordinate system is... The transformation from the camera coordinate system to the robot arm's end effector coordinate system is as follows: The transformation from the camera coordinate system to the robot arm's base coordinate system is as follows: For the first Group and No. Group calibration data meets The constraints are solved using the least squares method. Then by The pose transformation matrix between the camera coordinate system and the robot arm base coordinate system is obtained.

[0031] Among them, the calibration plate refers to a planar calibration device printed with known geometric patterns, the robot arm pose refers to the position and orientation of the robot arm end in the robot arm base coordinate system, and the least squares method refers to a mathematical method of solving unknown parameters by constructing an overdetermined system of equations and using the minimum sum of squared residuals as the criterion.

[0032] Specifically, a checkerboard calibration plate is fixedly mounted on the end flange of the robotic arm. The robotic arm is controlled to move to no fewer than 15 different postures. At each posture, images of the calibration plate are simultaneously acquired and the pose data of the robotic arm's end effector is recorded. Corner pixel coordinates are extracted from each calibration plate image, and the camera's extrinsic parameters, i.e., the pose of the calibration plate in the camera coordinate system, are solved using the known physical dimensions of the checkerboard. Let the... The pose of the robotic arm's end effector in the robotic arm's base coordinate system in the dataset is as follows: The pose of the calibration board in the camera coordinate system is The geometric relationships defined by the eye on the outside of the hand are: ,in, Let be the transformation matrix from the camera coordinate system to the robot arm's end effector coordinate system. Given the transformation matrix from the camera coordinate system to the robot arm's base coordinate system, we construct an overdetermined system of equations by simultaneously solving all sets of data, and then solve it using the least squares method. Then by Solve .

[0033] In one specific implementation: Using The checkerboard calibration board has a grid side length of 20mm. The robotic arm collects 20 sets of calibration data in different postures. After calibration, the hand-eye calibration residual is less than 0.5mm.

[0034] S2. Based on the bounding box, extract the circuit board area point cloud from the point cloud and perform plane fitting to obtain the plane equation and three-dimensional pose. Construct the optical forbidden volume based on the parameter template. Based on the point cloud, cluster the points that exceed the preset threshold to generate the high component volume. Take the union of the optical forbidden volume and the high component volume to obtain the global forbidden volume. Construct the edge band volume at the edge of the circuit board and perform a difference operation with the global forbidden volume to obtain the grabbable edge volume.

[0035] Among them, the circuit board plane equation refers to the mathematical equation describing the plane on which the circuit board is located, obtained by performing plane fitting on the point cloud of the circuit board region.

[0036] Three-dimensional pose refers to the normal vector and position parameters of the circuit board plane in the camera coordinate system.

[0037] Optical forbidden volume refers to the area in three-dimensional space around an optical lens that prevents clamps from entering.

[0038] The height of a component refers to the three-dimensional space occupied by components on a circuit board whose height exceeds a preset threshold.

[0039] The global no-touch volume refers to the union of the optical no-touch volume and the height volume. The edge band volume refers to the strip-shaped three-dimensional region constructed after offsetting the edge of the circuit board inward by a preset distance; its thickness and height are determined according to the size of the electric parallel gripper and the safety clearance.

[0040] The grabbable edge volume refers to the connected region remaining in the edge band volume of the circuit board after removing the part that intersects with the global no-touch volume.

[0041] Specifically, the point cloud of the circuit board region is extracted from the depth image or point cloud based on the bounding box of the target circuit board. The point cloud of the region is then fitted with a plane to obtain the plane equation of the circuit board and the three-dimensional pose of the circuit board in the camera coordinate system. Combined with the shape contour parameters in the parameter template, the geometric contour of the circuit board is transformed from the circuit board coordinate system to the camera coordinate system to construct the three-dimensional geometric model of the circuit board in the camera coordinate system.

[0042] Based on the optical lens center coordinates and optical forbidden area parameters in the parameter template, one or more optical forbidden two-dimensional regions are constructed on the circuit board plane. Taking the height of the circuit board plane as a reference, each optical forbidden two-dimensional region is stretched into a three-dimensional optical forbidden volume by extending upward in the height direction by a preset safety height margin. The total optical forbidden volume is obtained by performing a union of multiple optical forbidden volumes.

[0043] The vertical height of each point from the circuit board plane is calculated using point cloud data. Points exceeding a preset component height threshold are marked as high-point points. Connected-domain clustering is performed on the high-point point cloud to calculate the 3D bounding boxes of each connected domain, forming the high-point volume. The optical no-touch volume and the high-point volume are then combined to obtain the global no-touch volume. An edge band volume is constructed by offsetting each edge of the circuit board inwards by a preset distance. The edge band volume and the global no-touch volume are then geometrically differ to remove the parts intersecting with the no-touch volume; the remaining connected regions are the graspable edge volumes.

[0044] In one specific implementation: On circuit board model A, there is an optical lens located at (25, 30). The optical forbidden area is a circle with a radius of 15mm and a height safety margin of 5mm. The system constructs a circular forbidden area with a radius of 15mm on the circuit board plane with (25, 30) as the center and stretches it upward by 5mm along the normal of the circuit board to obtain a cylindrical optical forbidden volume. A tall part region with a height exceeding the preset threshold of 3mm is detected in the point cloud. After performing connected component clustering on the point cloud of this region, the bounding box size is calculated as follows: As the high component volume, the global no-touch volume is obtained by taking the union of the optical no-touch volume and the high component volume. An edge band volume with a thickness of 8mm and a height of 10mm is constructed by offsetting 5mm inward from the edge of the circuit board. After performing a difference operation on the edge band volume and the global no-touch volume, the remaining two connected regions located on the long side are the grabbable edge volumes.

[0045] Furthermore, before constructing the optically forbidden volume in S2, an outlier removal operation based on statistical filtering is performed on the point cloud of the circuit board region. The average distance between each point and its set of neighboring points is calculated. When the average distance exceeds a preset multiple of the sum of the global mean and the standard deviation, the point is marked as an outlier and removed. The plane fitting is then re-performed on the point cloud after removing outliers.

[0046] Among them, statistical filtering refers to an algorithm that identifies and removes noise points based on the statistical characteristics of the local neighborhood of a point cloud; the neighboring point set refers to a subset of neighboring points selected within a preset radius or by the number of nearest neighbors, with each point as the center; the global mean refers to the arithmetic mean of the average neighborhood distance of all points; the standard deviation refers to the standard deviation of the average neighborhood distance of all points; and the preset multiplier refers to the adjustment parameter used to control the strictness of the filtering.

[0047] Specifically, for each point in the point cloud of the extracted circuit board area, its... Find the nearest neighbor and calculate the distance from that point to the nearest neighbor. The average Euclidean distance between the nearest neighbors, where The value of is between 5 and 20. Calculate the global mean of the neighborhood average distance of all points. and standard deviation The average neighborhood distance is greater than Points marked as outliers are removed from the point cloud, where, The value ranges from 1.0 to 3.0. Plane fitting is re-performed on the point cloud after outlier removal to obtain a more accurate circuit board plane equation and 3D pose. In one specific implementation: Set to 10, Set to 2.0, the original number of points in the point cloud of the A circuit board area is 15,000. After statistical filtering, 1,200 outliers are removed, leaving 13,800 valid points for plane fitting. After fitting, the angle error between the plane normal vector and the true normal vector is reduced from 1.5 degrees before filtering to 0.3 degrees.

[0048] Furthermore, the generation of the optical forbidden volume includes constructing a circular, elliptical, or polygonal forbidden area on the circuit board plane with the center of the optical lens as the reference, and extending it upwards by a preset safety height margin in the height direction, stretching the two-dimensional forbidden area into a three-dimensional cylinder or polyhedron, and performing a union operation on the optical forbidden volumes corresponding to multiple optical lenses to obtain the total optical forbidden volume.

[0049] The geometry of the optical forbidden volume depends on the planar shape of the optical forbidden area and the value of the safety margin in the height direction. A circular forbidden area is stretched into a cylinder, an elliptical forbidden area is stretched into an elliptical cylinder, and a polygonal forbidden area is stretched into a prism.

[0050] Specifically, a two-dimensional forbidden area is constructed on the circuit board plane based on the center coordinates of the optical lens. The geometric boundary of the forbidden area is determined according to the shape type and size parameters defined in the parameter template. A preset safety height margin is extended upward along the normal direction of the circuit board plane. The two-dimensional forbidden area is stretched into a three-dimensional cylinder or polyhedron along the normal direction. When there are multiple optical lenses on the circuit board, a Boolean union operation is performed on the corresponding optical forbidden volume to obtain the total optical forbidden volume.

[0051] In one specific implementation: There are two optical lenses on the model B circuit board, with center coordinates of (20, 25) and (60, 25) respectively. Each of them has a restricted area of ​​12 mm radius and a safety margin of 4 mm. The system constructs circular restricted areas of 12 mm radius with (20, 25) and (60, 25) as the center respectively and stretches them upward along the normal direction by 4 mm to form two cylindrical optical restricted volumes. The total optical restricted volume is obtained by performing a Boolean union operation on the two cylinders.

[0052] Furthermore, the generation of the high component volume includes calculating the vertical height of each point in the point cloud to the circuit board plane, marking a high component point when the height exceeds a preset component height threshold, performing connected component clustering on the high component point cloud, and calculating the three-dimensional bounding box or convex hull of each connected component as the high component volume.

[0053] Among them, the preset component height threshold is the height threshold used to distinguish between ordinary components and high-profile components on the circuit board. High-profile points are points in the point cloud whose vertical height exceeds the threshold. Connected component clustering is an algorithm that groups high-profile points into independent connected regions based on the spatial proximity relationship of the point cloud. The 3D bounding box is the smallest cuboid that can enclose all points in each connected region. The convex hull is the smallest convex polyhedron that can enclose all points in each connected region.

[0054] Specifically, the preset component height threshold ranges from 2mm to 5mm. The vertical distance from each point in the point cloud to the circuit board plane is calculated. Points with distances exceeding the threshold are extracted as a subset of the high component point cloud. The high component point cloud is grouped using a connected component clustering algorithm based on Euclidean distance. The clustering distance threshold is set to 1mm to 3mm. The axial bounding box or convex hull of each connected component is calculated as the high component volume corresponding to that connected component.

[0055] In one specific implementation: the preset component height threshold is set to 3mm, the clustering distance threshold is set to 2mm, and the bounding box size after clustering a tall component area detected in the point cloud of circuit board model A is [missing information]. Two high-profile areas were detected in the point cloud of circuit board model B. After clustering, the bounding box size was [missing information]. and .

[0056] Furthermore, after constructing the edge band volume in S2, edges that are not suitable for grabbing are ignored according to the grabbing priority in the parameter template, and then the edge band volume and the global forbidden volume are subjected to geometric difference operation.

[0057] Here, gripping priority refers to the gripping priority level defined for each side of the circuit board in the parameter template. Ignoring unsuitable gripping edges means excluding the edge band volume corresponding to edges with a priority lower than the preset level from subsequent calculations.

[0058] Specifically, the crawling priority code of each edge in the parameter template is read, and the edges with priority codes lower than the preset threshold are marked as edges that are not suitable for crawling. When constructing the edge band volume, only the unmarked edges are offset and the band volume is constructed. Then, the constructed edge band volume and the global untouchable volume are subjected to geometric difference operation to obtain the crawlable edge volume.

[0059] In one specific implementation: In the parameter template of model A, the priority of the long side is coded as 1 and the priority of the short side is coded as 2. The preset priority threshold is set to 2. Only the edge band volume is constructed on the long side and the difference operation is performed with the global forbidden volume. In the parameter template of model B, the priority of the short side is coded as 1 and the priority of the long side is coded as 2. Only the edge band volume is constructed on the short side and the difference operation is performed with the global forbidden volume.

[0060] S3. Sample candidate gripping points within the grippable edge volume, generate trajectories by solving joint poses based on inverse kinematics, perform collision detection on the trajectory and the global no-touch volume to eliminate candidate gripping points with collision risks, construct a comprehensive scoring function for the remaining candidate gripping points according to preset weights, and select the candidate gripping point with the highest score as the target edge gripping point and the target gripper pose.

[0061] Among them, the candidate gripping point refers to the position point sampled at a preset interval within the grippable edge volume, and each candidate gripping point corresponds to the contact position between a pair of parallel grippers and the edge of the circuit board.

[0062] Gripper posture refers to the orientation and angle parameters of the gripper relative to the edge of the circuit board at the candidate gripping point.

[0063] The comprehensive scoring function refers to a function that evaluates candidate gripping points based on multiple indicators, including the effective contact length between the gripper and the edge of the circuit board. Distance between gripping point and circuit board center of gravity Minimum safe distance between the capture trajectory and the global no-touch volume Robotic arm joint posture margin The normalized sub-items are linearly weighted according to preset weights to obtain the comprehensive score.

[0064] It refers to the length of the line segment that actually contacts the inner side of the parallel gripper and the edge of the circuit board. It refers to the Euclidean distance from the candidate grab point to the geometric centroid of the circuit board.

[0065] It refers to the minimum Euclidean distance between each discrete posture on the gripper profile and motion trajectory and the boundary of the global no-touch volume. It refers to the minimum angular margin between each joint of the robotic arm and its joint limit.

[0066] Specifically, multiple candidate gripping points are sampled at preset intervals within the graspable edge volume. Each candidate gripping point corresponds to the contact position between a pair of parallel grippers and the edge of the circuit board, and the corresponding gripper posture is determined so that the inner side of the parallel grippers is basically parallel to the edge of the circuit board. The inverse kinematics of the robotic arm is called to solve the joint posture of the robotic arm at each candidate gripping point, generating a trajectory from the standby position to the pre-grip position, the gripping position, and the departure position. The point cloud data is used to perform 3D collision detection on the gripper contours and the global no-touch volume under each discrete posture in the trajectory. If there is overlap, the corresponding candidate gripping point is removed. For the candidate gripping points that are not removed, calculation is performed. , , and A comprehensive scoring function is constructed according to preset weights, and the candidate point with the highest comprehensive score is selected as the target edge grasping point and the corresponding target gripper posture is determined.

[0067] In one specific implementation: within the grippable edge volume of the model A circuit board, three candidate gripping points P1, P2, and P3 are sampled at 10mm intervals, corresponding to the effective contact length. 25mm 18mm It is 22mm, the distance from the center of gravity. 30mm 45mm The safe distance is 38mm. 8mm 12mm The joint posture margin is 6mm. 15 degrees 20 degrees The angle is 10 degrees. After 3D collision detection, P3 is removed because its trajectory overlaps with the global no-touch volume. P1 and P2 are weighted and their comprehensive scores are calculated. P1 has the highest comprehensive score and is selected as the target edge grab point.

[0068] Furthermore, before constructing the comprehensive scoring function in S3, a gripper reachability check is performed on the candidate gripping points that pass the collision detection. The gripper posture of the candidate gripping point is substituted into the inverse kinematics solver of the robotic arm. If the solver returns no solution or any joint angle exceeds the physical limit range of the joint, the candidate gripping point is marked as unreachable and is removed.

[0069] Among them, the gripper reachability verification refers to the verification step that further verifies whether the robotic arm can reach the candidate gripping point with an effective joint angle after the collision detection is passed. The inverse kinematics solver is a mathematical solver that solves the angles of each joint given the end pose. The physical limit range refers to the minimum and maximum angle range that each joint is allowed to rotate in the mechanical structure.

[0070] Specifically, for each candidate gripping point that passes collision detection, its gripper posture and position parameters are combined into a target end-effector pose matrix and input into the inverse kinematics solver of the robotic arm. If the solver returns no solution, it means that the pose is outside the workspace of the robotic arm, and the candidate gripping point is directly marked as unreachable and eliminated. If the solver returns a solution, it checks whether each joint angle falls within the physical limit range of the joint. If any joint angle exceeds the physical limit range, the candidate gripping point is also marked as unreachable and eliminated. Only candidate gripping points whose joint angles are all within the physical limit range are retained for subsequent comprehensive scoring.

[0071] In one specific implementation: After collision detection, the candidate grab points P1 and P2 of the circuit board of model A undergo accessibility verification. In the inverse kinematics solution of P2, the angle of the fourth joint is -175 degrees, which exceeds the physical limit range of -170 degrees to +170 degrees of the joint. P2 is marked as unreachable and eliminated. Only P1 passes the accessibility verification and enters the comprehensive score.

[0072] Furthermore, the comprehensive scoring function includes the following normalized sub-item: effective contact length between the gripper and the edge of the circuit board. Distance between gripping point and circuit board center of gravity Minimum safe distance between the capture trajectory and the global no-touch volume Robotic arm joint posture margin The scores are then linearly weighted according to preset weights to obtain a comprehensive score.

[0073] in, , , and All values ​​are dimensionless values ​​after normalization. The normalization method uses min-max normalization to map each sub-item to the interval between 0 and 1. The preset weights are the weight coefficients of each sub-item in the comprehensive score, and the sum of the weight coefficients is 1. Specifically, for each candidate crawl point that passes collision detection and reachability verification, the following calculations are performed: , , and The original values ​​are then normalized by performing min-max normalization on all candidate capture points in the current batch to obtain normalized values, and then normalized according to preset weights. , , , The comprehensive score is obtained by linearly weighting and summing the normalized values. The formula for calculating the comprehensive score is: ,in, The value ranges from 0.2 to 0.4. The value ranges from 0.1 to 0.3. The value ranges from 0.2 to 0.4. The value ranges from 0.1 to 0.3.

[0074] In one specific implementation: Set it to 0.3. Set it to 0.2. Set it to 0.3. Set the value to 0.2, calculate the overall score for P1, and P1's... The normalized value is 1.0. The normalized value is 0.0. It is 1.0. The normalized value is 0.0. The normalized value is 1.0, and the P1 composite score is... P1 was selected as the target edge grab point.

[0075] S4. Transform the target edge gripping point and target gripper posture from the camera coordinate system to the robot arm base coordinate system through the hand-eye calibration relationship to obtain the target gripping posture. Plan a collision-free trajectory under kinematic constraints and control the end gripper of the robot arm to complete the gripping and loading / unloading operations.

[0076] The target grasping pose refers to the pose description of the target edge grasping point and the target gripper posture in the robot arm's base coordinate system. Kinematic constraints refer to the collective term for the robot arm's joint limits, velocity, and acceleration constraints. Specifically, the target edge grasping point and the target gripper posture are transformed from the camera coordinate system to the robot arm's base coordinate system using a hand-eye calibration matrix to obtain the target grasping pose. Based on the robot arm's kinematic and dynamic constraints, a collision-free trajectory is planned from the standby position to the pre-grabbing position, grasping position, departure position, and placement position. Under the premise of satisfying the joint limit, velocity, and acceleration constraints, the robot arm is controlled to drive the end-effector electric parallel gripper along this trajectory to complete the grasping and unloading operations at the edge of the circuit board. The electric parallel gripper only grips in the edge area of ​​the circuit board and does not contact the optical surface of the optical lens.

[0077] In one specific implementation: the robotic arm is a six-axis robotic arm with an electric parallel gripper with a stroke of 40mm installed at the end. The RGB-D camera is installed 500mm above the worktable. The processor is an industrial PC that communicates with the RGB-D camera and the robotic arm control cabinet via Ethernet. After the processor completes all the above calculations, it sends the target grasping pose to the robotic arm control cabinet via Ethernet. The robotic arm control cabinet plans the joint spatial trajectory based on this and drives the joint motors to perform grasping and loading / unloading actions.

[0078] Furthermore, the present invention also proposes an edge gripping and loading system for optical lens circuit boards. The system includes an image and depth data acquisition module, a deep learning recognition and model matching module, a 3D modeling and untouchable volume generation module, a grippable edge volume generation module, a candidate gripping point planning and comprehensive scoring module, and a robotic arm trajectory planning and control module.

[0079] The image and depth data acquisition module is responsible for acquiring RGB images, depth images, and point cloud data and outputting them to the deep learning recognition and model matching module. The deep learning recognition and model matching module uses an object detection network to output the circuit board bounding box and category identifier, and after completing the template index, it outputs them to the 3D modeling and no-touch volume generation module. The 3D modeling and no-touch volume generation module constructs the circuit board plane, optical no-touch volume, high component volume, and global no-touch volume based on the point cloud and template, and outputs them to the graspable edge volume generation module. The graspable edge volume generation module obtains the graspable edge volume by performing geometric difference operations on the edge band volume and the global no-touch volume, and outputs it to the candidate grasping point planning and comprehensive scoring module. The candidate grasping point planning and comprehensive scoring module generates candidate grasping points within the graspable edge volume, completes collision detection and multi-index scoring, and outputs them to the robotic arm trajectory planning and control module. The robotic arm trajectory planning and control module plans a collision-free trajectory based on the target grasping pose and sends it to the robotic arm controller to realize the circuit board edge grasping and loading / unloading operations. Specifically, the system uses an industrial PC as the processor to run the software implementation of the above modules. The modules transfer data through shared memory or message queues. The image and depth data acquisition module acquires raw image data from an RGB-D camera through a USB or Ethernet interface. The deep learning recognition and model matching module calls the pre-loaded YOLOv13 model weight file for inference. The 3D modeling and no-touch volume generation module performs plane fitting and volume calculation based on the PCL point cloud library. The candidate grasping point planning and comprehensive scoring module performs 3D collision detection based on the FCL collision detection library. The robotic arm trajectory planning and control module generates joint space trajectories through a kinematic solver and trajectory optimizer and sends them to the robotic arm control cabinet via Ethernet.

[0080] In one specific implementation: the system is deployed on a mobile phone camera module assembly line. The workstation is equipped with a six-axis robotic arm, an RGB-D camera and an industrial PC. The industrial PC is connected to the camera and the robotic arm control cabinet via gigabit Ethernet. When the two types of circuit boards, model A and model B, are produced in mixed production, the system automatically switches parameter templates through deep learning recognition. It can complete the edge gripping and loading / unloading operations of the two types of circuit boards without manual replacement of fixtures.

[0081] The above are merely preferred embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A method for edge-gripping and unloading optical lens circuit boards, characterized in that, include: S1. Fix the camera outside the workspace of the robotic arm, solve the pose transformation matrix between the camera coordinate system and the base coordinate system of the robotic arm through hand-eye calibration, establish a circuit board model library and parameter template, collect RGB images and depth images to generate point cloud data, input the RGB images into the target detection model to obtain the circuit board bounding box and category label, and read the parameter template accordingly. S2. Based on the bounding box, the point cloud of the circuit board area is extracted from the point cloud and the plane is fitted to obtain the plane equation and three-dimensional pose. The optical forbidden volume is constructed based on the parameter template. The points exceeding the preset threshold are clustered based on the point cloud to generate the high component volume. The optical forbidden volume and the high component volume are combined to obtain the global forbidden volume. The edge band volume is constructed at the edge of the circuit board and the difference operation is performed with the global forbidden volume to obtain the grabbable edge volume. S3. Sample candidate grab points within the grabbable edge volume, solve the joint posture based on inverse kinematics to generate the trajectory, perform collision detection on the trajectory and the global no-touch volume to remove candidate grab points with collision risk, and construct a comprehensive scoring function according to preset weights for the remaining candidate grab points to select the candidate grab point with the highest score as the target edge grab point and the target gripper posture. S4. Transform the target edge gripping point and target gripper posture from the camera coordinate system to the robot arm base coordinate system through the hand-eye calibration relationship to obtain the target gripping posture. Plan a collision-free trajectory under kinematic constraints and control the end gripper of the robot arm to complete the gripping and loading / unloading operations.

2. The method for edge gripping and unloading optical lens circuit boards according to claim 1, characterized in that, Before constructing the optically forbidden volume in S2, an outlier removal operation based on statistical filtering is performed on the point cloud of the circuit board region. The average distance between each point and its set of neighboring points is calculated. When the average distance exceeds a preset multiple of the sum of the global mean and the standard deviation, the point is marked as an outlier and removed. The plane fitting is then re-performed on the point cloud after removing outliers.

3. The method for edge gripping and unloading optical lens circuit boards according to claim 1, characterized in that, Before constructing the comprehensive scoring function in S3, the reachability of the gripper is checked for the candidate gripping points that pass the collision detection. The gripper posture of the candidate gripping point is substituted into the inverse kinematics solver of the robot arm. If the solver returns no solution or any joint angle exceeds the physical limit range of the joint, the candidate gripping point is marked as unreachable and is removed.

4. The method for edge gripping and unloading optical lens circuit boards according to claim 1, characterized in that, The generation of the optical forbidden volume includes constructing a circular, elliptical, or polygonal forbidden area on the circuit board plane with the center of the optical lens as the reference, and extending it upwards by a preset safety height margin in the height direction, stretching the two-dimensional forbidden area into a three-dimensional cylinder or polyhedron, and performing a union operation on the optical forbidden volumes corresponding to multiple optical lenses to obtain the total optical forbidden volume.

5. The method for edge gripping and unloading optical lens circuit boards according to claim 1, characterized in that, The generation of the high component volume includes calculating the vertical height of each point in the point cloud to the circuit board plane. When the height exceeds a preset component height threshold, it is marked as a high component point. The high component point cloud is clustered by connected components, and the three-dimensional bounding box or convex hull of each connected component is calculated as the high component volume.

6. The method for edge gripping and unloading optical lens circuit boards according to claim 1, characterized in that, The comprehensive scoring function includes the following normalized sub-item: effective contact length between the gripper and the edge of the circuit board. Distance between gripping point and circuit board center of gravity Minimum safe distance between the capture trajectory and the global no-touch volume Robotic arm joint posture margin The scores are then linearly weighted according to preset weights to obtain a comprehensive score.

7. The method for edge gripping and unloading optical lens circuit boards according to claim 1, characterized in that, After constructing the edge band volume in S2, edges that are not suitable for grabbing are ignored according to the grabbing priority in the parameter template, and then the edge band volume and the global forbidden volume are subjected to geometric difference operation.

8. The method for edge gripping and unloading optical lens circuit boards according to claim 1, characterized in that, Solving the pose transformation matrix in S1 includes mounting a calibration plate on the end effector of the robotic arm, setting the motion trajectory of the robotic arm, acquiring calibration images and corresponding robotic arm poses in multiple postures, and assuming the pose of the end effector of the robotic arm in the robotic arm base coordinate system is... The transformation from the camera coordinate system to the robot arm's end effector coordinate system is as follows: The transformation from the camera coordinate system to the robot arm's base coordinate system is as follows: For the first Group and No. Group calibration data meets The constraints are solved using the least squares method. Then by The pose transformation matrix between the camera coordinate system and the robot arm base coordinate system is obtained.

9. A system for edge gripping and unloading optical lens circuit boards, characterized in that, include: Image and depth data acquisition module, used to acquire RGB images, depth images and point cloud data; The deep learning recognition and model matching module is used to output the circuit board bounding box and category identifier using the object detection network and to complete the parameter template indexing. The 3D modeling and touch-free volume generation module is used to construct circuit board planes, optical touch-free volumes, high-component volumes, and global touch-free volumes based on point clouds and parameter templates. The grabbable edge volume generation module is used to obtain the grabbable edge volume by performing a geometric difference operation on the edge band volume and the global untouchable volume; The candidate grab point planning and comprehensive scoring module is used to generate candidate grab points within the grabbable edge volume and complete collision detection and multi-index scoring. The robotic arm trajectory planning and control module is used to plan a collision-free trajectory based on the target grasping posture and send it to the robotic arm controller to realize the edge grasping and loading / unloading operations of the circuit board.

Images