A laser blank sorting system and method based on machine vision

Abstract

The application discloses a laser blanking sorting system and method based on machine vision and belongs to the technical field of robots. The method comprises the following steps: performing coarse positioning on a plate; performing fine positioning on a cutting piece; performing abnormality detection on the cutting piece and skipping the grabbing of an abnormal cutting piece; and planning an overall path and completing grabbing. The application can realize three-dimensional accurate pose recognition of a workpiece, intelligent detection of an abnormal state, and fast and accurate recognition of a large-size workpiece, so that the automation level and the intelligent degree of a laser blanking production line are comprehensively improved.

Description

Technical Field

[0001] This invention relates to the field of robotics, and in particular to a laser-based material sorting system and method based on machine vision. Background Technology

[0002] Laser blanking technology, as an advanced sheet metal processing method, has gradually replaced some traditional die-stamping blanking techniques. It utilizes a high-power laser to cut parts of the desired shape from continuously fed sheet metal, offering advantages such as high flexibility, no need for molds, and short preparation cycles. However, after laser cutting, efficiently and accurately separating the parts from the scrap skeleton and stacking them has become a key factor restricting the automation level and production efficiency of the entire production line. Currently, this is mainly done manually, which has drawbacks such as poor safety, low efficiency, and high labor intensity. Furthermore, manual handling can easily damage parts, affecting product quality.

[0003] To address these issues, those skilled in the art have developed automated sorting methods using industrial robots. Early robotic sorting systems typically employed "teach-in programming" or "offline programming" modes. However, due to positional and angular deviations in the sheet metal during transport, the actual and theoretical positions of parts often differ, leading to frequent gripping failures. When the cut part is tilted, rigid gripping can cause the part to be forcibly dragged, resulting in sheet metal deformation or robot damage. Forcing the robot to grasp a fallen part also reduces overall efficiency. Furthermore, changes in production orders or part models require tedious reprogramming and path planning, resulting in insufficient flexibility and poor adaptability.

[0004] Chinese invention patent CN114529612A discloses a visual positioning and locating device and method for sorting large-size workpieces. The device includes a camera, an end effector, and a control system. Both the camera and the end effector are communicatively connected to the control system. The camera acquires at least one image containing the corner points of the steel plate and at least one image containing a portion of the steel plate's edge, and sends them to the control system. The control system compares the images with a preset steel plate nesting diagram to obtain the steel plate's position information and the corresponding position information of each workpiece on the steel plate. It also drives the end effector to pick up the workpiece based on the position information of each workpiece. However, this method is based on a 2D vision system for plate positioning and cannot obtain the plate's height information and three-dimensional orientation. In actual production, the plate may tilt due to unevenness in the plate chain, and the 2D camera's installation orientation cannot be completely perpendicular to the plate, all of which affect the positioning accuracy and reliability.

[0005] For large workpieces, image or point cloud stitching methods are generally used. For example, Chinese invention patent CN114289332A acquires multiple images from multiple camera points and stitches them together to obtain a panoramic view, thus determining the workpiece's position and orientation. However, these methods are computationally intensive and time-consuming, making it difficult to meet the strict cycle time requirements of automated production lines. Furthermore, the stitching process introduces cumulative errors, leading to a decrease in the overall workpiece pose accuracy. This loss of accuracy is unacceptable, especially for complex workpieces requiring high-precision positioning for accurate grasping. Summary of the Invention

[0006] This invention provides a laser blanking and sorting system and method based on machine vision. Its advantages are that it can realize the three-dimensional accurate pose recognition of workpieces and can quickly and accurately identify large-sized workpieces, thereby comprehensively improving the automation level and intelligence level of the laser blanking production line.

[0007] The technical solution of the present invention is as follows:

[0008] On one hand, the present invention provides a laser-based material sorting system based on machine vision, comprising:

[0009] A sheet metal processing unit, comprising a laser cutting machine, for cutting the sheet metal according to a workpiece nesting diagram;

[0010] The sorting robot has an actuator at its end for gripping and cutting parts, and a ground rail at its bottom for movement.

[0011] The vision system includes a 3D camera and multiple built-in 2D cameras for acquiring three-dimensional point cloud data and two-dimensional image data of the workpiece. The 3D camera is installed at the end of the sorting robot.

[0012] And a central control system, which is communicatively connected to the sheet metal processing unit, the sorting robot and the vision system.

[0013] In another aspect, the present invention provides a laser-based material sorting method based on machine vision, implemented using the laser-based material sorting system described above, comprising the following steps:

[0014] S1: Obtain the workpiece nesting diagram, parse the nesting diagram to obtain the dimensions of each cut part and its position in the nesting diagram;

[0015] S2: Rough positioning of the board material;

[0016] Obtain the transformation matrix between the sheet metal coordinate system and the robot base coordinate system, and determine the pose of each cut part in the robot base coordinate system based on the sheet metal edge shrinkage transformation matrix and the nesting diagram analysis results;

[0017] S3: Precision positioning of the cut parts;

[0018] The point cloud information of the cut part is acquired and a plane is fitted. The perspective distortion of the cut part image is corrected by fitting the plane to obtain the perspective distortion corrected image and perspective distortion correction parameters. The pose of the cut part in the robot base coordinate system is calculated based on the perspective distortion correction parameters. The sorting robot is controlled to grasp the cut part to achieve sorting.

[0019] Furthermore, step S1 also includes the following steps:

[0020] S11: Obtain actuator parameters and related parameters, including the transformation matrix between the robot base coordinate system and the laser cutting machine coordinate system. 2D texture camera hand-eye calibration matrix 2D texture camera internals 2D Depth Camera Internals Transformation matrix between 2D texture camera coordinate system and 2D depth camera coordinate system Camera shooting height The shooting height is The field of view of the camera visual overlap rate Maximum allowable tilt angle threshold Maximum allowable drop ratio threshold ;

[0021] S12: Analyze the nesting diagram to obtain the dimensions and pose of each cut part in the nesting diagram, and plan the cutting part gripping sequence;

[0022] S13: Based on the nesting diagram analysis results and actuator information, obtain the preset gripping points for each cutting part;

[0023] S14: The laser cutting machine completes the cutting and sends a three-point calibration conversion matrix to the central control system. Transformation matrix for edge shrinkage of sheet metal .

[0024] Furthermore, step S2 includes:

[0025] S21: Camera at shooting height Capable of capturing the position of at least one corner of the board, and determining the transformation matrix between the board coordinate system and the 2D texture camera coordinate system. ,according to , , The robot's end-effector pose is determined by the following formula:

[0026]

[0027] The robot is controlled to move to the pose and sends a data acquisition signal to the vision system, which then completes the data acquisition of the corner point position of the board.

[0028] S22: Camera at shooting height Capable of capturing at least one edge location of the board material, determining the transformation matrix between the board material coordinate system and the 2D texture camera coordinate system. ,according to , , The robot's end-effector pose is determined by the following formula:

[0029]

[0030] The robot is controlled to move to this position and a data acquisition signal is sent to the vision system. The vision system then completes the data acquisition of the edge position of the board material.

[0031] S23. Process the two-dimensional image acquired in step S21 to obtain the corner points of the board and Precise location of directional edge points on a 2D image and ;

[0032] S24. Perform plane fitting on the point cloud acquired in step S22 to obtain the plane equation:

[0033]

[0034] , , , These are the equation parameters;

[0035] S25. Process the two-dimensional image acquired in step S23 to obtain the board material. Precise location of directional edge points on a 2D image ;

[0036] S26. Perform plane fitting on the point cloud acquired in step S23 to obtain the plane equation:

[0037]

[0038] , , , These are the equation parameters;

[0039] S27. Based on the calculation results of steps S24-S26 and , To obtain the corner points of the board, Directional edge point, 3D coordinates of the directional edge point in the 2D texture camera coordinate system , and ;

[0040] S28. Based on the robot end-effector pose obtained in steps S21-S22 and To obtain the corner points of the board, Directional edge point, 3D coordinates of the directional edge point in the robot base coordinate system , and Then, the transformation matrix between the plate coordinate system and the robot base coordinate system is obtained. ;

[0041] S29. Based on the plate edge indentation transformation matrix Find the transformation matrix between the nesting diagram coordinate system and the robot base coordinate system. ;

[0042] S210. Based on the calculation results of S29 and the analysis results of the nesting diagram, the pose of each cutting part in the robot's base coordinate system is obtained.

[0043] Furthermore, step S3 includes:

[0044] S30: Determine whether each cut piece is a large-sized workpiece based on the camera's field of view, where a large-sized workpiece is defined as a workpiece whose size exceeds the field of view of a single camera.

[0045] Furthermore, step S3 also includes:

[0046] S31. Position the cutting parts sequentially according to the grasping order planned in step S12, and execute the corresponding process according to the type of cutting parts: if the cutting parts are not large-sized workpieces, execute S32-S36; if the cutting parts are large-sized workpieces and anomaly detection is required, execute S37-S312; if the cutting parts are large-sized workpieces and anomaly detection is not required, execute S313-S317.

[0047] S32. By camera at altitude The entire cut part can be captured in time, and the transformation matrix between the nesting diagram coordinate system and the 2D texture camera coordinate system can be determined based on the nesting diagram analysis results. ,according to , Determine the robot's end-effector pose, i.e.:

[0048]

[0049] The central control system guides the robot to this position and sends a data acquisition signal to the vision system, which then completes the data acquisition of the cutting part's position.

[0050] S33. Perform plane fitting on the point cloud of the cut piece position collected in step S32 to obtain the plane equation:

[0051]

[0052] , , , These are the equation parameters;

[0053] S34. Use the plane equation obtained in step S33 to perform perspective distortion correction on the image acquired in step S32 to obtain the perspective distortion corrected image;

[0054] S35. Based on the analysis results of the nesting diagram, , , , Obtain the search range of the cut parts on the two-dimensional image;

[0055] S36. Search for the cutting position within the search range obtained in step S35, and then based on the nesting drawing analysis results and perspective distortion correction parameters, , , Obtain the pose of the cut part in the robot's base coordinate system. ;

[0056] S37. Camera at altitude It is possible to capture the entire cut part in time, based on the analysis results of the nesting diagram. , Determine the minimum number of photos. The transformation matrix between the overlay coordinate system and the 2D texture camera coordinate system during each photo capture. ;according to , Determine the robot's end-effector pose for each photo taken, i.e.:

[0057]

[0058] The main control system controls the robot to move to each pose in sequence and sends acquisition signals to the vision system, which then completes the data acquisition of the position of the cut part.

[0059] S38. According to step S37 Nesting diagram analysis results , , , The pose with the highest regional score is selected as the optimal photo pose. The formula for the regional score is as follows:

[0060]

[0061] in Indicates the regional score. Indicates the average curvature of the region. Indicates the standard deviation of the curvature of the region. Indicates the number of feature points in the region. Indicates the number of region primitive types. , , , These are the weighting coefficients;

[0062] S39. Regarding step S37 The point cloud collected in this second acquisition is fitted to a plane to obtain the plane equation:

[0063]

[0064] , , , These are the equation parameters;

[0065] S310. Using the plane equation obtained in step S39, apply the equation obtained in step S37... The images acquired in the previous step were subjected to perspective distortion correction to obtain the perspective distortion corrected image;

[0066] S311. Based on the analysis results of the nesting diagram, , , , Obtain the search range of the local cut-out part on the two-dimensional image;

[0067] S312. Search for the location of the local cut part within the search range obtained in step S311, and then based on the nesting drawing analysis results and perspective distortion correction parameters, , , Obtain the pose of the cut part in the robot's base coordinate system. ;

[0068] S313. Based on the analysis results of the nesting diagram, Calculate an optimal shooting pose The optimal image pose is used for data collection and precise positioning. The selection is made by traversing the cutting part kit diagram. The main control system controls the robot to move to the position and sends a collection signal to the vision system. The vision system completes the data collection at that position.

[0069] S314. Perform planar processing on the point cloud acquired in step S313 to obtain the plane equation:

[0070]

[0071] , , , These are the equation parameters;

[0072] S315. Use the plane equation obtained in step S314 to perform perspective distortion correction on the image acquired in step S313 to obtain the perspective distortion corrected image.

[0073] S316. Based on the analysis results of the nesting diagram, , , , The search range of a local cut-off piece on a two-dimensional image can be obtained;

[0074] S317. Search for the location of the local cut part within the search range obtained in step S316, and then based on the nesting drawing analysis results and perspective distortion correction parameters, , , The pose of the cut part in the base coordinate system can be obtained. .

[0075] Furthermore, for the cut parts that require anomaly detection in step S3, step S4 is also included: performing anomaly detection on the cut parts;

[0076] S41. Perform anomaly detection on the cut parts sequentially according to the gripping order planned in step S12;

[0077] S42. Based on the analysis results of the nesting diagram, or , , , , , or Find the transformation matrix between the nesting map coordinate system and the 2D depth camera coordinate system. The 2D depth camera coordinate system is the point cloud coordinate system;

[0078] S43. The result obtained from step S42 If the part to be cut is not a large workpiece, align the nesting diagram with the point cloud; if the part to be cut is a large workpiece, align the nesting diagram with the spliced ​​point cloud.

[0079] S44. Project the nesting pattern outline onto the 2D depth camera plane using cone projection, with the following equation:

[0080]

[0081] in The depth value of a pixel. For 2D depth camera intrinsic parameters, = The corresponding cutting mask is generated using the nesting diagram information;

[0082] S45. Based on the cutting mask generated in step S44, extract the corresponding region from the point cloud and fit the plane equation:

[0083]

[0084] , , , These are the equation parameters;

[0085] Another point cloud surrounding this point cloud is extracted as a reference point cloud, and the equation of the reference plane is obtained by fitting the data.

[0086]

[0087] , , , These are the equation parameters;

[0088] The tilt angle of the cut piece can be calculated from this:

[0089]

[0090] S46. Based on the cutting mask generated in step S44, extract the corresponding region from the point cloud and calculate the number of effective point clouds. and the ideal number of point clouds Therefore, the proportion of the cut piece that fell off can be calculated:

[0091]

[0092] S47. The central control system will and Comparison: If If the condition is normal, the cutting part is considered normal; otherwise, it is considered abnormal. The central control system will... and Comparison: If If the condition is normal, the cut part is considered to be in normal condition; otherwise, it is considered abnormal.

[0093] S48. When an abnormal status is detected in the cutting part, the central control system skips the current cutting part grabbing.

[0094] Furthermore, it also includes step S5:

[0095] S51. Follow the grabbing sequence planned in step S12;

[0096] S52. For cut parts in normal condition, the central control system will calculate the values ​​in step S3. Combined with the pre-set grasping points in step S14, a series of joint angles required by the robot end effector are calculated using the robot inverse kinematics algorithm, thereby forming a collision-free and optimal grasping path from the waiting position to the grasping position and then to the placement position.

[0097] Furthermore, it also includes step S6: sorting and stacking the cut parts;

[0098] S61. The central control system sends the final planned path instructions to the robot;

[0099] S62. The robot moves according to instructions, the actuator grabs the workpiece, and places it precisely into the designated stacking position or downstream equipment.

[0100] In summary, the beneficial effects of the present invention are as follows:

[0101] 1. A machine vision-based laser blanking and sorting system architecture is proposed, forming a complete process from visual perception to anomaly detection, path planning, and stacking of cut parts.

[0102] 2. The fusion of 2D and 3D vision reduces the dependence of pure 2D vision on the vertical relationship between camera pose and board pose, and improves the accuracy and reliability of board positioning.

[0103] 3. Anomaly detection gives the system autonomous judgment. By judging the abnormal state of the cut parts tilting and falling, it not only improves the reliability and safety of production, but also improves the overall efficiency of the system.

[0104] 4. When processing large-sized workpieces, the adaptive photography strategy not only improves the recognition accuracy of the cut parts, but also speeds up the system's operation cycle and reduces the overall computational load, meeting the system's rapid response requirements. Attached Figure Description

[0105] Figure 1 This is a schematic diagram of the overall system in a specific embodiment of the present invention;

[0106] Figure 2 This is a structural diagram of the overall system in a specific embodiment of the present invention;

[0107] Figure 3 This is a flowchart of the overall system in one specific embodiment of the present invention;

[0108] Figure 4 This is a flowchart of the coarse positioning stage in a specific embodiment of the present invention;

[0109] Figure 5 This is a flowchart of the fine positioning stage in a specific embodiment of the present invention;

[0110] Figure 6 This is a flowchart of the anomaly detection stage in a specific embodiment of the present invention;

[0111] Figure 7 This is a schematic diagram illustrating the transformation between different coordinate systems in a specific embodiment of the present invention.

[0112] In the diagram, 1. Sorting robot; 2. Vision system; 3. Laser cutting machine; 4. Workpiece nesting diagram; 5. Ground rail; 6. Unloading pallet. Detailed Implementation

[0113] The specific embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0114] Example: This embodiment of the invention provides a laser-based material sorting method using machine vision, implemented through a laser material sorting system. This system mainly includes:

[0115] The central control system, acting as the system's brain, is typically an industrial computer integrating control software, including vision modules, nesting diagram analysis modules, path planning modules, collision detection modules, magnetization algorithm modules, and 3D offline simulation modules.

[0116] The vision system includes a 3D camera and multiple built-in 2D cameras for acquiring 3D point cloud data and 2D image data of the workpiece. The camera is mounted at the end of the sorting robot and may be equipped with a light source depending on the situation.

[0117] Sorting robots, such as six-axis industrial robots, have actuators at their ends, such as vacuum suction cups or magnetic grippers.

[0118] Sheet metal processing unit, including laser cutting machine, workpiece nesting diagram, etc.

[0119] Supporting equipment, such as floor rails and material unloading pallets.

[0120] The communication bus connects all the above components, enabling real-time interaction of data and instructions.

[0121] Figure 3 The following is a flowchart of the overall process of the laser blanking and sorting system. The steps of this method are explained in detail below:

[0122] S1. Initialize the system, the specific steps are as follows:

[0123] S11. Turn on the central control system;

[0124] S12. The central control system acquires the nesting diagram path, environment model, gripper information, and relevant parameters, including the transformation matrix between the robot's base coordinate system and the laser cutting machine's coordinate system. 2D texture camera hand-eye calibration matrix 2D texture camera internals 2D Depth Camera Internals Transformation matrix between 2D texture camera coordinate system and 2D depth camera coordinate system Camera shooting height The shooting height is The field of view of the camera visual overlap rate Maximum allowable tilt angle threshold Maximum allowable drop ratio threshold ;

[0125] S13. The nesting diagram parsing module parses the nesting diagram to obtain the size of each cut part and its pose in the nesting diagram. The vision module confirms whether each cut part is a large-sized workpiece based on the camera's field of view. A large-sized workpiece is defined as a workpiece whose size exceeds the field of view of a single camera. The path planning module plans the order of grabbing the cut parts.

[0126] S14. The magnetization algorithm module obtains the preset gripping point for each cutting part based on the nesting diagram analysis results and actuator information;

[0127] S15. The laser cutting machine completes the cutting and sends a three-point calibration conversion matrix to the central control system. Transformation matrix for edge shrinkage of sheet metal ;

[0128] S2. Perform rough positioning of the board material, such as... Figure 4 As shown, the specific steps are as follows:

[0129] S21. To ensure the camera is at an altitude By capturing the corner positions of the board material in real time, the transformation matrix between the board material coordinate system and the 2D texture camera coordinate system can be determined. ,according to , , Determine the robot's end-effector pose, i.e.:

[0130]

[0131] Between the robot's base coordinate system and the laser cutting machine's coordinate system

[0132] Three-point calibration transformation matrix

[0133] Between the sheet metal coordinate system and the 2D texture camera coordinate system

[0134] 2D texture camera hand-eye calibration matrix

[0135] The main control system controls the robot to move to this pose and sends a data acquisition signal to the vision system, which then completes the data acquisition of the corner position.

[0136] S22. Similar to step S21, to ensure the camera is at the altitude By capturing the edge position of the board material in real time, the transformation matrix between the board material coordinate system and the 2D texture camera coordinate system can be determined. ,according to , , Determine the robot's end-effector pose, i.e.:

[0137]

[0138] The main control system guides the robot to move to this pose and sends a data acquisition signal to the vision system, which then completes the data acquisition of the edge position.

[0139] S23. The vision module processes the two-dimensional image acquired in step S21 to obtain the corner points of the board material and... Precise location of directional edge points on a 2D image and ;

[0140] S24. The vision module performs plane fitting on the point cloud acquired in step S22 to obtain the plane equation:

[0141]

[0142] , , , These are the equation parameters;

[0143] S25. The vision module processes the two-dimensional image acquired in step S23 to obtain the board material. Precise location of directional edge points on a 2D image ;

[0144] S26. The vision module performs plane fitting on the point cloud acquired in step S23 to obtain the plane equation:

[0145]

[0146] , , , These are the equation parameters;

[0147] S27. Based on the calculation results of steps S24-S26 and , It can obtain the corner points of the board, Directional edge point, 3D coordinates of the directional edge point in the 2D texture camera coordinate system , and ;

[0148] S28. Based on the robot end-effector pose obtained in steps S21-S22 and It can obtain the corner points of the board, Directional edge point, 3D coordinates of the directional edge point in the robot base coordinate system , and Then, the transformation matrix between the plate coordinate system and the robot base coordinate system is obtained. ;

[0149] S29. Based on the plate edge indentation transformation matrix The transformation matrix between the nesting diagram coordinate system and the robot base coordinate system can be calculated. ;

[0150] S210. Based on the calculation results of S29 and the analysis results of the nesting diagram, the pose of each cutting part in the robot's base coordinate system can be obtained;

[0151] S3. Precisely position the cut parts, such as... Figure 5 As shown, the specific steps are as follows:

[0152] S31. Position the cutting parts sequentially according to the grasping order planned in step S13, and execute the corresponding process according to the type of cutting parts: if the cutting parts are not large-sized workpieces, execute S32-S36; if the cutting parts are large-sized workpieces and anomaly detection is required, execute S37-S312; if the cutting parts are large-sized workpieces and anomaly detection is not required, execute S313-S317.

[0153] S32. To ensure the camera is at a certain altitude The entire cut part was captured in real time, and based on the nesting diagram analysis results, the transformation matrix between the nesting diagram coordinate system and the 2D texture camera coordinate system can be determined. ,according to , Determine the robot's end-effector pose, i.e.:

[0154]

[0155] The central control system guides the robot to this position and sends a data acquisition signal to the vision system, which then completes the data acquisition of the cutting part's position.

[0156] S33. The vision module performs plane fitting on the point cloud of the cut piece position acquired in step S32 to obtain the plane equation:

[0157]

[0158] , , , These are the equation parameters;

[0159] S34. The vision module uses the plane equation obtained in step S33 to perform perspective distortion correction on the image acquired in step S32, and obtains the perspective distortion corrected image.

[0160] S35. The vision module analyzes the nesting diagram results... , , , The search range of the cut parts on the two-dimensional image can be obtained;

[0161] S36. The vision module searches for the cutting position within the search range obtained in step S35, and then, based on the nesting drawing analysis results and perspective distortion correction parameters, , , The pose of the cut part in the robot's base coordinate system can be obtained. ;

[0162] S37. To ensure the camera is at a certain altitude Take a full picture of the entire cut part, based on the analysis results of the nesting diagram. , This allows us to determine the minimum number of photos required. The transformation matrix between the overlay coordinate system and the 2D texture camera coordinate system during each photo capture. .according to , Determine the robot's end-effector pose for each photo taken, i.e.:

[0163]

[0164] The main control system controls the robot to move to each pose in sequence and sends acquisition signals to the vision system, which then completes the data acquisition of the position of the cut part.

[0165] S38. The vision module, according to step S37... Nesting diagram analysis results , , , Select the optimal shooting pose for precise positioning. The core objective of this optimal photo pose selection is to provide stable and reliable data for image matching. Specifically, it prioritizes regions with rich geometric features, such as high-curvature areas or complex contour areas like holes, grooves, and protrusions. Ideally, these regions should contain point, line, and surface features simultaneously, while avoiding the selection of featureless areas. The region scoring formula is as follows:

[0166]

[0167] in Indicates the regional score. Indicates the average curvature of the region. Indicates the standard deviation of the curvature of the region. Indicates the number of feature points in the region. Indicates the number of region primitive types. , , , These are the weighting coefficients;

[0168] S39. The vision module corresponds to step S37. The point cloud collected in this second acquisition is fitted to a plane to obtain the plane equation:

[0169]

[0170] , , , These are the equation parameters.

[0171] S310. The vision module uses the plane equation obtained in step S39 to apply the equation obtained in step S37. The images acquired in the previous step were subjected to perspective distortion correction to obtain the perspective distortion corrected image;

[0172] S311. The vision module analyzes the nesting diagram results... , , , The search range of a local cut-off piece on a two-dimensional image can be obtained;

[0173] S312. The vision module searches for the location of the local cut part within the search range obtained in step S311, and then, based on the nesting drawing analysis results and perspective distortion correction parameters, , , The pose of the cut part in the robot's base coordinate system can be obtained. ;

[0174] S313. The vision module analyzes the nesting diagram results... An optimal shooting pose can be calculated. The optimal image pose is used for data collection and precise positioning. The selection is performed by traversing the cutting part kit diagram. The selected target is the same as in step S38. The main control system controls the robot to move to the position and sends a collection signal to the vision system. The vision system completes the data collection at that position.

[0175] S314. The vision module performs planar processing on the point cloud acquired in step S313 to obtain the plane equation:

[0176]

[0177] , , , These are the equation parameters;

[0178] S315. The vision module uses the plane equation obtained in step S314 to perform perspective distortion correction on the image acquired in step S313, and obtains the perspective distortion corrected image.

[0179] S316. The vision module analyzes the nesting diagram results... , , , The search range of a local cut-off piece on a two-dimensional image can be obtained;

[0180] S317. The vision module searches for the location of the local cut part within the search range obtained in step S316, and then, based on the nesting drawing analysis results and perspective distortion correction parameters, , , The pose of the cut part in the base coordinate system can be obtained. ;

[0181] S4. Perform anomaly detection on the cut parts, such as... Figure 6 As shown, the specific steps are as follows:

[0182] S41. Perform anomaly detection on the cut parts sequentially according to the gripping order planned in step S13;

[0183] S42. The vision module analyzes the nesting diagram results... or , , , , , or The transformation matrix between the nesting map coordinate system and the 2D depth camera coordinate system can be calculated. The 2D depth camera coordinate system is the point cloud coordinate system;

[0184] S43. The result obtained from step S42 If the part to be cut is not a large workpiece, align the nesting diagram with the point cloud; if the part to be cut is a large workpiece, align the nesting diagram with the spliced ​​point cloud.

[0185] S44. Project the nesting pattern outline onto the 2D depth camera plane using cone projection, with the following equation:

[0186]

[0187] in The depth value of a pixel. For 2D depth camera intrinsic parameters, = The corresponding cutting mask can be generated using the nesting diagram information;

[0188] S45. Based on the cutting mask generated in step S44, extract the corresponding region from the point cloud and fit the plane equation:

[0189]

[0190] , , , These are the equation parameters;

[0191] Another point cloud surrounding this point cloud is extracted as a reference point cloud, and the equation of the reference plane is obtained by fitting the data.

[0192]

[0193] , , , These are the equation parameters;

[0194] The tilt angle of the cut piece can then be calculated:

[0195]

[0196] S46. Based on the cutting mask generated in step S44, extract the corresponding region from the point cloud and calculate the number of effective point clouds. and the ideal number of point clouds Therefore, the proportion of the cut piece that fell off can be calculated:

[0197]

[0198] S47. The central control system will and Comparison: If If the condition is normal, the cutting part is considered normal; otherwise, it is considered abnormal. The central control system will... and Comparison: If If the condition is normal, the cut part is considered to be in normal condition; otherwise, it is considered abnormal.

[0199] S48. When an abnormal status is detected in the cutting part, the central control system will skip the current cutting part grabbing;

[0200] S5. Plan the overall route, with the following specific steps:

[0201] S51. Follow the grabbing sequence planned in step S13;

[0202] S52. For cut parts in normal condition, the central control system will calculate the values ​​in step S3. Combined with the pre-set grasping points in step S14, a series of joint angles required by the robot end effector are calculated using the robot inverse kinematics algorithm, thereby forming a collision-free and optimal grasping path from the waiting position to the grasping position and then to the placement position.

[0203] S6. Sorting and stacking the cut parts, the specific steps are as follows:

[0204] S61. The central control system sends the final planned path instructions to the robot;

[0205] S62. The robot moves according to instructions, the actuator grabs the workpiece, and places it precisely into the designated stacking position or downstream equipment.

[0206] The above description is only a preferred embodiment of the present invention. It should be noted that those skilled in the art can make several modifications and improvements without departing from the inventive concept of the present invention, and these all fall within the protection scope of the present invention.

Claims

1. A laser-based material sorting system based on machine vision, characterized in that, include: A sheet metal processing unit, comprising a laser cutting machine, for cutting the sheet metal according to a workpiece nesting diagram; The sorting robot has an actuator at its end for gripping and cutting parts, and a ground rail at its bottom for movement. The vision system includes a 3D camera and multiple built-in 2D cameras for acquiring three-dimensional point cloud data and two-dimensional image data of the workpiece. The 3D camera is installed at the end of the sorting robot. And a central control system, which is communicatively connected to the sheet metal processing unit, the sorting robot and the vision system.

2. A laser-based material sorting method based on machine vision, characterized in that, The laser-assisted material sorting system as described in claim 1 is used to achieve this, comprising the following steps: S1: Obtain the workpiece nesting diagram, parse the nesting diagram to obtain the dimensions of each cut part and its position in the nesting diagram; S2: Rough positioning of the board material; Obtain the transformation matrix between the sheet metal coordinate system and the robot base coordinate system, and determine the pose of each cut part in the robot base coordinate system based on the sheet metal edge shrinkage transformation matrix and the nesting diagram analysis results; S3: Precision positioning of the cut parts; The point cloud information of the cut part is acquired and a plane is fitted. The perspective distortion of the cut part image is corrected by fitting the plane to obtain the perspective distortion corrected image and perspective distortion correction parameters. The pose of the cut part in the robot base coordinate system is calculated based on the perspective distortion correction parameters. The sorting robot is controlled to grasp the cut part to achieve sorting.

3. The laser-based material sorting method based on machine vision according to claim 2, characterized in that, Step S1 also includes the following steps: S11: Obtain actuator parameters and related parameters, including the transformation matrix between the robot base coordinate system and the laser cutting machine coordinate system. 2D texture camera hand-eye calibration matrix 2D texture camera internals 2D Depth Camera Internals Transformation matrix between 2D texture camera coordinate system and 2D depth camera coordinate system Camera shooting height The shooting height is The field of view of the camera visual overlap rate Maximum allowable tilt angle threshold Maximum allowable drop ratio threshold ; S12: Analyze the nesting diagram to obtain the dimensions and pose of each cut part in the nesting diagram, and plan the cutting part gripping sequence; S13: Based on the nesting diagram analysis results and actuator information, obtain the preset gripping points for each cutting part; S14: The laser cutting machine completes the cutting and sends a three-point calibration conversion matrix to the central control system. Transformation matrix for edge shrinkage of sheet metal .

4. The laser-based material sorting method based on machine vision according to claim 3, characterized in that, Step S2 includes: S21: Camera at shooting height Capable of capturing the position of at least one corner of the board, and determining the transformation matrix between the board coordinate system and the 2D texture camera coordinate system. ,according to , , The robot's end-effector pose is determined by the following formula: The robot is controlled to move to the pose and sends a data acquisition signal to the vision system, which then completes the data acquisition of the corner point position of the board. S22: Camera at shooting height Capable of capturing at least one edge location of the board material, determining the transformation matrix between the board material coordinate system and the 2D texture camera coordinate system. ,according to , , The robot's end-effector pose is determined by the following formula: The robot is controlled to move to this position and a data acquisition signal is sent to the vision system. The vision system then completes the data acquisition of the edge position of the board material. S23. Process the two-dimensional image acquired in step S21 to obtain the corner points of the board and Precise location of directional edge points on a 2D image and ; S24. Perform plane fitting on the point cloud acquired in step S22 to obtain the plane equation: , , , These are the equation parameters; S25. Process the two-dimensional image acquired in step S23 to obtain the board material. Precise location of directional edge points on a 2D image ; S26. Perform plane fitting on the point cloud acquired in step S23 to obtain the plane equation: , , , These are the equation parameters; S27. Based on the calculation results of steps S24-S26 and , To obtain the corner points of the board, Directional edge point, 3D coordinates of the directional edge point in the 2D texture camera coordinate system , and ; S28. Based on the robot end-effector pose obtained in steps S21-S22 and To obtain the corner points of the board, Directional edge point, 3D coordinates of the directional edge point in the robot base coordinate system , and Then, the transformation matrix between the plate coordinate system and the robot base coordinate system is obtained. ; S29. Based on the plate edge indentation transformation matrix Find the transformation matrix between the nesting diagram coordinate system and the robot base coordinate system. ; S210. Based on the calculation results of S29 and the analysis results of the nesting diagram, the pose of each cutting part in the robot's base coordinate system is obtained.

5. The laser-based material sorting method based on machine vision according to claim 4, characterized in that, Step S3 includes: S30: Determine whether each cut piece is a large-sized workpiece based on the camera's field of view, where a large-sized workpiece is defined as a workpiece whose size exceeds the field of view of a single camera.

6. The laser-based material sorting method based on machine vision according to claim 5, characterized in that, Step S3 also includes: S31. Position the cutting parts sequentially according to the grasping order planned in step S12, and execute the corresponding process according to the type of cutting parts: if the cutting parts are not large-sized workpieces, execute S32-S36; if the cutting parts are large-sized workpieces and anomaly detection is required, execute S37-S312; if the cutting parts are large-sized workpieces and anomaly detection is not required, execute S313-S317. S32. By camera at altitude The entire cut part can be captured in time, and the transformation matrix between the nesting diagram coordinate system and the 2D texture camera coordinate system can be determined based on the nesting diagram analysis results. ,according to , Determine the robot's end-effector pose, i.e.: The central control system guides the robot to this position and sends a data acquisition signal to the vision system, which then completes the data acquisition of the cutting part's position. S33. Perform plane fitting on the point cloud of the cut piece position collected in step S32 to obtain the plane equation: , , , These are the equation parameters; S34. Use the plane equation obtained in step S33 to perform perspective distortion correction on the image acquired in step S32 to obtain the perspective distortion corrected image; S35. Based on the analysis results of the nesting diagram, , , , Obtain the search range of the cut parts on the two-dimensional image; S36. Search for the cutting position within the search range obtained in step S35, and then based on the nesting drawing analysis results and perspective distortion correction parameters, , , Obtain the pose of the cut part in the robot's base coordinate system. ; S37. Camera at altitude It is possible to capture the entire cut part in time, based on the analysis results of the nesting diagram. , Determine the minimum number of photos. The transformation matrix between the overlay coordinate system and the 2D texture camera coordinate system during each photo capture. ;according to , Determine the robot's end-effector pose for each photo taken, i.e.: The main control system controls the robot to move to each pose in sequence and sends acquisition signals to the vision system, which then completes the data acquisition of the position of the cut part. S38. According to step S37 Nesting diagram analysis results , , , The pose with the highest regional score is selected as the optimal photo pose. The formula for the regional score is as follows: in Indicates the regional score. Indicates the average curvature of the region. Indicates the standard deviation of the curvature of the region. Indicates the number of feature points in the region. Indicates the number of region primitive types. , , , These are the weighting coefficients; S39. Regarding step S37 The point cloud collected in this second acquisition is fitted to a plane to obtain the plane equation: , , , These are the equation parameters; S310. Using the plane equation obtained in step S39, apply the equation obtained in step S37... The images acquired in the previous step were subjected to perspective distortion correction to obtain the perspective distortion corrected image; S311. Based on the analysis results of the nesting diagram, , , , Obtain the search range of the local cut-out part on the two-dimensional image; S312. Search for the location of the local cut part within the search range obtained in step S311, and then based on the nesting drawing analysis results and perspective distortion correction parameters, , , Obtain the pose of the cut part in the robot's base coordinate system. ; S313. Based on the analysis results of the nesting diagram, Calculate an optimal shooting pose The optimal image pose is used for data collection and precise positioning. The selection is made by traversing the cutting part kit diagram. The main control system controls the robot to move to the position and sends a collection signal to the vision system. The vision system completes the data collection at that position. S314. Perform planar processing on the point cloud acquired in step S313 to obtain the plane equation: , , , These are the equation parameters; S315. Use the plane equation obtained in step S314 to perform perspective distortion correction on the image acquired in step S313 to obtain the perspective distortion corrected image. S316. Based on the analysis results of the nesting diagram, , , , The search range of a local cut-off piece on a two-dimensional image can be obtained; S317. Search for the location of the local cut part within the search range obtained in step S316, and then based on the nesting drawing analysis results and perspective distortion correction parameters, , , The pose of the cut part in the base coordinate system can be obtained. .

7. The laser-based material sorting method based on machine vision according to claim 6, characterized in that, For the cut parts that require anomaly detection in step S3, step S4 is also included: the cut parts undergo anomaly detection. S41. Perform anomaly detection on the cut parts sequentially according to the gripping order planned in step S12; S42. Based on the analysis results of the nesting diagram, or , , , , , or Find the transformation matrix between the nesting map coordinate system and the 2D depth camera coordinate system. The 2D depth camera coordinate system is the point cloud coordinate system; S43. The result obtained from step S42 If the part to be cut is not a large workpiece, then align the nesting diagram with the point cloud; If the part to be cut is a large workpiece, align the nesting diagram with the stitched point cloud; S44. Project the nesting pattern outline onto the 2D depth camera plane using cone projection, with the following equation: in The depth value of a pixel. For 2D depth camera intrinsic parameters, = The corresponding cutting mask is generated using the nesting diagram information; S45. Based on the cutting mask generated in step S44, extract the corresponding region from the point cloud and fit the plane equation: , , , These are the equation parameters; Another point cloud surrounding this point cloud is extracted as a reference point cloud, and the equation of the reference plane is obtained by fitting the data. , , , These are the equation parameters; The tilt angle of the cut piece can be calculated from this: S46. Based on the cutting mask generated in step S44, extract the corresponding region from the point cloud and calculate the number of effective point clouds. and the ideal number of point clouds Therefore, the proportion of the cut piece that fell off can be calculated: S47. The central control system will and Comparison: If If the condition is normal, the cutting part is considered normal; otherwise, it is considered abnormal. The central control system will... and Comparison: If If the condition is normal, the cut part is considered to be in normal condition; otherwise, it is considered abnormal. S48. When an abnormal status is detected in the cutting part, the central control system skips the current cutting part grabbing.

8. The laser-based material sorting method based on machine vision according to claim 2, characterized in that, It also includes step S5: S51. Follow the grabbing sequence planned in step S12; S52. For cut parts in normal condition, the central control system will calculate the values ​​in step S3. Combined with the pre-set grasping points in step S14, a series of joint angles required by the robot end effector are calculated using the robot inverse kinematics algorithm, thereby forming a collision-free and optimal grasping path from the waiting position to the grasping position and then to the placement position.

9. The laser-based material sorting method based on machine vision according to claim 2, characterized in that, It also includes step S6: sorting and stacking the cut parts; S61. The central control system sends the final planned path instructions to the robot; S62. The robot moves according to instructions, the actuator grabs the workpiece, and places it precisely into the designated stacking position or downstream equipment.

Images