Workpiece sorting system based on 2D vision and control method and control device thereof

Abstract

The present invention discloses a workpiece sorting system based on 2D vision and a control method and a control device thereof. The workpiece sorting system based on 2D vision includes: a production line and a conveyor line; a ground rail robot, which is used to grab steel plate cutting parts on the production line; a two-dimensional camera, which is arranged on the ground rail robot and is used to collect two-dimensional images of steel plate cutting parts on the production line, wherein when the ground rail robot grabs the steel plate cutting parts on the production line, the field of view of the two-dimensional camera is projected onto the steel plate cutting parts on the production line; and a control device, which is connected to the two-dimensional camera and the ground rail robot and is used to obtain a two-dimensional image, and control the ground rail robot to grab the steel plate cutting parts on the production line according to the two-dimensional image. By adopting the workpiece sorting system based on 2D vision, the cost can be reduced, the grabbing range can be increased, and the sorting efficiency can be improved.

Description

Technical Field

[0001] The present invention relates to the technical field of machine vision, and in particular to a workpiece sorting system based on 2D vision and a control method and a control device thereof. Background Art

[0002] With the development of science and technology, the level of automation in industrial production is also constantly improving. In the production of workpieces, it is usually necessary to sort the cut workpieces from the steel plates. For this purpose, three-dimensional vision is usually used to realize the recognition and grasping of the workpieces. Specifically, the robot is fixed on the robot base, and the three-dimensional camera is fixedly installed on a bracket independent of the robot. After the three-dimensional camera takes a picture of the steel plate, the image obtained by the photo is processed to identify the workpiece to be grasped, and after obtaining the identified grasping point, it is converted to the robot coordinate system for grasping. However, the above-mentioned method of using a three-dimensional camera to realize workpiece recognition and grasping is costly and has low sorting efficiency. In addition, in the above-mentioned method, multiple robots need to be set up on the entire production line and the robot positions are relatively fixed, and the robot grasping range is small. Summary of the invention

[0003] The present invention aims to solve at least one of the technical problems existing in the prior art. To this end, one purpose of the present invention is to propose a workpiece sorting system based on 2D vision, by adopting the workpiece sorting system based on 2D vision, the cost can be reduced, the grasping range can be increased, and the sorting efficiency can be improved.

[0004] A second objective of the present invention is to provide a control method for a workpiece sorting system based on 2D vision.

[0005] A third objective of the present invention is to provide a control device for a workpiece sorting system based on 2D vision.

[0006] A fourth objective of the present invention is to provide a computer storage medium.

[0007] In order to solve the above problems, the first aspect of the present invention provides a workpiece sorting system based on 2D vision, including: a production line and a conveyor line, the production line is used to produce steel plate cutting parts, and the conveyor line is used to convey steel plate cutting parts; a ground rail robot, the ground rail robot includes a ground rail component, a robot and an end picker, the ground rail component is arranged between the production line and the conveyor line, the robot is movably arranged on the ground rail component, the end picker is connected to the robot, and the ground rail robot is used to grab the steel plate cutting parts on the production line; a two-dimensional camera, the two-dimensional camera is arranged on the ground rail robot, and is used to collect two-dimensional images of the steel plate cutting parts on the production line, wherein, when the ground rail robot grabs the steel plate cutting parts on the production line, the field of view of the two-dimensional camera is projected onto the steel plate cutting parts on the production line; a control device, the control device is connected to the two-dimensional camera and the ground rail robot, and is used to obtain the two-dimensional image, and control the ground rail robot to grab the steel plate cutting parts on the production line according to the two-dimensional image.

[0008] According to the 2D vision-based workpiece sorting system of the embodiment of the present invention, a ground rail robot is adopted. Compared with the fixed setting of the robot, when the ground rail robot is working, the ground rail component can drive the robot to move, thereby expanding the activity range of the ground rail robot and improving the grasping range of the robot. In addition, a two-dimensional camera is used to collect two-dimensional images of steel plate cutting parts on the production line. Compared with the use of a three-dimensional camera, it can reduce costs, reduce collection time, and improve sorting efficiency.

[0009] In some embodiments, the end picker includes: an end picker body; a bracket, the bracket is connected to the end picker body, and the two-dimensional camera is arranged on the bracket; wherein, when the end picker body grabs the steel plate cutting piece on the production line, the field of view of the two-dimensional camera is projected onto the steel plate cutting piece.

[0010] In some embodiments, the 2D vision-based workpiece sorting system further includes: an annular light source, which is disposed on the bracket and surrounds the two-dimensional camera.

[0011] A second aspect of the present invention provides a control method for a 2D vision-based workpiece sorting system, which is used for the 2D vision-based workpiece sorting system provided in the above embodiment, and the control method includes: acquiring a two-dimensional image of steel plate cutting parts on a production line in the 2D vision-based workpiece sorting system; preprocessing the two-dimensional image to obtain a preprocessed image; obtaining an actual homography matrix based on the preprocessed image and a preset template image; obtaining an initial image to be matched based on the preprocessed image, the actual homography matrix and a calibrated homography matrix; determining a target grasping point of a part to be grasped in the initial image to be matched based on the initial image to be matched and the preset template image; and controlling the ground rail robot of the 2D vision-based workpiece sorting system to grasp the part to be grasped according to the target grasping point.

[0012] According to the control method of the workpiece sorting system based on 2D vision in the embodiment of the present invention, after obtaining the two-dimensional image of the steel plate cutting part on the production line in the workpiece sorting system based on 2D vision and preprocessing it, the actual homography matrix is ​​obtained according to the preprocessed image, and the initial image to be matched is obtained by the preprocessed image, the actual homography matrix and the calibrated homography matrix. That is to say, during the normal operation of the workpiece sorting system based on 2D vision, the homography matrix of converting the preprocessed image into the initial image to be matched, that is, the actual homography matrix, can be calculated in real time. When the height of the steel plate cutting part is fixed, the initial image to be matched is obtained by the calibrated homography matrix, and when the height of the steel plate cutting part changes, the initial image to be matched is obtained by the actual homography matrix. Therefore, without using a three-dimensional camera, the workpiece sorting system based on 2D vision can effectively meet the accuracy requirements caused by the change in the height of the steel plate cutting part by using a two-dimensional camera, thereby reducing costs and improving sorting efficiency.

[0013] In some embodiments, an initial image to be matched is obtained according to the preprocessed image, the actual homography matrix and the calibrated homography matrix, including: calculating a first matrix modulus according to the actual homography matrix, and calculating a second matrix modulus according to the calibrated homography matrix; determining that the difference between the first matrix modulus and the second matrix modulus is less than or equal to a preset difference, then obtaining the initial image to be matched according to the preprocessed image and the calibrated homography matrix; determining that the difference between the first matrix modulus and the second matrix modulus is greater than the preset difference, then obtaining the initial image to be matched according to the preprocessed image and the actual homography matrix.

[0014] In some embodiments, the preset template image includes a local template image of a steel plate and a template image of a part to be grasped, and a target grasping point of the part to be grasped in the initial image to be matched is determined according to the initial image to be matched and the preset template image, including: globally matching the initial image to be matched with the local template image of the steel plate to determine the distribution position of each part in the initial image to be matched; locally matching the distribution position of each part with the template image of the part to be grasped to determine the target grasping point of the part to be grasped in the initial image to be matched.

[0015] In some embodiments, controlling the rail robot of the 2D vision-based workpiece sorting system to grab the part to be grabbed according to the target grabbing point includes: acquiring a coordinate conversion matrix; converting the target grabbing point from the image coordinate system of the two-dimensional camera of the 2D vision-based workpiece sorting system to an actual grabbing point in the robot coordinate system of the rail robot according to the coordinate conversion matrix; and controlling the rail robot of the 2D vision-based workpiece sorting system to grab the part to be grabbed according to the actual grabbing point.

[0016] In some embodiments, the two-dimensional image is preprocessed to obtain a preprocessed image, including: distorting the two-dimensional image according to a distortion correction method to obtain a distorted two-dimensional image; performing edge detection on the distorted two-dimensional image according to an edge feature extraction method to obtain an edge feature image; and enhancing the distorted two-dimensional image according to the edge feature image to obtain the preprocessed image.

[0017] According to a third aspect of the present invention, there is provided a control device for a 2D vision-based workpiece sorting system, comprising: at least one processor; and a memory communicatively connected to at least one of the processors; wherein the memory stores a computer program executable by at least one of the processors, and when at least one of the processors executes the computer program, the control method for the 2D vision-based workpiece sorting system described in the above embodiment is implemented.

[0018] According to the control device for the workpiece sorting system based on 2D vision according to the embodiment of the present invention, the control method of the workpiece sorting system based on 2D vision provided by the above embodiment is executed by the processor. Without the need for a three-dimensional camera, the workpiece sorting system based on 2D vision can effectively meet the accuracy requirements caused by changes in the height of steel plate cutting parts by using a two-dimensional camera, thereby reducing costs and improving sorting efficiency.

[0019] A fourth aspect of the present invention provides a computer storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the control method of the 2D vision-based workpiece sorting system described in the above embodiment.

[0020] Additional aspects and advantages of the present invention will be given in part in the following description and in part will be obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The above and / or additional aspects and advantages of the present invention will become apparent and easily understood from the description of the embodiments in conjunction with the following drawings, in which:

[0022] Figure 1 is a schematic diagram of a workpiece sorting system based on 2D vision according to an embodiment of the present invention;

[0023] Figure 2 is a schematic diagram of a robot according to an embodiment of the present invention;

[0024] Figure 3 is a schematic diagram of an end picker according to an embodiment of the present invention;

[0025] Figure 4 is a flow chart of a control method of a workpiece sorting system based on 2D vision according to an embodiment of the present invention;

[0026] Figure 5 is a workflow diagram of a 2D vision-based workpiece sorting system according to an embodiment of the present invention;

[0027] Figure 6 4 is a structural block diagram of a control device for a 2D vision-based workpiece sorting system according to an embodiment of the present invention.

[0028] Reference numerals:

[0029] 2D vision-based workpiece sorting system 100;

[0030] Production line 10; conveyor line 20; ground rail robot 30; two-dimensional camera 40; control device 50;

[0031] Ground rail assembly 1; robot 2; end picker 3; end picker body 4; bracket 5; annular light source 6;

[0032] Processor 7; Memory 8. DETAILED DESCRIPTION

[0033] Embodiments of the present invention are described in detail below. The embodiments described with reference to the accompanying drawings are exemplary. Embodiments of the present invention are described in detail below.

[0034] In order to solve the above problems, a first embodiment of the present invention proposes a workpiece sorting system based on 2D vision. By adopting the workpiece sorting system based on 2D vision, the cost can be reduced, the grasping range can be increased, and the sorting efficiency can be improved.

[0035] Reference below Figure 1-Figure 3 A 2D vision-based workpiece sorting system according to an embodiment of the present invention is described.

[0036] like Figure 1 As shown, the 2D vision-based workpiece sorting system 100 includes a production line 10, a conveyor line 20, a ground rail robot 30, a two-dimensional camera 40 and a control device 50 (not shown in the figure).

[0037] The production line 10 is used to produce steel plate cutting parts, and the conveyor line 20 is used to convey the steel plate cutting parts; the ground rail robot 30 includes a ground rail component 1, a robot 2 and an end picker 3, the ground rail component 1 is arranged between the production line 10 and the conveyor line 20, the robot 2 is movably arranged on the ground rail component 1, the end picker 3 is connected to the robot 2, and the ground rail robot 30 is used to grab the steel plate cutting parts on the production line 10; Figure 2 As shown, the two-dimensional camera 40 is arranged on the ground rail robot 30, and is used to collect the two-dimensional image of the steel plate cutting piece on the production line 10, wherein when the ground rail robot 30 grabs the steel plate cutting piece on the production line 10, the field of view of the two-dimensional camera 40 is projected onto the steel plate cutting piece on the production line 10; the control device 50 is connected to the two-dimensional camera 40 and the ground rail robot 30, and is used to obtain the two-dimensional image, and control the ground rail robot 30 to grab the steel plate cutting piece on the production line 10 according to the two-dimensional image, so as to place the grabbed steel plate cutting piece on the conveyor line 20.

[0038] In the embodiment, since the position of the robot is fixed in the existing visual sorting system, the robot's grasping range is small when sorting steel plate cutting parts, and multiple robots need to be set up on the same production line to meet the sorting requirements. In this regard, the embodiment of the present invention adopts a ground rail robot 30, that is, the robot 2 is movably set on the ground rail assembly 1, so that when the ground rail robot 30 is working, the ground rail assembly 1 can drive the robot 2 to move, thereby expanding the activity range of the ground rail robot 30 and improving the grasping range of the robot 2. At the same time, for the same production line, the number of ground rail robots 30 used can also be appropriately reduced.

[0039] Furthermore, since three-dimensional cameras are used in existing visual sorting systems to realize workpiece recognition and grasping, but the use cost of three-dimensional cameras is relatively high, and the camera field of view is small and the acquisition time is relatively long, in this regard, a two-dimensional camera 40 is used in the embodiment of the present invention to acquire a two-dimensional image of the steel plate cutting parts on the production line 10. Compared with the method of using a three-dimensional camera, the cost can be reduced, and for steel plate cutting parts of the same height, the field of view of the two-dimensional camera 40 is significantly larger than that of the three-dimensional camera. Therefore, the workpiece sorting system 100 based on 2D vision in the embodiment of the present invention adopts the two-dimensional camera 40, which can effectively increase the imaging field of view, which is beneficial to improving the automatic grasping efficiency of the workpiece sorting system 100 based on 2D vision, and because the shooting time of the two-dimensional camera 40 is also significantly faster than that of the three-dimensional camera, therefore, the workpiece sorting system 100 based on 2D vision in the embodiment of the present invention adopts the two-dimensional camera 40, which can also effectively improve the image acquisition time, improve the efficiency of identifying and grasping the grasping points of the steel plate cutting parts, and improve the sorting efficiency.

[0040] In addition, in the embodiment of the present invention, the two-dimensional camera 40 is set on the rail robot 30, so there is no need to separately set up a mounting bracket for the two-dimensional camera 40, which is beneficial to saving costs. In addition, the two-dimensional camera 40 can move with the movement of the rail robot 30, which is beneficial to improving the flexibility of the two-dimensional camera 40 in capturing images.

[0041] In some embodiments, the 2D vision-based workpiece sorting system 100 may include multiple ground rail robots 30 according to actual needs, without limitation.

[0042] According to the 2D vision-based workpiece sorting system 100 of the embodiment of the present invention, by adopting the ground rail robot 30, compared with the fixed setting of the robot, when the ground rail robot 30 is working, the ground rail component 1 can drive the robot 2 to move, thereby expanding the activity range of the ground rail robot 30 and improving the grasping range of the robot 2. In addition, the two-dimensional camera 40 is used to collect the two-dimensional image of the steel plate cutting parts on the production line 10. Compared with the use of a three-dimensional camera, it can reduce costs, reduce collection time, and improve sorting efficiency.

[0043] In some embodiments, Figure 3 As shown, the end tool 3 includes an end tool body 4 and a bracket 5 .

[0044] The bracket 5 is connected to the end picker body 4, the two-dimensional camera 40 is arranged on the bracket 5, and when the end picker body 4 grabs the steel plate cutting part on the production line 10, the field of view of the two-dimensional camera 40 is projected onto the steel plate cutting part.

[0045] Among them, there is no restriction on the connection method between the bracket 5 and the end picker body 4. For example, the bracket 5 can be fixedly connected to the end picker body 4, so that it can play a fixing role when the two-dimensional camera 40 is installed on the bracket 5, thereby improving the stability of the two-dimensional camera 40 in capturing two-dimensional images. Alternatively, the bracket 5 and the end picker body 4 are detachably connected to facilitate later maintenance.

[0046] In some embodiments, Figure 3 As shown, the 2D vision-based workpiece sorting system 100 further includes an annular light source 6 , which is disposed on the bracket 5 and surrounds the two-dimensional camera 40 , so as to provide a lighting effect when the two-dimensional camera 40 captures a two-dimensional image.

[0047] Based on the 2D vision-based workpiece sorting system provided in the above embodiment, a second aspect of the present invention provides a control method for the 2D vision-based workpiece sorting system, such as Figure 4 As shown, the control method at least includes steps S1 to S6.

[0048] Step S1, obtaining a two-dimensional image of a steel plate cut part on a production line in a workpiece sorting system based on 2D vision.

[0049] The application scenario of the workpiece sorting system based on 2D vision is as follows: in the heavy industry, a whole steel plate is placed on the production line. After the steel plate is automatically cut by the cutting machine, various parts of different shapes and sizes will be generated. All parts are kept in the steel plate before sorting, wherein the steel plate is regarded as a whole steel plate cut piece after cutting and before sorting. Then, under the condition of knowing the nesting diagram of the steel plate cut piece, the different parts in the steel plate cut piece are accurately located and identified, thereby guiding the ground rail robot to perform precise grasping to complete the sorting work.

[0050] Specifically, refer to Figure 1 As shown, when the workpiece sorting system based on 2D vision sorts steel plate cutting parts, the two-dimensional camera is set on the ground rail robot. Therefore, based on the placement range of the parts to be grabbed on the production line determined according to the nesting diagram of the overall steel plate cutting parts, the ground rail robot is controlled to move to the designated position, i.e., the photo point of the two-dimensional camera, and the two-dimensional camera is controlled to start to collect the two-dimensional image of the steel plate cutting parts within the camera field of view at this position, and the collected two-dimensional image is sent to the control device of the workpiece sorting system based on 2D vision. It can be understood that the two-dimensional image is obtained based on the field of view of the two-dimensional camera, so the two-dimensional image of the steel plate cutting parts collected by the two-dimensional camera is an image of a local area of ​​the overall steel plate cutting parts.

[0051] Step S2, preprocessing the two-dimensional image to obtain a preprocessed image.

[0052] Specifically, since there may be some interference in the two-dimensional image captured by the two-dimensional camera, the two-dimensional image needs to be preprocessed to remove interference factors, avoid interference in the subsequent identification and positioning of the parts to be grasped, and improve the accuracy of grasping the parts to be grasped.

[0053] Step S3, obtaining an actual homography matrix according to the preprocessed image and the preset template image.

[0054] Among them, homography refers to the transformation from one point in an image to the corresponding point in another image. For a workpiece sorting system based on 2D vision, any two images in the same plane in space are associated through homography. Based on this, when accurately locating and identifying different parts in steel plate cutting parts, the embodiment of the present invention uses the homography matrix to transform the pre-processed image so that the converted pre-processed image is more in line with the actual scene, thereby facilitating the accurate grasping of the parts to be grasped.

[0055] Among them, the preset template image is a nesting diagram of various steel plate cutting parts pre-stored in a database. For example, the preset template image at least includes an overall steel plate cutting part nesting diagram, a plurality of local steel plate cutting part nesting diagrams intercepted from the overall steel plate cutting part nesting diagram according to the field of view of the two-dimensional camera, namely, a local template diagram of the steel plate and a template diagram of each part in the steel plate cutting part.

[0056] Specifically, feature points are extracted from the preprocessed image and the preset template image respectively, such as feature point extraction algorithms such as SIFT, SURF, and ORB, and feature points extracted from the preprocessed image are matched with feature points extracted from the preset template image to obtain a set of matching point pairs, and then the matching point set can be fitted with algorithms such as the random sampling consensus algorithm or the RANSAC algorithm to obtain the actual homography matrix. Therefore, each time a part is grasped, the homography matrix used to convert the preprocessed image is calculated in real time based on the acquired preprocessed image to improve the accuracy of subsequent grasping of the parts to be grasped.

[0057] Step S4, obtaining an initial image to be matched according to the preprocessed image, the actual homography matrix and the calibrated homography matrix.

[0058] Among them, the calibrated homography matrix is ​​the homography matrix set when the distance between the steel plate cutting part and the two-dimensional camera, that is, the height of the steel plate cutting part is constant. That is to say, the distance between the steel plate cutting part and the two-dimensional camera is pre-set in the workpiece sorting system based on 2D vision, so that the height of the steel plate cutting part is fixed, and the homography matrix for converting the preprocessed image at the height is calculated through calibration, that is, the calibrated homography matrix is ​​obtained, which is used as the homography matrix used when the height of the steel plate on the production line is normal.

[0059] The initial image to be matched is an image obtained by converting the preprocessed image according to the actual homography matrix or the calibrated homography matrix. The positions of various parts in the steel plate cutting part can be identified through the initial image to be matched.

[0060] Specifically, in actual industrial production, since the distance between the steel plate cutting part and the two-dimensional camera will change, the existing visual sorting system solves the above problem by adopting a three-dimensional camera. However, the use cost of the three-dimensional camera is high and the efficiency is low. Therefore, in the embodiment of the present invention, a two-dimensional camera is used to realize the recognition and positioning of the parts to reduce the cost and improve the sorting efficiency. At the same time, for the situation where the height of the steel plate cutting part changes, in the embodiment of the present invention, the preset processing image can be converted by the actual homography matrix or the calibrated homography matrix, that is, when the height of the steel plate cutting part is fixed, the initial image to be matched is obtained by the calibrated homography matrix, and when the height of the steel plate cutting part changes, the calibrated homography matrix is ​​no longer used to convert the pre-processed image, but the actual homography matrix calculated in real time is used to obtain the initial image to be matched, so as to adapt to the situation where the height of the steel plate cutting part changes. Therefore, without the need to use a three-dimensional camera, the workpiece sorting system based on 2D vision can effectively meet the accuracy requirements caused by the change in the height of the steel plate cutting part using a two-dimensional camera, thereby improving the accuracy of subsequent grasping of the parts to be grasped.

[0061] Step S5, determining a target grasping point of the part to be grasped in the initial image to be matched according to the initial image to be matched and the preset template image.

[0062] Specifically, since the initial image to be matched is obtained based on a two-dimensional image captured by a two-dimensional camera, and the two-dimensional image is an image of a local area of ​​the overall steel plate cutting part, the initial image to be matched contains images of all parts in the local area. Therefore, in order to accurately locate the parts to be grasped, the initial image to be matched is matched with the preset template image to determine the target grasping point of the parts to be grasped in the initial image to be matched.

[0063] Step S6, controlling the ground rail robot of the 2D vision-based workpiece sorting system to grab the parts to be grabbed according to the target grabbing point.

[0064] Specifically, after determining the target grasping point of the part to be grasped on the steel plate cutting piece in the initial image to be matched, the ground rail robot can be controlled to move to the corresponding position to accurately grasp the part to be grasped, and the grasped steel plate cutting piece can be placed on the conveyor line, thereby realizing the sorting function of the workpiece sorting system based on 2D vision.

[0065] According to the control method of the workpiece sorting system based on 2D vision in the embodiment of the present invention, after obtaining the two-dimensional image of the steel plate cutting part on the production line in the workpiece sorting system based on 2D vision and preprocessing it, the actual homography matrix is ​​obtained according to the preprocessed image, and the initial image to be matched is obtained by the preprocessed image, the actual homography matrix and the calibrated homography matrix. That is to say, during the normal operation of the workpiece sorting system based on 2D vision, the homography matrix of converting the preprocessed image into the initial image to be matched, that is, the actual homography matrix, can be calculated in real time. When the height of the steel plate cutting part is fixed, the initial image to be matched is obtained by the calibrated homography matrix, and when the height of the steel plate cutting part changes, the initial image to be matched is obtained by the actual homography matrix. Therefore, without using a three-dimensional camera, the workpiece sorting system based on 2D vision can effectively meet the accuracy requirements caused by the change in the height of the steel plate cutting part by using a two-dimensional camera, thereby reducing costs and improving sorting efficiency.

[0066] In some embodiments, a first matrix modulus is calculated according to an actual homography matrix, and a second matrix modulus is calculated according to a calibrated homography matrix. If it is determined that the difference between the first matrix modulus and the second matrix modulus is less than or equal to a preset difference, it means that the height of the steel plate cutting part has not changed. Therefore, an initial image to be matched is obtained according to the preprocessed image and the calibrated homography matrix, that is, the preprocessed image is converted into the initial image to be matched through the calibrated homography matrix; if it is determined that the difference between the first matrix modulus and the second matrix modulus is greater than the preset difference, it means that the height of the steel plate cutting part has changed. Therefore, an initial image to be matched is obtained according to the preprocessed image and the actual homography matrix, that is, the preprocessed image is converted into the initial image to be matched through the actual homography matrix, so as to solve the accuracy problem caused by the change in the height of the steel plate cutting part.

[0067] In some embodiments, since the positions of various parts on the local steel plate cutting piece can be roughly located through the initial image to be matched, if all parts are flat on the steel plate as a whole, such as the parts are not warped, global matching can obtain accurate matching results. However, in actual situations, due to the jitter and vibration of the steel plate during transportation, it is difficult for parts in different areas to remain on the same plane. This will cause large deviations in some parts after global matching, making it difficult for the ground rail robot to achieve accurate sorting. In this regard, the embodiment of the present invention performs global matching on the initial image to be matched to obtain the positions of various parts on the local steel plate cutting piece, and then performs precise matching, i.e. local matching, on each part in the initial image to be matched, thereby determining the target grasping point of the part to be grasped.

[0068] Specifically, the preset template image includes a local template image of a steel plate and a template image of a part to be grasped. The initial image to be matched is globally matched with the local template image of the steel plate to determine the distribution position of each part on the steel plate cut piece in the initial image to be matched; the distribution position of each part is locally matched with the template image of the part to be grasped to determine the distribution position of the part to be grasped in the initial image to be matched, and the target grasping point is determined in combination with the distribution position of the part to be grasped. Therefore, the embodiment of the present invention can effectively improve the success rate of part matching by combining the overall information of the steel plate cut piece with the local features of the part to determine the target grasping point of the part to be grasped.

[0069] It is understandable that, since the initial image to be matched is an image of a local area of ​​the whole steel plate cut part, when the initial image to be matched is globally matched, the local template image of the steel plate used should correspond to the local area presented by the initial image to be matched. Also, when the initial image to be matched is locally matched, the template image of the part to be grasped used should correspond to the part to be grasped.

[0070] In some embodiments, since the image coordinate system when the two-dimensional camera captures the two-dimensional image is not unified with the robot coordinate system of the ground rail robot, if the ground rail robot is directly controlled to grab the parts to be grabbed by the target grabbing point, the ground rail robot will not be able to achieve the desired action. Therefore, the target grabbing point identified in the image coordinate system needs to be converted to the robot coordinate system of the ground rail robot. Specifically, a coordinate conversion matrix is ​​obtained, wherein the coordinate conversion matrix can be obtained by hand-eye calibration according to the image coordinate system corresponding to the two-dimensional camera and the robot coordinate system of the ground rail robot, and the obtained coordinate conversion matrix is ​​stored for subsequent use; then, according to the coordinate conversion matrix, the target grabbing point is converted from the image coordinate system of the two-dimensional camera in the workpiece sorting system based on 2D vision to the actual grabbing point in the robot coordinate system of the ground rail robot, so that the ground rail robot of the workpiece sorting system based on 2D vision is controlled by the actual grabbing point to grab the parts to be grabbed, thereby realizing the sorting function of the workpiece sorting system based on 2D vision.

[0071] In some embodiments, in order to avoid the interference factors in the collected two-dimensional image affecting the grasping result, the embodiment of the present invention can perform distortion processing on the two-dimensional image according to the distortion correction method to obtain the two-dimensional image after distortion processing; perform edge detection on the two-dimensional image after distortion processing according to the edge feature extraction method to obtain the edge feature image; perform enhancement processing on the two-dimensional image after distortion processing according to the edge feature image to obtain the preprocessed image. Thus, by preprocessing the two-dimensional image to remove interference factors, interference is avoided in the subsequent identification and positioning of the parts to be grasped, and the accuracy of grasping the parts to be grasped is improved.

[0072] Please refer to the following Figure 5 The working process of the workpiece sorting system based on 2D vision according to an embodiment of the present invention is described, and the specific steps are as follows.

[0073] Step S7, obtaining the running status of the ground rail robot.

[0074] Step S8, judging whether the running state of the ground rail robot is normal. If so, executing step S9; if not, ending the control of the ground rail robot.

[0075] Step S9, controlling the ground rail robot to move to a designated position to collect a two-dimensional image.

[0076] Step S10, preprocessing the two-dimensional image to obtain a preprocessed image.

[0077] Step S11, identifying the part to be grasped according to the preprocessed image.

[0078] Step S12, converting the target grasping point of the part to be grasped into the robot coordinate system.

[0079] Step S13, controlling the ground rail robot to grasp the parts to be grasped according to the actual grasping points and placing them on the conveyor line.

[0080] In summary, according to the control method of the workpiece sorting system based on 2D vision according to the embodiment of the present invention, the workpiece sorting system based on 2D vision adopts a two-dimensional camera. When the height of the steel plate cutting part is fixed, the initial image to be matched is obtained by calibrating the homography matrix. When the height of the steel plate cutting part changes, the initial image to be matched is obtained by using the actual homography matrix. Therefore, without using a three-dimensional camera, the workpiece sorting system based on 2D vision can effectively meet the accuracy requirements caused by the change in the height of the steel plate cutting part by using a two-dimensional camera, thereby reducing costs and improving sorting efficiency.

[0081] A third aspect of the present invention provides a control device for a workpiece sorting system based on 2D vision, such as Figure 6 As shown, the control device 50 includes at least one processor 7 and a memory 8 communicatively connected to the at least one processor 7 .

[0082] The memory 8 stores a computer program that can be executed by at least one processor 7, and when the at least one processor 7 executes the computer program, the control method of the 2D vision-based workpiece sorting system provided in the above embodiment is implemented.

[0083] It should be noted that the specific implementation method of the control device 50 of the embodiment of the present invention is similar to the specific implementation method of the control method of the 2D vision-based workpiece sorting system of any of the above-mentioned embodiments of the present invention. Please refer to the description of the method part for details. In order to reduce redundancy, it will not be repeated here.

[0084] According to the control device 50 for the workpiece sorting system based on 2D vision according to the embodiment of the present invention, the control method of the workpiece sorting system based on 2D vision provided by the above embodiment is executed by the processor 7. In this way, the workpiece sorting system based on 2D vision can effectively meet the accuracy requirements caused by changes in the height of steel plate cutting parts by using a two-dimensional camera without using a three-dimensional camera, thereby reducing costs and improving sorting efficiency.

[0085] A fourth aspect of the present invention provides a computer storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the control method of the 2D vision-based workpiece sorting system provided in the above embodiment is implemented.

[0086] In the description of this specification, any process or method description in a flowchart or otherwise described herein may be understood to represent a module, fragment or portion of code including one or more executable instructions for implementing the steps of a custom logical function or process, and the scope of the preferred embodiments of the present invention includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of the present invention belong.

[0087] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in combination with these instruction execution systems, devices or apparatuses. For the purpose of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in combination with these instruction execution systems, devices or apparatuses. More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (electronic device), a portable computer disk box (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), a fiber optic device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium and then editing, interpreting or processing in other suitable ways if necessary, and then stored in a computer memory.

[0088] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, a plurality of steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0089] A person skilled in the art may understand that all or part of the steps in the method for implementing the above-mentioned embodiment may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, which, when executed, includes one or a combination of the steps of the method embodiment.

[0090] In addition, each functional unit in each embodiment of the present invention may be integrated into a processing module, or each unit may exist physically separately, or two or more units may be integrated into one module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium.

[0091] The storage medium mentioned above may be a read-only memory, a magnetic disk or an optical disk, etc. Although the embodiments of the present invention have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limiting the present invention. A person of ordinary skill in the art may change, modify, replace and modify the above embodiments within the scope of the present invention.

[0092] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "illustrative embodiments", "examples", "specific examples", or "some examples" means that the specific features, structures, materials, or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example.

[0093] Although the embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the claims and their equivalents.

Claims

1. A workpiece sorting system based on 2D vision, characterized in that: include: A production line and a conveyor line, wherein the production line is used to produce steel plate cut pieces, and the conveyor line is used to convey the steel plate cut pieces; A ground rail robot, the ground rail robot comprising a ground rail assembly, a robot and an end picker, the ground rail assembly being arranged between the production line and the conveyor line, the robot being movably arranged on the ground rail assembly, the end picker being connected to the robot, and the ground rail robot being used to grab steel plate cutting parts on the production line; A two-dimensional camera, which is arranged on the ground rail robot and is used to collect a two-dimensional image of the steel plate cutting piece on the production line, wherein when the ground rail robot grabs the steel plate cutting piece on the production line, the field of view of the two-dimensional camera is projected onto the steel plate cutting piece on the production line; A control device is connected to the two-dimensional camera and the ground rail robot, and is used to acquire the two-dimensional image, pre-process the two-dimensional image to obtain a pre-processed image, obtain an actual homography matrix according to the pre-processed image and a preset template image, and obtain an initial image to be matched according to the pre-processed image, the actual homography matrix and the calibrated homography matrix, wherein a first matrix modulus is calculated according to the actual homography matrix, and a second matrix modulus is calculated according to the calibrated homography matrix, and it is determined that the difference between the first matrix modulus and the second matrix modulus is less than or equal to a preset difference, then the initial image to be matched is obtained according to the pre-processed image and the calibrated homography matrix, and it is determined that the difference between the first matrix modulus and the second matrix modulus is greater than the preset difference, then the initial image to be matched is obtained according to the pre-processed image and the actual homography matrix, and a target grasping point of a part to be grasped in the initial image to be matched is determined according to the initial image to be matched and the preset template image, and the ground rail robot is controlled to grasp the steel plate cutting parts on the production line according to the target grasping point.

2. The 2D vision-based workpiece sorting system according to claim 1, characterized in that: The end picker comprises: End picker body; A bracket, the bracket is connected to the end picker body, and the two-dimensional camera is arranged on the bracket; Wherein, when the end picker body grabs the steel plate cut piece on the production line, the field of view of the two-dimensional camera is projected onto the steel plate cut piece.

3. The 2D vision-based workpiece sorting system according to claim 2, characterized in that: The 2D vision-based workpiece sorting system also includes: An annular light source is disposed on the bracket and surrounds the two-dimensional camera.

4. A workpiece sorting control method based on 2D vision, characterized in that: The control method comprises: Acquire a two-dimensional image of a steel plate cut piece on a production line in the 2D vision-based workpiece sorting system; Control the 2D vision-based workpiece sorting system described in any one of claims 1-3 to grasp the parts to be grasped.

5. The workpiece sorting control method based on 2D vision according to claim 4 is characterized in that: The preset template image includes a local template image of a steel plate and a template image of a part to be grasped, and determining a target grasping point of the part to be grasped in the initial image to be matched according to the initial image to be matched and the preset template image includes: Performing global matching between the initial image to be matched and the local template image of the steel plate to determine the distribution position of each part in the initial image to be matched; The distribution position of each part is locally matched with the template image of the part to be grasped to determine the target grasping point of the part to be grasped in the initial image to be matched.

6. The workpiece sorting control method based on 2D vision according to claim 4 is characterized in that: Controlling the ground rail robot of the 2D vision-based workpiece sorting system to grasp the part to be grasped according to the target grasping point includes: Get the coordinate transformation matrix; According to the coordinate conversion matrix, the target grasping point is converted from the image coordinate system of the two-dimensional camera in the workpiece sorting system based on 2D vision to an actual grasping point in the robot coordinate system of the ground rail robot; The ground rail robot of the 2D vision-based workpiece sorting system is controlled according to the actual grasping point to grasp the part to be grasped.

7. The workpiece sorting control method based on 2D vision according to claim 4 is characterized in that: Preprocessing the two-dimensional image to obtain a preprocessed image includes: Performing distortion processing on the two-dimensional image according to a distortion correction method to obtain a two-dimensional image after distortion processing; Performing edge detection on the distorted two-dimensional image according to an edge feature extraction method to obtain an edge feature image; The two-dimensional image after the distortion processing is enhanced according to the edge feature image to obtain the pre-processed image.

8. A control device for a workpiece sorting system based on 2D vision, characterized in that: include: at least one processor; a memory communicatively coupled to at least one of the processors; Wherein, the memory stores a computer program that can be executed by at least one of the processors, and when at least one of the processors executes the computer program, the 2D vision-based workpiece sorting control method described in any one of claims 4 to 7 is implemented.

9. A computer storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the 2D vision-based workpiece sorting control method described in any one of claims 4 to 7 is implemented.

Images