A flexible sorting method, a flexible sorting system

Abstract

The application discloses a flexible sorting method and a flexible sorting system. The flexible sorting method comprises the following steps: acquiring image information of a target area collected by a 3D vision system, wherein the image information comprises a plurality of workpieces to be sorted; identifying the image information to obtain attribute parameters of each workpiece to be sorted and a pose matrix of the workpiece to be sorted; determining a target position of sorting and placing the workpiece to be sorted according to the attribute parameters of the workpiece to be sorted; sorting a plurality of workpieces to be sorted according to the attribute parameters of the workpieces to be sorted to obtain a grabbing sequence; and controlling a manipulator to grab, move and place the workpiece to be sorted at the target position according to the grabbing sequence and the pose matrix of the corresponding workpiece to be sorted.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of visual sorting, and particularly relates to a flexible sorting method and a flexible sorting system. BACKGROUND

[0002] Under the impetus of economy, China's industrial automation has made great progress, and various industries are increasingly developing towards automation and intelligence. In recent years, the update iteration and retirement of electrical equipment and electrical and electronic products have led to a sharp increase in the scale of waste electronic components, causing huge environmental pressure. With the increasing requirement of the state for energy saving and emission reduction year by year, green manufacturing technology is becoming increasingly important. The disassembly, sorting and recycling of parts of waste electrical and electronic products is an important part of green manufacturing, which can greatly reduce resource consumption and improve resource utilization. However, electronic components are of various types, and sorting is a major problem in the recycling process.

[0003] At present, the recycling of electrical equipment still mainly adopts manual methods, which has high working intensity and low sorting efficiency. At the same time, for some high-value parts and reusable parts, manual sorting is prone to cause damage to the parts due to factors such as violent sorting. Therefore, the development of a flexible sorting system will promote the development of green manufacturing technology and is of great significance to green manufacturing. SUMMARY

[0004] The present application aims to solve the above-mentioned technical problems of manual sorting, and provides a flexible sorting method and a flexible sorting system.

[0005] In order to solve the above-mentioned problems, the present application is implemented according to the following technical solutions:

[0006] In a first aspect, the present application provides a flexible sorting method, which comprises the following steps:

[0007] acquiring image information of a target area collected by a 3D vision system, the image information comprising a plurality of workpieces to be sorted;

[0008] identifying the image information to obtain attribute parameters of each workpiece to be sorted and a pose matrix for grasping the workpiece to be sorted;

[0009] determining a target position for sorting and placing the workpiece to be sorted according to the attribute parameters of the workpiece to be sorted;

[0010] sorting a plurality of workpieces to be sorted according to the attribute parameters of the workpieces to be sorted to obtain a grasping sequence;

[0011] According to the grabbing sequence and the pose matrix of the corresponding workpiece to be sorted, the robot is controlled to grab, move and place the workpiece to be sorted at the target position.

[0012] In combination with the first aspect, the first specific implementation of the first aspect is provided, in particular, the image information of the target area collected by the 3D vision system is acquired, specifically including the following steps:

[0013] The conveyor is controlled to operate, and the conveyor stops after transporting the plurality of workpieces to be sorted to the target area;

[0014] The 3D vision system is instructed to collect the target area to obtain the image information.

[0015] In combination with the first aspect, the second specific implementation of the first aspect is provided, in particular, the image information is identified to obtain the attribute parameters of each workpiece to be sorted, specifically including the following steps:

[0016] The shape features of the plurality of workpieces to be sorted are extracted from the image information;

[0017] The shape features of the workpieces to be sorted are matched in a preset feature model library to obtain the part types of the workpieces to be sorted.

[0018] In combination with the first aspect, the third specific implementation of the first aspect is provided, in particular, the plurality of workpieces to be sorted are sorted according to the attribute parameters of the workpieces to be sorted to obtain a grabbing sequence, specifically including the following steps:

[0019] A first gripper currently loaded by the robot is acquired;

[0020] A preset gripper replacement sequence is acquired, the gripper replacement sequence is a sequential replacement sequence of a plurality of second grippers, and the second gripper is a gripper that needs to be replaced by the robot to complete the sorting operation;

[0021] The first gripper and the part types of the workpieces to be sorted are matched to obtain a first priority workpiece to be sorted;

[0022] Each second gripper of the gripper replacement sequence and the part model of the workpieces to be sorted are matched to obtain an N+1 priority workpiece to be sorted, N is a positive integer;

[0023] The plurality of workpieces to be sorted are sorted according to the first priority and the N+1 priority to obtain a grabbing sequence.

[0024] In combination with the first aspect, the present application further provides a fourth specific implementation of the first aspect. Specifically, the image information is identified to obtain a pose matrix of each of the workpieces to be sorted for grasping the workpieces to be sorted, and specifically includes the following steps:

[0025] The image information is identified and fused to obtain point cloud set information, and the image information includes a raster image;

[0026] According to the point cloud set information, a first pose matrix of each of the workpieces to be sorted in a preset grasping space is obtained;

[0027] A second pose matrix of the manipulator is obtained;

[0028] A third pose matrix of the 3D vision system relative to the manipulator is obtained;

[0029] A fourth pose matrix of a preset grasping point of the manipulator for grasping each workpiece is called;

[0030] According to the first pose matrix, the second pose matrix, the third pose matrix and the fourth pose matrix, a fifth pose matrix of each of the workpieces to be sorted for grasping the workpieces to be sorted is calculated;

[0031] The first pose matrix is a pose matrix in a camera coordinate system, and the second pose matrix, the third pose matrix, the fourth pose matrix and the fifth pose matrix are all pose matrices in a manipulator coordinate system.

[0032] In combination with the first aspect, the present application further provides a fifth specific implementation of the first aspect. Specifically, the flexible sorting method further includes:

[0033] The base of the 3D vision system and the manipulator is fixedly arranged;

[0034] The coordinate transformation relationship between the base of the 3D vision system and the manipulator is set.

[0035] The second aspect relates to a flexible sorting system, which comprises a host computer, a 3D vision system and a manipulator, wherein the host computer is connected with the 3D vision system and the manipulator respectively;

[0036] The host computer comprises a processor and a memory, wherein the processor is connected with the memory, and the memory stores machine readable instructions executable by the processor. When the host computer is running, the machine readable instructions are executed by the processor to execute the flexible sorting method as described in the first aspect.

[0037] In combination with the second aspect, the present application further provides a first specific implementation of the second aspect, in particular, the flexible sorting system further comprises a conveyor, the conveyor stops after transporting the plurality of workpieces to be sorted to the target area.

[0038] The 3D vision system has a mounting bracket located on one side of the conveyor, and the 3D vision system is fixedly suspended above the conveyor.

[0039] The mechanical hand is located on the other side of the conveyor and is arranged opposite to the 3D vision system.

[0040] In combination with the second aspect, the present application further provides a second specific implementation of the second aspect, in particular, a plurality of material frames with open tops are arranged on both sides of the mechanical hand, one material frame corresponds to one type of workpiece to be sorted, and the inner cavity of the material frame is a target position for placing the corresponding workpiece to be sorted.

[0041] In combination with the second aspect, the present application further provides a third specific implementation of the second aspect, in particular, the mechanical hand is provided with a plurality of quick exchange clamps, each quick exchange clamp corresponds to at least one type of workpiece to be sorted.

[0042] The quick exchange clamp comprises a clamp body and a quick exchange female seat, and the quick exchange female seat is connected to the upper part of the clamp body.

[0043] The movable end of the mechanical hand is provided with a quick exchange male seat, and the quick exchange male seat can be adaptively connected with the quick exchange female seats of all quick exchange clamps.

[0044] Compared with the prior art, the present application has the following advantages:

[0045] The present application provides a flexible sorting method, which comprises the following steps: acquiring image information of a target area collected by a 3D vision system, the image information comprising a plurality of workpieces to be sorted; identifying the image information to obtain attribute parameters of each workpiece to be sorted and a pose matrix for grasping the workpiece to be sorted; determining a target position for sorting and placing the workpiece to be sorted according to the attribute parameters of the workpiece to be sorted; sorting a plurality of workpieces to be sorted according to the attribute parameters of the workpieces to be sorted to obtain a grasping sequence; and controlling a mechanical hand to grasp, move and place the workpiece to be sorted at the target position according to the grasping sequence and the pose matrix for grasping the corresponding workpiece to be sorted.

[0046] The present application can adapt to different workpiece sorting under the combination of a 3D vision system and a manipulator, and based on the 3D vision system, the pose information of the workpiece to be sorted can be quickly recognized, and finally the intelligent control of the manipulator is realized, so as to realize flexible sorting. On the other hand, according to the attribute parameters of the workpiece to be sorted, the plurality of workpieces to be sorted are sorted, and are sequentially grabbed according to the grabbing order, so as to improve the sorting efficiency and realize efficient sorting of workpieces of different shapes. BRIEF DESCRIPTION OF DRAWINGS

[0047] The specific embodiments of the present application will be further described in detail below with reference to the accompanying drawings, in which:

[0048] Figure 1 is a flowchart of a flexible sorting method of the present application;

[0049] Figure 2 is a composition diagram of the flexible sorting system of the present application;

[0050] Figure 3 is an assembly diagram of the clamp of the present application;

[0051] Figure 4 is a related basic coordinate system in the camera imaging process of the present application;

[0052] In the drawings:

[0053] 10 - manipulator, 11 - quick exchange male seat;

[0054] 20 - clamp, 21 - quick exchange female seat;

[0055] 30 - 3D vision system;

[0056] 40 - conveyor;

[0057] 50 - material frame. DETAILED DESCRIPTION

[0058] The preferred embodiments of the present application will be described below with reference to the accompanying drawings, and it should be understood that the preferred embodiments described herein are only used to illustrate and explain the present application, and are not used to limit the present application.

[0059] The state's control of electronic pollutants is becoming more and more strict, and relevant policies such as "Electronic Industry Pollutant Discharge Standard" and "Electronic Industry Waste Gas Treatment Engineering Design Standard" have been introduced to control the recycling method and processing specification of electronic components. From this point of view, the recycling of waste electronic components has become an inevitable trend. How to efficiently sort waste electronic components from waste terminals for recycling has become a research hotspot in the field.

[0060] The sorting task is a key step of electronic component recycling, and the work intensity is large and the repeatability is high. At present, the sorting task mainly relies on manual sorting, and the labor cost is high, the accuracy is low and the speed is slow. For some high-value parts, reusable parts and the like, violent sorting and other factors are prone to occur in the manual sorting link, which can cause damage to the parts.

[0061] Therefore, the industrial robot is applied to the field, the automatic sorting of the disassembled electronic parts is realized, and the intelligent degree of the waste electronic component recycling and sorting can be significantly improved. The application develops a robot recognition and sorting system based on three-dimensional point cloud, establishes three-dimensional information of the articles through a 3D vision system, uses a point cloud feature matching method for target recognition and pose estimation, calculates the grabbing pose of the robot and performs the sorting task.

[0062] Embodiment one

[0063] Figure 2 The flexible sorting system shown in embodiment one of the application, the flexible sorting system includes a conveyor, an upper computer, a 3D vision system, a mechanical hand and a material frame.

[0064] The upper computer is connected with the 3D vision system and the mechanical hand respectively; the upper computer includes a processor and a memory, the processor is connected with the memory, the memory stores machine readable instructions executable by the processor, when the upper computer runs, the machine readable instructions are executed by the processor to execute the flexible sorting method as described below.

[0065] The sorting process of the application is as follows, the conveyor transports a plurality of workpieces to be sorted to the target area and stops. The sensor of the 3D vision system is used as the eyes of the robot to obtain external information, the end effector of the robot is used as the hands to realize the grabbing, moving and placing of the workpieces to be sorted, and various functions are realized to complete the workpiece sorting work. The 3D vision system can help the robot to complete the sorting work more quickly and perfectly.

[0066] According to the different relative positions of the 3D vision camera sensor and the robot end effector, it can be divided into two types of eyes on the hand and eyes outside the hand. The camera sensor is fixedly installed at the robot end effector and the relative position relationship with the robot end is unchanged in the movement process. The hand-eye system structure is relatively simple, the visual space of the camera is large, but the field of view of the camera is affected by many factors.

[0067] Therefore, the application adopts the eye-outside-hand type, such as Figure 2As shown, the 3D vision system used in the present application has a mounting bracket located on one side of the conveying belt, and the 3D vision system is fixedly suspended above the conveying belt; and the mechanical hand is located on the other side of the conveying belt and is arranged opposite to the 3D vision system.

[0068] The present application adopts a technical solution of installing the 3D vision system outside the robot body and keeping the position unchanged, and a form of relative position relationship between the 3D vision system and the robot base is unchanged. Although the system structure is relatively complex and requires an additional camera fixing device, the field of view of the camera is larger and the relative accuracy is higher, which more meets the application scenarios of sorting and recycling electronic components, recycling and reusing high-value reusable components.

[0069] In a specific implementation, at the end of the mechanical hand, it is necessary to first establish a data transmission channel between the mechanical hand controller and the host computer. In this paper, the TCP / IP interfaces of the controller and the host computer are connected by network cables to realize data sending and receiving. At the host computer end, a launch file for starting the robot control interface and related simulation programs needs to be written. After the settings of the robot end and the host computer end are completed, starting the completed launch file can realize the communication between the robot and the host computer.

[0070] In a preferred embodiment, in order to reduce the wiring complexity between the vision system and the mechanical arm control system, a wireless connection is adopted in this embodiment, that is, the vision system has a wireless communication function, and the vision system and the mechanical hand control system interact data through a wireless network.

[0071] Embodiment two

[0072] Embodiment two provides a flexible sorting method, which can be executed by the host computer of the flexible sorting system. The host computer can be realized in the form of hardware and / or software. Figure 1 is a flowchart of a flexible sorting method of embodiment one of the present application. The present embodiment can be applied to the sorting and recycling of electronic components.

[0073] As Figure 1 shown, the flexible sorting method comprises the following steps:

[0074] S100: acquiring image information of a target area collected by a 3D vision system, the image information comprising a plurality of workpieces to be sorted;

[0075] S200: identifying the image information to obtain attribute parameters of each workpiece to be sorted and a pose matrix for grabbing the workpiece to be sorted;

[0076] S300: determining a target position for placing the workpiece to be sorted according to the attribute parameters of the workpiece to be sorted.

[0077] S400: According to the attribute parameters of the workpieces to be sorted, the plurality of workpieces to be sorted are sorted to obtain a grabbing sequence;

[0078] S500: According to the grabbing sequence and the pose matrix of the workpiece to be grabbed, the robot is controlled to grab, move and place the workpiece to be sorted at the target position.

[0079] Specifically, the flexible sorting method is described in detail as follows.

[0080] S100: Image information of a target region collected by a 3D vision system is obtained, and the image information includes a plurality of workpieces to be sorted.

[0081] In the present application, the image information is a raster image of the workpiece to be grabbed. The raster image is also called a bitmap, a dot matrix image, a pixel image, and only point information.

[0082] In the embodiment of the present application, the 3D vision system includes a light projector and an image sensor. The image sensor is used to collect image information of the workpiece to be grabbed, and the light projector is used to provide a light source for the image sensor. The image sensor can use a laser scanner, a linear array and a surface array CCD camera or a TV camera, or a newly developed digital camera, and the light projector can be a laser emitter, etc.

[0083] In a specific implementation, a Kinect2.0 depth camera produced by Microsoft can be used as a three-dimensional point cloud data collection device. The Kinect2.0 depth camera has the advantages of compact structure, high frame rate, insensitivity to light, and high cost performance, and is widely used in the fields of robot vision and control, 3D modeling and education and teaching. The Kinect2.0 camera is composed of a color camera, a depth infrared camera and an infrared generating device.

[0084] In a specific implementation, before the sorting operation is performed, the relationship between the pixel coordinate system and the robot end coordinate system needs to be solved, that is, the hand-eye calibration. Specifically, the flexible sorting method further comprises:

[0085] S01: The base of the 3D vision system and the robot is fixedly arranged.

[0086] S02: The coordinate transformation relationship between the 3D vision system and the base of the robot is set.

[0087] In the embodiment of the present application, there are generally three main coordinate systems in the complete robot system, namely a pixel coordinate system established with the first pixel point of the image acquired by the camera as the origin, a camera coordinate system established with the optical center of the camera as the origin, and a robot coordinate system established with the base of the robot as the origin.

[0088] Since the motion of the robot is described based on the robot coordinate system, and the image acquired by the camera is generally described based on the pixel coordinate system, if the target image acquired by the camera is used to plan and control the motion trajectory of the robot, the pixel coordinate system and the camera coordinate system must be unified to the robot coordinate system.

[0089] The unification of the three coordinate systems can be mainly divided into two parts. Firstly, the pixel coordinate system is converted to the camera coordinate system, which is called camera calibration or camera intrinsic parameter calibration. Secondly, the camera coordinate system is converted to the robot coordinate system, which is called robot hand-eye calibration or camera extrinsic parameter calibration. Through camera calibration and robot hand-eye calibration, the two-dimensional coordinates in the image can be converted to three-dimensional coordinates in the robot coordinate system, thereby guiding the motion of the robot.

[0090] The hand-eye calibration of the robot is a technical means that can be realized and implemented by those skilled in the art, and needs to be set according to specific camera parameters and robot parameters. The present application does not make too many explanations.

[0091] In the embodiment of the present application, the flexible sorting system further comprises a conveyor. Therefore, in the flexible sorting method of the present application, the image information of the target area collected by the 3D vision system is acquired, specifically comprising the following steps:

[0092] S03: Control the operation of the conveyor belt, and stop the conveyor belt after transporting a plurality of to-be-sorted workpieces to the target area.

[0093] S04: Instruct the 3D vision system to collect the target area to obtain the image information.

[0094] Specifically, through the circulation operation of the conveyor belt, after the electronic components in the target area are sorted, the conveyor can be controlled to transport the next batch of disassembled electronic components to the target area, so as to realize automatic sorting operation in a cycle.

[0095] S200: Identify the image information to obtain the attribute parameters of each to-be-sorted workpiece and the pose matrix of the to-be-sorted workpiece.

[0096] In an embodiment, the attribute parameter of the workpiece can be a workpiece type, a workpiece shape, a workpiece size parameter, etc., which can be set according to sorting requirements. In an embodiment, in order to improve sorting efficiency and identify the placement position of the corresponding workpiece, the attribute parameter of the workpiece is specifically a workpiece type. The name of each workpiece feature model is pre-set to name and divide the workpiece.

[0097] In an embodiment of the present application, the image information is identified to obtain the attribute parameter of each of the workpieces to be sorted, specifically including the following steps:

[0098] S211: Shape features of a plurality of the workpieces to be sorted are extracted from the image information.

[0099] In an embodiment, the existing image recognition technology can be used to identify and extract the shape features of the workpieces to be sorted.

[0100] S212: The shape features of the workpieces to be sorted are matched in a preset feature model library to obtain the part type of the workpieces to be sorted.

[0101] Specifically, the preset feature model library is provided in the storage module in the host computer, and the preset feature model library contains feature model libraries of all workpieces in the sorting operation. The extracted part shape features are compared with the part feature models in the storage module one by one, and the type of the part is determined according to the comparison result. The purpose is to select the corresponding clamp according to the sorting priority in the subsequent process, control the mechanical arm to the workpiece placement coordinates, and transport the grabbed part to the target coordinates.

[0102] In an embodiment of the present application, the image information is identified to obtain the pose matrix of each of the workpieces to be sorted, specifically including the following steps:

[0103] S221: Point cloud set information is obtained by identifying and fusing the image information, and the image information includes a raster image.

[0104] In an embodiment, the 3D vision system projects a plurality of groups of different phase gratings to the workpiece area (target area) in the field of view and obtains the corresponding raster images. Further, the raster images are identified, and the coordinates of each feature point are calculated by using the triangular positioning method to obtain the point cloud set information.

[0105] S222: A first pose matrix of each of the workpieces to be sorted in a preset grabbing space is obtained according to the point cloud set information.

[0106] S223: A second pose matrix of the mechanical hand is obtained.

[0107] In the present application, the second pose matrix of the manipulator, i.e. the pose matrix of the manipulator base, is fixed.

[0108] S224: Obtain the third pose matrix of the 3D vision system relative to the manipulator.

[0109] In specific implementation, the present application adopts the technical solution of installing the 3D vision system outside the robot body and keeping the position unchanged. For this purpose, the third pose matrix of the 3D vision system relative to the manipulator can be directly calculated and pre-stored in the upper computer.

[0110] S225: Retrieve the fourth pose matrix of the preset gripping point of the manipulator for gripping each workpiece.

[0111] In the present application, the specific shape of the workpiece to be sorted is generally unchanged, i.e. each workpiece has its own gripping point coordinates. By pre-setting the fourth pose matrix of the gripping point of the workpiece for gripping, the end action position of the manipulator for gripping the workpiece can be quickly calculated.

[0112] S226: Calculate the fifth pose matrix of each of the workpieces to be sorted for gripping the workpiece to be sorted according to the first pose matrix, the second pose matrix, the third pose matrix and the fourth pose matrix.

[0113] The first pose matrix is a pose matrix in the camera coordinate system, and the second pose matrix, the third pose matrix, the fourth pose matrix and the fifth pose matrix are all pose matrices in the manipulator coordinate system.

[0114] In specific implementation, according to the coordinate conversion relationship of the hand-eye calibration, the fifth pose matrix of each of the workpieces to be sorted for gripping the workpiece to be sorted can be calculated through the first pose matrix, the second pose matrix, the third pose matrix and the fourth pose matrix.

[0115] S300: Determine the target position of sorting and placing the workpiece to be sorted according to the attribute parameter of the workpiece to be sorted.

[0116] In specific implementation, when the type of the workpiece to be sorted is determined, the pre-stored target position is called, for example, the workpiece A is stored in the material frame A. The pose matrix of the pre-stored material frame A is retrieved to calculate the motion trajectory of the manipulator and the final target position of sorting and placing.

[0117] S400: Sort a plurality of workpieces to be sorted according to the attribute parameter of the workpiece to be sorted to obtain a gripping sequence.

[0118] In a specific implementation, the number of clamps is determined according to the types, shapes, sizes, weights and other characteristics of the workpieces to be sorted. However, it is difficult for a clamp to be compatible with workpieces of various shapes and weights. Therefore, each clamp has certain limitations and compatibility, and can be compatible with workpieces within a certain range, but it is difficult to be universal.

[0119] Under the premise, the manipulator often needs to replace different types of clamps for operation. However, if disordered sorting is performed, there may be problems such as frequent replacement of clamps and large amount of calculation. In order to comprehensively consider the sorting efficiency and improve the sorting efficiency, the adaptive grasping sequence is arranged according to the contour characteristics of the workpiece, so as to reduce the clamp replacement time and improve the grasping efficiency.

[0120] In a specific implementation, the plurality of workpieces to be sorted are sorted according to the attribute parameters of the workpieces to be sorted, and a grasping sequence is obtained, which specifically includes the following steps:

[0121] S410: Obtain a first clamp currently loaded by the manipulator.

[0122] S420: Obtain a preset clamp replacement sequence, the clamp replacement sequence being a sequential replacement sequence of a plurality of second clamps, the second clamps being clamps that need to be replaced by the manipulator to complete the sorting operation.

[0123] S430: Match the first clamp and the part type of the workpiece to be sorted to obtain a first priority workpiece to be sorted.

[0124] S440: Match each second clamp of the clamp replacement sequence and the part model of the workpiece to be sorted to obtain an N+1 priority workpiece to be sorted, N being a positive integer.

[0125] In a specific implementation, the number of clamps is determined according to the types, shapes, sizes, weights and other characteristics of the workpieces to be sorted. Each clamp has certain compatibility and can be compatible with workpieces within a certain range. By setting the workpiece type matched by each clamp in advance, the part type of the workpiece to be sorted obtained by identification can be matched with the first / second clamp.

[0126] S450: Sort the plurality of workpieces to be sorted according to the first priority and the N+1 priority to obtain a grasping sequence.

[0127] In the present application, the workpieces suitable for sorting by using the same grasping clamp within the field of view are set as the same level, and the same as the active grasping clamp is the first priority, and then the second level and the third level are set according to the grasping clamp replacement sequence set by the system. The classification of grasping levels is based on shape and size as the judgment basis, thereby improving the sorting efficiency.

[0128] With the device obtained by disassembling the waste charging pile as an example, the flexible sorting system includes a first clamp, a second clamp 1 and a second clamp 2 in the clamping replacement sequence. The charging module, the switch module, the State Grid billing unit, the display screen, the AC contactor, and the molded case circuit breaker have regular shapes and close areas, and are rectangular. They are matched with the first clamp and are determined as the first priority in the same field of view. The DC contactor, DC fuse and other small parts have smaller and irregular shapes, and are matched with the second clamp 1 and are determined as the same priority in the same field of view, as the second priority. By analogy, the others are matched with the second clamp 2, as the third priority. According to the grasping sequence, the pose matrix of the current workpiece to be grasped is determined, and the pose matrix is converted into the target grasping coordinates of the mechanical arm coordinate system in combination with the hand-eye calibration matrix, and is sent to the mechanical arm; the mechanical hand target grasping coordinates perform grasping operation on the workpiece to be grasped.

[0129] S500: According to the grasping sequence and the pose matrix of the corresponding workpiece to be sorted, the mechanical hand is controlled to grasp, move and place the workpiece to be sorted in the target position.

[0130] Embodiment three

[0131] The third embodiment of the present application provides a preferred embodiment of a flexible sorting system. Specifically, the system structure, sorting method and principle of the third embodiment are exactly the same as those of the first and second embodiments.

[0132] As shown in Figure 2 and Figure 3 , a plurality of material frames with open top structure are arranged on both sides of the mechanical hand, one material frame corresponds to one kind of workpiece to be sorted, and the inner cavity of the material frame is the target position for placing the corresponding workpiece to be sorted.

[0133] In a preferred embodiment, the mechanical hand is provided with a plurality of quick exchange clamps, each quick exchange clamp corresponding to at least one kind of workpiece to be sorted; the quick exchange clamp includes a clamp body and a quick exchange female seat, and the quick exchange female seat is connected to the upper part of the clamp body.

[0134] Among them, the movable end of the mechanical hand is provided with a quick exchange male seat, and the quick exchange male seat can be adaptively connected with the quick exchange female seats of all quick exchange clamps.

[0135] In the present application, the quick exchange female seat is provided with a connecting flange, and the connecting flange is fixedly connected with the manipulator. Different clamps are each provided with a quick exchange female seat, and the number of the quick exchange female seats is configured according to the number of the clamps. The number of the clamps is determined according to the types, shapes, sizes, weights and other characteristics of the workpieces to be sorted, each clamp has a certain compatibility, can be compatible with workpieces within a certain range, and the clamping force of the clamp can be controlled, so that the workpieces are not damaged by clamping.

[0136] Embodiment four

[0137] The embodiment four provides a preferred embodiment of a flexible sorting method, and specifically, the method and principle of the embodiment four are completely same as those of the embodiment two. The present application gives specific and detailed steps of hand-eye calibration.

[0138] Several basic coordinate systems involved in the camera imaging process are shown in Fig. 1. Figure 4 Firstly, a rectangular coordinate system (u, v) is defined in the image, and the coordinates (u, v) represent a pixel coordinate system in units of pixels, which represents the row number and column number of each pixel point in the image array. However, in order to express the actual position of each pixel point in the image with physical quantities, an image coordinate system (x, y) in units of millimeters also needs to be established, and the x-axis and y-axis of the image coordinate system are parallel to the u-axis and v-axis of the pixel coordinate system respectively, and the origin point Oi is defined as the intersection of the camera optical axis and the image plane, which is generally located at the center position of the image plane.

[0139] However, in actual situation, there will be a certain amount of deviation, and the deviation in the pixel coordinate system is (cx, cy), and the physical dimensions of each pixel in the x-axis and y-axis directions are defined as dx and dy respectively. The conversion relationship between the pixel coordinate system and the image coordinate system can be obtained as follows:

[0140]

[0141] For the convenience of subsequent operation, the above formula is rewritten in the form of homogeneous matrix as follows:

[0142]

[0143] A camera coordinate system Oc-Xc, Yc, Zc is constructed at the position of the camera, the Oc point coincides with the optical center of the camera, the Xc axis and the Yc axis are parallel to the x-axis and the y-axis of the image coordinate respectively, the Zc axis is perpendicular to the image plane, and the length of the line segment OcOi represents the focal length f of the camera.

[0144] Since the installation position of the camera in space is arbitrary, a reference coordinate system is needed to describe the position of the camera, and the position of any object in space can also be represented by the reference coordinate system, which is called a world coordinate system, and the origin Ow and the Xw, Yw, Zw axes constitute the world coordinate system.

[0145] The world coordinate system and the camera coordinate system satisfy the rotation and translation relationship between rigid bodies. If the coordinates of a point in the world coordinate system are (X w ,Y w ,Z w ), and the coordinates in the camera coordinate system are (X c ,Y c ,Z c ), the conversion between the two coordinates satisfies the following relationship:

[0146]

[0147] Where R represents the rotation matrix, T represents the translation matrix, T = [t x t y t z ] T ;

[0148] The conversion relationship between the pixel coordinate system and the image coordinate system, and the conversion relationship between the world coordinate system and the camera coordinate system have been established through the above coordinate transformation. In order to simplify the problem studied, a pinhole imaging model is used as the final imaging model. The pinhole imaging model, also known as the linear camera model, is the most basic camera imaging model. It is assumed that there is a point P in space, and the coordinates in the camera coordinate system are (X c ,Y c ,Z c ), and the imaging point of the space point in the image can be approximately represented as (x, y), and the imaging point is the intersection point of the line OcP connecting the space point P and the camera optical center point Oc and the image plane.

[0149] To solve the x-axis direction coordinate value of the image point, project the three-dimensional imaging model to the y-axis direction to obtain a two-dimensional pinhole imaging model in the x-axis direction, and according to the similar triangle principle, the expression of the x-axis direction coordinate value can be calculated. The solving process of the y-axis direction coordinate value of the image point is exactly the same as the above process, and the following normalized expression is obtained.

[0150]

[0151] Rewrite the above expression into a homogeneous matrix form:

[0152]

[0153] The conversion relationship between the world coordinate system and the image pixel coordinate (u, v) can be obtained finally by combining the conversion relationship between each coordinate system:

[0154]

[0155] The matrix M1 is called the internal parameter matrix of the camera, and includes four parameters, i.e., focal length f x , focal length f y , and principal point coordinates (c x , c y ). After calibration, the internal parameter matrix is a fixed value. The matrix M2 is called the external parameter matrix of the camera, and is composed of a rotation matrix R and a translation matrix T. The matrix M 12 is called the projection matrix of the camera.

[0156] The position conversion relationship between the two coordinate systems {X} and {Y} is represented by the matrix . Since the eye-in-hand method is used to build the hand-eye system, the relative position relationship between the robot coordinate system and the camera coordinate system is fixed and unchangeable. The conversion relationship between two positions in all motion postures of the robot can be represented by the following formula: The above formula can be transformed to obtain the following results:

[0157] The final purpose of the robot hand-eye calibration is to obtain the conversion relationship between the robot coordinate system and the camera coordinate system. By recording the parameters on the robot, the conversion relationship between the end effector coordinate system and the robot coordinate system can be obtained. That is, the robot hand-eye calibration is completed.

[0158] The other structures of the flexible sorting method and the flexible sorting system described in the embodiment are known in the art.

[0159] The above description is only a preferred embodiment of the present application, and does not limit the present application in any form. Any modification, equivalent change and modification of the above embodiment based on the technical essence of the present application, without departing from the technical solution of the present application, are still within the scope of the present application.

Claims

1. A flexible sorting method, characterized in that, The flexible sorting method comprises the following steps: acquiring image information of a target area collected by a 3D vision system, the image information comprising a plurality of workpieces to be sorted; identifying the image information to obtain attribute parameters of each workpiece to be sorted and a pose matrix for grasping the workpiece to be sorted; determining a target position for placing the workpiece to be sorted according to the attribute parameters of the workpiece to be sorted; sorting the plurality of workpieces to be sorted according to the attribute parameters of the workpieces to be sorted to obtain a grasping sequence, wherein the sorting comprises: acquiring a first gripper currently loaded by a manipulator; acquiring a preset gripper replacement sequence, the gripper replacement sequence being a sequential replacement sequence of a plurality of second grippers, the second grippers being grippers that need to be replaced by the manipulator to complete a sorting task; matching the first gripper and a part type of the workpiece to be sorted to obtain a first-priority workpiece to be sorted; matching each second gripper of the gripper replacement sequence and a part model of the workpiece to be sorted to obtain an N+1th-priority workpiece to be sorted, N being a positive integer; and sorting the plurality of workpieces to be sorted according to the first priority and the N+1th priority to obtain the grasping sequence; controlling the manipulator to grasp, move and place the workpiece to be sorted at the target position according to the grasping sequence and the pose matrix for grasping the workpiece to be sorted; wherein the number of grippers is determined according to the types, shapes, sizes, weight characteristics and other properties of the workpieces to be sorted, each type of gripper is preset to match a corresponding type of workpiece, and the part type of the workpiece to be sorted obtained through identification is matched with the first gripper or the second gripper, workpieces that can be sorted by the same grasping gripper within the field of view are set to the same level, the first priority is the same as the active grasping gripper, and the second level and the third level are set according to the grasping gripper replacement sequence set by the system, and the grasping level is classified according to shape and size.

2. A flexible sorting method according to claim 1, characterized in that, acquiring image information of a target area collected by a 3D vision system, specifically comprising the following steps: controlling the conveyor to operate, and stopping the conveyor after transporting a plurality of workpieces to be sorted to the target area; instructing the 3D vision system to collect the target area to obtain the image information.

3. A flexible sorting method according to claim 1, characterized in that, identifying the image information to obtain attribute parameters of each workpiece to be sorted, specifically comprising the following steps: extracting shape features of a plurality of workpieces to be sorted from the image information; matching the shape features of the workpieces to be sorted in a preset feature model library to obtain part types of the workpieces to be sorted.

4. The flexible sorting method of claim 1, wherein, identifying the image information to obtain a pose matrix for grasping each workpiece to be sorted, specifically comprising the following steps: identifying and fusing the image information to obtain point cloud set information, the image information comprising a raster image; acquiring a first pose matrix of each workpiece to be sorted in a preset grasping space according to the point cloud set information; acquiring a second pose matrix of the manipulator; acquiring a third pose matrix of the 3D vision system relative to the manipulator; retrieve a fourth pose matrix of a preset gripping point of the manipulator for gripping each workpiece; calculate a fifth pose matrix of gripping each of the workpieces to be sorted according to the first pose matrix, the second pose matrix, the third pose matrix, and the fourth pose matrix; The first pose matrix is a pose matrix in a camera coordinate system, and the second pose matrix, the third pose matrix, the fourth pose matrix, and the fifth pose matrix are pose matrices in a manipulator coordinate system.

5. A flexible sorting method according to claim 4, characterized in that, The flexible sorting method further comprises: The 3D vision system and the base of the manipulator are fixedly arranged; The coordinate transformation relationship between the 3D vision system and the base of the manipulator is set.

6. A flexible sorting system characterized in that, The flexible sorting system comprises an upper computer, a 3D vision system, and a manipulator, and the upper computer is connected with the 3D vision system and the manipulator respectively; The upper computer comprises a processor and a memory, the processor is connected with the memory, and the memory stores machine readable instructions executable by the processor; when the upper computer is running, the machine readable instructions are executed by the processor to execute the flexible sorting method of any one of claims 1-5.

7. The flexible sorting system of claim 6, wherein: The flexible sorting system further comprises a conveyor, and the conveyor stops after transporting a plurality of workpieces to be sorted to the target area; The 3D vision system has a mounting bracket, the mounting bracket is located on one side of the conveyor, and the 3D vision system is fixedly suspended above the conveyor; The manipulator is located on the other side of the conveyor and is arranged opposite to the 3D vision system.

8. The flexible sorting system of claim 7, wherein: A plurality of material frames with open top structure are arranged on both sides of the manipulator, one material frame corresponds to one type of workpiece to be sorted, and the inner cavity of the material frame is a target position for placing the corresponding workpiece to be sorted.

9. The flexible sorting system of claim 6, wherein: The manipulator is provided with a plurality of quick exchange clamps, each quick exchange clamp corresponds to at least one type of workpiece to be sorted; The quick exchange clamp comprises a clamp body and a quick exchange female seat, and the quick exchange female seat is connected to the upper part of the clamp body; The movable end of the manipulator is provided with a quick exchange male seat, and the quick exchange male seat can be adaptively connected with the quick exchange female seat of all quick exchange clamps.

Images