Calibration device

Abstract

This calibration device comprises: a first member (2) disposed in an imaging area (21), which is imaged by a camera (28), in a transport path (16); a second member (3) fixed to the first member (2); a marker (41) for putting a mark (S1, 1~SM, N) on the second member (3); an attachment member (42) for attaching the marker (41) to a support member (34) supporting a grasping unit (51) that grasps, in a grasping area (22) in the transport path (16) different from the imaging area (21), an object transported along the transport path (16); and a positional relationship calculation unit (56) for calculating a positional relationship between the imaging area (21) and the grasping unit (51).

Description

Technical Field

[0001] The technology of the present disclosure relates to a calibration device.

Background Art

[0002] There is known a resource waste automatic sorting device that automatically sorts resource waste from a plurality of wastes conveyed along a conveyance path. The resource waste automatic sorting device includes a camera that images an image of the waste disposed in a sorting area of the conveyance path, and a robot that removes the resource waste from a removal area on the downstream side of the sorting area of the conveyance path based on the position of the resource waste calculated based on the image of the waste. At this time, the position relationship between the camera and the robot is measured in advance in the resource waste automatic sorting device, and the robot is controlled based on the position relationship so that the resource waste is removed. (Japanese Patent Application Laid-Open No. 2019-141935).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, when the position relationship between the camera and the robot is not appropriately measured in the resource waste automatic sorting device, the robot may not appropriately grasp the resource waste based on the image of the waste, and the resource waste may not be removed from the conveyance path.

[0005]

Means for Solving the Problems

[0006] A calibration device according to one aspect of the present disclosure includes a first member positioned in an imaging area of ​​a transport path that is imaged by a camera, a second member fixed to the first member, a marker for marking the second member, a mounting member for attaching the marker to a support member that supports a gripping portion for gripping an object transported along the transport path in a gripping area different from the imaging area of ​​the transport path, and a positional relationship calculation unit for calculating the positional relationship between the imaging area and the gripping portion. [Effects of the Invention]

[0007] The disclosed calibration device can accurately measure the positional relationship between a camera that captures an image of an object and a robot that processes the object. [Brief explanation of the drawing]

[0008] [Figure 1] Figure 1 is a plan view showing the calibration jig provided in the calibration device of the embodiment. [Figure 2] Figure 2 is a plan view showing an automated resource waste sorting system in which the calibration device of the embodiment is used. [Figure 3] Figure 3 is a perspective view showing an automated waste sorting system. [Figure 4] Figure 4 is a cross-sectional view showing an automated waste sorting device. [Figure 5] Figure 5 is a block diagram of the object processing device. [Figure 6] Figure 6 is a flowchart showing the process of calculating the positional relationship between the imaging area and the gripping part. [Figure 7] Figure 7 is a plan view showing a calibration jig properly positioned in the transport path. [Figure 8] Figure 8 is a plan view showing a calibration jig in which the second component is properly positioned in the imaging area. [Figure 9] Figure 9 is a plan view showing one of several marks. [Figure 10]Figure 10 is a side view of the robot body showing the Y-axis rotation angle, where the robot coordinate system is rotated around the Y-axis of the jig coordinate system relative to the jig coordinate system. [Figure 11] Figure 11 is another side view of the robot body showing the X-axis rotation angle, where the robot coordinate system is rotated around the X-axis of the jig coordinate system relative to the jig coordinate system. [Figure 12] Figure 12 is a plan view showing a step chart with multiple markings. [Figure 13] Figure 13 is a plan view showing a calibration jig in which the second member is positioned in a different orientation within the imaging area. [Modes for carrying out the invention]

[0009] The calibration device according to the embodiments disclosed herein will be described below with reference to the drawings. However, the technology of this disclosure is not limited by the following description. Furthermore, the same reference numerals are used for identical components, and redundant explanations are omitted. [Examples]

[0010] The calibration device of the embodiment is provided with a calibration jig 1, as shown in Figure 1. Figure 1 is a plan view showing the calibration jig 1 provided in the calibration device of the embodiment. The calibration jig 1 comprises a first member 2, a second member 3, and a frame 5. The first member 2 is formed in the shape of a plate. A pattern chart 6 is drawn on the first member 2. The pattern chart 6 is a test chart used when measuring the aberrations of an optical system, for example, a chessboard test chart. The pattern chart 6 further indicates a jig coordinate system, showing the jig coordinate system origin Oj, the jig coordinate system X-axis direction Xj, and the jig coordinate system Y-axis direction Yj. The jig coordinate system X-axis direction Xj is parallel to the plane along which the surface of the first member 2 on which the pattern chart 6 is drawn is aligned. The jig coordinate system Y-axis direction Yj is parallel to the plane along which the surface of the first member 2 on which the pattern chart 6 is drawn is aligned, and is perpendicular to the jig coordinate system X-axis direction Xj.

[0011] The second member 3 is formed in a plate shape. A notched chart 7 and arrows 8 are drawn on the second member 3. Multiple X-axis lines and multiple Y-axis lines are drawn on the notched chart 7. The multiple X-axis lines are parallel to each other and arranged at equal intervals. The multiple Y-axis lines are parallel to each other, parallel to the direction perpendicular to the lines along which the multiple X-axis lines are aligned, and arranged at equal intervals. In other words, the notched chart 7 is tiled with multiple rectangles separated by multiple X-axis lines and multiple Y-axis lines.

[0012] Frame 5 is formed from multiple rods. The first member 2 is fixed to frame 5. The second member 3 is positioned such that the first member 2 and the second member 3 are aligned along the Y-axis direction Yj of the jig coordinate system. The second member 3 is positioned such that multiple X-axis lines are parallel to the X-axis direction Xj of the jig coordinate system, multiple Y-axis lines are parallel to the Y-axis direction Yj of the jig coordinate system, and arrow 8 indicates the direction from pattern chart 6 to marking chart 7. The second member 3 is fixed to frame 5 and fixed to the first member 2 via frame 5. Therefore, the Y-coordinates of multiple positions where multiple X-axis lines are positioned, and the X-coordinates of multiple positions where multiple Y-axis lines are positioned, can be read from the image showing the marking chart 7 and arrow 8.

[0013] The calibration device of the embodiment is used in the automatic waste sorting device 10 shown in Figure 2. Figure 2 is a plan view showing the automatic waste sorting device 10 in which the calibration device of the embodiment is used. The automatic waste sorting device 10 comprises a conveying device 11 and an object processing device 12. The conveying device 11 is formed from a belt conveyor and comprises a belt conveyor frame 14 and a belt 15, and a belt drive device (not shown). The belt conveyor frame 14 is placed on the installation surface 17 in which the automatic waste sorting device 10 is installed. The belt 15 is formed from a flexible material and is formed in a loop-shaped band.

[0014] A conveying path 16 is formed in the conveying device 11. The conveying path 16 follows another plane that is parallel to the plane along which the installation surface 17 follows, and the straight line along which the conveying path 16 follows is parallel to the conveying direction 18, which is parallel to the plane along which the installation surface 17 follows. The belt 15 has a conveying path opposing portion 19. The conveying path opposing portion 19 is located below the conveying path 16 and follows the conveying path 16. The belt 15 is further supported by a belt conveyor frame 14 so as to be movable. A belt drive device moves the belt 15 so that the conveying path opposing portion 19 moves in the conveying direction 18.

[0015] The transport path 16 includes an imaging area 21 and a gripping area 22, and also includes an object supply area (not shown). The gripping area 22 is located downstream of the imaging area 21 in the transport direction 18. The object supply area is located upstream of the imaging area 21 in the transport direction 18.

[0016] The object processing device 12 comprises an object recognition device 23 and a robot 24. The object recognition device 23 is positioned near the imaging area 21. The object recognition device 23 includes a housing 25. The housing 25 is made of a light-shielding material and is generally box-shaped. The housing 25 is positioned above the imaging area 21 of the transport path 16 and covers the imaging area 21. The housing 25 is fixed to the belt conveyor frame 14 and fixed to the mounting surface 17 via the belt conveyor frame 14. The robot 24 is positioned near the gripping area 22.

[0017] Figure 3 is a perspective view showing the automated recyclable waste sorting device 10. The robot 24 comprises a robot body 26 and a robot cover 27. The robot body 26 grasps objects placed in the gripping area 22, removes the grasped objects from the transport path 16, and moves the grasped objects to a recyclable waste storage area (not shown). The robot cover 27 is generally box-shaped. The robot cover 27 is positioned over the gripping area 22 such that the gripping area 22 of the transport path 16 and the robot body 26 are positioned inside the robot cover 27. The robot cover 27 is fixed to the belt conveyor frame 14 and fixed to the mounting surface 17 via the belt conveyor frame 14. The robot cover 27 prevents objects being transported along the transport path 16 from scattering from the gripping area 22 when the robot body 26 handles objects in the gripping area 22.

[0018] Figure 4 is a cross-sectional view showing the automatic resource waste sorting device 10. The object recognition device 23 further includes an imaging camera 28. The imaging camera 28 is directed towards the imaging area 21, positioned above the imaging area 21, and located inside the housing 25. The imaging camera 28 is fixed to the housing 25 and, via the housing 25, to the belt conveyor frame 14. By being positioned in this manner, the imaging camera 28 captures images of objects placed in the imaging area 21 as if viewed from directly above.

[0019] The robot body 26 comprises a first support member 32, X-axis and Y-axis actuators 33, a second support member 34, a Z-axis actuator 35, and a gripping part mounting member 36. The first support member 32 is supported on a base fixed to the installation surface 17 so as to be able to translate parallel to the X-axis direction Xr or the Y-axis direction Yr of the robot coordinate system via the X-axis and Y-axis actuators 33. The Y-axis direction Yr of the robot coordinate system is generally parallel to the transport direction 18, that is, generally parallel to the plane along which the transport path 16 follows. The X-axis direction Xr of the robot coordinate system is perpendicular to the Y-axis direction and is generally parallel to the plane along which the transport path 16 follows. The X-axis and Y-axis actuators 33 translate the first support member 32 parallel to the X-axis direction Xr of the robot coordinate system, and translate the first support member 32 parallel to the Y-axis direction Yr of the robot coordinate system. The second support member 34 is supported by the first support member 32 via a Z-axis actuator 35 so as to be able to translate parallel to the Z-axis direction Zr of the robot coordinate system. The Z-axis direction Zr of the robot coordinate system is perpendicular to the X-axis direction Xr of the robot coordinate system and perpendicular to the Y-axis direction Yr of the robot coordinate system, that is, it is approximately perpendicular to the plane along which the transport path 16 is traversed. The Z-axis actuator 35 causes the second support member 34 to translate parallel to the Z-axis direction Zr of the robot coordinate system relative to the first support member 32.

[0020] The calibration device comprises a marker 41 and a marker mounting member 42. The marker 41 is formed from a pen. The marker 41 marks the second member 3 by contacting it. The marker 41 is detachably attached to the second support member 34 via the marker mounting member 42 and is fixed to the second support member 34 when attached to it.

[0021] The object processing device 12 further comprises a gripping unit 51, a notification device 52, and a control device 53, as shown in Figure 5. Figure 5 is a block diagram of the object processing device 12. The gripping unit 51 is detachably attached to the second support member 34 via a gripping unit mounting member 36 and is fixed to the second support member 34 when attached to the second support member 34. The gripping unit 51 is controlled by the control device 53 to grip an object or release a gripped object. The notification device 52 comprises a display device and an audio device. The notification device 52 is controlled by the control device 53 to display a screen created by the control device 53 and to speak an audio message created by the control device 53.

[0022] The control device 53 is a computer and includes a storage device 54 and a CPU (Central Processing Unit) 55. The storage device 54 records computer programs installed in the control device 53 and records information used by the CPU 55. The CPU 55 processes information by executing computer programs installed in the control device 53, controls the storage device 54, and controls the imaging camera 28, the X-axis and Y-axis actuators 33, the Z-axis actuator 35, the gripping unit 51, and the notification device 52.

[0023] The computer program installed in the control device 53 includes multiple computer programs that implement multiple functions in the control device 53. These multiple functions include a position relationship calculation unit 56, a sorting unit 57, and a robot control unit 58.

[0024] The positional relationship calculation unit 56 controls the imaging camera 28 of the object recognition device 23 so that a pattern chart image is captured showing the pattern chart 6 of the first member 2 placed in the imaging area 21 of the transport path 16. Based on the pattern chart image, the positional relationship calculation unit 56 calculates the relative position of the calibration jig 1 relative to the imaging area 21 and calculates camera image-jig coordinate transformation information.

[0025] The position relationship calculation unit 56 controls the X-axis, Y-axis actuators 33 and the Z-axis actuator 35 so that when the marker 41 is attached to the second support member 34, multiple marks corresponding to multiple target positions are placed on the second member 3 positioned in the gripping area 22. That is, the position relationship calculation unit 56 transmits movement signals to the X-axis and Y-axis actuators 33 so that the first support member 32 is positioned at multiple target positions. The position relationship calculation unit 56 controls the Z-axis actuator 35 so that when the first support member 32 is positioned at one of the multiple target positions, a mark is placed on the second member 3 positioned in the gripping area 22 of the transport path 16.

[0026] The positional relationship calculation unit 56 controls the imaging camera 28 of the object recognition device 23 so that it captures a marking chart image showing the images of multiple marks attached to the marking chart 7 of the second member 3, which is placed in the imaging area 21 of the transport path 16. The positional relationship calculation unit 56 processes the marking chart image to calculate multiple mark coordinates corresponding to multiple marks and calculates the relative position in which the gripping unit 51 is positioned relative to the calibration jig 1. The mark coordinates corresponding to a particular mark among the multiple mark coordinates indicate the coordinates in the jig coordinate system of the position where the center of that mark is located on the marking chart 7. The positional relationship calculation unit 56 calculates jig-robot coordinate transformation information based on the multiple target positions and the multiple mark coordinates. The positional relationship calculation unit 56 calculates the relative position in which the gripping unit 51 is positioned relative to the imaging area 21 based on the camera image-jig coordinate transformation information and the jig-robot coordinate transformation information and calculates camera image-robot coordinate transformation information.

[0027] The sorting unit 57 controls the imaging camera 28 of the object recognition device 23 so that it captures a waste sorting image in which multiple images of multiple objects placed in the imaging area 21 of the transport path 16 are each captured. The sorting unit 57 processes the waste sorting image and calculates multiple sorting data.

[0028] The robot control unit 58 calculates multiple control data based on the camera image-robot coordinate transformation information calculated by the position relationship calculation unit 56 and multiple sorting data calculated by the sorting unit 57. Based on the multiple control data, the robot control unit 58 controls the robot 24 so that multiple recyclable waste items to be processed are removed from the transport path 16 and moved to the recyclable waste disposal area from among the multiple objects placed in the gripping area 22 of the transport path 16.

[0029] [Operation of the automatic resource waste sorting device 10] The operation of the automatic resource waste sorting device 10 includes the operation of calculating the positional relationship between the imaging area 21 and the gripping unit 51, the operation of transporting multiple waste items along the transport path 16, and the operation of removing multiple resource waste items to be processed from the transport path 16.

[0030] Figure 6 is a flowchart showing the operation for calculating the positional relationship between the imaging area 21 and the gripping unit 51. The operation for calculating the positional relationship between the imaging area 21 and the gripping unit 51 is performed before the operation of transporting multiple pieces of waste along the transport path 16 and the operation of removing multiple recyclable waste items from the transport path 16. In the operation for calculating the positional relationship between the imaging area 21 and the gripping unit 51, first, the user detaches the gripping unit 51 from the second support member 34 of the robot body 26 by operating the gripping unit mounting member 36. After the gripping unit 51 has been detached from the second support member 34, the user attaches the marker 41 to the second support member 34 by operating the marker mounting member 42 (step S1).

[0031] The user then places the calibration jig 1 on the transport path opposing portion 19 so that the first member 2 is positioned in the imaging area 21 and the second member 3 is positioned in the gripping area 22, as shown in Figure 7 (step S2). Figure 7 is a plan view showing the calibration jig 1 properly positioned on the transport path 16. By being placed on the transport path opposing portion 19 in this manner, the calibration jig 1 is properly positioned on the transport path 16.

[0032] The control device 53 controls the imaging camera 28 when the calibration jig 1 is properly positioned on the transport path 16, thereby capturing a pattern chart image in which the pattern chart 6 is visible (step S3). The control device 53 processes the pattern chart image to calculate the relative position of the calibration jig 1 relative to the imaging area 21, and calculates jig position information (step S4). The jig position information indicates the position and orientation in which the image of the pattern chart 6 is projected onto the pattern chart image. Specifically, the jig position information indicates the coordinates of the jig coordinate system origin Oj in the camera image coordinate system, the direction of the jig coordinate system X-axis direction Xj in the camera image coordinate system, and the direction of the jig coordinate system Y-axis direction Yj in the camera image coordinate system.

[0033] The control device 53 further calculates the positional relationship between the camera image coordinate system and the jig coordinate system based on the jig position information, and calculates camera image-jig coordinate transformation information (step S5). The camera image-jig coordinate transformation information consists of the camera image-jig rotation matrix Rcj and the camera image-jig translation matrix Tcj, which are shown in equation (1) below.

number

[0034] The control device 53 has multiple target positions and target counts pre-set. The target count is equal to the number of target positions. The control device 53 places the first support member 32 at the first of the multiple target positions by transmitting a movement signal to the X-axis and Y-axis actuators 33 (step S6). When the first support member 32 is positioned at the target position (step S7, Yes), the control device 53 lowers the second support member 34 parallel to the Z-axis direction Zr of the robot coordinate system by controlling the Z-axis actuator 35. As the second support member 34 lowers, the marker 41 contacts the second member 3 of the calibration jig 1 and marks the marking chart 7 (step S8). After the marker 41 contacts the second member 3, the control device 53 raises the second support member 34 parallel to the Z-axis direction Zr of the robot coordinate system by controlling the Z-axis actuator 35.

[0035] When the number of times the first support member 32 has been placed at the target position is less than the target number (step S9, No), the control device 53 sends the next movement signal to the X-axis and Y-axis actuators 33, thereby placing the first support member 32 at the next target position (step S10). After the first support member 32 has been placed at the next target position, the control device 53 repeats the processes of steps S7 to S9.

[0036] When the number of times the control device 53 places the first support member 32 at the target position reaches the target number of times (step S9, Yes), the control device 53 controls the notification device 52 to display a predetermined screen and emit an audio message (step S11). The screen and the audio message indicate an instruction to re-place the calibration jig 1 on the conveyance path opposing portion 19 so that the scale chart 7 and the arrow 8 are disposed in the imaging region 21. The user translates the calibration jig 1 in the direction opposite to the conveyance direction 18, and re-places the calibration jig 1 on the conveyance path opposing portion 19 so that the second member 3 is disposed in the imaging region 21 as shown in FIG. 8. FIG. 8 is a plan view showing the calibration jig 1 in which the second member 3 is appropriately disposed in the imaging region 21. The second member 3 is appropriately disposed in the imaging region 21 when the scale chart 7 and the arrow 8 are disposed in the imaging region 21.

[0037] When the second member 3 is appropriately disposed in the imaging region 21, the control device 53 controls the imaging camera 28 to image a scale chart image in which the scale chart 7 and the arrow 8 are captured (step S12). The control device 53 performs image processing on the scale chart image and calculates a plurality of mark coordinates corresponding to a plurality of marks S m,n ~S M,N (step S13). FIG. 9 is a plan view showing one of the plurality of marks S 1,1 ~S M,N . The mark coordinates corresponding to the mark S among the plurality of mark coordinates indicate the coordinates in the jig coordinate system at the center position of the mark S m,n . For example, the control device 53 calculates the coordinates in the jig coordinate system at the center position of the mark S m,n based on the distance from the center of S m,n to a straight line in the X-axis direction whose Y coordinate is known, and the distances from the center of S m,n to a plurality of straight lines in the Y-axis direction whose X coordinate is known. m,n The center of the mark S m,n

[0038] FIG. 10 shows a Y-axis rotation angle θ Y m at which the robot coordinate system is rotated about the Y axis of the jig coordinate system.This is a side view of the robot body 26. Mark S m,1 The first support member 32 moves to the target position G m,1 This was attached to the step chart 7 when it was positioned there. Mark S m,2 The first support member 32 moves to the target position G m,2 This was attached to the step chart 7 when it was positioned there. Mark S m,N The first support member 32 moves to the target position G m,N This is what was attached to the step chart 7 when it was positioned there. Multiple target positions G m,1 ~G m,N The Y coordinates in the robot coordinate system are equal to each other. The control device 53 uses the following equation (2) to determine the Y-axis rotation angle θ Y m Calculate the following multiple Y-axis rotation angles θ Y 1 ~θ Y M Calculate.

number

[0039] Figure 11 shows the X-axis rotation angle θ, where the robot coordinate system is rotated around the X-axis of the jig coordinate system relative to the jig coordinate system. X n This is another side view of the robot body 26. Mark S 1,n The first support member 32 moves to the target position G 1,n This was attached to the step chart 7 when it was positioned there. Mark S 2,n The first support member 32 moves to the target position G 2,n This was attached to the step chart 7 when it was positioned there. Mark S M,n The first support member 32 moves to the target position G M,nThis is what was attached to the step chart 7 when it was positioned there. Multiple target positions G 1,n ~G M,n The X coordinates in the robot coordinate system are equal to each other. The control device 53 uses the following equation (3) to determine the X-axis rotation angle θ X n Calculate the following multiple X-axis rotation angles θ X 1 ~θ X N Calculate.

number

[0040] When the number of times the first support member 32 has been placed in the target position reaches the target number (step S9, Yes), the step chart 7 will have multiple marks S as shown in Figure 12. 1,1 ~S M,N It is marked. Figure 12 shows multiple marks S 1,1 ~S M,N This is a plan view showing the step chart 7 with multiple marks S. 1,1 ~S 1,N When the markings were placed on chart 7, the multiple target positions where the first support member 32 was positioned had different X coordinates in the robot coordinate system and equal Y coordinates in the robot coordinate system. M,1 ~S M,N When the markings were placed on the chart 7, the multiple target positions where the first support member 32 was positioned had different X coordinates in the robot coordinate system and equal Y coordinates in the robot coordinate system. The control device 53 controls the multiple marks S 1,1 ~S 1,N Based on this, the regression line L 1 Calculate the following, and similarly, multiple regression lines L 1 ~L MThe control device 53 further calculates the regression line L. 1 The rotation angle θ between the Z axis and the X axis in the Zig coordinate system. Z 1 Calculate the multiple Z-axis rotation angles θ Z 1 ~θ Z M Calculate.

[0041] The robot coordinate system may be rotated by an X-axis rotation angle θX around the X-axis of the jig coordinate system with respect to the jig coordinate system due to errors when the robot body 26 is installed. The robot coordinate system may be rotated by a Y-axis rotation angle θY around the Y-axis of the jig coordinate system with respect to the jig coordinate system due to errors when the robot body 26 is installed. The robot coordinate system may be rotated by a Z-axis rotation angle θZ around the Z-axis of the jig coordinate system with respect to the jig coordinate system due to errors when the robot body 26 is installed. The control device 53 calculates the X-axis rotation angle θX, the Y-axis rotation angle θY, and the Z-axis rotation angle θZ using the following equation (4).

number

[0042] The control device 53 calculates the relative position in which the gripping unit 51 is positioned relative to the calibration jig 1 based on multiple mark coordinates, multiple target positions, the X-axis rotation angle θX, the Y-axis rotation angle θY, and the Z-axis rotation angle θZ, and calculates jig-robot coordinate transformation information (step S14). The jig-robot coordinate transformation information represents the jig-robot rotation matrix Rjr and the jig-robot translation matrix Tjr shown in the following equation (5).

number

number

[0043] The control device 53 calculates camera image-robot coordinate transformation information based on camera image-jig coordinate transformation information and jig-robot coordinate transformation information (step S15). The camera image-robot coordinate transformation information consists of the camera image-robot rotation matrix Rcr and the camera image-robot translation matrix Tcr, which are shown in equation (7) below.

number

[0044] The calibration device, by using the calibration jig 1, can measure the positional relationship between the imaging camera 28 and the robot body 26 regardless of the transport speed at which the transport device 11 transports objects. Therefore, the calibration device can measure the positional relationship between the imaging camera 28 and the robot body 26 with high accuracy even when the transport speed at which the transport device 11 transports objects fluctuates.

[0045] In the operation of transporting multiple pieces of waste along the transport path 16, the user first activates the transport device 11 by operating it. The belt drive unit of the transport device 11 moves the belt 15 at a predetermined constant transport speed so that the section 19 facing the transport path translates in the transport direction 18.

[0046] The user further places multiple pieces of waste on the portion of the belt 15 facing the transport path 19 that is opposite the object supply area. The multiple pieces of waste include multiple recyclable waste items. These recyclable waste items are those that need to be removed from the transport path 16 and that need to be moved to the recyclable waste disposal area. Examples of multiple recyclable waste items include bottles made from glass colored in a predetermined color (for example, brown). The multiple pieces of waste placed on the portion of the transport path 19 are moved along the transport path 16 as the belt 15 moves. prescribedThe debris is transported in the transport direction 18 at the transport speed. Multiple pieces of debris transported along the transport path 16 are placed in the imaging area 21. Multiple pieces of debris placed in the imaging area 21 are further transported along the transport path 16 and placed in the gripping area 22 of the housing 25. Multiple pieces of debris placed in the gripping area 22 are further transported along the transport path 16 and placed in the area downstream of the gripping area 22.

[0047] The operation to remove multiple recyclable waste items from the transport path 16 is performed in parallel with the operation to transport multiple waste items along the transport path 16. The control device 53 controls the imaging camera 28 to capture a waste sorting image showing multiple waste items placed in the imaging area 21. The control device 53 records the waste sorting image in the storage device 54, corresponding to the time the waste sorting image was captured. By processing the waste sorting image, the control device 53 selects multiple recyclable waste items from the multiple waste items shown in the waste sorting image, calculates the positions of the multiple recyclable waste items, and creates multiple sorting data. The multiple sorting data corresponds to multiple recyclable waste items among the multiple waste items shown in the waste sorting image. The sorting data corresponding to a particular recyclable waste item among the multiple sorting data indicates the time of capture and the centroid coordinates. The time of capture indicates the time the waste sorting image was captured. The centroid coordinates indicate the coordinates in the camera image coordinate system of the centroid position where the centroid of the waste being processed is located at the time of imaging.

[0048] The control device 53 further calculates multiple control data corresponding to multiple recyclable waste items based on multiple sorting data and camera image-robot coordinate transformation information. Among the multiple control data, the control data corresponding to a particular recyclable waste item indicates the gripping timing and gripping coordinates. The gripping timing indicates the timing at which the recyclable waste item passes through the gripping area 22. The gripping coordinates indicate the coordinates in the robot coordinate system of the position where the center of gravity of the recyclable waste item is located at the gripping timing. By controlling the robot 24 based on the multiple control data, the control device 53 grips the recyclable waste item located at the gripping coordinates at the gripping timing, removes it from the transport path 16, and moves the recyclable waste item to the recyclable waste storage area.

[0049] The automated resource waste sorting device 10 can appropriately calculate the gripping position based on camera image-robot coordinate transformation information, as the positional relationship between the imaging camera 28 and the robot body 26 is measured with high precision by a calibration device. By appropriately calculating the gripping position, the automated resource waste sorting device 10 can appropriately grip the resource waste to be processed and appropriately remove the resource waste to be processed from the transport path 16.

[0050] [Effects of the calibration device in the example] The calibration device of this embodiment comprises a first member 2, a second member 3, a marker 41, and a marker mounting member 42. The first member 2 is positioned in the imaging area 21 of the transport path 16 that is imaged by the imaging camera 28. The second member 3 is fixed to the first member 2. The marker 41 marks the second member 3. The marker mounting member 42 attaches the marker 41 to a second support member 34 that supports a gripping part 51 that grips an object being transported along the transport path 16 in a gripping area 22 different from the imaging area 21 of the transport path 16.

[0051] In this case, the calibration device of the embodiment uses a calibration jig 1 that is not being transported, so it can measure the positional relationship between the imaging camera 28 and the robot body 26 regardless of the transport speed at which the transport device 11 transports objects. For this reason, the calibration device can measure the positional relationship between the imaging camera 28 and the robot body 26 with high accuracy even when the transport speed at which the transport device 11 transports objects fluctuates. The automatic resource waste sorting device 10 can appropriately calculate the gripping position for gripping the resource waste to be processed by accurately measuring the positional relationship between the imaging camera 28 and the robot body 26, and can appropriately remove the resource waste to be processed from the transport path 16.

[0052] Furthermore, the calibration device of the embodiment further includes a positional relationship calculation unit 56. The positional relationship calculation unit 56 controls the imaging camera 28 so that a pattern chart image showing the first member 2 is captured, and calculates a first relative position in which the first member 2 is positioned relative to the imaging area 21 based on the pattern chart image. The positional relationship calculation unit 56 further controls the X-axis, Y-axis actuators 33 and the Z-axis actuator 35 to move the second support member 34 so that a first mark is placed on the second member 3. The positional relationship calculation unit 56 further calculates a second relative position in which the gripping part 51 is positioned relative to the first member 2 based on the mark position on the second member 3 where the first mark is placed. The positional relationship calculation unit 56 further calculates a third relative position in which the gripping part 51 is positioned relative to the imaging area 21 based on the first relative position and the second relative position. The calibration device of this embodiment eliminates the need for the user to operate the imaging camera 28, the X-axis and Y-axis actuators 33, and the Z-axis actuator 35, or to calculate relative positions, thereby reducing the user's workload and minimizing human error.

[0053] Incidentally, in the calibration device of the embodiment described above, the control device 53 controls the imaging camera 28, the X-axis and Y-axis actuators 33, and the Z-axis actuator 35 to calculate the relative position, but this can also be done by the user. Even in this case, the calibration device of the embodiment can measure the positional relationship between the imaging camera 28 and the robot body 26 with high precision by using a calibration jig 1 that is not transported.

[0054] Furthermore, the positional relationship calculation unit 56 of the calibration device in the embodiment controls the imaging camera 28 so that when the second member 3 is placed in the imaging area 21, a marking chart image showing the second member 3 is captured, and calculates the mark position based on the marking chart image. In this case, the calibration device in the embodiment does not require the user to measure the mark position, which reduces the user's workload and reduces human error.

[0055] Incidentally, in the calibration device of the embodiment described above, the position of the marks is calculated using a marked chart image, but the user may measure the position of the marks. Even in this case, the calibration device of the embodiment can measure the positional relationship between the imaging camera 28 and the robot body 26 with high precision by using a calibration jig 1 that is not being transported.

[0056] Furthermore, the positional relationship calculation unit 56 of the calibration device in the embodiment controls the X-axis, Y-axis actuators 33 and the Z-axis actuator 35 so that a second mark is placed at a mark position different from the mark position of the second member 3. The positional relationship calculation unit 56 calculates the direction in which the second support member 34 moves from when the first mark is placed to when the second mark is placed, based on the marking chart image. In this case, the calibration device in the embodiment can calculate the relative positions of multiple positions where the gripping unit 51 is placed with respect to the imaging camera 28 by placing multiple marks on the marking chart image. The automatic resource waste sorting device 10 can appropriately grip the resource waste to be processed at multiple positions in the gripping area 22 by calculating the relative positions of multiple positions where the gripping unit 51 is placed with respect to the imaging camera 28.

[0057] Incidentally, in the calibration device of the embodiment described above, the relative position is calculated based on multiple marks placed on the marking chart 7, but the relative position may also be calculated based on the position of a single mark placed on the marking chart 7. Even in this case, the calibration device of the embodiment can measure the positional relationship between the imaging camera 28 and the robot body 26 with high precision by using a calibration jig 1 that is not transported.

[0058] By the way, in the calibration device of the embodiment described above, the second member 3 is positioned in the imaging area 21 by translating the calibration jig 1 in the opposite direction to the transport direction 18, but the second member 3 may be positioned in the imaging area 21 in other orientations. Figure 13 is a plan view showing the calibration jig 1 with the second member 3 positioned in the imaging area 21 in other orientations. The calibration jig 1 is positioned such that the second member 3 is appropriately positioned in the imaging area 21, and the first member 2 is positioned downstream of the second member 3 in the transport direction 18. At this time, the control device 53 calculates multiple mark coordinates based on the position or orientation in which the image of the arrow 8 is projected onto the engraved chart image. For example, the control device 53 identifies multiple X-axis line lines and multiple Y-axis line lines projected onto the engraved chart image based on the position or orientation of the arrow 8, and calculates multiple mark coordinates based on the multiple X-axis line lines and multiple Y-axis line lines. Even when the second member 3 is placed in the imaging area 21 without regard to its orientation, the calibration device can determine the orientation of the second member 3 based on the direction of the arrow 8 shown in the marking chart image, and can appropriately calculate the position of the mark.

[0059] By the way, although the second member 3 of the calibration device in the embodiment described above is marked with an arrow 8, the arrow 8 may be omitted. If the arrow 8 is omitted, the control device 53 can, for example, identify multiple X-axis straight lines and multiple Y-axis straight lines that appear in the marked chart image based on the position or orientation of the frame 5 that appears in the marked chart image.

[0060] By the way, in the calibration jig 1 of the calibration device described above, the first member 2 and the second member 3 are fixed to each other via a frame 5, but the frame 5 may be omitted. When the frame 5 is omitted, the calibration jig 1 is formed by directly joining the first member 2 and the second member 3, or by forming the first member 2 and the second member 3 as a single integrated member. The calibration device can calculate the relative position of the gripping part 51 with respect to the imaging camera 28 with high accuracy by using the calibration jig without the frame 5 in the same way as the calibration jig 1 described above.

[0061] By the way, in the calibration device of the embodiment described above, the computer that implements the position relationship calculation unit 56 is the same as the computer that implements the sorting unit 57 and the robot control unit 58, but it may be different. Even if the computer that implements the position relationship calculation unit 56 is different from the computer that implements the sorting unit 57 and the robot control unit 58, the calibration device can similarly calculate the relative position of the gripping unit 51 with respect to the imaging camera 28 with high accuracy.

[0062] Although examples have been described above, the examples are not limited to those described above. Furthermore, the components described above include those that can be easily imagined by a person skilled in the art, those that are substantially the same, and those that fall within the so-called equivalent range. Moreover, the components described above can be combined as appropriate. Furthermore, at least one of various omissions, substitutions, and modifications of the components can be made without departing from the gist of the examples. [Explanation of symbols]

[0063] 1: Calibration jig 2: First member 3: Second member 6: Pattern Chart 7: Step-by-step chart 8: Arrow 10: Automatic waste sorting device 11: Conveyor equipment 12: Object Processing Device 16: Conveyor Route 18: Conveying direction 21: Imaging area 22: Gripping area 28: Imaging camera 33: X-axis and Y-axis actuators 34: Second support member 35: Z-axis actuator 41: Marker 42: Marker mounting component 51:Gripping part 56: Positional Relationship Calculation Unit

Claims

1. A camera that images the imaging area of ​​the transport path that transports objects, A calibration jig having a first member on which a chart showing a jig coordinate system is drawn, and a second member fixed to the first member on which another chart showing a plurality of X-axis line lines and a plurality of Y-axis line lines is drawn, A robot having a movable support part to which a marker for marking the second member and a gripping part to which the marker is attached and which grips the object in a gripping area that does not overlap with the imaging area of ​​the transport path are detachably attached, A positional relationship calculation unit that calculates the positional relationship between the camera and the robot, Equipped with, The position relationship calculation unit, Based on a first image captured by the camera of the first member located in the imaging area, camera image-to-jig coordinate transformation information is calculated to transform the camera image coordinate system, which is the coordinate system of the camera, into the jig coordinate system, which is the coordinate system of the jig. When the marker is attached to the support, the support is controlled such that it moves to a plurality of target positions on the support, and a mark is placed on the second member located in the gripping area when the support moves to each of the plurality of target positions. Based on a second image captured by the camera of the second member having multiple marks located in the imaging area, the position of the marks on the second member is calculated. Based on the multiple target positions and the mark positions calculated for each of the marks, jig-to-robot coordinate transformation information is calculated to transform the jig coordinate system into the robot coordinate system, which is the coordinate system of the robot. Based on the camera image-jig coordinate transformation information and the jig-robot coordinate transformation information, the camera image coordinate system is transformed into the robot coordinate system, and camera image-robot coordinate transformation information indicating the positional relationship between the camera and the robot is calculated. Calibration device.

2. The chart depicted on the first member is a pattern chart, The other chart depicted on the second member is a step chart, The first image is an image of the pattern chart of the first member, The second image is an image captured of the multiple marks attached to the notched chart of the second member. The calibration apparatus according to claim 1.

3. The second member is marked with a symbol indicating its orientation. The calibration apparatus according to claim 1 or 2.

4. The position relationship calculation unit, Based on the second image, the direction in which the support part moves from when the first mark is made to when a second mark, different from the first mark, is made is calculated. The calibration apparatus according to claim 1 or 2.

Images