Device for detecting work site position, robot system, method, and computer program

Abstract

There are instances when prescribed work is to be performed on an article of unknown dimensions which is conveyed by a conveyance device. In such instances, a means is required for quickly and accurately detecting the position of the conveyed article.　This device 60 for detecting the position of a work site comprises: a feature position acquisition unit 62 for acquiring the position of a detection target feature of an article 100 on the basis of image data of the article captured by cameras 32A, 32B; a work position acquisition unit 64 for acquiring a position Q acquired by the feature position acquisition unit 62 and the position of the work site in a direction orthogonal to the height direction of a conveyance device 12; a positioning execution unit 66 for executing a positioning operation for arranging a height sensor 20 at the position acquired by the work position acquisition unit 64 by operating a robot 18 which moves the height sensor 20; and a detection execution unit 68 for executing a detection operation for detecting the position of the work site in the height direction by operating the height sensor 20 which was positioned by the positioning operation.

Description

Device, robotic system, method, and computer program for detecting the location of a work area

[0001] This disclosure relates to an apparatus, robotic system, method, and computer program for detecting the location of a work area.

[0002] A robotic system is known that measures the distance to an object to be worked on using a laser sensor moved by the robot (for example, Patent Document 1).

[0003] Japanese Patent Publication No. 2019-63955

[0004] In some cases, specific tasks are performed on items of unknown dimensions that are being transported by conveying equipment. In such situations, there is a need for technology that can detect the position of the transported items quickly and with high precision.

[0005] In one embodiment of the present disclosure, a device for detecting the position of a work location on an article being transported by a transport device in order to perform a predetermined operation on the work location, comprises: a feature position acquisition unit that acquires the position of a detection target feature of the article captured in image data based on image data of the article captured by a camera; a work position acquisition unit that acquires the position of the work location in a direction perpendicular to the height direction of the transport device based on the position acquired by the feature position acquisition unit and a known positional relationship between the work location and the detection target feature; a positioning execution unit that operates a robot to move a height sensor for detecting the position of an article in the height direction and performs a positioning operation to place the height sensor at the position acquired by the work position acquisition unit; and a detection execution unit that operates the height sensor placed at the position by the positioning operation and performs a detection operation to detect a first position of the work location in the height direction.

[0006] In another embodiment of the present disclosure, a method for detecting the location of a work location on an article being transported by a transport device in order to perform a predetermined operation on the work location is as follows: Based on image data of the article captured by a camera, the location of a detection target feature of the article captured in the image data is obtained; Based on the obtained location and a known positional relationship between the work location and the detection target feature, the location of the work location in a direction perpendicular to the height direction of the transport device is obtained; A robot is operated to move a height sensor for detecting the position of the article in the height direction to perform a positioning operation to place the height sensor at the obtained work location; and the height sensor placed at the position by the positioning operation is operated to perform a detection operation to detect the location of the work location in the height direction.

[0007] This is a schematic diagram of a robot system according to one embodiment. This is a block diagram of the robot system shown in Figure 1. This is a perspective view of the robot for moving the height sensor shown in Figure 1. This is a diagram of a workpiece according to one embodiment, where region (a) shows a side view of the article and region (b) shows a top view of the article. This is a flowchart of an example of the operation flow of the robot system shown in Figure 1. This is a flowchart of an example of the flow of step S1 in Figure 5. This shows the state in which an article is placed on the transport device. This shows an example of image data of an article captured by a camera. This shows the state in which a transport coordinate system has been set for the transport device shown in Figure 7. This shows an example of a position database. This is a flowchart of an example of the flow of step S3 in Figure 5. This shows the state in which the position sensor has been positioned by step S21 in Figure 11. This shows the state in which the article shown in Figure 12 has been transported by the transport device. This shows the position database updated in step S25 in Figure 11. This is a flowchart of an example of the flow of step S5 in Figure 5. This shows the position database updated after the work is completed. This schematically shows an example of a candidate work location. This is an example of image data of an article captured by a camera, with a candidate work location shown in Figure 17 superimposed on it. This is an enlarged view of a part of the image data shown in Figure 18. This is a schematic diagram of a robot system according to another embodiment. This is a block diagram of the robot system shown in Figure 20. This is a perspective view of the robot for moving the height sensor shown in Figure 20. This is a flowchart of another example of the flow of step S3 in Figure 5. This shows the position database updated in step S49 in Figure 23. This is a flowchart of yet another example of the flow of step S3 in Figure 5. This is a schematic diagram of a robot system according to yet another embodiment. This is a block diagram of the robot system shown in Figure 26. This is a perspective view of the robot for moving the occupancy sensor shown in Figure 26. This is a flowchart of yet another example of the flow of step S1 in Figure 5. This is a diagram illustrating the relative movement trajectory of the occupancy sensor with respect to an item.

[0008] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings. In the various embodiments described below, the same reference numerals will be used for similar elements, and redundant explanations will be omitted. First, a robot system 10 according to one embodiment will be described with reference to Figures 1 to 3. The robot system 10 includes a transport device 12, a transport sensor 14, a vision sensor 16, a robot 18 for sensor movement, a height sensor 20 (Figures 2 and 3), a work robot 22, and a control device 24.

[0009] The conveying device 12 is, for example, a belt conveyor that conveys the article 100 in the conveying direction Dc. Specifically, the conveying device 12 has a base portion 26, a movable portion 28, and a servo motor 30 (Figure 2). The base portion 26 is fixed to the floor of the work cell. The movable portion 28 is provided on the base portion 26 so as to be movable in the conveying direction Dc, and the article 100 is placed on the movable portion 28.

[0010] The servo motor 30 drives the movable part 28 and transports the article 100 placed on the movable part 28 in the transport direction Dc. The transport device 12 has a transport direction Dc, a width direction Dw, and a height direction Dh. The width direction Dw is perpendicular to the transport direction Dc. The height direction Dh is, for example, parallel to the vertical direction and perpendicular to the width direction Dw and the transport direction Dc.

[0011] The transport sensor 14 detects the amount δ of the article 100 transported by the transport device 12. In this embodiment, the transport sensor 14 is provided on the base portion 26 and includes a rotating roller that contacts the movable portion 28, and an encoder (or Hall element) that detects the rotation speed of the rotating roller. The transport sensor 14 repeatedly detects the amount δ of the article 100 transported by the transport device 12 in the transport direction Dc at a predetermined control period τ (for example, τ = 10 [msec]), and sequentially supplies the detected transport amount δ data to the control device 24. The transport sensor 14 may also be provided on the servo motor 30 of the transport device 12.

[0012] The visual sensor 16 is positioned upstream of the robot 18 (for example, near the upstream end of the transport device 12). In this embodiment, the visual sensor 16 has a pair of cameras 32A and 32B (i.e., stereo cameras). Each of the cameras 32A and 32B is a two-dimensional camera having an imaging sensor (CCD, CMOS, etc.) and an optical lens (collimating lens, focusing lens, etc.) that guides the subject image to the imaging sensor. Cameras 32A and 32B each capture images of the article 100 being transported by the transport device 12 and supply them to the control device 24 as a pair of image data 200A (Figure 8) and 200B, respectively.

[0013] The robot 18 moves the height sensor 20. As shown in Figure 3, the robot 18 has a robot base 34, a movable part 36 that is movably mounted on the robot base 34, and a moving mechanism 38 that moves the movable part 36. The robot base 34 is fixed to the floor of the work cell. The movable part 36 is movably mounted in the width direction Dw of the conveying device 12.

[0014] In this embodiment, the moving mechanism 38 is a ball screw mechanism comprising a ball screw 40 extending in the width direction Dw and a servo motor 42 that rotationally drives the ball screw 40. The servo motor 42 rotates the ball screw 40 in the forward and reverse directions in response to a command from the control device 24, thereby causing the movable part 36 that engages with the ball screw 40 to reciprocate in the width direction Dw.

[0015] The height sensor 20 is used to detect the position Ph of the article 100 in the height direction Dh and is fixed to the movable part 36 of the robot 18. In this embodiment, the height sensor 20 is a laser sensor that measures the distance ds to an object by irradiating the object with a single-axis laser beam. More specifically, the height sensor 20 has a light-emitting unit that emits laser light along a single optical axis A1 and a light-receiving unit (neither shown) that receives the reflected light of the laser beam. The height sensor 20 is fixed to the movable part 36 such that its optical axis A1 is parallel to the height direction Dh. The height sensor 20 supplies the measured distance ds data to the control device 24. Note that the height sensor 20 is not limited to a laser sensor, but may be any type of sensor for detecting the height of the article 100.

[0016] Robot 22 is installed downstream of robot 18 and performs predetermined operations WK on articles 100 that are transported by transport device 12. In this embodiment, robot 22 is a vertical articulated robot and comprises a robot base 44, a movable part 46 movably mounted on the robot base 44, and a moving mechanism 48 for moving the movable part 46. The moving mechanism 48 has a plurality of links 50 that are rotatably connected to each other, and a plurality of servo motors 52 (Figure 2) that rotate the links 50. Each servo motor 52 rotates the links 50 in response to a command from the control device 24, thereby moving the movable part 46 to an arbitrary position.

[0017] An end effector 49 is detachably attached to the movable part 46. The end effector 49 is configured to perform a predetermined operation WK, such as operation WK1, which involves printing (or coating) on ​​the surface 104 of the article 100, or operation WK2, which involves forming an imprint on the surface 104 of the article 100. The robot 22 moves the end effector 49 by the moving mechanism 48 and performs the operation WK (printing operation WK1, imprinting operation WK2, etc.) on the article 100 using the end effector 49.

[0018] The control device 24 controls the operation of the transport device 12, transport sensor 14, vision sensor 16, height sensor 20, and robots 18 and 22. As shown in Figure 2, the control device 24 is a computer having a processor 54, memory 56, and I / O interface 58. The processor 54 has a CPU or GPU, etc., and is communicated with the memory 56 and I / O interface 58 via a bus 59.

[0019] The processor 54 performs calculations to realize the functions of detecting the position P of the article 100 and executing operations WK on the article 100, while communicating with the memory 56 and the I / O interface 58. The memory 56 has RAM or ROM, etc., and stores various data temporarily or permanently. The memory 56 may be a computer-readable recording medium such as a semiconductor memory, a magnetic recording medium, or an optical recording medium.

[0020] The I / O interface 58 has, for example, an Ethernet® port, a USB port, an optical fiber connector, or an HDMI® terminal, and communicates data with external devices via wired or wireless connection under the command of the processor 54. The transport device 12 (servo motor 30), transport sensor 14, vision sensor 16 (cameras 32A and 32B), robot 18 (servo motor 42), height sensor 20, and robot 22 (servo motor 52) are all connected to the I / O interface 58 via wired or wireless communication.

[0021] As shown in Figure 1, a robot coordinate system C1 is set on the robot 22. In this embodiment, the robot coordinate system C1 is fixed to the robot base 44 such that its x-axis is parallel to the transport direction Dc, its y-axis is parallel to the width direction Dw, and its z-axis is parallel to the height direction Dh.

[0022] As shown in Figures 1 and 3, a height robot coordinate system C2 is set on the robot 18. In this embodiment, the height robot coordinate system C2 is fixed to the robot base 34 such that its x-axis is parallel to the transport direction Dc, its y-axis is parallel to the width direction Dw, and its z-axis is parallel to the height direction Dh.

[0023] On the other hand, a height sensor coordinate system C3 is set for the height sensor 20. The height sensor coordinate system C3 is a moving coordinate system that moves in the robot coordinate systems C1 and C2 along with the movement of the height sensor 20 by the robot 18. The height sensor coordinate system C3 defines the position and orientation of the height sensor 20 in the robot coordinate systems C1 and C2 (i.e., the position and direction of the optical axis A1). In this embodiment, the height sensor coordinate system C3 is fixedly set with respect to the height sensor 20 such that its origin is located at the center of the light-emitting part of the height sensor 20, and its z-axis is parallel to (specifically coincides with) the optical axis A1.

[0024] As shown in Figure 1, a visual sensor coordinate system C4 is set for the visual sensor 16. The visual sensor coordinate system C4 defines the position and orientation of the visual sensor 16 in the robot coordinate systems C1 and C2 (i.e., the position and direction of the lines of view of cameras 32A and 32B). In this embodiment, the origin of the visual sensor coordinate system C4 is located at the midpoint of the pair of cameras 32A and 32B (or at the center of the imaging sensor of camera 32A or 32B), and its z-axis is fixedly set relative to the visual sensor 16 so that it is parallel to the line of view of camera 32A or 32B. In this embodiment, the z-axis of the visual sensor coordinate system C4 is positioned parallel to the height direction Dh.

[0025] A transport coordinate system C5 is set for the transport device 12. The transport coordinate system C5 defines the transport direction Dc, width direction Dw, and height direction Dh of the transport device 12. In this embodiment, the x-axis direction of the transport coordinate system C5 defines the transport direction Dc, the y-axis direction defines the width direction Dw, and the z-axis direction defines the height direction Dh. The origin of the transport coordinate system C5 may be located on the upper surface of the movable part 28. In this embodiment, the transport coordinate system C5 is a moving coordinate system that moves in the transport direction Dc in the robot coordinate systems C1 and C2 according to the transport amount δ of the item 100 transported by the transport device 12.

[0026] The processor 54 automatically controls the movements of robots 18 and 22, height sensor 20, vision sensor 16, transport device 12, and transport sensor 14 based on the work robot coordinate system C1, height robot coordinate system C2, height sensor coordinate system C3, vision sensor coordinate system C4, and transport coordinate system C5, and executes work WK on item 100. Therefore, the work robot coordinate system C1, height robot coordinate system C2, height sensor coordinate system C3, vision sensor coordinate system C4, and transport coordinate system C5 constitute the control coordinate system C set for work WK.

[0027] Figure 4 shows an example of article 100. Article 100 is, for example, an irregularly shaped bag-like article with a roughly rectangular outer edge 102 when viewed from above. Article 100 has a defined length direction B1 and a width direction B2. The length direction B1 and the width direction B2 represent the orientation of article 100.

[0028] Furthermore, article 100 has a visually recognizable detection target feature 106. In this embodiment, the detection target feature 106 is a print applied to the surface 104 of article 100, or an engraving formed on the surface 104, and has a substantially rectangular shape. The detection target feature 106 is defined in a length direction E1 and a width direction E2. The length direction E1 and the width direction E2 represent the orientation of the detection target feature 106. The detection target feature 106 also has a feature point Q. The feature point Q may be, for example, the center point or vertex of the outer edge of the detection target feature 106, or the midpoint of one side defining the outer edge. Figure 4 shows the case where the feature point Q is the center point of the outer edge of the detection target feature 106.

[0029] The item 100 has a work area 108 where the robot 22 performs the task WK. The positional relationship PR between the work area 108 and the detection target feature 106 is known in advance. This positional relationship PR can be predetermined, for example, as a positional relationship where the work target point P representing the position of the work area 108 is separated from the feature point Q of the detection target feature 106 by a distance α1 [mm] in one direction in the length direction B1 (or E1) and a distance α2 [mm] in one direction in the width direction B2 (or E2).

[0030] The target point P can be defined, for example, as the center point or vertex of the outer edge of the detected target feature 106, or as the midpoint of one side defining the outer edge. Data indicating the positional relationship PR (for example, data of directions B1 and B2 or E1 and E2, and distances α1 and α2) is stored in memory 56 beforehand. Note that the positional relationship PR is not limited to the above example and can be defined arbitrarily.

[0031] In this embodiment, the dimensions DM (height, width, length) of the article 100 are unknown. The robot system 10 detects the position P of the work area 108 of the article 100, whose dimensions DM are unknown, and performs the work WK on the work area 108. The operation of the robot system 10 will now be described with reference to Figure 5. When the processor 54 receives a work start command from the operator, the higher-level controller, or the computer program PG, it starts the flow shown in Figure 5.

[0032] After the start of the flow shown in Figure 5, the processor 54 starts the first detection process of step S1. Step S1 will be explained with reference to Figure 6. After the start of step S1, in step S11, the processor 54 determines whether or not the article 100 has arrived at the upstream end of the transport device 12. For example, a sensor SN (not shown) capable of detecting the presence of the article 100, such as a presence sensor or proximity sensor, is provided at the upstream end of the transport device 12.

[0033] The processor 54 can determine whether or not the article 100 has arrived at the upstream end of the transport device 12 based on the output signal of the sensor SN. If the processor 54 determines that the answer is YES, it proceeds to step S12; otherwise, it proceeds to step S16. In this embodiment, as shown in Figure 7, multiple types of articles 100 having various dimensions DM are continuously placed on the movable part 28 in any orientation by another robot or operator.

[0034] In step S12, the processor 54 operates the visual sensor 16 (Figure 1) to image the article 100. Specifically, the processor 54 transmits an imaging command to the visual sensor 16, and in response to the imaging command, the visual sensor 16 images the article 100 with cameras 32A and 32B and acquires a pair of image data 200A and 200B. An example of image data 200A is shown in Figure 8. Each pixel PX constituting the captured image data 200A and 200B is defined as a coordinate in the visual sensor coordinate system C4. The visual sensor 16 supplies the image data 200A and 200 captured by cameras 32A and 32B to the control device 24.

[0035] In step S13, the processor 54 acquires the position Q of the detected target feature 106 based on the image data 200A and 200B of the item 100 captured by cameras 32A and 32B. Specifically, the processor 54 determines the parallax DP of cameras 32A and 32B from the difference in coordinates in the visual sensor coordinate system C4 between the pixel PX that captures the item 100 in image data 200A (Figure 8) and the pixel PX that captures the item 100 in image data 200B.

[0036] Meanwhile, the processor 54 performs image analysis on the image data 200A to detect the outer edge 102 and the target feature 106 of the article 100 depicted in the image data 200A, and identifies the length direction B1 or width direction B2 (or length direction E1 or width direction E2) and the feature point Q of the target feature 106 in the visual sensor coordinate system C4. In order to detect the outer edge 102 and the target feature 106 from the image data 200A, the shapes of the outer edge 102 and the target feature 106 may be taught in advance. Then, based on the parallax DP described above, the processor 54 obtains the coordinates Q4 (x, y, z) of the feature point Q in the visual sensor coordinate system C4.

[0037] Meanwhile, the processor 54 sets the transport coordinate system C5 in the robot coordinate systems C1 and C2 each time it captures image data 200A and 200B in step S12. At this time, the origin of the transport coordinate system C5 is set at a predetermined initial position IPc in the robot coordinate systems C1 and C2. Figure 9 schematically shows the transport coordinate system C5 set at the initial position IPc.

[0038] Here, the positional relationships between the robot coordinate systems C1 and C2, the sensor coordinate systems C3 and C4, and the transport coordinate system C5 located at the initial position IPc are known through calibration. Therefore, these control coordinate systems C1 to C5 can be transformed into each other's coordinates via a known coordinate transformation matrix MX (such as a homogeneous transformation matrix).

[0039] The processor 54 converts the coordinates Q4 of the feature point Q of the detected target feature 106 in the visual sensor coordinate system C4 to the coordinates Q5 (x, y, z) in the carrier coordinate system C5. In this way, the processor 54 acquires coordinates Q4 and Q5 as the position Q of the detected target feature 106 based on the image data 200A and 200B captured by cameras 32A and 32B. Therefore, the processor 54 functions as a feature position acquisition unit 62 (Figure 2) that acquires the position Q.

[0040] Referring again to Figure 6, in step S14, the processor 54 obtains the positions Pw and Pc of the work area 108 in directions Dw and Dc that are orthogonal to the height direction Dh, based on the position Q of the detected target feature 106 obtained in the previous step S13 and the known positional relationship PR between the work area 108 and the detected target feature 106.

[0041] Specifically, the processor 54 determines the work target point P in the transport coordinate system C5 based on the coordinates Q5 of the detected target feature 106 in the transport coordinate system C5, which were acquired in the previous step S13, and the positional relationship PR data described above. Then, the processor 54 acquires the coordinates P5(x,y) of the work target point P in the transport coordinate system C5.

[0042] The x-coordinate of coordinate P5 indicates the position of the work target point P in the x-axis direction of the transport coordinate system C5, in other words, the position Pc (third position) of the work area 108 in the transport direction Dc. The y-coordinate of coordinate P5 indicates the position of the work target point P in the y-axis direction of the transport coordinate system C5, in other words, the position Pw (second position) of the work area 108 in the width direction Dw.

[0043] In this embodiment, since the height dimension DM1 of the article 100 is unknown, it is not possible to accurately determine the position of the work location 108 (work target point P) in the z-axis direction (height direction Dh) of the transport coordinate system C5 (z-coordinate of coordinate P5) from the position Q (coordinate Q5) of the detected target feature 106 acquired in step S13.

[0044] Thus, in step S14, the processor 54 acquires the positions Pw and Pc of the work area 108 in directions Dw and Dc orthogonal to the height direction Dh, based on the position Q and positional relationship PR of the detected target feature 106, as coordinates P5(x,y). Therefore, the processor 54 functions as a work position acquisition unit 64 (Figure 2) that acquires the positions Pw and Pc of the work area 108.

[0045] In step S15, the processor 54 functions as a work position acquisition unit 64 and stores the coordinates P5 acquired in the previous step S14 in the position database 202. This position database 202 is for managing the position P of the work location 108 in the control coordinate system C and is stored in memory 56. An example of the data structure of the position database 202 is schematically shown in Figure 10. In the flow in Figure 6, the loop of steps S11 to S16 is repeatedly executed while the result is NO in step S16, which will be described later. Therefore, each time step S14 is executed, the processor 54 acquires new coordinates P5 _ｎ This will involve repeatedly obtaining values ​​for n (1, 2, 3, ...).

[0046] In step S14, the processor 54 generates a new coordinate P5 _ｎ Each time a coordinate P5 is obtained, in step S15, _ｎ to the coordinate P5 _ｎ The data obtained from the transport coordinate system C5 is associated with the data and sequentially stored in the position database 202. In the position database 202 shown in Figure 10, column 204 indicates the order "n" in which the coordinates P5 were obtained (in the example in Figure 10, n = 1, 2, 3, 4). Also, column 206 indicates the stored coordinates P5 _ｎ This shows (x, y).

[0047] On the other hand, column 208, "Status," shows the progress of work WK for item 100. For example, "Waiting for detection" is at coordinate P5 _ｎ This indicates that the z-coordinate (i.e., the position Ph of the work location 108 in the height direction Dh) has not been detected. Thus, in step S15, the processor 54 uses the positions Pw and Pc of the work location 108 acquired in the previous step S14 to determine the coordinates P5 of the transport coordinate system C5. _ｎ The location is stored in the position database 202 as (x, y).

[0048] Again, referring to FIG. 6, in step S16, the processor 54 determines whether it has received an end-of-operation command from the operator, the upper controller, or the computer program PG. If the processor 54 determines YES, it ends the flow shown in FIG. 6. On the other hand, if it determines NO, it returns to step S11. Thus, while the processor 54 determines NO in step S16, it repeatedly executes the loop of steps S11 to S16.

[0049] Again, referring to FIG. 5, after starting step S1, in step S2, the processor 54 determines whether the new coordinate P5 _ｎ has been stored in the position database 202 in step S15 described above. If the processor 54 determines YES, it starts the second detection process in step S3 and proceeds to step S4. On the other hand, if the processor 54 determines NO, it proceeds to step S6. Hereinafter, step S3 will be described with reference to FIG. 11.

[0050] After starting step S3, in step S21, the processor 54 operates the robot 18 to execute a positioning operation PO for arranging the height sensor 20 at the position Pw acquired in step S14. Specifically, the processor 54 refers to the position database 202 (FIG. 10) stored in the memory 56 at the current time, and the "status" in column 208 is "detection waiting", and the "No" in column 204 is the topmost coordinate P5 _ｎ is read out. For example, in the case of the example shown in FIG. 10, the processor 54 reads the coordinate P5 _１ (x, y) of No. 1 (n = 1).

[0051] Next, the processor 54 converts the read coordinate P5 _ｎ to the coordinate P2 _ｎ (x, y) in the height robot coordinate system C2. This coordinate P2 _ｎ indicates the positions Pw and Pc in the height robot coordinate system C2 of the work location 108 (work target point P) acquired in step S14. In the present embodiment, since the y-axis of the height robot coordinate system C2 is arranged parallel to the width direction Dw, the coordinate P2 _ｎThe y-coordinate indicates the position Pw in the width direction Dw of the work area 108.

[0052] Processor 54 controls coordinate P2 _ｎ The processor 54 generates commands (position command, velocity command, torque command) for the servo motor 42 of the robot 18 to position the height sensor 20 at the y-coordinate of P2. The processor 54 operates the robot 18's movement mechanism 38 according to these commands, thereby moving the height sensor 20 in the y-axis direction (i.e., the width direction Dw) of the height robot coordinate system C2 to coordinate P2. _ｎ Position it at the y-coordinate of [the object].

[0053] As a result, the height sensor 20 is positioned at position Pw in the width direction Dw of the work area 108 acquired in step S14, as shown in Figure 12. At this time, the origin O of the height sensor coordinate system C3 is set to coordinate O2(x,y,z) of the height robot coordinate system C2. The y coordinate of this coordinate O2 is coordinate P2 _ｎ This will be the same as the y-coordinate.

[0054] After positioning the height sensor 20 at position Pw, the processor 54 stops the operation of the robot 18 and has the height sensor 20 wait at position Pw. Thus, in this embodiment, the processor 54 functions as a positioning execution unit 66 (Figure 2) that executes a positioning operation PO to place the height sensor 20 at position Pw.

[0055] In step S22, the processor 54 updates the position Pc of the work area 108 in the transport direction Dc in the control coordinate system C. Specifically, the processor 54 displaces the origin of the transport coordinate system C5 in the transport direction Dc by the transport amount δ obtained from the transport sensor 14. As a result, as shown in Figure 13, the transport coordinate system C5 is displaced in the transport direction Dc by the transport amount δ from the initial position IPc in the robot coordinate systems C1 and C2.

[0056] In the transport coordinate system C5 after displacement, a work target point P is set, and the coordinates of the work target point P in the transport coordinate system C5 are P5 _ｎThis remains constant. On the other hand, in robot coordinate systems C1 and C2, the work target point P is displaced by the transport amount δ in the transport direction Dc. Also, when the transport coordinate system C5 is displaced by the transport amount δ, the positional relationship between the transport coordinate system C5 and robot coordinate systems C1 and C2 also changes, and as a result, the parameters of the coordinate transformation matrix MX between the transport coordinate system C5 and robot coordinate systems C1 and C2 change according to the transport amount δ.

[0057] Each time step S22 is executed, the processor 54 displaces the origin of the transport coordinate system C5 in the robot coordinate systems C1 and C2 by a transport amount δ in the transport direction Dc, thereby updating the position Pc of the work location 108 (work target point P) in the robot coordinate systems C1 and C2 to be displaced in the transport direction Dc.

[0058] As described above, the position P of the work area 108 is the coordinate P5 of the transport coordinate system C5, which moves within the robot coordinate systems C1 and C2. _ｎ This is managed in the location database 202. As a result, even if the location Pc is updated in step S22, the coordinates P5 stored in the location database 202 are managed. _ｎ Since this is constant, it is possible to easily manage the position P of the work area 108.

[0059] In step S23, the processor 54 determines whether the work area 108 has reached directly below the height sensor 20, which is waiting at position Pw, due to the transport operation of the transport device 12. Specifically, the processor 54 checks the position database 202 for the coordinates P5 read in the most recent step S21. _ｎ Then, using the coordinate transformation matrix MX between the transport coordinate system C5 and the height robot coordinate system C2 at this point, the coordinate P5 _ｎ The height of the robot coordinate system C2 is coordinate P2 _ｎ Convert to (x, y).

[0060] Meanwhile, the processor 54 obtains the current coordinates O2 (x, y, z) of the height sensor 20 (specifically, the origin O of the height sensor coordinate system C3) in the height robot coordinate system C2 based on feedback from the encoder (or Hall element) provided on the servo motor 42 of the moving mechanism 38. Then, the processor 54 obtains the converted coordinates P2 _ｎ Determine whether the x-coordinate of point P2 matches the x-coordinate of point O2. _ｎ The x-coordinate of O2 indicates the current position of the work target point P in the transport direction Dc, and the x-coordinate of O2 indicates the current position of the height sensor 20 in the transport direction Dc.

[0061] For example, processor 54 controls coordinate P2 _ｎ The difference Δx between the x-coordinate of coordinate O2 is calculated, and if the difference Δx is less than or equal to a predetermined threshold Δth, then coordinate P2 _ｎ It may also be determined that the x-coordinates of O2 and O2 coincide. The processor 54 determines coordinate P2 _ｎ If the x-coordinates of O2 and O2 match, it is determined that the work area 108 has reached directly below the height sensor 20 (i.e., YES), and the process proceeds to step S24. On the other hand, if the processor 54 determines NO, it returns to step S22.

[0062] Thus, while the processor 54 determines NO in step S23, it repeatedly executes the loop of steps S22 and S23, and each time step S22 is executed, it updates the position Pc of the work area 108 in the control coordinate system C (robot coordinate systems C1 and C2). The processor 54 may also repeatedly execute the loop of steps S22 and S23 with a control period τ (specifically, in synchronization with the operation of the transport sensor 14 detecting the transport amount δ).

[0063] In step S24, the processor 54 operates the height sensor 20 positioned at position Pw by the positioning operation PO to perform a detection operation DO to detect the position Ph (first position) of the work area 108 in the height direction Dh. Specifically, the processor 54 sends a detection command to the height sensor 20 which is waiting at position Pw (coordinate O2).

[0064] In accordance with the detection command, the height sensor 20 irradiates a single-axis laser beam toward the item 100. At this time, the laser beam is directed toward the work target point P. The height sensor 20 then measures the distance ds to the item 100 based on the reflected light from the item 100. This distance ds represents the distance from the height sensor 20 (origin O of the height sensor coordinate system C3) to the work target point P, and corresponds to the z-coordinate of the work target point P in the height sensor coordinate system C3.

[0065] The processor 54 obtains the z-coordinate of the work target point P in the transport coordinate system C5 based on the distance ds measured by the height sensor 20. This z-coordinate indicates the position Ph of the work area 108 (work target point P) in the height direction Dh. Thus, the processor 54 executes a detection operation DO to detect the position Ph of the work area 108. Therefore, the processor 54 functions as a detection execution unit 68 (Figure 2) that executes the detection operation DO.

[0066] In step S25, the processor 54 functions as a work position acquisition unit 64 and updates the position database 202 by adding the position Ph detected by the detection operation DO to the position database 202 as coordinates of the control coordinate system C. Specifically, the processor 54 updates the position database 202 currently stored in memory 56 by adding the coordinates P5 read in the most recent step S21. _ｎ By adding the z coordinate of the transport coordinate system C5 detected in the previous step S24 to (x, y), the coordinate P5 _ｎ Update to (x, y, z). As a result, the updated coordinates P5 _ｎ ' is stored in the location database 202. This coordinate P5 _ｎ ' indicates the positions Pc, Pw, and Ph of the work area 108 (work target point P) in the transport direction Dc, width direction Dw, and height direction Dh.

[0067] Then, the processor 54 detects the coordinates P5 in the position database 202 where position Ph was detected. _ｎ The "Status" of ' is changed to "Waiting for work". The updated location database 202 is shown in Figure 14. In the example shown in Figure 14, coordinates P5 of No. 1 _１The "Status" of ' has been changed to "Waiting for work".

[0068] This "waiting for work" is at coordinates P5 of No. 1. _１ The z-coordinate (i.e., the position Ph of the work location 108 in the height direction Dh) is detected, indicating that work WK is possible on item 100, which is labeled No. 1. In this way, each time step S25 is executed, the processor 54 updates the position database 202 by adding the position Ph (z-coordinate of the transport coordinate system C5) detected in the previous step S24.

[0069] In step S26, the processor 54 refers to the current location database 202 and finds coordinates P5 whose "Status" is "Waiting for detection". _ｎ The processor determines whether or not there is a condition. If it determines that the condition is YES, the processor 54 returns to step S21; if it determines that the condition is NO, it terminates the flow shown in Figure 11. In this way, the processor 54 repeatedly executes the loop of steps S21 to S26 while it determines that the condition is YES in step S26.

[0070] Referring again to Figure 5, in step S4, the processor 54 refers to the current location database 202 (for example, Figure 14) and finds coordinate P5 where the "Status" is "Waiting for work". _ｎ The processor determines whether or not there is a condition. If it determines that there is a condition, the processor 54 starts step S5 and proceeds to step S6, while if it determines that there is a condition, it proceeds to step S7.

[0071] In step S5, the processor 54 performs the operation WK on the article 100. Step S5 will be described below with reference to Figure 15. After the start of step S5, in step S31, the processor 54 operates the movement mechanism 48 of the robot 22 to move the end effector 49 to the work location 108 on the article 100.

[0072] Specifically, the processor 54 refers to the current location database 202 (for example, Figure 14), and if the "Status" column 204 is "Waiting for work" and the "No." column 204 is the highest coordinate P5_ｎ (For example, coordinates P5 of No. 1) _１ The processor 54 then reads the read coordinate P5 _ｎ 'Then, using the coordinate transformation matrix MX between the transport coordinate system C5 and the work robot coordinate system C1 at this point, the coordinate P5 _ｎ '(x, y, z) are coordinates P1 of the working robot coordinate system C1 _ｎ Convert to (x, y, z). This coordinate P1 _ｎ In the coordinate system C1 of the work robot, the positions Pc, Pw, and Ph of the work location 108 (work target point P) in the transport direction Dc, width direction Dw, and height direction Dh are shown.

[0073] Then, the processor 54 controls coordinate P1 _ｎ The processor 54 generates commands (position command, speed command, torque command) for the servo motor 52 of the robot 22 to position the end effector 49. The processor 54 operates the robot 22 according to these commands, thereby moving the end effector 49 toward the work location 108 (specifically, the work target point P).

[0074] In step S32, the processor 54 determines whether the end effector 49 has reached the work area 108. Specifically, the processor 54 determines the coordinates R1 of the end effector 49 in the current work robot coordinate system C1 based on feedback from encoders (or Hall elements) provided on each servo motor 52 of the robot 22.

[0075] Then, the processor 54 determines that coordinate R1 is the coordinate P1 of the current work target point P. _ｎ The processor 54 determines whether or not it matches the given condition. If it determines YES, the processor 54 proceeds to step S34; if it determines NO, it proceeds to step S33. When it determines YES in step S32, the end effector 49 can be considered to have been positioned at the work location 108 (work target point P).

[0076] In step S33, the processor 54 updates the position Pc of the work area 108 in the control coordinate system C by displacing the origin of the transport coordinate system C5 in the transport direction Dc, similar to step S22 described above. Then, the processor 54 returns to step S31. In this way, the processor 54 repeatedly executes the loop of steps S31 to S33 while the determination in step S32 is NO. The processor 54 may also repeatedly execute the loop of steps S31 to S33 with a control period τ (specifically, in synchronization with the operation of the transport sensor 14 detecting the transport amount δ).

[0077] In step S34, the processor 54 operates the end effector 49 to perform a work WK on the item 100. Specifically, the processor 54 activates the end effector 49 and performs a work WK such as printing work WK1 or engraving work WK2 on the work area 108. In this way, the work WK on the work area 108 is completed.

[0078] When the operation WK is properly completed, the processor 54 retrieves the coordinates P5 referenced in the most recent step S31 in the position database 202. _ｎ The "Status" of ' is changed to "Task Completed". The modified location database 202 is shown in Figure 16. In the example shown in Figure 16, coordinates P5 of No. 1 _１ This indicates that work WK for item 100 has been completed. Meanwhile, coordinate P5 of No. 2 _２ 'and coordinates P5 of No. 3 _３ The ' indicates the state after step S24 in Figure 11 has been completed ("waiting for work").

[0079] In step S35, the processor 54 refers to the current location database 202 and finds coordinate P5 whose "status" is "waiting for work". _ｎ The processor determines whether or not there is a condition. If it determines that the condition is YES, the processor 54 returns to step S31; if it determines that the condition is NO, it terminates the flow shown in Figure 15. In this way, the processor 54 repeatedly executes the loop of steps S31 to S35 while it determines that the condition is YES in step S35.

[0080] Referring again to Figure 5, in step S6, the processor 54 determines whether or not it has received a work completion command, similar to step S16 in Figure 6. If the processor 54 determines it is YES, it terminates the flow shown in Figure 5; otherwise, it returns to step S2. On the other hand, if it determines it is NO in step S4, in step S7, the processor 54 determines whether or not it has received a work completion command, similar to step S6. If the processor 54 determines it is YES, it terminates the flow shown in Figure 5; otherwise, it returns to step S4.

[0081] As described above, in this embodiment, the processor 54 functions as a feature position acquisition unit 62, a work position acquisition unit 64, a positioning execution unit 66, and a detection execution unit 68 to detect the position P (Pc, Pw, Ph) of the work location 108 on the article 100 in order to perform work WK on the work location 108. Therefore, the feature position acquisition unit 62, the work position acquisition unit 64, the positioning execution unit 66, and the detection execution unit 68 constitute a device 60 (Figure 2) for detecting the position P of the work location 108.

[0082] In this device 60, the feature position acquisition unit 62 acquires the position Q (coordinates Q4 and Q5) of the detection target feature 106 of the article 100 captured in the image data 200A and 200B of the article 100 captured by the cameras 32A and 32B (step S13). The work position acquisition unit 64 acquires the positions Pc and Pw of the work area 108 in directions Dc and Dw perpendicular to the height direction Dh, based on the position Q and positional relationship PR acquired by the feature position acquisition unit 62 (step S14).

[0083] Then, the positioning execution unit 66 operates the robot 18 that moves the height sensor 20 to perform a positioning operation PO to place the height sensor 20 at the position Pw acquired by the work position acquisition unit 64 (step S21). The detection execution unit 68 operates the height sensor 20 that was placed at position Pw by the positioning operation PO to perform a detection operation DO to detect the first position Ph (z coordinate of the transport coordinate system C5) of the work area 108 in the height direction Dh (step S24).

[0084] This configuration allows for the rapid acquisition of the first position Ph of the work area 108 of an item 100 whose height dimension DM1 is unknown. Conventionally, a three-dimensional visual sensor system has been used to detect an item 100 whose dimension DM is unknown. The three-dimensional visual sensor system irradiates the item 100 with patterned light from a projector, then images the item 100 and generates three-dimensional point cloud image data that represents the visual features of the item 100 as a three-dimensional point cloud. In the case of such a three-dimensional visual sensor system, the computational load for detecting the position of the item 100 is large, and the processing takes time.

[0085] In this embodiment, the first position Ph of the work area 108 of an item 100 whose dimensions DM are unknown can be detected with high accuracy simply by using a pair of cameras 32A and 32B and a height sensor 20. Therefore, compared to the case where a three-dimensional visual sensor system such as the one described above is used, the amount of computation required for the process of detecting the positions Pc, Pw, and Ph of the work area 108 can be reduced, thereby shortening the cycle time. Such high-speed detection processing is particularly advantageous in applications where a large number of items 100 whose dimensions DM are unknown are continuously transported by a transport device 12.

[0086] Furthermore, in the device 60, the robot 18 is configured to move the height sensor 20 in the width direction Dw, and the work position acquisition unit 64 acquires the second position Pw (y coordinates of coordinates P1, P2, and P5) of the work location 108 in the width direction Dw. Then, in the positioning execution unit 66, in the positioning operation PO, moves the height sensor 20 in the width direction Dw by the movement of the robot 18 and places the height sensor 20 in standby position Pw.

[0087] Then, the detection execution unit 68 executes the detection operation DO when the work area 108 reaches directly below the height sensor 20 waiting at the second position Pw due to the transport operation of the transport device 12 (YES in step S23). With this configuration, the height sensor 20 can be positioned at position Pw by a single-axis robot 18, and the first position Ph of the work area 108 can be detected. As a result, the structure of the robot 18 can be simplified, which reduces costs and simplifies the control of the robot 18. This makes it possible to more effectively reduce the amount of computation required for position detection processing.

[0088] Furthermore, in the device 60, the work position acquisition unit 64 acquires the second position Pw, as well as the third position Pc of the work location 108 in the transport direction Dc (for example, the x-coordinates of coordinates P1, P2, and P5), and stores the acquired second position Pw and third position Pc as coordinates P5 of the control coordinate system C (transport coordinate system C5) set for work WK in the position database 202 (Figures 10, 14, and 16). Then, the positioning execution unit 66 executes the positioning operation PO based on the coordinates P5 stored in the position database 202. With this configuration, the acquired positions Pw and Pc of the work location 108 can be effectively managed.

[0089] Furthermore, in the device 60, the control coordinate system C has a transport coordinate system C5 that moves in the transport direction Dc according to the transport amount δ of the transport device 12 transporting the article 100, and the work position acquisition unit 64 stores the second position Pw and the third position Pc as coordinates P5 of the transport coordinate system C5 in the position database 202. With this configuration, as described above, the coordinates P5 stored in the position database 202 remain unchanged, so the management of the position P of the work location 108 can be made more effective and easier.

[0090] Furthermore, in the device 60, the work position acquisition unit 64 updates the position database 202 by adding the first position Ph detected by the detection operation DO to the position database 202 as the coordinate of the control coordinate system C (z coordinate of the transport coordinate system C5) (step S25). With this configuration, the coordinates of the position Ph detected with high accuracy using the height sensor 20 can be added to the position database 202 and managed in the position database 202 for work WK.

[0091] Furthermore, in the device 60, the feature position acquisition unit 62 acquires the position Q of a print on the article 100 or an engraving formed on the article 100 as a detection target feature 106. Here, the article 100, as a product, is often provided with a print or engraving to identify the article 100. According to this embodiment, the position Q can be acquired by utilizing such a print or engraving.

[0092] Next, another example of step S14 in Figure 6 will be described. In this embodiment, a plurality of candidate work locations 108A, 108B, 108C, and 108D are predetermined for the article 100. Examples of candidates 108A, 108B, 108C, and 108 are shown in Figure 17. In the example shown in Figure 17, a total of four candidates 108A, 108B, 108C, and 108D are determined around the detection target feature 106.

[0093] The positional relationship PR between each of the candidates 108A, 108B, 108C, and 108D and the detection target feature 106 is known. For example, candidate 108A is defined as a positional relationship PR in which its work target point P is separated from the feature point Q of the detection target feature 106 by a distance β1 in one direction in the length direction B1 (or E1). Candidate 108C is defined as a positional relationship PR in which its work target point P is separated from the feature point Q by a distance β2 in the other direction in the length direction B1. On the other hand, candidate 108B is defined as a positional relationship PR in which its work target point P is separated from the feature point Q by a distance β3 in one direction in the width direction B2 (or E2). Candidate 108D is defined as a positional relationship PR in which its work target point P is separated from the feature point Q by a distance β4 in the other direction in the width direction B2.

[0094] In this embodiment, in step S14, the processor 54 functions as a work position acquisition unit 64 and selects one of a plurality of candidates 108A, 108B, 108C, and 108D based on the image data 200A or 200B captured in step S12. Figure 18 shows the captured image data 200A.

[0095] Based on the coordinates Q4 (x, y, z) of the feature point Q of the detection target feature 106 in the visual sensor coordinate system C4 and the positional relationship PR data acquired in the previous step S13, the processor 54 defines the working target points P for candidates 108A, 108B, 108C, and 108D in the image data 200A (i.e., the visual sensor coordinate system C4). Then, in the image data 200A, the processor 54 selects one candidate 108A, 108B, 108C, or 108D based on the pixel value PV (for example, brightness from 0 to 255 levels) of a pixel PX that captures at least a portion of the interval 210 from the detection target feature 106 to candidates 108A, 108B, 108C, and 108D.

[0096] Figure 19 shows an enlarged view of the detection target feature 106 captured in the image data 200A. The processor 54 defines an interval 210 in the visual sensor coordinate system C4 from the feature point Q of the detection target feature 106 to the working target point P of candidate 108A. The processor 54 then analyzes the pixel value PV of the pixel PX corresponding to at least a portion of the interval 210 to determine whether candidate 108A is selectable.

[0097] In the example shown in Figure 19, candidate 108A is located outside the image area that captures the surface 104 of the article 100 in the visual sensor coordinate system C4. If such candidate 108A is selected, it is not possible to perform the operation WK on the article 100 appropriately in step S5 (Figure 15) described above. Therefore, it is necessary to exclude candidate 108A. In the example shown in Figure 19, the section 210 includes a region 212 of pixels PX that captures the surface 104 of the article 100 and a region 214 of pixels PX that captures the background BG other than the article 100 (in this embodiment, the surface of the movable part 28). The pixel value PV1 of region 212 and the pixel value PV2 of region 214 are different values ​​from each other.

[0098] As an example, the processor 54 acquires the pixel values ​​PV of all pixels PX included in the interval 210 defined in the sensor coordinate system C4. Then, if the acquired pixel values ​​PV include the pixel values ​​PV2 of region 214, the processor 54 determines that candidate 108A is not selectable. As another example, the processor 54 acquires the pixel values ​​PV of all pixels PX included in the interval 210 and calculates the ratio Rp = N1 / N2 (or N2 / N1) of the number of pixels PX with pixel value PV1 N1 to the number of pixels PX with pixel value PV2 N2. Then, the processor 54 determines that the calculated ratio Rp is equal to a predetermined threshold R ｔｈ Candidate 108A is deemed unselectable if the following conditions (or conditions) are met.

[0099] As yet another example, the processor 54 obtains the pixel value PV of a pixel PX in the work target point P or its neighboring region within the interval 210, and determines whether the pixel value PV is the same as pixel value PV2. If the pixel value PV is the same as pixel value PV2, the processor 54 determines that candidate 108A is not selectable. As yet another example, the difference ρ between the pixel values ​​PV of adjacent pixels PX within the interval 210 is calculated, and the difference ρ is a predetermined threshold ρ ｔｈ If the above conditions are met, candidate 108A is determined to be unselectable.

[0100] Thus, the processor 54 determines whether candidate 108A is selectable based on the pixel value PV of the pixel PX that captures at least a portion of the section 210. In the example shown in Figure 19, the processor 54 determines that candidate 108A is not selectable. Note that the pixel value PV1 that captures the surface 104 of the article 100 and the pixel value PV2 that captures the background BG (movable part 28) may be taught in advance.

[0101] Similarly, the processor 54 determines whether the other candidates 108B, 108C, and 108D are also selectable. For example, in the case of the example in Figure 18, the processor 54 determines that candidate 108B is not selectable, just like candidate 108A. On the other hand, in the case of candidates 108C and 108D, the pixel values ​​PV of all pixels PX within the interval 210 from feature point Q to the respective work target points P of candidates 108C and 108D become the pixel value PV1 that captures the item 100. Therefore, the processor 54 determines that candidates 108C and 108D are selectable.

[0102] The processor 54 may determine whether candidates 108A, 108B, 108C, and 108D are selectable in a predetermined order (for example, candidate 108A → 108B → 108C → 108D). Then, the processor 54 selects the first candidate 108A, 108B, 108C, or 108D that it determined to be selectable. For example, in the example shown in Figure 18, if the determination was made in the order of candidate 108A → 108B → 108C → 108D, the processor 54 would select candidate 108C from among the multiple candidates 108A, 108B, 108C, and 108D.

[0103] In this way, the processor 54 functions as a work position acquisition unit 64 and selects one candidate 108C based on the pixel values ​​PV (PV1, PV2) of pixels PX that capture at least a portion of the interval 210 from the detected target feature 106 (feature point Q) to candidates 108A, 108B, 108C, and 108D in the image data 200A.

[0104] Next, the processor 54 obtains the positions Pc and Pw of the selected candidate 108C in directions Dw and Dc that are orthogonal to the height direction Dh. Specifically, the processor 54 obtains the coordinates P4(x,y) of the work target point P of the selected candidate 108C in the visual sensor coordinate system C4, as in the embodiment described above, and converts the coordinates P4 to coordinates P5(x,y) in the transport coordinate system C5. In this way, the processor 54 obtains the positions Pc and Pw (coordinates P5) of the selected candidate 108C (work target point P) in the width direction Dw and the transport direction Dc.

[0105] As described above, in the apparatus 60 according to this embodiment, a plurality of candidate work locations 108, 108A, 108B, 108C, and 108D, are predetermined for the article 100, each having a known positional relationship PR with the detected target feature 106 (feature point Q). The work location acquisition unit 64 then selects one of the plurality of candidates 108A, 108B, 108C, and 108D (for example, candidate 108C) based on the image data 200A, and acquires the positions Pc and Pw (coordinates P5) of the selected candidate 108C in directions Dc and Dw. With this configuration, it is possible to avoid candidates 108A and 108B that cannot be worked on, thus preventing the work WK at the work location 108 from failing.

[0106] Furthermore, in the apparatus 60 according to this embodiment, the work position acquisition unit 64 defines a plurality of candidates 108A, 108B, 108C, and 108D in the image data 200A, and selects one candidate 108C based on the pixel value PV of a pixel PX that captures at least a portion of the section 210 from the detected target feature 106 (feature point Q) to candidates 108A, 108B, 108C, and 108D (work target point P) in the image data 200A. With this configuration, it becomes possible to determine from the pixel value PV whether the pixel PX corresponding to candidates 108A, 108B, 108C, and 108D captures a background BG other than the item 100. Therefore, candidates 108A and 108B that cannot be worked on can be reliably avoided.

[0107] Note that candidates 108A, 108B, 108C, and 108D shown in Figure 17 are just examples, and any number of candidates may be defined in any positional relationship PR with respect to the detection target feature 106. Also, the section 210 shown in Figure 19 may be defined in any size and shape. For example, the section 210 may be defined as a region having a predetermined area as shown in Figure 19, or it may be defined as a line from feature point Q to work target point P.

[0108] Next, with reference to Figures 20 and 21, a robot system 80 according to another embodiment will be described. The robot system 80 differs from the robot system 10 described above in the robot 82 for moving the sensor. The robot 82 moves the height sensor 20 in the width direction Dw and the transport direction Dc. Specifically, as shown in Figure 22, the robot 82 comprises a robot base 84, a movable part 36 provided on the robot base 84 so as to be movable in the width direction Dw and the transport direction Dc, and a moving mechanism 38 and 86 for moving the movable part 36. The robot base 84 is a four-legged member and is fixed to the floor of the work cell.

[0109] The moving mechanism 86 is a ball screw mechanism, comprising a ball screw (not shown) extending in the transport direction Dc, and a servo motor 42B that rotationally drives the ball screw. The servo motor 42B rotates the ball screw in the forward and reverse directions in response to a command from the control device 24, thereby causing the moving mechanism 38, which engages with the ball screw, to reciprocate in the transport direction Dc.

[0110] The moving mechanism 38 is connected to the moving mechanism 86 so as to be able to reciprocate in the transport direction Dc. The servo motor 42A of the moving mechanism 38, similar to the robot 18 described above, rotates the ball screw 40 (Figure 3) in the forward and reverse directions to reciprocate the movable part 36 in the width direction Dw. In this way, in this embodiment, the robot 82 moves the height sensor 20 attached to the movable part 36 in the width direction Dw and the transport direction Dc by the operation of the moving mechanisms 38 and 86. In addition, the height robot coordinate system C2 described above is set for the robot 82.

[0111] In this embodiment, the device 60 further includes a position determination unit 70 in addition to the feature position acquisition unit 62, work position acquisition unit 64, positioning execution unit 66, and detection execution unit 68 described above. The operation of the robot system 80 will now be described. The processor 54 executes the flow shown in Figure 5. At the start of the flow in Figure 5, the moving mechanism 86 has positioned the moving mechanism 38 and the height sensor 20 at an initial position IP1 in the transport direction Dc (Figures 20 and 22). This initial position IP1 is represented as the x-coordinate of the height robot coordinate system C2.

[0112] In this embodiment, the processor 54 executes the flow shown in Figure 23 as step S3 in Figure 5. In the flow shown in Figure 23, the same step numbers are used for processes that are the same as those in the flow in Figure 11, and redundant explanations are omitted. After the start of step S3, in step S41, the processor 54 functions as a positioning execution unit 66 and executes a first positioning operation PO1 to place the height sensor 20 at the position Pw (second position) acquired in the most recent step S14 (Figure 6).

[0113] Specifically, the processor 54, similar to step S21 described above, searches the current location database 202 for a location where the "Status" in column 208 is "Waiting for detection" and the "No." in column 204 is the highest coordinate P5. _ｎ The processor 54 then reads the read coordinates P5 _ｎ The height of the robot coordinate system C2 is coordinate P2 _ｎ It is converted to (x, y). Then, the processor 54 operates the movement mechanism 38 to move the height sensor 20 in the y-axis direction (i.e., the width direction Dw) of the height robot coordinate system C2, to coordinate P2 _ｎ The operation to position the object at the y-coordinate position Pw (second position) is initiated.

[0114] After step S41, the processor 54 executes step S22 described above. Next, in step S42, the processor 54 determines whether or not the height sensor 20 has been placed at position Pw. Specifically, the processor 54 obtains the current coordinates O2(x,y,z) of the height sensor 20 in the height robot coordinate system C2 based on feedback from the encoder (or Hall element) provided on the servo motor 42A of the moving mechanism 38.

[0115] Then, the processor 54 determines that the y-coordinate of the acquired coordinate O2 is the same as the coordinate P2 acquired in the previous step S41. _ｎ Determine whether it matches the y-coordinate of coordinates O2 and P2. _ｎ If the y coordinates of the two points coincide, the processor 54 determines that the height sensor 20 has been placed at position Pw (i.e., YES) and proceeds to step S43.

[0116] On the other hand, if the processor 54 determines that the result is NO, it returns to step S22. In this way, the processor 54 repeatedly executes the loop of steps S22 and S42 while it determines that the result is NO in step S42. The processor 54 may also repeatedly execute the loop of steps S22 and S42 with a control period τ (specifically, in synchronization with the operation of the transport sensor 14 detecting the transport amount δ).

[0117] In step S43, the processor 54 determines whether the work area 108 of the item 100 has passed the height sensor 20 in the transport direction Dc. Specifically, the processor 54 determines the coordinate P5 read in the most recent step S41. _ｎ Then, using the coordinate transformation matrix MX between the transport coordinate system C5 and the height robot coordinate system C2 at this point, the coordinate P5 _ｎ The height of the robot coordinate system C2 is coordinate P2 _ｎ It is converted to (x, y). Then, the processor 54 processes the coordinate P2 _ｎ It is determined whether the x-coordinate of the object exceeds the x-coordinate of the height sensor 20's coordinate O2 in the height robot coordinate system C2.

[0118] Processor 54 controls coordinate P2 _ｎ If the x-coordinate of is greater than the x-coordinate of coordinate O2, it is determined that the work area 108 has passed the height sensor 20 in the transport direction Dc (i.e., YES), and the process proceeds to step S44. On the other hand, if the processor 54 determines NO, it proceeds to step S22 and executes steps S22 to S26 sequentially, similar to the flow in Figure 11.

[0119] Thus, in this embodiment, when the height sensor 20 is positioned at position Pw during the first positioning operation PO1, the processor 54 determines whether or not the work location 108 (work target point P) has passed the height sensor 20 in the transport direction Dc. Therefore, the processor 54 functions as a positioning determination unit 70 (Figure 21) that determines whether or not the work location 108 has passed the height sensor 20.

[0120] In step S44, the processor 54 functions as a positioning execution unit 66 and executes a second positioning operation PO2 to position the height sensor 20 at position Pc (third position) of the work area 108. Specifically, the processor 54 operates the moving mechanism 86 to move the moving mechanism 38 and the height sensor 20 in the x-axis direction (i.e., the transport direction Dc) of the height robot coordinate system C2.

[0121] After step S44, the processor 54 executes step S22 described above. Next, in step S45, the processor 54 determines whether or not the height sensor 20 has been placed at position Pc. Specifically, the processor 54 obtains the current coordinates O2(x,y,z) of the height sensor 20 in the height robot coordinate system C2 based on feedback from the encoder (or Hall element) provided on the servo motor 42B of the moving mechanism 86.

[0122] Meanwhile, the processor 54 read the coordinates P5 in the most recent step S41. _ｎ Then, using the coordinate transformation matrix MX between the transport coordinate system C5 and the height robot coordinate system C2 at this point, the coordinate P5 _ｎ The height of the robot coordinate system C2 is coordinate P2 _ｎ It is converted to (x, y). Then, the processor 54 determines that the x coordinate of coordinate O2 is the same as coordinate P2 _ｎ Determine whether or not it matches the x-coordinate.

[0123] The processor 54 controls coordinates O2 and P2 _ｎ If the x coordinates of the two points coincide, the processor determines that the height sensor 20 has been placed at position Pc (i.e., YES) and proceeds to step S24. On the other hand, if the processor determines that the result is NO, it proceeds to step S46. Thus, when the processor 54 has placed the height sensor 20 at positions Pw and Pc by the first positioning operation PO1 and the second positioning operation PO2, it executes the detection operation DO in step S24.

[0124] In step S46, the processor 54 determines whether the height sensor 20 has reached the end of the movement stroke SE of the movement mechanism 86. Here, as shown in Figures 20 and 22, the movement mechanism 86 is configured such that the movement mechanism 38 and the height sensor 20 reciprocate between an initial position IP1 and a predetermined end of the movement stroke SE. This end of the movement stroke SE is represented as the x-coordinate of the height robot coordinate system C2.

[0125] The processor 54 determines whether the x-coordinate of coordinate O2 in the height robot coordinate system C2 of the height sensor 20 at this current time matches the x-coordinate of the end of the movement stroke SE. If the x-coordinate of coordinate O2 matches the x-coordinate of the end of the movement stroke SE, the processor 54 determines that the height sensor 20 has reached the end of the movement stroke SE (i.e., YES) and proceeds to step S47. On the other hand, if the processor 54 determines NO, it returns to step S22.

[0126] Thus, while the processor 54 determines NO in steps S45 and S46, it repeatedly executes the loop of steps S22, S45, and S46. The processor 54 may also repeatedly execute the loop of steps S22, S45, and S46 with a control period τ (specifically, in synchronization with the operation of the transport sensor 14 detecting the transport amount δ).

[0127] In step S47, the processor 54 functions as a positioning execution unit 66 and executes a return operation RO to return the height sensor 20 to its initial position IP1. Specifically, the processor 54 operates the moving mechanism 86 to move the height sensor 20 toward the x-coordinate position of the height robot coordinate system C2 corresponding to the initial position IP1, in the direction Dc' opposite to the transport direction Dc. After step S47, the processor 54 executes the above-described step S22.

[0128] Next, in step S48, the processor 54 determines whether the recovery operation RO has been completed. Specifically, the processor 54 determines whether the x-coordinate of coordinate O2 in the height robot coordinate system C2 of the height sensor 20 at the current time matches the x-coordinate representing the initial position IP1. If the x-coordinate of coordinate O2 matches the x-coordinate of the initial position IP1, the processor 54 determines that the recovery operation RO has been completed (i.e., YES) and proceeds to step S49. On the other hand, if the processor 54 determines NO, it returns to step S22.

[0129] Thus, while the processor 54 determines NO in step S48, it repeatedly executes the loop of steps S22 and S48. The processor 54 may also repeatedly execute the loop of steps S22 and S48 with a control period τ (specifically, in synchronization with the operation of the transport sensor 14 detecting the transport amount δ).

[0130] In step S49, the processor 54 updates the location database 202. Specifically, the processor 54 updates the coordinates P5 whose "status" is "waiting for detection" in the location database 202 currently stored in memory 56. _ｎ Read the coordinates, and use the coordinate transformation matrix MX between the transport coordinate system C5 and the height robot coordinate system C2 at this point, and the coordinate P5 _ｎ The height of the robot coordinate system C2 is coordinate P2 _ｎ Convert to (x, y).

[0131] Then, the processor 54 calculates the transformed coordinates P2 _ｎ The processor determines whether the x-coordinate of is greater than the x-coordinate of the initial position IP1. _ｎ Change the "Status" to "Detection Failed". This "Detection Failed" is for coordinate P5 _ｎ This indicates that the position Ph of the work location 108, which was acquired, cannot be detected.

[0132] For example, at the start of step S49, the position database 202 shown in Figure 16 is stored in memory 56, and coordinates P5 of No. 4 _４Assume that the x-coordinate of No. 4 exceeds the initial position IP1. In this case, the processor 54 will move to coordinate P5 of No. 4 as shown in Figure 24. _４ Change the "Status" to "Detection Failed".

[0133] Furthermore, in the position database 202 at the start of step S49, coordinate P5 has a "Status" of "Waiting for detection". _ｎ If multiple coordinates P5 exist, the processor 54 will process the multiple coordinates P5 in the order specified in "No" of column 204. _ｎ Read the coordinate P5 _ｎ Each time the x coordinate is read, it may be determined sequentially whether or not it has exceeded the x coordinate of the initial position IP1. Also, the processor 54 will check the coordinate P5 _ｎ If it is determined that the x-coordinate of the coordinate P5 exceeds the x-coordinate of the initial position IP1, then the coordinate P5 _ｎ Instead of changing the "status" of the coordinate P5 _ｎ The data may be removed from the location database 202. In this way, the processor 54 updates the location database 202. After step S49, the processor 54 proceeds to step S26.

[0134] As described above, in this embodiment, the robot 82 is configured to move the height sensor 20 in the width direction Dw and the transport direction Dc, and the work position acquisition unit 64 acquires the second position Pw of the work location 108 in the width direction Dw and the third position Pc of the work location 108 in the transport direction Dc (step S14 in Figure 6).

[0135] Then, in positioning operation PO1, the positioning execution unit 66 moves the height sensor 20 in the width direction Dw toward the second position Pw by the operation of the robot 82. If the height sensor 20 cannot be positioned at the second position Pw before the work area 108 passes the height sensor 20 in the transport direction Dc (YES in step S43), the robot 82 moves the height sensor 20 toward the transport direction Dc (step S44). With this configuration, even in applications where a large number of items 100 are continuously transported by the transport device 12, the position Ph of the item 100 in the height direction Dh can be detected by tracking the item 100 in the transport direction Dc and executing the detection operation DO.

[0136] Furthermore, in this embodiment, when the positioning determination unit 70 places the height sensor 20 at the second position Pw during the positioning operation PO1 (YES in step S42), it determines whether the work area 108 has passed the height sensor 20 in the transport direction Dc (step S43). Then, if the positioning execution unit 66 determines that the work area 108 has passed the height sensor 20, it moves the height sensor 20 in the transport direction Dc by the operation of the robot 82 (step S44). With this configuration, the height sensor 20 placed at the second position Pw is moved in the transport direction Dc, tracks the item 100 at the second position Pw, and when it is placed at the third position Pc of the work area 108, the detection operation DO can be executed.

[0137] Next, with reference to Figure 25, another example of step S3 performed in the robot system 80 will be described. After the start of step S3, in step S51, the processor 54 reads the positions Pc and Pw of the work location 108. Specifically, the processor 54 reads the position database 202 currently stored in memory 56 where the "Status" in column 208 is "Waiting for detection" and the "No" in column 204 is the highest coordinate P5 _ｎ Read it out.

[0138] In step S52, the processor 54 functions as a positioning determination unit 70 and determines whether the height sensor 20 can be positioned at position Pw of the work area 108 in the width direction Dw before the work area 108 passes the height sensor 20. Specifically, the processor 54 obtains the current coordinates O2 (x, y, z) of the height sensor 20 in the height robot coordinate system C2 based on feedback from encoders (or Hall elements) provided on the servo motors 42A and 42B. The y coordinate of coordinate O2 indicates the current position of the height sensor 20 in the width direction Dw.

[0139] Meanwhile, the processor 54 read the coordinates P5 in the previous step S51. _ｎ Then, using the coordinate transformation matrix MX between the transport coordinate system C5 and the height robot coordinate system C2 at this point, the coordinate P5 _ｎto the coordinates P2 in the height robot coordinate system C2 _ｎ is converted to (x, y, z). The y coordinate of the coordinate P2 _ｎ indicates the current position of the working location 108 (working target point P) in the width direction Dw.

[0140] Then, the processor 54 uses the moving speed V1 of the moving mechanism 38, the conveyance amount δ of the conveyance device 12, the coordinates O2 of the height sensor 20, and the coordinates P2 of the working location 108 _ｎ to determine whether the height sensor 20 can be arranged at the position Pw of the working location 108 before the working location 108 passes through the height sensor 20. Here, even if the processor 54 operates the moving mechanism 38 to move the height sensor 20 from the current position (coordinates O2) to the width direction Dw, before the working location 108 passes, the height sensor 20 may not be arranged at the position Pw (that is, the position of the y coordinate of the coordinate P2 _ｎ ) of the working target point P in the width direction Dw.

[0141] The processor 54 determines whether the height sensor 20 can be arranged at the position Pw before the working location 108 passes, based on the moving speed V1, the conveyance amount δ, the coordinates O2, and P2 _ｎ . If the processor 54 determines YES, it proceeds to step S41 and executes the first positioning operation PO1. After step S41, the processor 54 sequentially executes steps S22 to S26 in the same flow as in FIG. 11. On the other hand, if the processor 54 determines NO, it proceeds to step S53.

[0142] In step S53, the processor 54 functions as the positioning execution unit 66 and executes in parallel the first positioning operation PO1 that moves the height sensor 20 in the width direction Dw by the operation of the moving mechanism 38 and the second positioning operation PO2 that moves the height sensor 20 in the conveyance direction Dc by the operation of the moving mechanism 86. After that, the processor 54 executes step S22.

[0143] Next, in step S54, the processor 54 determines whether the height sensor 20 has been positioned at positions Pc and Pw of the work area 108. Specifically, based on feedback from encoders (or Hall elements) provided on the servo motors 42A and 42B, the processor 54 obtains the current coordinates O2(x, y, z) of the height sensor 20 in the height robot coordinate system C2.

[0144] Meanwhile, the processor 54 read the coordinates P5 in the most recent step S51. _ｎ Then, using the coordinate transformation matrix MX between the transport coordinate system C5 and the height robot coordinate system C2 at this point, the coordinate P5 _ｎ The height of the robot coordinate system C2 is coordinate P2 _ｎ It is converted to (x, y). Then, the processor 54 converts the x and y coordinates of coordinate O2 to coordinate P2 _ｎ Determine whether the x and y coordinates match, respectively.

[0145] Coordinates O2 and P2 _ｎ If the x and y coordinates of the sensor match, the processor 54 determines that the height sensor 20 is positioned at positions Pc and Pw (i.e., YES) and proceeds to step S24. On the other hand, if the processor 54 determines that the result is NO, it proceeds to step S46 and sequentially executes steps S46, S47, S22, S48, and S49, similar to the flow shown in Figure 23.

[0146] While the processor 54 determines NO in steps S54 and S46, it repeatedly executes the loop of steps S22, S54, and S46. The processor 54 may also repeatedly execute the loop of steps S22, S54, and S42 with a control period τ (specifically, in synchronization with the operation of the transport sensor 14 detecting the transport amount δ).

[0147] Thus, in this embodiment, the positioning determination unit 70 determines the second position Pw and the third position Pc (coordinate P2) of the work location 108. _ｎ Based on the current position of the height sensor (coordinate O2) and the work area 108, it is determined whether the height sensor 20 can be positioned at the second position Pw before the work area 108 passes the height sensor 20 (step S52).

[0148] Then, if the positioning execution unit 66 determines that the height sensor 20 cannot be positioned at the second position Pw, the robot 82 moves the height sensor 20 in the width direction Dw and the transport direction Dc. With this configuration, the necessity of the second positioning operation PO2 to move the height sensor 20 in the transport direction Dc can be determined in advance, and the second positioning operation PO2 can be started quickly together with the first positioning operation PO1.

[0149] Next, with reference to Figures 26 and 27, a robot system 90 according to yet another embodiment will be described. In addition to the transport device 12, transport sensor 14, vision sensor 16, robots 18 and 22, height sensor 20, and control device 24 described above, the robot system 90 includes a presence sensor 92 and a robot 94 (second robot) that moves the presence sensor 92. As shown in Figure 28, the robot 94, like the robot 82 described above, includes a robot base 84, a movable part 36, and moving mechanisms 38 and 86, and moves the presence sensor 92 in the width direction Dw and the transport direction Dc.

[0150] The presence sensor 92 detects the presence or absence of an item 100 on the movable part 28 of the transport device 12. For example, the presence sensor 92 is a photoelectric sensor or the like, which irradiates an object with electromagnetic waves (visible light or infrared light) and detects the presence of the item 100 based on the reflected waves from the object. More specifically, the presence sensor 92 has a light-emitting unit that emits electromagnetic waves along the optical axis A2 and a light-receiving unit (neither shown) that receives the reflected waves. Thus, in this embodiment, the presence sensor 92 is a different type of sensor (photoelectric sensor) from the height sensor 20 (laser sensor). However, the presence sensor 92 may be the same type of laser sensor as the height sensor 20.

[0151] The occupancy sensor 92 is fixed to the movable part 36 such that its optical axis A2 is parallel to the height direction Dh. The occupancy sensor 92 supplies detection data γ indicating the presence or absence of the detected item 100 to the control device 24. Specifically, when the occupancy sensor 92 detects the presence of the item 100, it transmits detection data γ1 (for example, an ON signal or a "1" signal) to the control device 24, while when it does not detect the presence of the item 100, it transmits detection data γ0 (for example, an OFF signal or a "0" signal) to the control device 24.

[0152] A robot coordinate system C6 is set on the robot 94. In this embodiment, the robot coordinate system C6 is fixed to the robot base 84 such that its x-axis is parallel to the transport direction Dc, its y-axis is parallel to the width direction Dw, and its z-axis is parallel to the height direction Dh.

[0153] On the other hand, the presence sensor 92 is assigned a presence sensor coordinate system C7. The presence sensor coordinate system C7 is a moving coordinate system that moves in the robot coordinate systems C1, C2, and C6 along with the movement of the presence sensor 92 by the robot 94. The presence sensor coordinate system C7 defines the position and orientation of the presence sensor 92 in the robot coordinate systems C1, C2, and C6 (i.e., the position and direction of the optical axis A2).

[0154] In this embodiment, the presence sensor coordinate system C7 is fixedly set relative to the presence sensor 92 such that its origin is located at the center of the light-emitting part of the presence sensor 92, and its z-axis is parallel to (specifically coincides with) the optical axis A2. The positional relationships between the robot coordinate systems C1, C2, and C6, the sensor coordinate systems C3, C4, and C7, and the transport coordinate system C5 located at the initial position IPc are known through calibration. Therefore, these control coordinate systems C1 to C7 can be coordinate-transformed from one another via a known coordinate transformation matrix MX.

[0155] In this embodiment, in addition to the feature position acquisition unit 62, work position acquisition unit 64, positioning execution unit 66, detection execution unit 68, and positioning determination unit 70 described above, the device 60 further includes a sensor movement unit 72, a dimension acquisition unit 74, and a dimension determination unit 76. The operation of the robot system 90 will now be described. The processor 54 executes the flow shown in Figure 5.

[0156] At the start of the flow shown in Figure 5, the moving mechanisms 38 and 86 of the robot 94 position the occupancy sensor 92 at an initial position IP2 in the transport direction Dc and the width direction Dw. This initial position IP2 is determined, for example, as the position of one end of the movement stroke of the occupancy sensor 92 by the moving mechanism 38, and the position of the end of the movement stroke of the occupancy sensor 92 by the moving mechanism 86 in the direction Dc' opposite to the transport direction Dc.

[0157] In this embodiment, the processor 54 executes the flow shown in Figure 29 as step S1 in Figure 5. In the flow shown in Figure 29, the same step numbers are used for processes that are the same as those in the flow in Figure 6, and redundant explanations are omitted. After the start of step S1, the processor 54 sequentially executes steps S11 to S13.

[0158] In this embodiment, in step S13, the processor 54 determines the angle θ between the length direction B1 or width direction B2 specified in the visual sensor coordinate system C4 and the transport direction Dc. For example, the processor 54 defines the length direction B1 or width direction B2 specified in the visual sensor coordinate system C4 in the transport coordinate system C5 using the coordinate transformation matrix MX between the visual sensor coordinate system C4 and the transport coordinate system C5 set at the initial position IPc. Then, the processor 54 determines the angle θ between the length direction B1 or width direction B2 and the x-axis direction of the transport coordinate system C5.

[0159] Figure 30 illustrates the angle θ between the length direction B1 and the x-axis direction of the transport coordinate system C5 (i.e., the transport direction Dc). This angle θ represents the orientation of the item 100 (specifically, the angle around the z-axis of the transport coordinate system C5). Thus, the processor 54 acquires the angle θ as orientation data for the item 100.

[0160] After step S13, in step S61, the processor 54 operates the robot 94 to move the occupancy sensor 92 in the width direction Dw and the transport direction Dc. Specifically, the processor 54 activates the occupancy sensor 92 to start detecting the presence or absence of the item 100, and moves the occupancy sensor 92 from the initial position IP2 to the width direction Dw and the transport direction Dc by the operation of the moving mechanisms 38 and 86.

[0161] In this case, the processor 54 may move the occupancy sensor 92 in the transport direction Dc at the same speed V2 as the transport speed V2 of the transport device 12 using the moving mechanism 86, based on the transport amount δ obtained from the transport sensor 14. Thus, in this embodiment, the processor 54 functions as a sensor moving unit 72 (Figure 27) that operates the robot 94 to move the occupancy sensor 92 in the width direction Dw and the transport direction Dc.

[0162] As a result of the robot 94's movements, the presence sensor 92 moves relative to the item 100 along the trajectory 110, as shown in Figure 30. While moving in this manner, the presence sensor 92 supplies detection data γ (ON / OFF signal or 0 / 1 signal) indicating the presence or absence of the item 100 to the control device 24.

[0163] For example, in the example shown in Figure 30, suppose the presence sensor 92 moves along the trajectory 110 in the positive y-axis direction of the presence robot coordinate system C6. In this case, when the presence sensor 92 reaches the position of point 112 on the outer edge 102 of the item 100, the detection data γ transmitted by the presence sensor 92 switches from detection data γ0 (OFF signal or "0" signal) indicating that the presence of the item 100 has not been detected to detection data γ1 (ON signal or "1" signal) indicating that the presence of the item 100 has been detected.

[0164] After that, when the occupancy sensor 92 crosses the article 100 and exceeds the point 114 on the outer edge 102, the detection data γ transmitted by the occupancy sensor 92 switches from the detection data γ1 to the detection data γ0. When the detection data γ acquired from the occupancy sensor 92 switches from the detection data γ0 to the detection data γ1 at the first time point, the processor 54 determines the coordinates U6 of the occupancy sensor 92 (specifically, the origin of the occupancy sensor coordinate system C7) in the occupancy robot coordinate system C6 _１ (x, y, z) is acquired.

[0165] Further, when the detection data γ acquired from the occupancy sensor 92 switches from the detection data γ1 to the detection data γ0 at the second time point, the processor 54 determines the coordinates U6 of the occupancy sensor 92 in the occupancy robot coordinate system C6 _２ (x, y, z) is acquired. The coordinates U6 _１ of the y coordinate and the coordinates U6 _２ of the y coordinate, the difference Δy therebetween becomes data indicating the distance Δy from the point 112 to the point 114 in FIG. 30.

[0166] In step S62, the processor 54 acquires the dimension DM of the article 100 based on the detection data γ detected by the occupancy sensor 92. Specifically, the processor 54 acquires the width dimension DM2 of the article 100 based on the distance Δy acquired as described above and the angle θ indicating the posture of the article 100. For example, the processor 54 obtains the width dimension DM2 from the formula DM2 = Δy / cos θ. Thus, in the present embodiment, the processor 54 functions as a dimension acquisition unit 74 (FIG. 27) that acquires the dimension DM (in the present embodiment, the width dimension DM2) of the article 100 based on the detection data γ.

[0167] In step S63, the processor 54 determines whether the dimension DM obtained in the previous step S62 satisfies the reference ST. This reference ST is predetermined as an acceptable range for dimension DM in order to select the item 100 to be worked on. In this embodiment, the processor 54 determines whether the width dimension DM2 obtained in the previous step S62 is within the acceptable range defined as the reference ST. If the width dimension DM2 is within the acceptable range, the processor 54 determines that dimension DM satisfies the reference ST (i.e., YES) and proceeds to step S14.

[0168] On the other hand, if the width dimension DM2 is outside the allowable range, the processor 54 determines that dimension DM does not meet the reference ST (i.e., NO) and proceeds to step S16. Thus, in this embodiment, the processor 54 functions as a dimension determination unit 76 (Figure 27) that determines whether or not dimension DM meets the reference ST.

[0169] If NO is determined in step S63, the processor 54 does not execute steps S14 and S15, and the coordinates P5 of the article 100, whose position Q of the detected target feature 106 was obtained in step S13, are not executed. _ｎ The item 100 is not stored in the location database 202. As a result, the processor 54 does not perform steps S3 (detection operation DO) and S5 (work WK) in Figure 5 for the item 100.

[0170] As described above, in the apparatus 60 according to this embodiment, the sensor moving unit 72 operates the second robot 94 to move the occupancy sensor 92 in the width direction Dw (step S61). The dimension acquisition unit 74 acquires the dimension DM (width dimension DM2) of the article 100 based on the detection data γ (γ1 or γ2) detected by the occupancy sensor 92 while it is being moved in the width direction Dw by the second robot 94 (step S62).

[0171] Furthermore, the dimension determination unit 76 determines whether the dimension DM obtained by the dimension acquisition unit 74 satisfies the reference ST (step S63). The detection execution unit 68 then does not perform the detection operation DO (step S24) on articles 100 having dimension DM that the dimension determination unit 76 has determined does not satisfy the reference ST. With this configuration, articles 100 that are not subject to processing can be effectively identified, and the detection operation DO can be avoided from being performed on such articles 100. This reduces the amount of computation required for the detection operation DO, thereby shortening the cycle time.

[0172] In this embodiment, the case described is when the robot 94 moves the occupancy sensor 92 in the width direction Dw and the transport direction Dc. However, the robot 94 is not limited to this, and may have a robot base 34, a movable part 36 and a moving mechanism 38, similar to the robot 18 shown in Figure 3, and be configured to move the occupancy sensor 92 only in the width direction Dw. In this case as well, the processor 54 can obtain the width dimension DM2 of the article 100 in step S62 based on the transport amount δ obtained from the transport sensor 14, the distance Δy described above, and the angle θ.

[0173] Furthermore, in this embodiment, we have described the case in which the processor 54 obtains the width dimension DM2 shown in Figure 30 as the dimension DM of the article 100. However, it should be understood that the processor 54 can obtain the length dimension DM3 of the article 100 in the length direction B1 from the detection data γ of the occupancy sensor 92, depending on the angle θ indicating the orientation of the article 100. Note that the robot 94 may be applied to the robot system 10.

[0174] The processor 54 may execute the flow shown in Figure 5 according to the computer program PG stored in memory 56. Furthermore, the functions of the device 60 executed by the processor 54 (feature position acquisition unit 62, work position acquisition unit 64, positioning execution unit 66, detection execution unit 68, positioning determination unit 70, sensor movement unit 72, dimension acquisition unit 74, dimension determination unit 76) may be functional modules realized by the computer program PG.

[0175] Furthermore, the computer program PG may include a first computer program PG1 that causes the processor 54 to execute the main flow in Figure 5, a second computer program PG2 that causes the processor 54 to execute the subflow of step S1 (Figures 6 and 29), a third computer program PG3 that causes the processor 54 to execute the subflow of step S3 (Figures 11, 23, and 25), and a fourth computer program PG4 that causes the processor 54 to execute the subflow of step S5 (Figure 15).

[0176] In step S14 described above, the processor 54 may further acquire the z-coordinate of the work target point P in the transport coordinate system C5 as a provisional coordinate. In this case, the processor 54 acquires the coordinates P5 (x, y, z) of the work target point P. In this case, the processor 54 may set a predetermined coordinate value (for example, z = 0) or the same coordinate value as the z-coordinate of the coordinate Q5 acquired in step S13 as a provisional coordinate value for the z-coordinate of coordinate P5.

[0177] Then, the processor 54 stores the coordinates P5 (x, y, z) in the position database 202 in step S15, and when it obtains the z coordinate of the transport coordinate system C5 of the work target point P by the detection operation DO in step S24, it may replace the z coordinate of coordinates P5 stored in the position database 202 as a temporary coordinate value with the z coordinate obtained by the detection operation DO, thereby additionally storing the z coordinate obtained by the detection operation DO in the position database 202.

[0178] In the embodiments described above, the case where the detection target feature 106 is printed or engraved was mentioned. However, the detection target feature 106 is not limited to these. For example, the processor 54 may function as a feature position acquisition unit 62 and acquire the position Q of the outer edge 102 of the article 100 as the detection target feature 106. Specifically, in step S13 described above, the processor 54 may identify the outer edge 102 of the article 100 in the visual sensor coordinate system C4 as the detection target feature 106 and acquire the coordinates Q4 (x, y, z) of the feature point Q of the outer edge 102 (e.g., center point, vertex, midpoint of one side, etc.) in the visual sensor coordinate system C4.

[0179] In the above embodiment, the case in which the visual sensor 16 has a pair of cameras 32A and 32B was described, but the visual sensor 16 is not limited to this, and may have a single camera 32A (i.e., a monocular camera). In this case, in step S13, the processor 54 obtains the coordinates Q4 (x, y, z) of the feature point Q in the visual sensor coordinate system C4 based on the image dimension DMf of the detection target feature 106 captured in the image data 200A acquired in step S12.

[0180] Here, if the dimension DMf of the detection target feature 106 is known, the distance from the camera 32A (the origin of the visual sensor coordinate system C4) to the detection target feature 106 (i.e., the z-coordinate of the visual sensor coordinate system C4) can be determined from the dimension DMf of the detection target feature 106 in the image data 200A. Therefore, the processor 54 can obtain the coordinates Q4(x, y, z) of the feature point Q in the visual sensor coordinate system C4 based on the dimension DMf in the image data 200A.

[0181] In the above embodiment, the case in which the coordinates P5 of the transport coordinate system C5 are stored in the position database 202 was described. However, the processor 54 is not limited to this, and in step S14, it may acquire the positions Pw and Pc of the work location 108 as, for example, the coordinates P1 or P2 of the robot coordinate system C1 or C2, and store them in the position database 202 in step S15. In this case, each time the above step S22 is executed, the processor 54 updates the x-coordinate of the coordinates P1 or P2 stored in the position database 202 according to the transport amount δ. Furthermore, the transport coordinate system C5 may be a fixed coordinate system fixed to the base portion 26 of the transport device 12.

[0182] Furthermore, the position database 202 shown in Figures 10, 14, 16, and 24 is merely an example, and the processor 54 may create a position database 202 with any data structure. Alternatively, the processor 54 may store the coordinates of the positions Pw and Pc of the work location 108 acquired in step S14 in memory 56 in chronological order without creating a position database 202. Note that the shape of the detected target feature 106 is not limited to a rectangle, but may be any shape such as a polygon, circle, or ellipse. Also, robots 18, 22, 82, and 94 may be of any type. For example, robots 18, 82, or 94 may be parallel link robots or vertical articulated robots.

[0183] Although the present disclosure has been described in detail above, it is not limited to the individual embodiments described above. These embodiments can be added, replaced, modified, partially deleted, etc., in any way that does not depart from the gist of the present disclosure or from the spirit of the present disclosure derived from the claims and their equivalents. Furthermore, these embodiments can be implemented in combination. For example, the order of operations and processes in the embodiments described above are shown as examples only and are not limited thereto. The same applies when numerical values ​​or mathematical formulas are used in the description of the embodiments described above.

[0184] As described above, this disclosure describes the following embodiments. (Embodiment 1) A device 60 for detecting the position P of a work location 108 on an article 100 being transported by a transport device 12 in order to perform a predetermined operation WK on the work location 108, the device comprising: a feature position acquisition unit 62 that acquires the position Q of a detection target feature 106 of the article 100 as captured in the image data 200A, 200B of the article 100 based on image data 200A, 200B of the article 100 captured by cameras 32A, 32B; and a direction Dw perpendicular to the height direction Dh of the transport device 12, based on the position Q acquired by the feature position acquisition unit 62 and a known positional relationship PR between the work location 108 and the detection target feature 106, The apparatus 60 comprises a work position acquisition unit 64 that acquires the positions Pw and Pc of a work location 108 in Dc; a positioning execution unit 66 that operates robots 18 and 82 that move height sensors 20 for detecting the position Ph of an article 100 in the height direction Dh to perform a positioning operation PO to place the height sensors 20 at the positions Pw and Pc acquired by the work position acquisition unit 64; and a detection execution unit 68 that operates the height sensors 20 placed at positions Pw and Pc by the positioning operation PO to perform a detection operation DO to detect a first position Ph of the work location 108 in the height direction Dh. (Aspect 2) The apparatus 60 as in Aspect 1, wherein the robots 18 and 82 are configured to move the height sensor 20 in the width direction Dw of the transport device 12, the work position acquisition unit 64 acquires a second position Pw of the work location 108 in the width direction Dw, the positioning execution unit 66, in positioning operation PO, moves the height sensor 20 in the width direction Dw by the operation of the robots 18 and 82 and places the height sensor 20 waiting at the second position Pw, and the detection execution unit 68 executes detection operation DO when the work location 108 reaches directly below the height sensor 20 waiting at the second position Pw by the transport operation of the transport device 12.(Aspect 3) The apparatus 60 according to aspect 1 or 2, wherein the work position acquisition unit 64 acquires a second position Pw of the work location 108 in the width direction Dw of the conveying device 12 and a third position Pc of the work location 108 in the conveying direction Dc of the conveying device 12, stores the acquired second position Pw and third position Pc in the position database 202 as coordinates P1, P2, and P5 of the control coordinate system C set for work WK, and the positioning execution unit 66 executes a positioning operation PO based on the coordinates P1, P2, and P5 stored in the position database 202. (Aspect 4) The apparatus 60 according to aspect 3, wherein the control coordinate system C has a conveying coordinate system C5 that moves in the conveying direction Dc according to the conveying amount δ of the article 100 that the conveying device 12 conveys, and the work position acquisition unit 64 stores the second position Pw and third position Pc in the position database 202 as coordinates P5 of the conveying coordinate system C5. (Aspect 5) The apparatus 60 according to aspect 3 or 4, wherein the work position acquisition unit 64 updates the position database 202 by additionally storing the first position Ph detected by the detection operation DO as coordinates P1, P2, P5 of the control coordinate system C in the position database 202. (Aspect 6) The apparatus 60 according to any one of aspects 1 to 5, wherein the robot 82 is configured to move the height sensor 20 in the width direction Dw and the transport direction Dc of the transport device 12, the work position acquisition unit 64 acquires the second position Pw of the work location 108 in the width direction Dw and the third position Pc of the work location 108 in the transport direction Dc, and the positioning execution unit 66 moves the height sensor 20 toward the second position Pw in the width direction Dw by the operation of the robot 18 in the positioning operation PO, and if the height sensor 20 cannot be positioned at the second position Pw before the work location 108 passes the height sensor 20 in the transport direction Dc, the robot 18 moves the height sensor 20 toward the transport direction Dc. (Aspect 7) The apparatus 60 according to aspect 6, further comprising a positioning determination unit 70 that determines whether or not the work area 108 has passed the height sensor 20 in the transport direction Dc when the height sensor 20 is positioned at a second position Pw during the positioning operation PO, and the positioning execution unit 66 moves the height sensor 20 in the transport direction Dc by the operation of the robot 82 when it is determined that the work area 108 has passed the height sensor 20.(Aspect 8) The apparatus 60 according to aspect 6, further comprising a positioning determination unit 70 that determines whether the height sensor 20 can be positioned at the second position Pw before the work area 108 passes the height sensor 20, based on the second position Pw and the third position Pc and the current position of the height sensor 20, wherein the positioning execution unit 66 moves the height sensor 20 in the width direction Dw and the transport direction Dc by the operation of the robot 82 when it is determined that the height sensor 20 cannot be positioned at the second position Pw, and the apparatus 60 according to aspect 6. (Aspect 9) The apparatus 60 according to any one of aspects 1 to 8, wherein the article 100 has a predetermined number of candidate work locations 108, 108A, 108B, 108C, and 108D, whose positional relationship PR with the detection target feature 106 is known, and the work location acquisition unit 64 selects one of the candidate 108A, 108B, 108C, and 108D based on the image data 200A, and acquires the positions Pw and Pc of the selected candidate 108C in orthogonal directions Dw and Dc. (Aspect 10) The apparatus 60 according to aspect 9, wherein the work position acquisition unit 64 defines a plurality of candidates 108A, 108B, 108C, and 108D in the image data 200A, and selects one candidate 108C based on the pixel value PV of a pixel PX that captures at least a portion of the interval 210 from the detection target feature 106 to candidates 108A, 108B, 108C, and 108D in the image data 200A. (Aspect 11) The apparatus 60 according to any one of aspects 1 to 10, comprising: a sensor moving unit 72 that moves a presence sensor 92 for detecting the presence or absence of an article 100 in the width direction Dw of the transport device 12 by operating a second robot 94 that moves the presence sensor 92 in the width direction Dw; a dimension acquisition unit 74 that acquires the dimension DM of the article 100 based on detection data γ detected by the presence sensor 92 while it is being moved in the width direction Dw by the second robot 94; and a dimension determination unit 76 that determines whether the dimension DM acquired by the dimension acquisition unit 74 satisfies a reference ST, wherein the detection execution unit 68 does not perform the detection operation DO on an article 100 having a dimension DM that the dimension determination unit 76 has determined does not satisfy the reference ST. (Aspect 12) The apparatus 60 according to any one of aspects 1 to 11, wherein the feature position acquisition unit 62 acquires the position Q of the outer edge 102 of the article 100 as a detection target feature 106, such as printing on the article 100, an engraving formed on the article 100, or the position Q of the outer edge 102 of the article 100.(Aspect 13) A robot system 10, 80, 90 comprising cameras 32A, 32B for imaging articles 100 being transported by a transport device 12, a height sensor 20 for detecting the position Ph of the article 100 in the height direction Dh of the transport device 12, robots 18, 82 for moving the height sensor 20, and a device 60 as described in any of aspects 1 to 12. (Aspect 14) A method for detecting the position P of a work location 108 on an article 100 being transported by a transport device 12 in order to perform a predetermined operation WK on the work location 108, the method comprising: obtaining the position Q of a detection target feature 106 of the article 100 captured in the image data 200A, 200B of the article 100 by cameras 32A, 32B, and based on the obtained position Q and a known positional relationship PR between the work location 108 and the detection target feature 106, the position P of the work location 108 in the height direction Dh of the transport device 12 A method comprising: acquiring the positions Pw and Pc of a work location 108 in directions Dw and Dc perpendicular to h; operating a robot 18 to move a height sensor 20 for detecting the position Ph of an article 100 in the height direction Dh; performing a positioning operation PO to place the height sensor 20 at the acquired positions Pw and Pc of the work location 108; and performing a detection operation DO to operate the height sensor 20 placed at positions Pw and Pc by the positioning operation PO to detect the position Ph of the work location 108 in the height direction Dh. (Aspect 15) A computer program PG that causes a processor 54 to execute the method described in aspect 14.

[0185] 10, 80, 90 Robot system 12 Transport device 14 Transport sensor 16 Vision sensor 20 Height sensor 92 Occupancy sensor 18, 22, 82, 94 Robot 24 Control device 32A, 32B Camera 60 Device 62 Feature position acquisition unit 64 Work position acquisition unit 66 Positioning execution unit 68 Detection execution unit 70 Positioning determination unit 72 Sensor movement unit 74 Dimension acquisition unit 76 Dimension determination unit 100 Item 102 Outer edge 106 Detected target feature 108 Work location 200A Image data

Claims

1. A device for detecting the position of a work location on an article being transported by a transport device, the device comprising: a feature position acquisition unit that acquires the position of a detection target feature of the article captured in image data of the article by a camera; a work position acquisition unit that acquires the position of the work location in a direction perpendicular to the height direction of the transport device, based on the position acquired by the feature position acquisition unit and a known positional relationship between the work location and the detection target feature; a positioning execution unit that operates a robot to move a height sensor for detecting the position of the article in the height direction and performs a positioning operation to place the height sensor at the position acquired by the work position acquisition unit; and a detection execution unit that operates the height sensor placed at the position by the positioning operation and performs a detection operation to detect the first position of the work location in the height direction.

2. The apparatus according to claim 1, wherein the robot is configured to move the height sensor in the width direction of the transport device, the work position acquisition unit acquires the second position of the work location in the width direction, the positioning execution unit, in the positioning operation, moves the height sensor in the width direction by the movement of the robot and places the height sensor in standby position, and the detection execution unit performs the detection operation when the work location reaches directly below the height sensor waiting in the second position by the transport operation of the transport device.

3. The apparatus according to claim 1, wherein the work position acquisition unit acquires a second position of the work location in the width direction of the conveying device and a third position of the work location in the conveying direction of the conveying device, stores the acquired second position and third position as coordinates of a control coordinate system set for the work in a position database, and the positioning execution unit performs the positioning operation based on the coordinates stored in the position database.

4. The apparatus according to claim 3, wherein the control coordinate system has a transport coordinate system that moves in the transport direction according to the amount of transported goods transported by the transport device, and the work position acquisition unit stores the second position and the third position as the coordinates of the transport coordinate system in the position database.

5. The apparatus according to claim 3, wherein the work position acquisition unit updates the position database by additionally storing the first position detected by the detection operation as coordinates of the control coordinate system in the position database.

6. The apparatus according to claim 1, wherein the robot is configured to move the height sensor in the width direction and the transport direction of the transport device, the work position acquisition unit acquires a second position of the work location in the width direction and a third position of the work location in the transport direction, the positioning execution unit, in the positioning operation, moves the height sensor in the width direction toward the second position by the operation of the robot, and if the height sensor cannot be positioned at the second position before the work location passes the height sensor in the transport direction, the robot moves the height sensor toward the transport direction.

7. The apparatus according to claim 6, further comprising a positioning determination unit that determines whether the work location has passed the height sensor in the transport direction when the height sensor is positioned at the second position during the positioning operation, wherein the positioning execution unit moves the height sensor in the transport direction by the operation of the robot when it is determined that the work location has passed the height sensor.

8. The apparatus according to claim 6, further comprising a positioning determination unit that determines whether the height sensor can be positioned at the second position before the work area passes the height sensor, based on the second position, the third position and the current position of the height sensor, wherein if the positioning execution unit determines that the height sensor cannot be positioned at the second position, the robot moves the height sensor in the width direction and the transport direction.

9. The apparatus according to claim 1, wherein a plurality of candidate work locations are predetermined for the article, the positional relationship with the detection target feature is known, the work location acquisition unit selects one of the plurality of candidates based on the image data, and acquires the position of the selected candidate in the orthogonal direction.

10. The apparatus according to claim 9, wherein the work position acquisition unit defines the plurality of candidates in the image data and selects one candidate based on the pixel value of a pixel that captures at least a portion of the interval from the detection target feature to the candidate in the image data.

11. The apparatus according to claim 1, comprising: a sensor moving unit that moves a presence sensor for detecting the presence or absence of an article in the width direction of the transport device by operating a second robot that moves in the width direction of the transport device; a dimension acquisition unit that acquires the dimensions of the article based on detection data detected by the presence sensor while it is being moved in the width direction by the second robot; and a dimension determination unit that determines whether the dimensions acquired by the dimension acquisition unit meet a standard, wherein the detection execution unit does not perform the detection operation on an article having dimensions that the dimension determination unit has determined do not meet the standard.

12. The apparatus according to claim 1, wherein the feature position acquisition unit acquires the position of the printing on the article, the markings formed on the article, or the outer edge of the article as the detection target feature.

13. A robot system comprising: a camera for imaging an article being transported by a transport device; a height sensor for detecting the position of the article in the height direction of the transport device; a robot for moving the height sensor; and the device according to claim 1.

14. A method for detecting the position of a work location on an article being transported by a transport device in order to perform a predetermined operation on the work location, the method comprising: obtaining the position of a detection target feature of the article captured in the image data based on image data of the article captured by a camera; obtaining the position of the work location in a direction perpendicular to the height direction of the transport device based on the obtained position and a known positional relationship between the work location and the detection target feature; operating a robot to move a height sensor for detecting the position of the article in the height direction to perform a positioning operation to place the height sensor at the obtained position of the work location; and operating the height sensor placed at the position by the positioning operation to perform a detection operation to detect the position of the work location in the height direction.

15. A computer program that causes a processor to perform the method described in claim 14.

Images