Device for detecting the position of a work site, robotic system, method and computer program

Abstract

Sometimes, a prescribed work is performed on an article of unknown size carried by a conveyance device. At this time, a technique is required to detect the position of the carried article at high speed and high accuracy. A device (60) that detects the position of a work site has: a characteristic position acquisition section (62) that acquires the position of a detection target characteristic of an article (100) from image data of the article captured by a camera (32A, 32B); a work position acquisition section (64) that acquires the position of the work site in a direction orthogonal to the height direction of the conveyance device (12) from the position (Q) acquired by the characteristic position acquisition section (62); a positioning execution section (66) that causes a robot (18) that moves a height sensor (20) to act to perform a positioning action that configures the height sensor (20) at the position acquired by the work position acquisition section (64); and a detection execution section (68) that causes the height sensor (20) configured by the positioning action to act to perform a detection action that detects the position of the work site in the height direction.

Description

Technical Field

[0001] This disclosure relates to apparatus, robot systems, methods, and computer programs for detecting the location of work sites. Background Technology

[0002] Robotic systems that utilize laser sensors moved by the robot to measure the distance to the work object are known (e.g., Patent Document 1).

[0003] Existing technical documents

[0004] Patent documents

[0005] Patent Document 1: Japanese Patent Application Publication No. 2019-63955 Summary of the Invention

[0006] The problem that the invention aims to solve

[0007] Sometimes, prescribed operations are performed on items of unknown size that are being transported by a conveyor system. In such cases, a technology for detecting the position of the transported items at high speed and with high precision is required.

[0008] Methods for solving problems

[0009] In one aspect of this disclosure, an apparatus is provided for detecting the position of a work part on an item being transported by a conveying device in order to perform a prescribed operation on that work part. The apparatus comprises: a feature position acquisition unit that acquires the position of a detection target feature of the item as displayed in image data of the item captured by a camera; a work position acquisition unit that acquires the position of the work part in a direction orthogonal to the height direction of the conveying device based on the position acquired by the feature position acquisition unit and a known positional relationship between the work part and the detection target feature; a positioning execution unit that moves a robot that moves a height sensor to perform a positioning action that positions the height sensor at the position acquired by the work position acquisition unit, wherein the height sensor is used to detect the position of the item in the height direction; and a detection execution unit that moves the height sensor positioned at the location by the positioning action to perform a detection action that detects a first position of the work part in the height direction.

[0010] In other aspects of this disclosure, a method for detecting the position of a work part on an item being transported by a conveying device to perform a prescribed operation is provided. This method involves obtaining the position of a detection target feature of the item as shown in image data captured by a camera, obtaining the position of the work part in a direction orthogonal to the height direction of the conveying device based on the obtained position and a known positional relationship between the work part and the detection target feature, and then moving a robot with a movable height sensor to perform a positioning action that places the height sensor at the obtained position of the work part. The height sensor is used to detect the position of the item in the height direction, and the height sensor, positioned by the positioning action, is activated to perform a detection action that detects the position of the work part in the height direction. Attached Figure Description

[0011] Figure 1 This is a schematic diagram of a robot system according to one embodiment.

[0012] Figure 2 yes Figure 1 The diagram shows a block diagram of the robot system.

[0013] Figure 3 yes Figure 1 The image shown is a 3D view of a robot used for moving height sensors.

[0014] Figure 4 This is a diagram of an embodiment of the workpiece, with region (a) representing a side view of the workpiece and region (b) representing a top view of the workpiece.

[0015] Figure 5 It means Figure 1 The flowchart shows an example of the motion flow of the robot system.

[0016] Figure 6 It means Figure 5 The flowchart is an example of step S1 in the process.

[0017] Figure 7 This indicates the state in which an item is placed on a conveying device.

[0018] Figure 8 An example representing image data of an object captured by a camera.

[0019] Figure 9 Indicates to Figure 7 The conveying device shown has its conveying coordinate system set.

[0020] Figure 10 This represents an example of a location database.

[0021] Figure 11 It means Figure 5The flowchart is an example of step S3 in the process.

[0022] Figure 12 Indicates passage Figure 11 Step S21 in the process describes the positioning state of the position sensor.

[0023] Figure 13 express Figure 12 The item shown is being moved by a conveyor.

[0024] Figure 14 Indicates in Figure 11 The updated location database in step S25.

[0025] Figure 15 It means Figure 5 The flowchart is an example of step S5 in the process.

[0026] Figure 16 This indicates the location database updated after the task.

[0027] Figure 17 This is an example of a candidate for a work location.

[0028] Figure 18 This is an example of image data of an object captured by a camera, showing an overlay representation. Figure 17 The candidate work location shown.

[0029] Figure 19 It is Figure 18 The image shown is a magnified view of a portion of the image data.

[0030] Figure 20 This is a schematic diagram of a robot system with other implementation methods.

[0031] Figure 21 yes Figure 20 The diagram shows a block diagram of the robot system.

[0032] Figure 22 yes Figure 20 The image shown is a 3D view of a robot used for moving height sensors.

[0033] Figure 23 It means Figure 5 The flowchart of another example of the process of step S3 in the process.

[0034] Figure 24 Indicates in Figure 23 The updated location database in step S49.

[0035] Figure 25 It means Figure 5 The flowchart of another example of the process of step S3 in the process.

[0036] Figure 26 This is a schematic diagram of another robot system according to a different implementation.

[0037] Figure 27 yes Figure 26 The diagram shows a block diagram of the robot system.

[0038] Figure 28 yes Figure 26 The image shown is a 3D view of a robot used for moving in-situ sensors.

[0039] Figure 29 It means Figure 5 The flowchart of another example of the process of step S1 in the process.

[0040] Figure 30 This is a diagram used to illustrate the relative movement trajectory of an in-situ sensor relative to an object. Detailed Implementation

[0041] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the accompanying drawings. Furthermore, in the various embodiments described below, the same elements are labeled with the same reference numerals, and repeated descriptions are omitted. First, refer to... Figures 1-3 The robot system 10 according to one embodiment will be described. The robot system 10 includes: a conveying device 12, a conveying sensor 14, a vision sensor 16, a robot 18 for sensor movement, and a height sensor 20. Figure 2 , Figure 3 ), the robot 22 for operation and the control device 24.

[0042] The conveying device 12 is, for example, a belt conveyor, which conveys the item 100 in the conveying direction Dc. Specifically, the conveying device 12 includes: a base 26, a movable part 28, and a servo motor 30. Figure 2 The base 26 is fixedly mounted on the ground of the work unit. The movable part 28 is mounted on the base 26 so as to be movable in the conveying direction Dc, and the article 100 is placed on the movable part 28.

[0043] A servo motor 30 drives a movable part 28 to transport an article 100 placed on the movable part 28 along a transport direction Dc. The transport device 12 has a transport direction Dc, a width direction Dw, and a height direction Dh. The width direction Dw is orthogonal to the transport direction Dc. The height direction Dh is, for example, parallel to the vertical direction and orthogonal to the width direction Dw and the transport direction Dc.

[0044] The transport sensor 14 detects the transport amount δ of the article 100 transported by the transport device 12. In this embodiment, the transport sensor 14 is provided on the base portion 26 and has a rotating roller that abuts against the movable portion 28 and an encoder (or Hall element) for detecting the rotational speed of the rotating roller. The transport sensor 14 repeatedly detects the transport amount δ of the article 100 transported by the transport device 12 in the transport direction Dc at a predetermined control cycle τ (e.g., τ = 10 [msec]), and sequentially supplies the detected transport amount δ data to the control device 24. Alternatively, the transport sensor 14 may also be provided on the servo motor 30 of the transport device 12.

[0045] The vision sensor 16 is positioned upstream of the robot 18 (e.g., near the upstream end of the conveying device 12). In this embodiment, the vision sensor 16 has a pair of cameras 32A and 32B (i.e., stereo cameras). Cameras 32A and 32B are two-dimensional cameras, each having an image sensor (CCD, CMOS, etc.) and an optical lens (collimating lens, focusing lens, etc.) that guides the image of the subject to the image sensor. Cameras 32A and 32B respectively capture images of the item 100 conveyed by the conveying device 12, as a pair of image data 200A (…). Figure 8 ) and 200B are respectively supplied to the control device 24.

[0046] Robot 18 moves with height sensor 20. (For example...) Figure 3 As shown, the robot 18 includes: a robot base 34, a movable part 36 movably disposed on the robot base 34, and a moving mechanism 38 for moving the movable part 36. The robot base 34 is fixedly disposed on the ground of the work unit. The movable part 36 is configured to move along the width direction Dw of the conveying device 12.

[0047] In this embodiment, the moving mechanism 38 is a ball screw mechanism, and has a ball screw 40 extending along the width direction Dw and a servo motor 42 that drives the ball screw 40 to rotate. The servo motor 42, in response to commands from the control device 24, causes the ball screw 40 to rotate forward and backward, thereby causing the movable part 36 engaged with the ball screw 40 to reciprocate in the width direction Dw.

[0048] The height sensor 20 is used to detect the position Ph of the object 100 in the height direction Dh and is fixed to the movable part 36 of the robot 18. In this embodiment, the height sensor 20 is a laser sensor that measures the distance ds to the object by irradiating the object with a single-axis laser beam. More specifically, the height sensor 20 has: a light-emitting part that emits a laser beam along a single optical axis A1; and a light-receiving part that receives the reflected light of the laser beam (neither shown). The height sensor 20 is fixed to the movable part 36 with its optical axis A1 parallel to the height direction Dh. The height sensor 20 supplies the measured distance ds data to the control device 24. Furthermore, the height sensor 20 is not limited to a laser sensor and can be any type of sensor used to detect the height of the object 100.

[0049] Robot 22 is positioned downstream of robot 18 and performs a prescribed operation WK on the item 100 transported by conveying device 12. In this embodiment, robot 22 is a vertical articulated robot, which includes: a robot base 44, a movable part 46 movably disposed on the robot base 44, and a moving mechanism 48 for moving the movable part 46. The moving mechanism 48 includes: a plurality of links 50 rotatably connected to each other; and a plurality of servo motors 52 for rotating the links 50. Figure 2 Each servo motor 52 rotates the linkage 50 in response to commands from the control device 24, thereby moving the movable part 46 to any position.

[0050] An end effector 49 is detachably mounted on the movable part 46. The end effector 49 is configured to perform a specified operation WK, such as printing (or coating) on ​​the surface 104 of the article 100 or forming an engraving on the surface 104 of the article 100. The robot 22 moves the end effector 49 via the moving mechanism 48, and performs the operation WK (printing operation WK1, engraving operation WK2, etc.) on the article 100 through the end effector 49.

[0051] Control device 24 controls the movements of conveying device 12, conveying sensor 14, vision sensor 16, height sensor 20, robot 18, and robot 22. For example... Figure 2 As shown, the control device 24 is a computer having a processor 54, a memory 56, and an I / O interface 58. The processor 54 may have a CPU or GPU, and is communicatively connected to the memory 56 and the I / O interface 58 via a bus 59.

[0052] The processor 54 communicates with the memory 56 and the I / O interface 58, and performs arithmetic operations to achieve the following functions: detecting the position P of the item 100; and executing a task WK for the item 100. The memory 56 may have RAM or ROM, etc., which temporarily or permanently stores various data. The memory 56 may be a computer-readable storage medium such as semiconductor memory, magnetic storage medium, or optical storage medium.

[0053] I / O interface 58, for example, has an Ethernet (trademarked) port, a USB port, a fiber optic connector, or an HDMI (trademarked) terminal, and communicates with external devices via wired or wireless means under instructions from processor 54. The conveying device 12 (servo motor 30), the conveying sensor 14, the vision sensor 16 (cameras 32A and 32B), the robot 18 (servo motor 42), the height sensor 20, and the robot 22 (servo motor 52) are connected to I / O interface 58 in a manner capable of wired or wireless communication.

[0054] like Figure 1 As shown, a work robot coordinate system C1 is set for robot 22. In this embodiment, the work robot coordinate system C1 is fixedly set for robot base 44 with its x-axis parallel to the conveying direction Dc, its y-axis parallel to the width direction Dw, and its z-axis parallel to the height direction Dh.

[0055] like Figure 1 and Figure 3 As shown, a height robot coordinate system C2 is set for robot 18. In this embodiment, the height robot coordinate system C2 is fixedly set for robot base 34 with its x-axis parallel to the conveying direction Dc, its y-axis parallel to the width direction Dw, and its z-axis parallel to the height direction Dh.

[0056] On the other hand, a height sensor coordinate system C3 is set for the height sensor 20. The height sensor coordinate system C3 is a translation coordinate system that moves in the robot coordinate systems C1 and C2 together with the height sensor 20 moved by the robot 18. The height sensor coordinate system C3 defines the position and orientation (i.e., the position and direction of the optical axis A1) of the height sensor 20 in the robot coordinate systems C1 and C2. In this embodiment, the height sensor coordinate system C3 is fixedly set for the height sensor 20 with its origin located at the center of the light-emitting part of the height sensor 20 and its z-axis parallel (specifically, aligned) with the optical axis A1.

[0057] like Figure 1As shown, a vision sensor coordinate system C4 is established for the vision sensor 16. The vision sensor coordinate system C4 defines the position and orientation of the vision sensor 16 within the robot coordinate systems C1 and C2 (i.e., the position and direction of the lines of sight of cameras 32A and 32B). In this embodiment, the vision sensor coordinate system C4 is fixedly established for the vision sensor 16 with its origin located at the midpoint of a pair of cameras 32A and 32B (or the center of the imaging sensor of camera 32A or 32B) and its z-axis parallel to the lines of sight of camera 32A or 32B. In this embodiment, the z-axis of the vision sensor coordinate system C4 is arranged parallel to the height direction Dh.

[0058] A transport coordinate system C5 is established for the transport device 12. The transport coordinate system C5 defines the transport direction Dc, width direction Dw, and height direction Dh of the transport device 12. In this embodiment, the x-axis of the transport coordinate system C5 defines the transport direction Dc, the y-axis defines the width direction Dw, and the z-axis defines the height direction Dh. Furthermore, the origin of the transport coordinate system C5 can be located on the upper surface of the movable part 28. In this embodiment, the transport coordinate system C5 is a moving coordinate system that moves in the transport direction Dc within the robot coordinate systems C1 and C2 according to the transport amount δ of the item 100 transported by the transport device 12.

[0059] The processor 54 uses the work robot coordinate system C1, the height robot coordinate system C2, the height sensor coordinate system C3, the vision sensor coordinate system C4, and the transport coordinate system C5 as references to automatically control the movements of robots 18 and 22, height sensor 20, vision sensor 16, transport device 12, and transport sensor 14, and executes the operation WK for the item 100. Therefore, the work robot coordinate system C1, the height robot coordinate system C2, the height sensor coordinate system C3, the vision sensor coordinate system C4, and the transport coordinate system C5 constitute the control coordinate system C set for the operation WK.

[0060] Figure 4 This represents an example of item 100. Item 100 is, for example, an amorphous bag-shaped item with a generally rectangular outer edge 102 when viewed from above. A length direction B1 and a width direction B2 are defined for item 100. The length direction B1 and the width direction B2 represent the orientation of item 100.

[0061] Additionally, the article 100 has a visually identifiable detection target feature 106. In this embodiment, the detection target feature 106 is a printing or engraving applied to the surface 104 of the article 100, and has a generally rectangular shape. A length direction E1 and a width direction E2 are defined for the detection target feature 106. The length direction E1 and the width direction E2 represent the orientation of the detection target feature 106. Furthermore, the detection target feature 106 has a feature point Q. The feature point Q may be, for example, the center point or vertex of the outer edge of the detection target feature 106, or the midpoint of one side defining the outer edge. Figure 4 This indicates that feature point Q is the outer center point of the detected target feature 106.

[0062] Within item 100, determine the work area 108 where robot 22 will perform task WK. The positional relationship PR between work area 108 and the detection target feature 106 is known in advance. This positional relationship PR can be predetermined, for example, as the positional relationship of the work target point P representing the position of work area 108, which is a distance α1 [mm] away from feature point Q of detection target feature 106 in the length direction B1 (or E1) and a distance α2 [mm] away from feature point Q in the width direction B2 (or E2).

[0063] The target point P can be determined, for example, as the center point or vertex of the outer edge of the target feature 106, or the midpoint of one side of the outer edge. Data representing the positional relationship PR (e.g., directions B1 and B2 or E1 and E2, and distances α1 and α2) are pre-stored in memory 56. Furthermore, the positional relationship PR is not limited to the above examples and can be arbitrarily determined.

[0064] In this embodiment, the dimensions DM (height, width, and length) of the article 100 are unknown. The robot system 10 detects the position P of the working part 108 of this article 100 with unknown dimensions DM, and performs work WK on the working part 108. Hereinafter, refer to Figure 5 The operation of robot system 10 is described below. When processor 54 receives a job start command from the operator, the host controller, or the computer program PG, it begins... Figure 5 The process.

[0065] exist Figure 5 After the process begins, processor 54 starts the first detection process of step S1. (Refer to...) Figure 6 Step S1 will be described below. After step S1 begins, in step S11, the processor 54 determines whether the item 100 has reached the upstream end of the conveying device 12. For example, a sensor SN (not shown) capable of detecting the presence of the item 100, such as a position sensor or a proximity sensor, is installed at the upstream end of the conveying device 12.

[0066] The processor 54 can determine whether the item 100 has reached the upstream end of the conveying device 12 based on the output signal of the sensor SN. If the determination is yes, the processor 54 proceeds to step S12; otherwise, it proceeds to step S16. In this embodiment, as... Figure 7 As shown, various items 100 of different sizes DM are continuously placed on the movable part 28 in any posture by other robots or operators.

[0067] In step S12, the processor 54 activates the vision sensor 16 ( Figure 1 The processor 54 sends a shooting command to the vision sensor 16, and in response to the shooting command, the vision sensor 16 takes pictures of the object 100 through cameras 32A and 32B, acquiring a pair of image data 200A and 200B. Figure 8 This represents an example of image data 200A. Each pixel PX constituting the captured image data 200A and 200B is defined as the coordinates of the visual sensor coordinate system C4. The visual sensor 16 supplies the image data 200A and 200B captured by cameras 32A and 32B to the control device 24.

[0068] In step S13, the processor 54 obtains the position Q of the detected target feature 106 based on image data 200A and 200B of the object 100 captured by cameras 32A and 32B. Specifically, the processor 54 obtains the position Q of the detected target feature 106 based on the image data 200A and 200B of the object 100 captured by cameras 32A and 32B. Figure 8 The parallax DP of cameras 32A and 32B is calculated by finding the difference between the coordinates of pixel PX of object 100 displayed in image data 200B and the coordinates of pixel PX of object 100 displayed in image data 200B in the visual sensor coordinate system C4.

[0069] On the other hand, the processor 54 performs image analysis on the image data 200A, detects the outer edge 102 of the object 100 displayed in the image data 200A and detects the target feature 106, and determines the feature point Q of the target feature 106 in the length direction B1 or width direction B2 (or length direction E1 or width direction E2) in the visual sensor coordinate system C4. Furthermore, in order to detect the outer edge 102 and the target feature 106 from the image data 200A, the shapes of the outer edge 102 and the target feature 106 can be taught in advance. Then, the processor 54 obtains the coordinates Q4 (x, y, z) of the feature point Q in the visual sensor coordinate system C4 based on the aforementioned disparity DP.

[0070] On the other hand, whenever image data 200A and 200B are captured in step S12, the processor 54 sets the transport coordinate system C5 in the robot coordinate systems C1 and C2. At this time, the origin of the transport coordinate system C5 is set at the initial position IPc predetermined in the robot coordinate systems C1 and C2. Figure 9 The diagram schematically represents the transport coordinate system C5 set at the initial position IPC.

[0071] Here, the positional relationships between the robot coordinate systems C1 and C2, the sensor coordinate systems C3 and C4, and the transport coordinate system C5 positioned at the initial position IPc are known through calibration. Therefore, these control coordinate systems C1~C5 can be transformed into each other via a known coordinate transformation matrix MX (homogeneous transformation matrix, etc.).

[0072] The processor 54 transforms the coordinates Q4 of the feature point Q of the detected target feature 106 in the visual sensor coordinate system C4 into coordinates Q5 (x, y, z) in the transport coordinate system C5. Thus, based on the image data 200A and 200B captured by cameras 32A and 32B, the processor 54 obtains the coordinates Q4 and Q5 as the position Q of the detected target feature 106. Therefore, the processor 54 serves as the feature position acquisition unit 62 for obtaining the position Q. Figure 2 It can perform its functions.

[0073] Refer again Figure 6 In step S14, the processor 54 obtains the positions Pw and Pc of the working part 108 in the directions Dw and Dc orthogonal to the height direction Dh, based on the position Q of the detection target feature 106 obtained in the previous step S13 and the known positional relationship PR between the working part 108 and the detection target feature 106.

[0074] Specifically, the processor 54 determines the target point P in the transport coordinate system C5 based on the coordinates Q5 of the detected target feature 106 in the transport coordinate system C5 obtained in the previous step S13 and the aforementioned positional relationship PR data. Then, the processor 54 obtains the coordinates P5(x, y) of the target point P in the transport coordinate system C5.

[0075] The x-coordinate of coordinate P5 represents the position of the target point P in the x-axis direction of the conveying coordinate system C5. In other words, it represents the position Pc (third position) of the working part 108 in the conveying direction Dc. The y-coordinate of coordinate P5 represents the position of the target point P in the y-axis direction of the conveying coordinate system C5. In other words, it represents the position Pw (second position) of the working part 108 in the width direction Dw.

[0076] In this embodiment, the height dimension DM1 of the item 100 is unknown. Therefore, it is not possible to accurately determine the position (z coordinate of coordinate P5) of the work part 108 (work target point P) in the z-axis direction (height direction Dh) of the conveying coordinate system C5 based on the position Q (coordinate Q5) of the detection target feature 106 obtained in step S13.

[0077] Thus, in step S14, processor 54 obtains the positions Pw and Pc of the working part 108 in directions Dw and Dc orthogonal to the height direction Dh, as coordinates P5(x, y), based on the position Q and positional relationship PR of the detected target feature 106. Therefore, processor 54 serves as the working position acquisition unit 64 for acquiring the positions Pw and Pc of the working part 108. Figure 2 It can perform its functions.

[0078] In step S15, the processor 54 functions as the work position acquisition unit 64, storing the coordinates P5 acquired in the previous step S14 in the position database 202. This position database 202 manages the position P of the work unit 108 in the control coordinate system C and stores it in the memory 56. Figure 10 This schematically illustrates an example of the data construction of location database 202. Figure 6 In the process, during the period when the condition is not met in step S16 (described later), the loop of steps S11 to S16 is repeatedly executed. Therefore, each time step S14 is executed, the processor 54 repeatedly obtains the new coordinates P5. _n (n = 1, 2, 3, ...).

[0079] Whenever processor 54 obtains new coordinates P5 in step S14 _n In step S15, the coordinates P5 will be obtained. _n And obtain the coordinates P5 _n The data from the transport coordinate system C5 are linked together and stored sequentially in the location database 202. Figure 10 In the location database 202 shown, column 204 indicates the order "n" of obtaining coordinates P5 (in... Figure 10 In the example, n = 1, 2, 3, 4). Additionally, column 206 represents the stored coordinates P5. _n (x, y).

[0080] On the other hand, the "Status" column 208 indicates the progress of operation WK relative to item 100. For example, "Waiting for Inspection" indicates coordinate P5. _nThe z-coordinate (i.e., the position Ph of the work part 108 in the height direction Dh) is in an undetected state. Therefore, in step S15, the processor 54 uses the position Pw and Pc of the work part 108 obtained in the previous step S14 as the coordinate P5 of the transport coordinate system C5. _n (x, y) is stored in location database 202.

[0081] Refer again Figure 6 In step S16, the processor 54 determines whether a job termination command has been received from the operator, the host controller, or the computer program PG. If the determination is yes, the processor 54 terminates the job. Figure 6 The process shown, on the other hand, returns to step S11 if the determination is negative. Thus, during the period when the processor 54 determines negative in step S16, the loop of steps S11 to S16 is repeatedly executed.

[0082] Refer again Figure 5 After starting step S1, in step S2, processor 54 determines the new coordinates P5 in step S15 above. _n Whether it is stored in the location database 202. If the processor 54 determines yes, it begins the second detection process of step S3 and proceeds to step S4. On the other hand, if the processor 54 determines no, it proceeds to step S6. Hereinafter, refer to Figure 11 Step S3 will be explained.

[0083] After step S3 begins, in step S21, processor 54 causes robot 18 to move, performing a positioning action PO to position the height sensor 20 at the position Pw obtained in step S14. Specifically, processor 54 refers to the position database 202 stored in memory 56 at the current time. Figure 10 ), read column 208's "Status" as "Waiting for Detection" and column 204's "No" as the topmost coordinate P5 _n For example, in Figure 10 In the example shown, processor 54 reads the coordinates P5 of No.1 (n=1). _1 (x, y).

[0084] Next, processor 54 reads the coordinates P5 _n Transformed into coordinates P2 in the robot's height coordinate system C2 _n (x, y). The coordinates are P2. _n This indicates the positions Pw and Pc of the work area 108 (work target point P) obtained in step S14 within the height robot coordinate system C2. In this embodiment, the y-axis of the height robot coordinate system C2 is arranged parallel to the width direction Dw; therefore, coordinate P2 _nThe y-coordinate represents the position Pw of the work area 108 in the width direction Dw.

[0085] The processor 54 generates instructions (position instructions, speed instructions, torque instructions) for the servo motor 42 of the robot 18 in order to position the height sensor 20 at coordinate P2. _n The processor 54, following this instruction, causes the robot 18's movement mechanism 38 to move, thereby moving the height sensor 20 in the y-axis direction (i.e., the width direction Dw) of the height robot coordinate system C2 and positioning it at coordinate P2. _n The position of the y-coordinate.

[0086] As a result, Figure 12 As shown, the height sensor 20 is positioned at position Pw in the width direction Dw of the work area 108 obtained in step S14. At this time, the origin O of the height sensor coordinate system C3 is set at coordinate O2 (x, y, z) of the height robot coordinate system C2. The y-coordinate of this coordinate O2 is related to the coordinate P2. _n Their y-coordinates are the same.

[0087] After positioning the height sensor 20 at position Pw, the processor 54 stops the movement of the robot 18, leaving the height sensor 20 in standby mode at position Pw. Thus, in this embodiment, the processor 54 functions as a positioning execution unit 66 that performs the positioning action PO of placing the height sensor 20 at position Pw. Figure 2 It can perform its functions.

[0088] In step S22, the processor 54 updates the position Pc of the working part 108 in the conveying direction Dc in the control coordinate system C. Specifically, the processor 54 shifts the origin of the conveying coordinate system C5 by the conveying amount δ obtained from the conveying sensor 14 in the conveying direction Dc. As a result, Figure 13 As shown, the transport coordinate system C5 is displaced by a transport amount δ from its initial position IPc to the transport direction Dc in the robot coordinate systems C1 and C2.

[0089] In the transport coordinate system C5 after displacement, set the target point P, and define the coordinates of the target point P in the transport coordinate system C5 as P5. _n The displacement remains unchanged. On the other hand, in robot coordinate systems C1 and C2, the target point P is displaced by a displacement δ in the transport direction Dc. Furthermore, when the transport coordinate system C5 is displaced by the displacement δ, the positional relationship between the transport coordinate system C5 and robot coordinate systems C1 and C2 also changes. Consequently, the parameters of the coordinate transformation matrix MX between the transport coordinate system C5 and robot coordinate systems C1 and C2 change according to the displacement δ.

[0090] Whenever processor 54 executes step S22, it moves the origin of the transport coordinate system C5 in the robot coordinate system C1 and C2 by a transport amount δ in the transport direction Dc. As a result, it updates the position Pc of the working part 108 (working target point P) in the robot coordinate system C1 and C2 so that it moves in the transport direction Dc.

[0091] As described above, the position P of the working part 108 is the coordinate P5 of the transport coordinate system C5, which moves within the robot coordinate systems C1 and C2. _n The location database 202 manages the coordinates. Therefore, even if the location Pc is updated in step S22, the coordinates P5 stored in the location database 202 remain unchanged. _n It remains unchanged, thus simplifying the management of position P of work area 108.

[0092] In step S23, the processor 54 determines whether the work area 108 has reached directly below the height sensor 20 in position Pw standby mode via the conveying action of the conveying device 12. Specifically, the processor 54 uses the coordinates P5 read in the most recent step S21 in the position database 202. _n The coordinate transformation matrix MX between the current transport coordinate system C5 and the height robot coordinate system C2 will transform coordinate P5. _n Transformed into coordinates P2 in the robot's height coordinate system C2 _n (x, y).

[0093] On the other hand, processor 54 obtains the coordinates O2 (x, y, z) of the height sensor 20 (specifically, the origin O of the height sensor coordinate system C3) in the height robot coordinate system C2 at the current time point based on feedback from the encoder (or Hall element) of the servo motor 42 installed on the moving mechanism 38. Then, processor 54 determines the transformed coordinates P2. _n Does the x-coordinate of P2 match the x-coordinate of O2? At this point, the coordinate P2... _n The x-coordinate of O2 represents the current position of the target point P in the conveying direction Dc, and the x-coordinate of O2 represents the current position of the height sensor 20 in the conveying direction Dc.

[0094] For example, processor 54 can calculate coordinate P2 _n The difference Δx between the x-coordinate of point P1 and the x-coordinate of point O2, if Δx is below a specified threshold Δth, is determined to be the x-coordinate of point P2. _n The x-coordinates of O2 and O2 are consistent. Processor 54 is at coordinate P2. _nIf the x-coordinate of O2 is consistent with that of the height sensor 20, it is determined that the working part 108 has reached directly below the height sensor 20 (i.e., yes), and the process proceeds to step S24. On the other hand, if the determination is no, the processor 54 returns to step S22.

[0095] Thus, during the period when the processor 54 determines "no" in step S23, it repeatedly executes the loop of steps S22 and S23. Each time step S22 is executed, the position Pc of the working part 108 is updated in the control coordinate system C (robot coordinate systems C1 and C2). In addition, the processor 54 can also repeatedly execute the loop of steps S22 and S23 with a control cycle τ (specifically, synchronously with the action of the transfer sensor 14 detecting the transfer amount δ).

[0096] In step S24, the processor 54 activates the height sensor 20 located at position Pw via positioning action PO, and performs detection action DO to detect the position Ph (first position) of the working part 108 in the height direction Dh. Specifically, the processor 54 sends a detection command to the height sensor 20, which is in standby mode at position Pw (coordinate O2).

[0097] According to the detection command, the height sensor 20 irradiates a single-axis laser beam toward the object 100. At this time, the laser beam illuminates the target point P. Then, the height sensor 20 measures the distance ds to the object 100 based on the reflected light from the object 100. This distance ds represents the distance from the height sensor 20 (the origin O of the height sensor coordinate system C3) to the target point P, which is equivalent to the z-coordinate of the target point P in the height sensor coordinate system C3.

[0098] The processor 54 obtains the z-coordinate of the target point P in the transport coordinate system C5 based on the distance ds measured by the height sensor 20. This z-coordinate represents the position Ph of the work part 108 in the height direction Dh (target point P). Thus, the processor 54 executes a detection action DO to detect the position Ph of the work part 108. Therefore, the processor 54 acts as a detection execution unit 68 that executes the detection action DO. Figure 2 It can perform its functions.

[0099] In step S25, the processor 54 functions as the job position acquisition unit 64, adding the position Ph detected by the detection action DO as the coordinate of the control coordinate system C to the position database 202, thereby updating the position database 202. Specifically, in the position database 202 stored in the memory 56 at the current time point, the processor 54 updates the coordinate P5 read in the most recent step S21. _n (x, y) is added to the z-coordinate of the transport coordinate system C5 detected in the previous step S24, thus updating the coordinates to P5. _n(x, y, z). The updated coordinates are P5. _n 'Stored in location database 202. This coordinate is P5' _n 'Indicates the position of work location 108 (work target point P) in the conveying direction Dc, width direction Dw, and height direction Dh, Pc, Pw, and Ph respectively.

[0100] Then, the processor 54 will detect the coordinates P5 of position Ph in the position database 202. _n The status of ' has been changed to "Waiting for job". Figure 14 This indicates the updated location database 202. Figure 14 In the example shown, the coordinates of No.1 are P5. _1 The status of ' has been changed to "Waiting for job".

[0101] The "awaiting job" indicates: coordinate P5 for No.1. _1 'Detecting the z-coordinate (i.e., the position Ph of the work area 108 in the height direction Dh) allows the processor 54 to perform work WK on the item 100 marked No.1. Thus, whenever step S25 is executed, the processor 54 updates the position database 202 by appending the position Ph (z-coordinate of the transport coordinate system C5) detected in the previous step S24.

[0102] In step S26, the processor 54 refers to the current time point location database 202 to determine whether there is a coordinate P5 with a "status" of "waiting to be detected". _n If the determination is yes, processor 54 returns to step S21; otherwise, if the determination is no, it ends. Figure 11 The process is shown. Thus, during the period when the processor 54 determines "yes" in step S26, it repeatedly executes the loop of steps S21 to S26.

[0103] Refer again Figure 5 In step S4, the processor 54 refers to the location database 202 at the current time point (e.g., Figure 14 Determine if there exists a coordinate P5 with a "status" of "waiting for job". _n If the determination is yes, the processor 54 starts step S5 and proceeds to step S6; otherwise, if the determination is no, it proceeds to step S7.

[0104] In step S5, processor 54 executes job WK for item 100. See below for further details. Figure 15 Step S5 will be explained. After step S5 begins, in step S31, the processor 54 causes the movement mechanism 48 of the robot 22 to move, causing the end effector 49 to move toward the working part 108 on the item 100.

[0105] Specifically, processor 54 refers to the location database 202 at the current time (e.g., Figure 14 ), read column 208's "Status" as "Waiting for Job" and column 204's "No" as the topmost coordinate P5 _n (For example, the coordinates of No.1 are P5) _1 Next, processor 54 uses the read coordinates P5. _n The coordinate transformation matrix MX between the current time point's transport coordinate system C5 and the robot coordinate system C1 will be used to transform coordinate P5. _n Transform (x, y, z) into coordinates P1 in the robot's coordinate system C1. _n (x, y, z). The coordinates P1 _n In the work robot coordinate system C1, the positions Pc, Pw, and Ph of the work part 108 (work target point P) in the conveying direction Dc, the width direction Dw, and the height direction Dh are represented.

[0106] Then, the processor 54 generates instructions (position instructions, speed instructions, torque instructions) for the servo motor 52 of the robot 22 in order to position the end effector 49 at coordinate P1. _n The processor 54 causes the robot 22 to move according to the instruction, thereby moving the end effector 49 toward the work site 108 (specifically, the work target point P).

[0107] In step S32, the processor 54 determines whether the end effector 49 has reached the work area 108. Specifically, the processor 54 calculates the coordinates R1 of the end effector 49 in the work robot coordinate system C1 at the current time point based on feedback from the encoders (or Hall elements) of each servo motor 52 installed on the robot 22.

[0108] Then, processor 54 determines whether coordinate R1 is the same as the coordinate P1 of the target point P at the current time. _n Consistent. If the determination is yes, the processor 54 proceeds to step S34; otherwise, it proceeds to step S33. In step S32, if the determination is yes, the end effector 49 can be considered to be positioned at the work location 108 (work target point P).

[0109] In step S33, similar to step S22 described above, processor 54 updates the position Pc of the working part 108 in the control coordinate system C by displacing the origin of the transport coordinate system C5 in the transport direction Dc. Then, processor 54 returns to step S31. Thus, processor 54 repeatedly executes the cycle of steps S31 to S33 during the period when the determination in step S32 is negative. In addition, processor 54 can also repeatedly execute the cycle of steps S31 to S33 with a control cycle τ (specifically, synchronously with the action of the transport sensor 14 detecting the transport amount δ).

[0110] In step S34, the processor 54 activates the end effector 49 to perform operation WK on the article 100. Specifically, the processor 54 starts the end effector 49 to perform operation WK such as printing operation WK1 or engraving operation WK2 on the work area 108. In this way, the operation WK for the work area 108 is completed.

[0111] When job WK is properly completed, processor 54 will retrieve the coordinates P5 referenced in the most recent step S31 from the location database 202. _n The status of ' has been changed to "Operation Completed". Figure 16 This indicates the changed location database 202. Figure 16 In the example shown, P5 represents the coordinates for No.1. _1 The status of item 100's task WK is completed. On the other hand, regarding coordinates P5 for No. 2. _2 'and No.3's coordinates P5 _3 ',express Figure 11 The status after step S24 is completed (“Waiting for job”).

[0112] In step S35, the processor 54 refers to the location database 202 at the current time point to determine whether there is a coordinate P5 with a "status" of "waiting for job". _n If the determination is yes, processor 54 returns to step S31; otherwise, if the determination is no, it ends. Figure 15 The process is shown. Thus, during the period when the processor 54 determines "yes" in step S35, the loop of steps S31 to S35 is repeatedly executed.

[0113] Refer again Figure 5 In step S6, with Figure 6 Similar to step S16, processor 54 determines whether a job termination command has been accepted. If the processor 54 determines yes, it terminates the job. Figure 5The process shown, on the other hand, returns to step S2 if the determination is negative. On the other hand, if the determination is negative in step S4, in step S7, the processor 54 determines whether a job termination command has been accepted, similar to step S6. If the determination is positive, the processor 54 terminates the process. Figure 5 The process shown, on the other hand, returns to step S4 if the determination is negative.

[0114] As described above, in this embodiment, the processor 54 functions as a feature position acquisition unit 62, a work position acquisition unit 64, a positioning execution unit 66, and a detection execution unit 68. In order to perform work WK on the work part 108 on the article 100, it detects the position P (Pc, Pw, Ph) of the work part 108. Therefore, the feature position acquisition unit 62, the work position acquisition unit 64, the positioning execution unit 66, and the detection execution unit 68 constitute a device 60 for detecting the position P of the work part 108. Figure 2 ).

[0115] In this device 60, the feature position acquisition unit 62 acquires the position Q (coordinates Q4 and Q5) of the detection target feature 106 of the item 100 displayed in the image data 200A and 200B captured by the cameras 32A and 32B (step S13). Furthermore, the work position acquisition unit 64 acquires the positions Pc and Pw of the work part 108 in the directions Dc and Dw, which are orthogonal to the height direction Dh, based on the position Q and positional relationship PR acquired by the feature position acquisition unit 62 (step S14).

[0116] Then, the positioning execution unit 66 causes the robot 18, which moves the height sensor 20, to move and execute the positioning action PO of the height sensor 20 at the position Pw obtained by the work position acquisition unit 64 (step S21). In addition, the detection execution unit 68 causes the height sensor 20, which is positioned at the position Pw by the positioning action PO, to move and execute the detection action DO of the first position Ph (z coordinate of the transport coordinate system C5) of the work part 108 in the height direction Dh (step S24).

[0117] According to this structure, the first position Ph of the working part 108 of an object 100 with an unknown height dimension DM1 can be quickly obtained. Previously, a three-dimensional vision sensor system was used to detect an object 100 with an unknown dimension DM. The three-dimensional vision sensor system photographs the object 100 after illuminating it with a patterned beam from a projector, generating three-dimensional point cloud image data representing the visual features of the object 100 as a three-dimensional point cloud. In the case of such a three-dimensional vision sensor system, the processing computation for detecting the position of the object 100 is high, and the processing time is considerable.

[0118] In this embodiment, by using only a pair of cameras 32A and 32B and a height sensor 20, the first position Ph of the working part 108 of an article 100 with an unknown size DM can be detected with high precision. Therefore, compared with the use of a three-dimensional vision sensor system as described above, the amount of computation required to detect the positions Pc, Pw, and Ph of the working part 108 can be reduced, thereby shortening the cycle time. This high-speed detection processing is particularly advantageous in applications where a large number of articles 100 with unknown size DM are continuously transported by the conveying device 12.

[0119] Furthermore, in the device 60, the robot 18 is configured to move the height sensor 20 along the width direction Dw, and the work position acquisition unit 64 acquires the second position Pw (y-coordinates of coordinates P1, P2, and P5) of the work part 108 in the width direction Dw. Then, in the positioning action PO, the positioning execution unit 66 moves the height sensor 20 in the width direction Dw by the action of the robot 18, so that the height sensor 20 is in standby at the second position Pw.

[0120] Then, when the conveying action of the conveying device 12 causes the working part 108 to reach directly below the height sensor 20 in the second position Pw (as in step S23), the detection execution unit 68 performs the detection action DO. According to this structure, by positioning the height sensor 20 at position Pw using the single-axis robot 18, the first position Ph of the working part 108 can be detected. Therefore, the construction of the robot 18 can be simplified, thus reducing costs and simplifying the control of the robot 18. Consequently, the computational load required for position detection processing can be reduced more effectively.

[0121] In addition, in device 60, the work position acquisition unit 64 acquires not only the second position Pw, but also the third position Pc of the work part 108 in the conveying direction Dc (e.g., the x-coordinates of coordinates P1, P2, and P5), and stores the acquired second position Pw and third position Pc as coordinate P5 of the control coordinate system C (conveyor coordinate system C5) set for work WK in the position database 202. Figure 10 , Figure 14 , Figure 16 Then, the positioning execution unit 66 performs the positioning action PO according to the coordinates P5 stored in the position database 202. According to this structure, the positions Pw and Pc of the acquired work site 108 can be effectively managed.

[0122] Furthermore, in the device 60, the control coordinate system C includes a transport coordinate system C5 that moves in the transport direction Dc according to the transport amount δ of the transported item 100 by the transport device 12. The work position acquisition unit 64 stores the second position Pw and the third position Pc as coordinates P5 of the transport coordinate system C5 in the position database 202. According to this structure, as described above, the coordinates P5 stored in the position database 202 remain unchanged, thus, the management of the position P of the work area 108 can be simplified more effectively.

[0123] Furthermore, in device 60, the work position acquisition unit 64 adds the first position Ph detected by the detection action DO as the coordinate of the control coordinate system C (the z-coordinate of the transport coordinate system C5) to the position database 202, thereby updating the position database 202 (step S25). According to this structure, the coordinates of the position Ph, detected with high precision by the height sensor 20, can be added to the position database 202 for management in the position database 202 for work WK.

[0124] Furthermore, in the device 60, the feature position acquisition unit 62 acquires the position Q of a print on the article 100 or the position Q of an engraving formed on the article 100, as a detection target feature 106. Here, prints or engravings for identifying the article 100 are often provided on the article 100, which is a commodity. According to this embodiment, the position Q can be acquired using such prints or engravings.

[0125] Next, regarding Figure 6 Other examples of step S14 will be described. In this embodiment, multiple candidates 108A, 108B, 108C, and 108D for the work site 108 are predetermined for the article 100. Figure 17 This represents examples of candidates 108A, 108B, 108C, and 108. In Figure 17 In the example shown, a total of four candidates 108A, 108B, 108C and 108D were identified around the target feature 106.

[0126] The positional relationship PR between candidates 108A, 108B, 108C, and 108D and the detection target feature 106 is known. For example, candidate 108A is determined to be the positional relationship PR of its target point P being a distance β1 away from feature point Q of the detection target feature 106 in the length direction B1 (or E1). Similarly, candidate 108C is determined to be the positional relationship PR of its target point P being a distance β2 away from feature point Q in the length direction B1. On the other hand, candidate 108B is determined to be the positional relationship PR of its target point P being a distance β3 away from feature point Q in the width direction B2 (or E2). Finally, candidate 108D is determined to be the positional relationship PR of its target point P being a distance β4 away from feature point Q in the width direction B2.

[0127] In this embodiment, in step S14, the processor 54 functions as the job position acquisition unit 64, and selects one of a plurality of candidate 108A, 108B, 108C and 108D based on the image data 200A or 200B captured in step S12. Figure 18 This indicates the captured image data 200A.

[0128] Based on the coordinates Q4(x, y, z) and positional relationship PR of the feature point Q of the detected target feature 106 in the visual sensor coordinate system C4, obtained in the previous step S13, the processor 54 defines candidate target points P for 108A, 108B, 108C, and 108D in the image data 200A (i.e., the visual sensor coordinate system C4). Then, in the image data 200A, the processor 54 selects one of the candidate 108A, 108B, 108C, or 108D based on the pixel value PV (e.g., brightness of grayscale 0-255) of pixel PX, wherein pixel PX represents at least a portion of the interval 210 from the detected target feature 106 to the candidate 108A, 108B, 108C, and 108D.

[0129] Figure 19 This represents a magnified view of the detected target feature 106 displayed in image data 200A. The processor 54 defines an interval 210 in the visual sensor coordinate system C4, from the feature point Q of the detected target feature 106 to the target point P of the candidate 108A. Then, the processor 54 determines whether the candidate 108A can be selected by analyzing the pixel value PV of the pixel PX corresponding to at least a portion of the interval 210.

[0130] Here, in Figure 19 In the example shown, candidate 108A is located outside the image area of ​​surface 104 of the display object 100 in the visual sensor coordinate system C4. With such candidate 108A selected, in step S5 above... Figure 15 The system cannot properly perform operations (WK) on item 100. Therefore, candidate 108A needs to be excluded. Figure 19 In the example shown, region 212 within interval 210 contains pixel PX of the surface 104 of the display article 100 and region 214 contains pixel PX of the background BG (in this embodiment, the surface of the movable part 28) outside the display article 100. The pixel value PV1 of region 212 and the pixel value PV2 of region 214 are different values.

[0131] As an example, processor 54 obtains the pixel values ​​PV of all pixels PX contained in interval 210 defined in sensor coordinate system C4. Then, if the obtained pixel values ​​PV include the pixel value PV2 of region 214, processor 54 determines that it is an unselectable candidate 108A. As another example, processor 54 obtains the pixel values ​​PV of all pixels PX contained in interval 210, and calculates the ratio Rp = N1 / N2 (or N2 / N1) of the number of pixels PX of pixel value PV1 to the number of pixels PX of pixel value PV2. Then, when the calculated ratio Rp is equal to a predetermined threshold R... th When the following conditions are met (or above), processor 54 determines that candidate 108A is not an option.

[0132] As another example, processor 54 obtains the pixel value PV of the target point P or its vicinity in interval 210, and determines whether the pixel value PV is pixel value PV2. Then, if the pixel value PV is pixel value PV2, processor 54 determines that it is an unselectable candidate 108A. As another example, the difference ρ between the pixel values ​​PV of adjacent pixels PX in interval 210 is calculated, where the difference ρ is a predetermined threshold ρ. th In the above circumstances, it is determined that candidate 108A cannot be selected.

[0133] Thus, the processor 54 determines whether candidate 108A can be selected based on the pixel values ​​PV of at least a portion of the pixels PX in the display range 210. Figure 19 In the example shown, the processor 54 determines that candidate 108A is not selectable. In addition, the pixel value PV1 of the surface 104 of the display item 100 and the pixel value PV2 of the display background BG (movable part 28) can be taught in advance.

[0134] Similarly, the eligibility of selecting other candidates, 108B, 108C, and 108D, is also determined. For example, in Figure 18In the example case, regarding candidate 108B, processor 54 determines it as unselectable, just like with candidate 108A. On the other hand, in the case of candidates 108C and 108D, the pixel values ​​PV of all pixels PX within the interval 210 from feature point Q to the respective target point P of candidates 108C and 108D are the pixel values ​​PV1 of the displayed item 100. Therefore, for candidates 108C and 108D, processor 54 determines they are selectable.

[0135] Furthermore, processor 54 can also determine whether candidates 108A, 108B, 108C, and 108D can be selected in a predetermined order (e.g., candidate 108A→108B→108C→108D). Then, processor 54 selects one of the candidates 108A, 108B, 108C, or 108D that was initially determined to be selectable. Assuming that in... Figure 18 In the example shown, when the decision is made in the order of candidate 108A→108B→108C→108D, the processor 54 selects candidate 108C from the multiple candidates 108A, 108B, 108C and 108D.

[0136] In this way, the processor 54 functions as the job position acquisition unit 64, and selects a candidate 108C based on the pixel values ​​PV (PV1, PV2) of the pixels PX that are displayed in the image data 200A in at least a portion of the interval 210 from the detected target feature 106 (feature point Q) to the candidates 108A, 108B, 108C and 108D.

[0137] Next, the processor 54 obtains the positions Pc and Pw of the selected candidate 108C in directions Dw and Dc orthogonal to the height direction Dh. Specifically, as in the above embodiment, the processor 54 obtains the coordinates P4(x, y) of the target point P of the selected candidate 108C in the visual sensor coordinate system C4, and transforms these coordinates P4 into coordinates P5(x, y) in the transport coordinate system C5. In this way, the processor 54 obtains the positions Pc and Pw (coordinates P5) of the selected candidate 108C (target point P) in the width direction Dw and the transport direction Dc.

[0138] As described above, in the apparatus 60 of this embodiment, multiple candidates 108A, 108B, 108C, and 108D of the work area 108 for which the positional relationship PR with the detection target feature 106 (feature point Q) is known is predetermined for the article 100. Then, the work position acquisition unit 64 selects one of the multiple candidates 108A, 108B, 108C, and 108D (for example, candidate 108C) based on the image data 200A, and acquires the position Pc and Pw (coordinate P5) of the selected candidate 108C in the directions Dc and Dw. According to this structure, candidates 108A and 108B that cannot be worked on can be avoided, and therefore, work WK failure for the work area 108 can be avoided.

[0139] Furthermore, in the apparatus 60 of this embodiment, the work position acquisition unit 64 defines a plurality of candidates 108A, 108B, 108C, and 108D in the image data 200A, and selects a candidate 108C based on the pixel value PV of the pixel PX that displays at least a portion of the interval 210 from the detection target feature 106 (feature point Q) to the candidates 108A, 108B, 108C, and 108D (work target point P) in the image data 200A. According to this structure, it is possible to determine whether the pixel PX corresponding to the candidates 108A, 108B, 108C, and 108D displays a background BG other than the item 100 based on the pixel value PV. Therefore, it is possible to reliably avoid candidates 108A and 108B that cannot be used.

[0140] also, Figure 17 The candidates 108A, 108B, 108C, and 108D shown are examples, and any number of candidates can be determined for the target feature 106 based on any positional relationship PR. Furthermore, Figure 19 The interval 210 shown can be defined in any size and shape. For example, interval 210 can be defined as... Figure 19 The area with a specified area, as shown, can also be defined as a line from feature point Q to target point P.

[0141] Next, refer to Figure 20 and Figure 21 The robot system 80 in other embodiments will be described. The robot system 80 differs from the robot system 10 described above in that it is a robot 82 for moving sensors. The robot 82 moves the height sensor 20 along the width direction Dw and the conveying direction Dc. Specifically, as... Figure 22 As shown, the robot 82 includes: a robot base 84, a movable part 36 disposed on the robot base 84 in a manner capable of moving in the width direction Dw and the conveying direction Dc, and a moving mechanism 38 and 86 for moving the movable part 36. The robot base 84 is a four-legged component and is fixedly disposed on the ground of the work unit.

[0142] The moving mechanism 86 is a ball screw mechanism, which has a ball screw (not shown) extending along the conveying direction Dc and a servo motor 42B that drives the ball screw to rotate. The servo motor 42B, in response to commands from the control device 24, causes the ball screw to rotate forward and reverse, thereby causing the moving mechanism 38, which engages with the ball screw, to reciprocate in the conveying direction Dc.

[0143] The moving mechanism 38 is connected to the moving mechanism 86 in a manner capable of reciprocating in the conveying direction Dc. The servo motor 42A of the moving mechanism 38, like that of the robot 18 described above, operates via a ball screw 40 (… Figure 3 The forward and reverse rotations cause the movable part 36 to reciprocate in the width direction Dw. Thus, in this embodiment, the robot 82 moves the height sensor 20 mounted on the movable part 36 along the width direction Dw and the conveying direction Dc via the actions of the moving mechanisms 38 and 86. Furthermore, the aforementioned height robot coordinate system C2 is established for the robot 82.

[0144] In this embodiment, in addition to the feature position acquisition unit 62, the work position acquisition unit 64, the positioning execution unit 66, and the detection execution unit 68 described above, the device 60 also includes a positioning determination unit 70. The operation of the robot system 80 will be described below. The processor 54 executes... Figure 5 The process is shown below. At the beginning... Figure 5 At the specified time point in the process, the moving mechanism 86 positions the moving mechanism 38 and the height sensor 20 at the initial position IP1 on the conveying direction Dc. Figure 20 , Figure 22 The initial position IP1 is represented as the x-coordinate of the robot's height coordinate system C2.

[0145] Here, in this embodiment, as Figure 5 In step S3, processor 54 executes... Figure 23 The process is shown. Furthermore, in Figure 23 In the process shown, for and Figure 11 The same process flow is labeled with the same step numbers, and repeated descriptions are omitted. After step S3 begins, in step S41, the processor 54 functions as the positioning execution unit 66, executing the configuration of the height sensor 20 in the most recent step S14 ( Figure 6 The first positioning action PO1 is obtained from the position Pw (second position) in the system.

[0146] Specifically, similar to step S21 above, processor 54 reads from the current time position database 202 the "status" of column 208 as "waiting for detection" and the "No" of column 204 as the topmost coordinate P5. _n Next, processor 54 reads the coordinates P5._n Transformed into coordinates P2 in the robot's height coordinate system C2 _n (x, y). Then, the processor 54, by actuating the moving mechanism 38, begins to move the height sensor 20 toward the y-axis direction (i.e., the width direction Dw) of the height robot coordinate system C2, and positions it at coordinate P2. _n The action of the position Pw (second position) of the y coordinate.

[0147] After step S41, the processor 54 executes step S22 as described above. Next, in step S42, the processor 54 determines whether to configure the height sensor 20 at position Pw. Specifically, the processor 54 obtains the coordinates O2 (x, y, z) of the height sensor 20 in the height robot coordinate system C2 at the current time point based on feedback from the encoder (or Hall element) of the servo motor 42A provided on the moving mechanism 38.

[0148] Then, processor 54 determines whether the y-coordinate of the obtained coordinate O2 is the same as the coordinate P2 obtained in the previous step S41. _n The y-coordinates are consistent. At coordinates O2 and P2... _n If the y-coordinates are consistent with each other, the processor 54 determines that the height sensor 20 is configured at position Pw (i.e., yes), and proceeds to step S43.

[0149] On the other hand, if the determination is negative, the processor 54 returns to step S22. Thus, during the period when the determination is negative in step S42, the processor 54 repeatedly executes the loop of steps S22 and S42. In addition, the processor 54 can also repeatedly execute the loop of steps S22 and S42 with a control cycle τ (specifically, synchronously with the action of the conveying sensor 14 detecting the conveying amount δ).

[0150] In step S43, the processor 54 determines whether the working part 108 of the item 100 has passed the height sensor 20 in the transport direction Dc. Specifically, the processor 54 uses the coordinates P5 read in the most recent step S41. _n The coordinate transformation matrix MX between the current time-point transport coordinate system C5 and the height robot coordinate system C2 will be used to transform the coordinate P5. _n Transformed into coordinates P2 in the robot's height coordinate system C2 _n (x, y). Then, processor 54 determines the coordinates P2. _n Does the x-coordinate of the height sensor 20 exceed the x-coordinate of the coordinate O2 in the height robot coordinate system C2?

[0151] Processor 54 at coordinate P2 _nIf the x-coordinate of the workpiece exceeds the x-coordinate of coordinate O2, it is determined that the workpiece 108 in the conveying direction Dc has passed the height sensor 20 (i.e., yes), and the process proceeds to step S44. On the other hand, if the determination is no, the processor 54 proceeds to step S22, and... Figure 11 The process is the same, and steps S22 to S26 are executed sequentially.

[0152] Thus, in this embodiment, when the processor 54 positions the height sensor 20 at position Pw during the first positioning action PO1, it determines whether the working part 108 (the target point P) has passed the height sensor 20 in the conveying direction Dc. Therefore, the processor 54 serves as the positioning determination unit 70 for determining whether the working part 108 has passed the height sensor 20. Figure 21 It can perform its functions.

[0153] In step S44, the processor 54 functions as a positioning execution unit 66, executing a second positioning action PO2 to position the height sensor 20 at position Pc (third position) of the work area 108. Specifically, the processor 54 moves the moving mechanism 38 and the height sensor 20 in the x-axis direction (i.e., the conveying direction Dc) of the height robot coordinate system C2 by actuating the moving mechanism 86.

[0154] After step S44, processor 54 executes step S22 described above. Next, in step S45, processor 54 determines whether to configure height sensor 20 at position Pc. Specifically, processor 54 obtains the coordinates O2 (x, y, z) of height sensor 20 at the current time point in the height robot coordinate system C2 based on feedback from the encoder (or Hall element) of servo motor 42B provided on the moving mechanism 86.

[0155] On the other hand, processor 54 uses the coordinates P5 read in the most recent step S41. _n The coordinate transformation matrix MX between the current time-point transport coordinate system C5 and the height robot coordinate system C2 will be used to transform the coordinate P5. _n Transformed into coordinates P2 in the robot's height coordinate system C2 _n (x, y). Then, processor 54 determines whether the x-coordinate of coordinate O2 is related to coordinate P2. _n Their x-coordinates are consistent.

[0156] Processor 54 at coordinates O2 and P2 _nIf the x-coordinates are consistent, it is determined that the height sensor 20 is positioned at position Pc (i.e., yes), and the process proceeds to step S24. On the other hand, if the determination is no, the processor 54 proceeds to step S46. Thus, when the processor 54 positions the height sensor 20 at positions Pw and Pc through the first positioning action PO1 and the second positioning action PO2, it executes the detection action DO in step S24.

[0157] In step S46, the processor 54 determines whether the height sensor 20 has reached the travel end SE of the moving mechanism 86. Here, as... Figure 20 and Figure 22 As shown, the moving mechanism 86 is configured to cause the moving mechanism 38 and the height sensor 20 to reciprocate between an initial position IP1 and a predetermined travel end SE. This travel end SE is represented by the x-coordinate of the height robot coordinate system C2.

[0158] Processor 54 determines whether the x-coordinate of the height sensor 20 at the current time point in the robot's coordinate system C2 (coordinate O2) matches the x-coordinate of the travel end SE. If the x-coordinate of coordinate O2 matches the x-coordinate of the travel end SE, processor 54 determines that the height sensor 20 has reached the travel end SE (i.e., yes), and proceeds to step S47. On the other hand, if the determination is no, processor 54 returns to step S22.

[0159] Thus, during the period when the processor 54 determines no in steps S45 and S46, the loop of steps S22, S45, and S46 is repeatedly executed. In addition, the processor 54 can also repeatedly execute the loop of steps S22, S45, and S46 with a control cycle τ (specifically, synchronously with the action of the conveying sensor 14 detecting the conveying amount δ).

[0160] In step S47, the processor 54 functions as the positioning execution unit 66, executing a recovery action RO to restore the height sensor 20 to its initial position IP1. Specifically, the processor 54 actuates the moving mechanism 86, causing the height sensor 20 to move in the opposite direction Dc' to the x-coordinate of the height robot coordinate system C2 corresponding to the initial position IP1. After step S47, the processor 54 executes step S22 as described above.

[0161] Next, in step S48, the processor 54 determines whether the recovery action RO is completed. Specifically, the processor 54 determines whether the x-coordinate of the height sensor 20 at the current time point in the height robot coordinate system C2 is consistent with the x-coordinate representing the initial position IP1. If the x-coordinate of the coordinate O2 is consistent with the x-coordinate of the initial position IP1, the processor 54 determines that the recovery action RO is completed (i.e., yes) and proceeds to step S49. On the other hand, if the determination is no, the processor 54 returns to step S22.

[0162] Thus, during the period when the processor 54 determines that the result is negative in step S48, the loop of steps S22 and S48 is repeatedly executed. In addition, the processor 54 can also repeatedly execute the loop of steps S22 and S48 with a control cycle τ (specifically, synchronously with the action of the conveying sensor 14 detecting the conveying amount δ).

[0163] In step S49, the processor 54 updates the location database 202. Specifically, the processor 54 reads the coordinate P5 whose "status" is "waiting to be detected" from the location database 202 stored in the memory 56 at the current time. _n Using the coordinate transformation matrix MX between the current time-point transport coordinate system C5 and the height robot coordinate system C2, the coordinate P5 is changed. _n Transformed into coordinates P2 in the robot's height coordinate system C2 _n (x, y).

[0164] Then, processor 54 determines the transformed coordinates P2. _n The processor 54 determines whether the x-coordinate of the coordinate exceeds the x-coordinate of the initial position IP1. The processor 54 will then determine the coordinate P5 that exceeds the x-coordinate of the initial position IP1. _n The "Status" has changed to "Detection Failed". This "Detection Failed" indicates that the coordinates P5 could not be detected. _n The state of position Ph at work site 108.

[0165] For example, at the start time of step S49, Figure 16 The location database 202 shown is stored in memory 56, and the coordinates of No.4 are P5. _4 The x-coordinate exceeds the initial position IP1. In this case, such as Figure 24 As shown, processor 54 will set the coordinates P5 of No.4. _4 The "status" has changed to "detection failed".

[0166] Furthermore, in the location database 202 at the start time of step S49, there are multiple coordinates P5 with a "status" of "waiting for detection". _n In this case, processor 54 can read multiple coordinates P5 in the order specified by "No" in column 204._n Whenever the coordinate P5 is read _n At that time, it is sequentially determined whether the x-coordinate of the initial position IP1 has been exceeded. Additionally, the processor 54 can also determine the coordinate P5. _n If the x-coordinate of the position exceeds the x-coordinate of the initial position IP1, then change the x-coordinate of P5 instead. _n Instead of changing the "status", the coordinate P5 was deleted from the location database 202. _n Thus, processor 54 updates location database 202. After step S49, processor 54 proceeds to step S26.

[0167] As described above, in this embodiment, the robot 82 is configured to move the height sensor 20 along the width direction Dw and the conveying direction Dc, and the work position acquisition unit 64 acquires the second position Pw of the work part 108 in the width direction Dw and the third position Pc of the work part 108 in the conveying direction Dc. Figure 6 Step S14 in the process.

[0168] Then, in positioning action PO1, the positioning execution unit 66 moves the height sensor 20 toward the second position Pw along the width direction Dw by the action of the robot 82. If the height sensor 20 cannot be positioned at the second position Pw before the working part 108 passes the height sensor 20 in the transport direction Dc (as in step S43), the robot 82 moves the height sensor 20 toward the transport direction Dc (step S44). According to this structure, even in applications such as continuous transport of a large number of items 100 by the transport device 12, the position Ph of the item 100 in the height direction Dh can be detected by tracking the item 100 in the transport direction Dc and performing detection action DO.

[0169] In this embodiment, when the positioning determination unit 70 places the height sensor 20 at the second position Pw during positioning action PO1 (as in step S42), it determines whether the work part 108 has passed the height sensor 20 in the transport direction Dc (step S43). Then, if the positioning execution unit 66 determines that the work part 108 has passed the height sensor 20, it moves the height sensor 20 towards the transport direction Dc through the action of the robot 82 (step S44). According to this structure, the height sensor 20 placed at the second position Pw is moved towards the transport direction Dc, and the item 100 is tracked at the second position Pw. When placed at the third position Pc of the work part 108, the detection action DO can be performed.

[0170] Next, refer to Figure 25Further examples of step S3 performed by the robot system 80 will be described. After step S3 begins, in step S51, the processor 54 reads the positions Pc and Pw of the work area 108. Specifically, the processor 54 reads the position database 202 stored in the memory 56 at the current time point, and reads the coordinate P5 where the "status" of column 208 is "waiting for detection" and the "No" of column 204 is the highest coordinate P5. _n .

[0171] In step S52, the processor 54 functions as a positioning determination unit 70, determining whether the height sensor 20 can be positioned at a position Pw in the width direction Dw of the work part 108 before it passes the work part 108. Specifically, the processor 54 obtains the coordinates O2 (x, y, z) of the height sensor 20 in the height robot coordinate system C2 at the current time point based on feedback from the encoders (or Hall elements) installed on the servo motors 42A and 42B. The y-coordinate of coordinate O2 represents the current position of the height sensor 20 in the width direction Dw.

[0172] On the other hand, processor 54 uses the coordinates P5 read in the previous step S51. _n The coordinate transformation matrix MX between the current time-point transport coordinate system C5 and the height robot coordinate system C2 will be used to transform the coordinate P5. _n Transformed into coordinates P2 in the robot's height coordinate system C2 _n (x, y, z). The coordinates are P2. _n The y-coordinate represents the current position of work location 108 (work target point P) in the width direction Dw.

[0173] Then, the processor 54 calculates the moving speed V1 of the moving mechanism 38, the conveying amount δ of the conveying device 12, the coordinates O2 of the height sensor 20, and the coordinates P2 of the working part 108. _n The system determines whether the height sensor 20 can be positioned at position Pw on the work site 108 before it passes the work site 108. Here, even if the processor 54 actuates the moving mechanism 38 to move the height sensor 20 from its current position (coordinate O2) towards the width direction Dw, it is sometimes impossible to position the height sensor 20 at position Pw (i.e., coordinate P2) on the width direction Dw of the work target point P before the work site 108 passes. _n (The position of the y-coordinate).

[0174] Processor 54 determines the movement speed V1, transport amount δ, coordinates O2, and P2 based on the following parameters: _nThe processor 54 determines whether the height sensor 20 can be positioned at position Pw before the work area 108 is passed. If the determination is yes, the processor 54 proceeds to step S41 and executes the first positioning action PO1. After step S41, the processor 54 and... Figure 11 The process is the same, executing steps S22 to S26 sequentially. On the other hand, if the determination is negative, the processor 54 proceeds to step S53.

[0175] In step S53, the processor 54 functions as the positioning execution unit 66, executing in parallel a first positioning action PO1, which moves the height sensor 20 along the width direction Dw via the action of the moving mechanism 38, and a second positioning action PO2, which moves the height sensor 20 along the conveying direction Dc via the action of the moving mechanism 86. Afterwards, the processor 54 executes step S22.

[0176] Next, in step S54, the processor 54 determines whether to configure the height sensor 20 at positions Pc and Pw of the work area 108. Specifically, based on feedback from the encoders (or Hall elements) installed on the servo motors 42A and 42B, the coordinates O2 (x, y, z) of the height sensor 20 at the current time point in the height robot coordinate system C2 are obtained.

[0177] On the other hand, processor 54 uses the coordinates P5 read in the most recent step S51. _n The coordinate transformation matrix MX between the current time-point transport coordinate system C5 and the height robot coordinate system C2 will be used to transform the coordinate P5. _n Transformed into coordinates P2 in the robot's height coordinate system C2 _n (x, y). Then, processor 54 determines whether the x and y coordinates of coordinate O2 are respectively related to coordinate P2. _n The x-coordinate and y-coordinate are the same.

[0178] At coordinates O2 and P2 _n If the x and y coordinates are consistent, the processor 54 determines that the height sensor 20 should be positioned at positions Pc and Pw (i.e., yes), and proceeds to step S24. On the other hand, if the determination is no, the processor 54 proceeds to step S46, and... Figure 23 The process is the same as the previous one, proceeding through steps S46, S47, S22, S48 and S49 in sequence.

[0179] During the period when the processor 54 determines that the result is negative in steps S54 and S46, it repeatedly executes the loop of steps S22, S54 and S46. In addition, the processor 54 can also repeatedly execute the loop of steps S22, S54 and S42 with a control cycle τ (specifically, synchronously with the action of the conveying sensor 14 detecting the conveying amount δ).

[0180] Thus, in this embodiment, the positioning determination unit 70 determines the position based on the second position Pw and the third position Pc (coordinate P2) of the working part 108. _n The system uses the height sensor's current position (coordinate O2) and determines whether the height sensor 20 can be configured in the second position Pw before the work area 108 passes the height sensor 20 (step S52).

[0181] Then, if it is determined that the height sensor 20 cannot be positioned at the second position Pw, the positioning execution unit 66 moves the height sensor 20 in the width direction Dw and the transport direction Dc by the action of the robot 82. According to this structure, it is possible to determine in advance whether a second positioning action PO2 is needed to move the height sensor 20 in the transport direction Dc, and quickly start the first positioning action PO1 and the second positioning action PO2.

[0182] Next, refer to Figure 26 and Figure 27 Another embodiment of the robot system 90 will be described. In addition to the aforementioned conveying device 12, conveying sensor 14, vision sensor 16, robots 18 and 22, height sensor 20, and control device 24, the robot system 90 also includes an in-situ sensor 92 and a robot 94 (a second robot) that moves the in-situ sensor 92. Figure 28 As shown, robot 94, like robot 82 described above, has: robot base 84, movable part 36, and moving mechanism 38 and 86, which move the in-situ sensor 92 in the width direction Dw and the conveying direction Dc.

[0183] The presence sensor 92 detects whether an item 100 is on the movable part 28 of the conveying device 12. For example, the presence sensor 92 is a photoelectric sensor or the like, which irradiates electromagnetic waves (visible light or infrared light) onto the object and detects the presence of the item 100 based on the reflected waves from the object. More specifically, the presence sensor 92 has a light-emitting part that emits electromagnetic waves along the optical axis A2 and a light-receiving part that receives the reflected waves (neither shown). Thus, in this embodiment, the presence sensor 92 is a sensor (photoelectric sensor) of a different type from the height sensor 20 (laser sensor). However, the presence sensor 92 could also be a laser sensor of the same type as the height sensor 20.

[0184] The presence sensor 92 is fixed to the movable part 36 with its optical axis A2 parallel to the height direction Dh. The presence sensor 92 supplies detection data γ, indicating whether an item 100 is detected, to the control device 24. Specifically, when the presence of the item 100 is detected, the presence sensor 92 sends detection data γ1 (e.g., an ON signal or a "1" signal) to the control device 24; on the other hand, when the presence of the item 100 is not detected, it sends detection data γ0 (e.g., an OFF signal or a "0" signal) to the control device 24.

[0185] The robot 94 is set to an in-situ robot coordinate system C6. In this embodiment, the robot base 84 is fixedly set to the in-situ robot coordinate system C6 with its x-axis parallel to the conveying direction Dc, its y-axis parallel to the width direction Dw, and its z-axis parallel to the height direction Dh.

[0186] On the other hand, an in-situ sensor coordinate system C7 is set for the in-situ sensor 92. The in-situ sensor coordinate system C7 is a translation coordinate system that moves in the robot coordinate systems C1, C2, and C6 together with the in-situ sensor 92 moved by the robot 94. The in-situ sensor coordinate system C7 defines the position and orientation of the in-situ sensor 92 in the robot coordinate systems C1, C2, and C6 (i.e., the position and orientation of the optical axis A2).

[0187] In this embodiment, the in-situ sensor coordinate system C7 is fixedly set for the in-situ sensor 92 with its origin positioned at the center of the light-emitting part and its z-axis parallel (specifically, aligned) with the optical axis A2. The positional relationships between the robot coordinate systems C1, C2, and C6, the sensor coordinate systems C3, C4, and C7, and the transport coordinate system C5 positioned at the initial position IPc are known through calibration. Therefore, these control coordinate systems C1 to C7 can be transformed into each other via a known coordinate transformation matrix MX.

[0188] In this embodiment, in addition to the feature position acquisition unit 62, work position acquisition unit 64, positioning execution unit 66, detection execution unit 68, and positioning determination unit 70 described above, the device 60 also includes: a sensor movement unit 72, a size acquisition unit 74, and a size determination unit 76. The operation of the robot system 90 will be described below. The processor 54 executes... Figure 5 The process is shown below.

[0189] At the beginning Figure 5At the specified time point in the process, the moving mechanisms 38 and 86 of robot 94 will position the in-situ sensor 92 at an initial position IP2 in the conveying direction Dc and the width direction Dw. This initial position IP2 is, for example, determined as: the position at one end of the travel distance of the in-situ sensor 92 implemented by the moving mechanism 38, and the end position of the travel distance of the in-situ sensor 92 implemented by the moving mechanism 86 in the direction Dc' opposite to the conveying direction Dc.

[0190] In this embodiment, as Figure 5 In step S1, processor 54 executes... Figure 29 The process is shown. Furthermore, in Figure 29 In the process shown, for and Figure 6 The same process flow is labeled with the same step numbers, and repeated descriptions are omitted. After step S1 begins, processor 54 executes steps S11 to S13 sequentially.

[0191] In this embodiment, the processor 54 calculates the angle θ between the length direction B1 or width direction B2, defined in the visual sensor coordinate system C4, and the conveying direction Dc in step S13. For example, the processor 54 uses the coordinate transformation matrix MX between the visual sensor coordinate system C4 and the conveying coordinate system C5 set at the initial position IPc to define the length direction B1 or width direction B2, defined in the conveying coordinate system C5, in the visual sensor coordinate system C4. Then, the processor 54 calculates the angle θ between the length direction B1 or width direction B2 and the x-axis direction of the conveying coordinate system C5.

[0192] Figure 30 An example is given of the angle θ between the length direction B1 and the x-axis direction of the transport coordinate system C5 (i.e., the transport direction Dc). This angle θ represents the orientation of the item 100 (specifically, the angle about the z-axis of the transport coordinate system C5). Thus, the processor 54 obtains the angle θ as the orientation data of the item 100.

[0193] Following step S13, in step S61, processor 54 causes robot 94 to move, moving in-situ sensor 92 in the width direction Dw and the transport direction Dc. Specifically, processor 54 activates in-situ sensor 92 to begin detecting the presence or absence of item 100, and through the actions of moving mechanisms 38 and 86, moves in-situ sensor 92 from its initial position IP2 in the width direction Dw and the transport direction Dc.

[0194] At this time, the processor 54 can also, based on the transport amount δ obtained from the transport sensor 14, move the in-situ sensor 92 at a speed V2, the same as the transport speed V2 of the transport device 12, in the transport direction Dc via the moving mechanism 86. Thus, in this embodiment, the processor 54 functions as a sensor moving unit 72 that moves the in-situ sensor 92 in both the width direction Dw and the transport direction Dc to cause the robot 94 to move. Figure 27 It can perform its functions.

[0195] Through the actions of robot 94, such as Figure 30 As shown, the presence sensor 92 moves relative to the article 100 along the track 110. During this movement, the presence sensor 92 supplies detection data γ (ON / OFF signal or 0 / 1 signal) indicating the presence or absence of the article 100 to the control device 24.

[0196] For example, in Figure 30 In the example shown, the in-situ sensor 92 moves along track 110 in the positive y-axis direction of the in-situ robot coordinate system C6. In this case, when the in-situ sensor 92 reaches the position of point 112 on the outer edge 102 of the item 100, the detection data γ sent by the in-situ sensor 92 switches from detection data γ0 (OFF signal or "0" signal) indicating that the item 100 is not detected to detection data γ1 (ON signal or "1" signal) indicating that the item 100 is detected.

[0197] Subsequently, when the in-situ sensor 92 crosses the object 100 and exceeds point 114 on the outer edge 102, the detection data γ sent by the in-situ sensor 92 switches from detection data γ1 to detection data γ0. At the first time point when the detection data γ obtained from the in-situ sensor 92 switches from detection data γ0 to detection data γ1, the processor 54 obtains the coordinates U6 of the in-situ sensor 92 (specifically, the origin of the in-situ sensor coordinate system C7) in the in-situ robot coordinate system C6. _1 (x, y, z).

[0198] Furthermore, at the second time point when the detection data γ obtained from the in-situ sensor 92 switches from detection data γ1 to detection data γ0, the processor 54 obtains the coordinates U6 of the in-situ sensor 92 in the in-situ robot coordinate system C6. _2 (x, y, z). Coordinates U6 _1 The y-coordinate and coordinate U6 _2 The difference Δy between the y coordinates is represented by Figure 30 The data for the distance Δy between midpoint 112 and point 114.

[0199] In step S62, the processor 54 obtains the dimension DM of the article 100 based on the detection data γ detected by the in-situ sensor 92. Specifically, the processor 54 obtains the width dimension DM2 of the article 100 based on the distance Δy obtained as described above and the angle θ representing the orientation of the article 100. For example, the processor 54 calculates the width dimension DM2 using the formula DM2 = Δy / cosθ. Thus, in this embodiment, the processor 54 serves as the dimension acquisition unit 74 that obtains the dimension DM (width dimension DM2 in this embodiment) of the article 100 based on the detection data γ. Figure 27 It can perform its functions.

[0200] In step S63, the processor 54 determines whether the dimension DM obtained in the previous step S62 satisfies the reference ST. The reference ST is predetermined as the allowable range of dimension DM in order to select the item 100 as the work object. In this embodiment, the processor 54 determines whether the width dimension DM2 obtained in the previous step S62 is within the allowable range determined as the reference ST. If the width dimension DM2 is within the allowable range, the processor 54 determines that the dimension DM satisfies the reference ST (i.e., yes), and proceeds to step S14.

[0201] On the other hand, if the width dimension DM2 is outside the allowable range, the processor 54 determines that the dimension DM does not meet the reference ST (i.e., no), and proceeds to step S16. Thus, in this embodiment, the processor 54 serves as the dimension determination unit 76 for determining whether the dimension DM meets the reference ST. Figure 27 It can perform its functions.

[0202] If the determination in step S63 is negative, the processor 54 does not execute steps S14 and S15. Therefore, the coordinates P5 of the item 100 at the position Q of the detected target feature 106 are obtained in step S13. _n This is not stored in location database 202. As a result, for item 100, processor 54 does not execute. Figure 5 The steps S3 (Detect Action DO) and S5 (Operation WK) are included.

[0203] As described above, in the apparatus 60 of this embodiment, the sensor moving unit 72 causes the second robot 94 to move, causing the in-situ sensor 92 to move in the width direction Dw (step S61). In addition, the size acquisition unit 74 acquires the size DM (width size DM2) of the article 100 based on the detection data γ (γ1 or γ2) detected by the in-situ sensor 92 during the movement of the second robot 94 in the width direction Dw (step S62).

[0204] Furthermore, the size determination unit 76 determines whether the size DM obtained by the size acquisition unit 74 meets the reference ST (step S63). Then, for items 100 that have a size DM that the size determination unit 76 determines does not meet the reference ST, the detection execution unit 68 does not perform the detection action DO (step S24). According to this structure, items 100 that are not part of the work object can be effectively identified, and the detection action DO can be avoided for items 100 that are not part of the work object. As a result, the amount of computation required for the detection action DO can be reduced, thereby shortening the cycle time.

[0205] Furthermore, in this embodiment, the case where the robot 94 moves the in-situ sensor 92 in the width direction Dw and the conveying direction Dc is described. However, it is not limited to this; the robot 94 may also move with... Figure 3 The robot 18 shown is configured to have a robot base 34, a movable part 36, and a moving mechanism 38, and the in-situ sensor 92 moves only along the width direction Dw. In this case, the processor 54 can also obtain the width dimension DM2 of the item 100 in step S62 based on the conveying amount δ obtained from the conveying sensor 14, the aforementioned distance Δy, and the angle θ.

[0206] Furthermore, in this embodiment, the processor 54 is obtained Figure 30 The width dimension DM2 shown is described as the dimension DM of the item 100. However, the processor 54 can obtain the length dimension DM3 of the item 100 in the length direction B1 from the detection data γ of the in-situ sensor 92 based on the angle θ representing the orientation of the item 100. Furthermore, the robot 94 can also be applied to the robot system 10.

[0207] In addition, the processor 54 can also execute the computer program PG stored in the memory 56. Figure 5 The process is shown. In addition, the functions of the device 60 (feature position acquisition unit 62, work position acquisition unit 64, positioning execution unit 66, detection execution unit 68, positioning determination unit 70, sensor movement unit 72, size acquisition unit 74, size determination unit 76) executed by the processor 54 can also be functional modules implemented by the computer program PG.

[0208] Additionally, the computer program PG may also include: a first computer program PG1, which causes the processor 54 to execute... Figure 5 The main process; the second computer program PG2, which causes the processor 54 to execute the sub-process of step S1 ( Figure 6 , Figure 29 The third computer program PG3 causes processor 54 to execute the subprocess of step S3. Figure 11 , Figure 23 , Figure 25The fourth computer program PG4 causes processor 54 to execute the subprocess of step S5. Figure 15 ).

[0209] Furthermore, in step S14 described above, the processor 54 can further obtain the z-coordinate of the target point P in the transport coordinate system C5 as a temporary coordinate. In this case, the processor 54 obtains the coordinates P5 (x, y, z) of the target point P. In this case, the processor 54 can set a predetermined coordinate value (e.g., z = 0) or the same coordinate value as the z-coordinate of coordinate Q5 obtained in step S13 as the temporary coordinate value.

[0210] Then, in step S15, the processor 54 can store the coordinates P5 (x, y, z) in the location database 202. When the z coordinate of the work target point P in the transport coordinate system C5 is obtained through the detection action DO in step S24, the z coordinate of the coordinates P5 stored in the location database 202 as a temporary coordinate value is replaced with the z coordinate obtained through the detection action DO. Thus, the z coordinate obtained through the detection action DO is appended and stored in the location database 202.

[0211] Furthermore, in the above embodiment, the case where the detection target feature 106 is printed or engraved has been described. However, the detection target feature 106 is not limited to this. For example, the processor 54 may also function as a feature position acquisition unit 62, acquiring the position Q of the outer edge 102 of the article 100 as the detection target feature 106. Specifically, in step S13 described above, the processor 54 may determine the outer edge 102 of the article 100 as the detection target feature 106 in the visual sensor coordinate system C4, and acquire the coordinates Q4 (x, y, z) of the feature point Q (e.g., center point, vertex, midpoint of one side, etc.) of the outer edge 102 in the visual sensor coordinate system C4.

[0212] Furthermore, in the above embodiments, the vision sensor 16 has been described with a pair of cameras 32A and 32B, but it is not limited thereto; the vision sensor 16 may also have a single camera 32A (i.e., a monocular camera). In this case, in step S13, the processor 54 obtains the coordinates Q4(x, y, z) of the feature point Q in the vision sensor coordinate system C4 based on the size DMf of the detected target feature 106 displayed in the image data 200A captured in step S12.

[0213] Here, given that the size DMf of the detected target feature 106 is known, the distance from the camera 32A (the origin of the visual sensor coordinate system C4) to the detected target feature 106 (i.e., the z-coordinate of the visual sensor coordinate system C4) can be calculated based on the size DMf of the detected target feature 106 displayed in the image data 200A. Therefore, the processor 54 can obtain the coordinates Q4 (x, y, z) of the feature point Q in the visual sensor coordinate system C4 based on the size DMf within the image data 200A.

[0214] Furthermore, in the above embodiment, the case where the coordinates P5 of the transport coordinate system C5 are stored in the position database 202 has been described. However, it is not limited to this; the processor 54 may also, in step S14, obtain, for example, the positions Pw and Pc of the work area 108 as the coordinates P1 or P2 of the robot coordinate system C1 or C2, and store them in the position database 202 in step S15. In this case, whenever the processor 54 executes the above-described step S22, it updates the x-coordinate of the coordinates P1 or P2 stored in the position database 202 according to the transport amount δ. In addition, the transport coordinate system C5 may also be a fixed coordinate system fixed relative to the base portion 26 of the transport device 12.

[0215] in addition, Figure 10 , Figure 14 , Figure 16 and Figure 24 The location database 202 shown is one example; the processor 54 can create a location database 202 with arbitrary data construction. Alternatively, the processor 54 may not create a location database 202, but instead store the coordinates of the positions Pw and Pc of the work area 108 obtained in step S14 in the memory 56 in a time series. Furthermore, the shape of the detected target feature 106 is not limited to a rectangle; it can also be any shape such as a polygon, circle, or ellipse. Additionally, robots 18, 22, 82, and 94 can be any type of robot. For example, robots 18, 82, or 94 can be parallel linkage robots or vertical multi-joint robots.

[0216] The present disclosure has been described in detail above, but it is not limited to the various embodiments described above. Various additions, substitutions, modifications, and partial deletions can be made to these embodiments without departing from the core essence of the present disclosure, or without departing from the core essence of the present disclosure derived from the claims and their equivalents. Furthermore, these embodiments can also be implemented in combination. For example, in the above embodiments, the order of each action and the order of each process have been shown as an example, but are not limited thereto. The same applies to the use of numerical values ​​or mathematical formulas in the description of the above embodiments.

[0217] As described above, the following methods are described in this disclosure.

[0218] (Method 1) An apparatus 60, which detects the position P of a working part 108 on an item 100 transported by a conveying device 12 in order to perform a prescribed operation WK on the working part 108. The apparatus 60 includes: a feature position acquisition unit 62, which acquires the position Q of a detection target feature 106 of the item 100 shown in image data 200A and 200B of the item 100 captured by cameras 32A and 32B; and a working position acquisition unit 64, which acquires the position P of the working part 108 based on the position Q acquired by the feature position acquisition unit 62 and a known positional relationship PR between the working part 108 and the detection target feature 106. 08. Positions Pw and Pc in directions Dw and Dc orthogonal to the height direction Dh of the conveying device 12; Positioning execution unit 66, which moves the robots 18 and 82 that move the height sensor 20 to perform a positioning action PO of the height sensor 20 at positions Pw and Pc obtained by the work position acquisition unit 64, wherein the height sensor 20 is used to detect the position Ph of the item 100 in the height direction Dh; Detection execution unit 68, which moves the height sensor 20 positioned at positions Pw and Pc by the positioning action PO to perform a detection action DO, which detects the first position Ph of the work part 108 in the height direction Dh.

[0219] (Method 2) According to the device 60 of Method 1, the robots 18 and 82 are configured to move the height sensor 20 along the width direction Dw of the conveying device 12, the work position acquisition unit 64 acquires the second position Pw of the work part 108 in the width direction Dw, the positioning execution unit 66 moves the height sensor 20 in the width direction Dw by the action of the robots 18 and 82 in the positioning action PO, and makes the height sensor 20 standby in the second position Pw. When the work part 108 reaches directly below the height sensor 20 in the second position Pw standby by the conveying action of the conveying device 12, the detection execution unit 68 executes the detection action DO.

[0220] (Method 3) According to the device 60 described in Method 1 or 2, the work position acquisition unit 64 performs the following operations: acquires the second position Pw of the work part 108 in the width direction Dw of the conveying device 12 and the third position Pc of the work part 108 in the conveying direction Dc of the conveying device 12; stores the acquired second position Pw and third position Pc as coordinates P1, P2, P5 of the control coordinate system C set for work WK in the position database 202, and the positioning execution unit 66 performs the positioning operation PO according to the coordinates P1, P2, P5 stored in the position database 202.

[0221] (Method 4) According to the device 60 of Method 3, the control coordinate system C has a transport coordinate system C5 that moves in the transport direction Dc according to the transport amount δ of the transport device 12 transporting the item 100, and the work position acquisition unit 64 stores the second position Pw and the third position Pc as coordinates P5 of the transport coordinate system C5 in the position database 202.

[0222] (Method 5) According to the device 60 of Method 3 or 4, the working position acquisition unit 64 adds the first position Ph detected by the detection action DO as coordinates P1, P2, P5 of the control coordinate system C to the position database 202, thereby updating the position database 202.

[0223] (Method 6) The device 60 according to any one of Methods 1 to 5, wherein the robot 82 is configured to move the height sensor 20 along the width direction Dw and the transport direction Dc of the conveying device 12, the work position acquisition unit 64 acquires the second position Pw of the work part 108 in the width direction Dw and the third position Pc of the work part 108 in the transport direction Dc, and the positioning execution unit 66 performs the following actions in the positioning action PO: moves the height sensor 20 toward the second position Pw along the width direction Dw by the action of the robot 18; if the height sensor 20 cannot be positioned at the second position Pw before the work part 108 passes the height sensor 20 in the transport direction Dc, the height sensor 20 is moved toward the transport direction Dc by the action of the robot 18.

[0224] (Method 7) According to the device 60 of Method 6, the device 60 further includes: a positioning determination unit 70, which determines whether the working part 108 has passed the height sensor 20 in the conveying direction Dc when the height sensor 20 is placed in the second position Pw during the positioning action PO; if the working part 108 has passed the height sensor 20, the positioning execution unit 66 moves the height sensor 20 in the conveying direction Dc by the action of the robot 82.

[0225] (Method 8) According to the device 60 of Method 6, the device 60 further includes: a positioning determination unit 70, which determines whether the height sensor 20 can be positioned at the second position Pw before the work part 108 passes the height sensor 20 based on the second position Pw, the third position P and the current position of the height sensor 20; if it is determined that the height sensor 20 cannot be positioned at the second position Pw, the positioning execution unit 66 moves the height sensor 20 in the width direction Dw and the conveying direction Dc by the action of the robot 82.

[0226] (Method 9) The apparatus 60 according to any one of Methods 1 to 8, wherein a plurality of candidates 108A, 108B, 108C, 108D for the work position 108 are predetermined in the article 100, and the positional relationship PR between the candidate and the detection target feature 106 is known, and the work position acquisition unit 64 performs the following operation: based on the image data 200A, select one of the plurality of candidates 108A, 108B, 108C, 108D, and acquire the position Pw, Pc of the selected candidate 108C in the orthogonal directions Dw, Dc.

[0227] (Method 10) According to the apparatus 60 of Method 9, the working position acquisition unit 64 performs the following operations: defines a plurality of candidates 108A, 108B, 108C, 108D in image data 200A; selects a candidate 108C based on the pixel value PV of pixel PX, wherein pixel PX displays at least a portion of the interval 210 from the detected target feature 106 to the candidates 108A, 108B, 108C, 108D in image data 200A.

[0228] (Method 11) The apparatus 60 according to any one of Methods 1 to 10, wherein the apparatus 60 includes: a sensor moving unit 72, which moves a second robot 94 that moves an in-situ sensor 92 toward the width direction Dw of the conveying device 12, thereby moving the in-situ sensor 92 toward the width direction Dw, wherein the in-situ sensor 92 detects the presence or absence of an article 100; a size acquisition unit 74, which acquires the size DM of the article 100 based on detection data γ detected by the in-situ sensor 92 during the movement of the second robot 94 toward the width direction Dw; a size determination unit 76, which determines whether the size DM acquired by the size acquisition unit 74 satisfies the reference ST, and the detection execution unit 68 does not perform the detection action DO for articles having a size DM that is determined by the size determination unit 76 to not satisfy the reference ST.

[0229] (Method 12) The apparatus 60 according to any one of Methods 1 to 11, wherein the feature position acquisition unit 62 acquires the following positions as the detection target feature 106: the position Q of the printing on the article 100, the position Q of the engraving formed on the article 100, or the position Q of the outer edge 102 of the article 100.

[0230] (Method 13) A robot system 10, 80, 90, comprising: cameras 32A, 32B for photographing an item 100 transported by a conveying device 12; a height sensor 20 for detecting the position Ph of the item 100 in the height direction Dh of the conveying device 12; robots 18, 82 for moving the height sensor 20; and a device 60 as described in any one of methods 1 to 12.

[0231] (Method 14) A method for detecting the position P of a working part 108 on an item 100 transported by a conveying device 12 in order to perform a prescribed operation WK on the working part 108. The method involves obtaining the position Q of a detection target feature 106 of the item 100 as shown in image data 200A and 200B of the item 100 captured by cameras 32A and 32B, and obtaining the position P of the working part 108 relative to the conveying device based on the obtained position Q and the known positional relationship PR between the working part 108 and the detection target feature 106. The positions Pw and Pc of the height sensor 20 in the orthogonal directions Dw and Dc of the height direction Dh of the height sensor 12 cause the robot 18 to move and perform a positioning action PO to configure the height sensor 20 at the positions Pw and Pc of the acquired work site 108. The height sensor 20 is used to detect the position Ph of the item 100 in the height direction Dh. The height sensor 20 configured at the positions Pw and Pc by the positioning action PO is activated to perform a detection action DO. The detection action DO detects the position Ph of the work site 108 in the height direction Dh.

[0232] (Method 15) A computer program PG that causes processor 54 to execute the method described in Method 14.

[0233] Symbol Explanation

[0234] 10, 80, 90 Robot Systems

[0235] 12 Conveying device

[0236] 14. Transporting sensors

[0237] 16 Visual Sensors

[0238] 20 Height Sensors

[0239] 92 In-situ sensors

[0240] Robots 18, 22, 82, and 94

[0241] 24 Control Device

[0242] 32A and 32B cameras

[0243] 60 devices

[0244] 62 Feature Position Acquisition Unit

[0245] 64. Work Location Acquisition Department

[0246] 66 Positioning Execution Department

[0247] 68 Inspection and Testing Department

[0248] 70 Positioning Judgment Department

[0249] 72 Sensor Moving Part

[0250] 74 Dimension Acquisition Section

[0251] 76 Dimensioning Section

[0252] 100 items

[0253] 102 Outer edge

[0254] 106 Detection of target features

[0255] 108 Work Location

[0256] 200A image data.

Claims

1. An apparatus for detecting the position of a work area in order to perform a prescribed operation on a work area on an item being transported by a conveying device, characterized in that, The device has: The feature location acquisition unit acquires the location of the detection target feature of the object as shown in the image data of the object captured by the camera. The work position acquisition unit obtains the position of the work part in a direction orthogonal to the height direction of the conveying device based on the position obtained by the feature position acquisition unit and the known positional relationship between the work part and the detected target feature; A positioning execution unit that causes a robot with a movable height sensor to move, thereby performing a positioning action by configuring the height sensor at the position obtained by the work position acquisition unit, wherein the height sensor is used to detect the position of the object in the height direction; and The detection execution unit actuates the height sensor, which is positioned at the location by the positioning action, to perform a detection action that detects the first position of the working part in the height direction.

2. The apparatus according to claim 1, characterized in that, The robot is configured to move the height sensor along the width direction of the conveying device. The work position acquisition unit acquires the second position of the work part in the width direction. The positioning execution unit moves the height sensor in the width direction through the robot's movement during the positioning action, and puts the height sensor into standby position at the second position. When the working part reaches directly below the height sensor in the second position due to the conveying action of the conveying device, the detection execution unit performs the detection action.

3. The apparatus according to claim 1, characterized in that, The work position acquisition unit performs the following actions: The second position of the working part in the width direction of the conveying device and the third position of the working part in the conveying direction of the conveying device are obtained; The obtained second and third positions are stored in the position database as coordinates of the control coordinate system set for the operation. The positioning execution unit performs the positioning action based on the coordinates stored in the location database.

4. The apparatus according to claim 3, characterized in that, The control coordinate system includes a conveying coordinate system that moves in the conveying direction according to the amount of the item conveyed by the conveying device. The work location acquisition unit stores the second and third locations as coordinates of the transport coordinate system in the location database.

5. The apparatus according to claim 3, characterized in that, The work position acquisition unit adds the first position detected by the detection action as the coordinate of the control coordinate system to the position database, thereby updating the position database.

6. The apparatus according to claim 1, characterized in that, The robot is configured to move the height sensor along the width direction and the conveying direction of the conveying device. The work position acquisition unit acquires the second position of the work part in the width direction and the third position of the work part in the conveying direction. The positioning execution unit performs the following actions during the positioning operation: The robot moves the height sensor toward the second position along the width direction by its actions. If the height sensor cannot be positioned in the second position before the work part passes the height sensor in the conveying direction, the robot's movement causes the height sensor to move in the conveying direction.

7. The apparatus according to claim 6, characterized in that, The device further includes a positioning determination unit, which, during the positioning operation, when the height sensor is positioned at the second position, determines whether the working part has passed the height sensor in the conveying direction. If it is determined that the work area has passed the height sensor, the positioning execution unit moves the height sensor in the conveying direction by the robot's movement.

8. The apparatus according to claim 6, characterized in that, The device further includes a positioning determination unit, which determines, based on the second and third positions and the current position of the height sensor, whether the height sensor can be positioned at the second position before the work area passes the height sensor. If it is determined that the height sensor cannot be positioned in the second position, the positioning execution unit moves the height sensor in the width direction and the conveying direction by the robot's movement.

9. The apparatus according to claim 1, characterized in that, Multiple candidate locations for the work area are pre-determined within the article, and the positional relationship between these candidate locations and the target feature being detected is known. The work position acquisition unit performs the following actions: Based on the image data, one of the plurality of candidates is selected; Obtain the position of the selected candidate in the orthogonal direction.

10. The apparatus according to claim 9, characterized in that, The work position acquisition unit performs the following actions: The plurality of candidates are defined in the image data; The candidate is selected based on the pixel value of the pixel, wherein the pixel in the image data shows at least a portion of the interval from the detected target feature to the candidate.

11. The apparatus according to claim 1, characterized in that, The device has: A sensor moving unit causes a second robot to move an in-situ sensor in the width direction of the conveying device, thereby moving the in-situ sensor in the width direction, wherein the in-situ sensor detects the presence or absence of the item; A size acquisition unit acquires the size of the item based on detection data detected by the in-situ sensor during the movement of the second robot in the width direction; and The dimension determination unit determines whether the dimension obtained by the dimension acquisition unit meets the reference. The detection execution unit does not perform the detection action on the following items: items that have dimensions that are determined by the size determination unit to be non-compliant with the reference.

12. The apparatus according to claim 1, characterized in that, The feature location acquisition unit acquires the following locations as the detection target features: the location printed on the article, the location formed on the engraving of the article, or the location on the outer edge of the article.

13. A robot system, characterized in that, have: A camera that takes pictures of items being transported by a conveyor. A height sensor is used to detect the position of the item in the height direction of the conveying device; The robot moves the height sensor; and The apparatus according to claim 1.

14. A method for detecting the position of a work area on an article being transported by a conveying device in order to perform a prescribed operation on that work area, characterized in that, Based on the image data of the object captured by the camera, the location of the target feature of the object displayed in the image data is obtained. Based on the obtained location, the known positional relationship between the working part and the detected target feature, the position of the working part in a direction orthogonal to the height direction of the conveying device is obtained. The robot, equipped with a movable height sensor, performs a positioning action to place the height sensor at the acquired work site location, wherein the height sensor is used to detect the position of the object in the height direction. The height sensor, which is positioned at the location by the positioning action, is activated to perform a detection action that detects the position of the work part in the height direction.

15. A computer program, characterized in that, The computer program causes the processor to execute the method of claim 14.

Images