Control device, robot system, and sensor device
Patent Information
- Application Number
- PCT/JP2025/015186
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2025-04-18
- Publication Date
- 2026-10-22
Smart Images
Figure JP2025015186_22102026_PF_FP_ABST
Abstract
Description
Control device, robot system, and sensor device
[0001] The present disclosure relates to a control device, a robot system, and a sensor device.
[0002] Conventionally, a technique is known that causes the tip of a robot arm to follow a moving object based on a detection result of a sensor. For example, reference is made to Patent Documents 1 and 2. Further, a technique is known in which a hand camera is attached to the tip of a robot arm, and position control of the tip of the arm is performed using a detection result of the hand camera. For example, reference is made to Patent Document 3.
[0003] JP 2023-108062 A, JP 2021-24027 A, JP 2011-235386 A
[0004] In conventional techniques, a hand camera may be attached to the tip of an arm for follow-up control that causes the tip of the robot arm to follow a moving object. In this case, since the hand camera moves together with the tip of the arm, it is necessary to consider interference between the hand camera and surrounding obstacles or the target article when setting the motion of the arm. In addition, when the arm performs work on a work target, the work target or the followed target may become invisible from the hand camera, which makes setting of the follow-up control difficult in such cases.
[0005] In order to obtain position information of a target for follow-up control with accuracy equivalent to that of a hand camera, a camera may be movably supported and a rail arranged along a transport device may be provided. In this case, a drive device is provided that moves the camera along the rail so that the camera moves in synchronization with the object being moved by the transport device. The camera may be supported by another robot instead of a slide rail. In these cases, the configuration for supporting and moving the camera becomes large-scale, making it impossible to easily newly install or relocate the apparatus, and it is not easy to ensure the detection accuracy of the camera. There is a demand for a technique that enables control for aligning the position of the tip of an arm with a moving object while simplifying the configuration of the apparatus.
[0006] A first aspect of the present disclosure is a control device capable of controlling at least a drive unit that rotates a sensor about a rotation axis extending in a direction intersecting the direction of movement of a moving article, the control device comprising at least one processor, the at least one processor configured to perform sensor rotation processing to control the drive unit so that an object in the moving article is positioned within the detection range of the sensor, and arm control processing to control a robot arm using at least information of the rotation of the sensor about the rotation axis.
[0007] A second aspect of the present disclosure comprises a drive unit for rotating a sensor about a rotation axis extending in a direction intersecting the direction of movement of a moving article, a robot arm, and a control device for controlling the arm, wherein at least one processor of the control device is configured to perform a sensor rotation process that controls the drive unit based on the output of the sensor so that an object in the moving article is placed within the detection range of the sensor, and an arm control process that controls the robot arm using at least information of the rotation of the sensor about the rotation axis.
[0008] A third aspect of the present disclosure is a sensor device comprising: a sensor; a drive unit for rotating the sensor about a rotation axis extending in a direction intersecting the direction of movement of a moving article; and a sensor control device capable of controlling the drive unit, wherein the sensor control device is configured to perform: a sensor rotation process for controlling the drive unit so that an object on the moving article is positioned at a predetermined location within the detection range of the sensor; and a rotation information transmission process for transmitting information about the rotation of the sensor about the rotation axis to a control device of a robot in order to cause the robot arm to follow the moving object.
[0009] This is a schematic side view of the robot system of the first embodiment. This is a schematic top view of the robot system of the first embodiment. This is a schematic side view of the main part of the robot system of the first embodiment. This is an example of an image obtained by the sensor of the robot system of the first embodiment. This is a block diagram of the control device of the robot system of the first embodiment. This is a flowchart showing an example of processing performed by the processor second embodiment. This is a block diagram of a modified sensor control device of the robot system. This is a flowchart showing an example of processing performed by the processor of the modified sensor control device of the robot system. This is a schematic side view of the robot system of the third embodiment. This is a flowchart showing an example of processing performed by the processor of the control device of the robot system of the third embodiment.
[0010] The control device 30 and robot system 1 of the first embodiment will be described below with reference to the drawings. The robot system 1 of the first embodiment has one or more robots (machines) 10. In the first embodiment, each robot 10 is a vertical articulated robot and is equipped with an articulated arm 20 as shown in Figures 1 and 2. The robots 10 are connected to the control device 30, and in the first embodiment, a tool 40 is attached to the end of the arm 20.
[0011] The arm 20 comprises multiple arm members 21, 22, 23, 24, 25, 26 and multiple joints. Each of the multiple joints is rotatable around axes J1 to J6 shown in Figure 1. The arm 20 also comprises multiple motors 27, such as servo motors, to drive each of the multiple joints (Figure 2). Various types of motors, such as rotary motors and linear motors, can be used as each motor 27. Each motor 27 has an operating position detection device for detecting its operating position and operating speed, and the operating position detection device is an encoder as an example. The detected value of the operating position detection device is transmitted to the control device 30. The number of arm members and joints may be 5 or less, or 7 or more.
[0012] The arm 20 is not limited to any particular type. The arm 20 may be the arm of a horizontal articulated robot, the arm of a multi-link robot, etc. Furthermore, the arm 20 may be supported by a moving device such as a linear guide, an AGV (Automatic Guided Vehicle), a vehicle, a walking robot, etc.
[0013] Furthermore, in the first embodiment, the arm 20 may be a robot arm for a collaborative robot, or it may be a normal robot arm that is not for a collaborative robot. The collaborative robot has a function to detect contact between the arm 20 and people, objects, etc., using known sensors such as force sensors, contact sensors, and vision sensors, and to safely stop the arm 20 according to the detection result. In addition, the collaborative robot may also have a function to decelerate or stop the arm 20 immediately before it comes into contact with people, objects, etc.
[0014] As shown in Figure 1, the robot system 1 includes a transport device (transporting means) 2 for moving an article 100. The arm 20 performs a predetermined operation on an object 101 of the article 100 that is being transported by the transport device 2. In the first embodiment, the robot system 1 uses a sensor 60 capable of detecting the object 101 to make a tool 40, part 110, etc., attached to or supported at the tip of the arm 20 follow the object 101 being transported by the transport device 2.
[0015] Furthermore, while the tracking is being performed, as a predetermined operation, in the first embodiment, the arm 20 attaches the mounting portion 111 of the component 110 to the target 101. The mounting portion 111 is, in one example, a fitting portion such as a pin or shaft, and the target 101 is, in one example, a fitted portion having a hole or the like into which the pin or shaft fits, but is not limited to these.
[0016] In the first embodiment, the tool 40 is a hand that supports the part 110 using magnetic force, air suction, etc., or a hand that supports the part 110 by gripping with multiple claws. In Figure 1, the tool 40 supports the part 110 by suction.
[0017] Furthermore, tool 40 may be a tool used for other tasks, processing, etc. In this case, tool 40 may be a tool for turning fastening parts such as bolts, a welding tool, a painting tool, a cleaning tool, etc., or a tool for other purposes. The aforementioned other tasks may be the removal of articles being moved by the conveying device 2 (means of transport). In this case, tool 40 may be a tool capable of supporting articles, such as a hand.
[0018] Article 100 is not limited to a specific type of object. In the first embodiment, for example, article 100 is a car body. The conveying device 2 conveys article 100 by driving several of the multiple rollers 3 with a motor 2a, and in the first embodiment, the conveying device 2 conveys article 100 along the X axis in Figure 1.
[0019] In the first embodiment, a force sensor (force detection unit) 41 is attached to the tip of the arm 20. The force sensor 41 measures force or moment around the X axis, Y axis, and Z axis, for example, in the X axis direction, Y axis direction, and Z axis direction as shown in Figure 2. The force sensor 41 only needs to be capable of detecting the direction and degree of force applied to the tool 40 or the part 110 gripped by the tool 40. For this reason, in the first embodiment, the force sensor 41 is provided between the arm 20 and the tool 40, but the force sensor 41 may be provided inside the arm 20 or inside the tool 40. The force sensor 41 may not be provided at all.
[0020] In the first embodiment, the sensor 60 is a two-dimensional camera. The sensor 60 is capable of sequentially acquiring image data of an object 102, as shown in Figure 3, while the object 101 is within a predetermined range of the field of view. In some cases, a three-dimensional camera, a three-dimensional distance sensor, etc., may be used as the sensor 60.
[0021] Image data is data that can identify the position of one object or the position of at least one of multiple objects within the image data. The image data may also be data that can identify the position of each of one or more objects 102, 103 within the image data. Each object 102, 103 is a detection target used to detect the position and / or orientation of the work target 101, and each object 102, 103 moves together with the work target 101 and its relative position to object 101 does not change.
[0022] The control device 30 may also determine the positions of objects 101, 102, and 103 based on the position, shape, etc., of one or more characteristic parts of the article 100 in the image data. Furthermore, the control device 30 may determine the orientation of object 101 based on the positional relationship of multiple objects among objects 101, 102, and 103 in the image data. The control device 30 may also determine the orientation of objects 101, 102, and 103 based on the position, shape, etc., of characteristic parts in the image data. Examples of characteristic parts include distinctive elements such as corners of the article 100, and examples of objects 102 include marks, distinctive elements, etc., provided on the article 100 other than object 101. Considering that objects 102 and 103 are used to detect the position of object 101, it is preferable that objects 102 and 103 are closer to object 101.
[0023] As shown in Figures 1 and 3, the sensor 60 is supported by a support member 61 so as to be rotatable around a rotation axis 60a extending in the vertical direction. In the first embodiment, a drive unit 62 such as a servo motor is supported on one end of the support member 61, and the other end of the support member 61 is fixed to a predetermined position such as an installation surface. The sensor 60 is fixed to the output shaft 63 of the drive unit 62, and the sensor 60 rotates around the rotation axis 60a by the drive unit 62. In addition, it is also possible that the sensor 60 is fixed to a rotation shaft that is rotatably supported on one end of the support member 61 via a bearing or the like, and the rotation shaft is rotated by the drive unit 62.
[0024] The drive unit 62 is equipped with a detection device 63a (Figure 5), such as an encoder, capable of detecting at least one of the rotational position or rotational amount of the output shaft 63. Instead of detection by the detection device 63a, the processor 31 may detect at least one of the rotational position or rotational amount of the output shaft 63 based on a drive command to the drive unit 62, or at least one of the rotational position or rotational amount may be detected by other means. In the following description, the rotational position or at least one of them may be referred to as "rotation information". In the first embodiment, the drive unit 62 has a built-in detection device 63a, but a detection device 63a may be provided separately from the drive unit 62.
[0025] As shown in Figure 5, the control device 30 includes a processor 31 such as a CPU, a microcomputer processor, or a PLC processor, and a known display device 32 such as a liquid crystal display device. The control device 30 also includes a memory unit 33 having non-volatile storage such as RAM, and examples of non-volatile storage include hard disks, flash memory, and ROM. In some cases, the CPU, microcomputer, PLC, or storage device of an external computer may function as part of the memory unit 33. The control device 30 also includes an input device 34 which includes one of the following computers: a keyboard, touch panel, control panel, teaching control panel, or tablet computer, and a transmitting / receiving unit 35 for transmitting and receiving signals. The control device 30 also includes a servo controller 36 connected to each motor 27 and a controller 37 connected to the drive unit 62.
[0026] The teaching control panel is a portable control panel called a teach pendant. When the input device 34 is a computer such as a control panel, teaching control panel, or tablet computer, the input device 34 is equipped with a known display device 34A such as a liquid crystal display, and the processor of the input device 34 may function as part or all of the processor 31. The input device 34 and the transmitting / receiving unit 35 function as input units. The storage unit 33 stores an operation program 33A, and the processor 31 controls each motor 27 based on the operation program 33A and operation signals from the input device 34.
[0027] The memory unit 33 stores a path teaching program 33B, an object detection program 33C, a sensor rotation position control program 33D, an object position detection program 33E, a tracking control program 33F, and a force control program 33G. An example of the processing performed by the processor 31 based on these programs will be explained below with reference to the flowchart in Figure 6.
[0028] First, the processor 31 starts detecting the positions of objects 101, 102, and 103 in the image data (detection range) sequentially received from the sensor 60 based on the object detection program 33C (step S1-1). For example, the processor 31 starts detecting the positions of objects 102 and / or object 103 in the image data, as shown in Figures 3 and 4. Object 102 is a characteristic part such as the upper left corner of the protrusion on which object 101 is provided, and object 103 is a mark or the like provided on the article 100. The processor 31 detects the centroid positions of objects 102 and 103 using, for example, known image processing, but other positions of objects 102 and 103 may also be detected. In one example, the processor 31 starts the processing of step S1-1 when it receives a predetermined signal, when the sensor 60 receives the article 100, etc., and the same applies to the following embodiment.
[0029] Next, the processor 31 starts controlling the rotational position of the drive unit 62 based on the sensor rotational position control program 33D so that, for example, the detection position of the object 102 is always positioned at a predetermined location in the sequentially received image data (step S1-2). Instead of object 102, object 101 or object 103 may always be positioned at the predetermined location, or the centers of objects 102 and 103 may always be positioned at the predetermined location. In these cases as well, it can be said that the object is positioned at the predetermined location in the image data. As a result of step S1-2, the rotational direction of the sensor 60 follows the movement of the object 102 in the X-axis direction.
[0030] In the first embodiment, the object 102 is always positioned at a predetermined location in the image X direction in the image data, and the image X direction corresponds to the X-axis direction of the reference coordinate system 120 (Figures 1 and 2), which is the coordinate system of the arm 20. An example of a predetermined location is the center of the image data. For example, at each control cycle, the processor 31 sends a control command and / or current to the drive unit 62 to direct the position of the object 102 towards the predetermined location, according to the position of the object 102 in the image data at that time.
[0031] Next, based on the target position detection program 33E, the processor 31 starts detecting the X-axis position of the target 102, which is positioned at a predetermined location in the image data, while step S1-2 is being performed (step S1-3). More specifically, the processor 31 obtains the X-axis position of the target 102 relative to the sensor 60 or the rotation axis 60a based on the rotation information of the output shaft 63 and the sensor 60 around the rotation axis 60a detected by the detection device 63a, and the passage path information of the target 102. In the first embodiment, the passage position information is information indicating the Y-axis position at each position in the X-axis direction of the path shown by the dashed-dotted line in Figure 2. In Figure 2, the passage path of target 103 is shown by the dashed-dotted line as an example. In Figure 2, the passage path is a straight line, but the passage path may be a curved path that changes in the Y-axis direction. Even in this case, the processor 31 can obtain the X-axis position of the target 102 relative to the sensor 60 or the rotation axis 60a. The passage path may also change in the Z-axis direction.
[0032] Here, the control device 30 recognizes the position of the sensor 60 (rotation axis 60a) in the reference coordinate system 120. More specifically, the control device 30 recognizes the position of the rotation axis 60a in at least the X-axis direction and the Y-axis direction in the reference coordinate system 120. Therefore, in step S1-3, the control device 30 can obtain the position of the object 102 in the X-axis direction.
[0033] Next, the processor 31 starts tracking control based on the tracking control program 33F to make the tool 40 at the tip of the arm 20, the mounting portion 111 of the part 110 supported by the tool 40, etc., follow the target 101 (step S1-4). When the processor 31 moves the tip of the arm 20 by the operation program 33A, it corrects the position of the operation program 33A, for example, in the X-axis direction, based on the position of the target 102 in the X-axis direction detected in step S1-3. As a result, the tip of the arm 20 follows the target 101. In this embodiment, the operation program 33A is created based on the reference coordinate system 120.
[0034] As described above, instead of correcting the X-axis position of the operation program 33A, the processor 31 may perform the tracking control using a tracking coordinate system. In this case, the tracking coordinate system is a coordinate system with the reference coordinate system 120 as the reference position, and is a coordinate system that moves in the X-axis direction based on the X-axis position of the target detected in step S1-3. Furthermore, the tracking coordinate system is the coordinate system that serves as the reference for the movement of the arm 20. Therefore, when the processor 31 operates the arm 20 based on the operation program 33A, the tip of the arm 20 follows the target 101.
[0035] While follow-up control is in operation, when the processor 31 initiates fitting of the mounting portion 111 and the target 101 by the operation program 33A, the processor 31 starts force control based on the force control program 33G (step S1-5). In force control, the processor 31 changes the position and / or orientation of the tip of the arm 20 in a direction that reduces the detected value of the force sensor 41. Other known methods may also be used as force control.
[0036] Next, once the fitting process between the mounting portion 111 and the target 101 is complete (step S1-6), the processor 31 releases the support of the part 110 by the tool 40 and moves the arm 20 toward the part supply location to support the next part 110 (step S1-7).
[0037] In order to set the path information of the target 102 used in step S1-3 above, the processor performs the following processing based on the path teaching program 33B. An example of the processing performed by the processor 31 at this time will be explained below with reference to the flowchart in Figure 7. Note that if the path information of the target has already been set, the following processing is not necessary.
[0038] First, the processor 31 displays a path setting screen on the display devices 32 and 34A to indicate the path that objects 101 and 102 will take (S2-1). In the first embodiment, since the transport device 2 transports the article 100 in the X direction, the screen is for indicating the position in the Y direction that each object 101, 102, and 103 will take. In one example, the path is only taught for the object that the sensor 60 tracks as described above.
[0039] The processor 31 receives input from the user on the path setting screen and sets or changes the position of the path in the Y-axis direction based on the input (step S2-2). For example, the processor 31 displays a diagram as shown in Figure 2 on the path setting screen, and the user moves the dashed line in Figure 2 by drag-and-drop input. Alternatively, when the user inputs to select an arbitrary object, the processor 31 may move the dashed line in Figure 2 to a position that passes through the centroid of the selected object.
[0040] Alternatively, the passage path setting screen may instruct the user to attach a predetermined device or part to the tip of the arm 20. The predetermined device is a calibration jig, and the predetermined part is part 110, etc. In this case, the user places the attached tip of, for example, the calibration jig at the position of target 101, target 102, or target 103, and makes an input to set or change that position as the passage path. This input is done by pressing a predetermined button, etc. Since the control device 30 recognizes the position of the tip of the calibration jig in the reference coordinate system 120 of the arm 20, the passage path information in the Y-axis direction on the reference coordinate system 120 is set or changed by the user's input.
[0041] Furthermore, a path detection sensor may be provided above the transport device 2. The path detection sensor can be a two-dimensional sensor, a three-dimensional sensor, a three-dimensional distance sensor, etc. In this case, the path detection sensor sequentially detects the positions of objects such as 101 and 102, and the processor 31 can use the detected Y-axis position, etc., as path information. In contrast, steps S2-1 and S2-2 do not use a path detection sensor, resulting in a simpler system configuration.
[0042] In the first embodiment, the sensor 60 is rotated by the drive unit 62 around a rotation axis 60a that extends in the Z-axis direction, intersecting the direction of movement (X-axis direction) of the objects 101, 102, and 103 being moved by the transport device 2. The processor 31 controls the drive unit 62 so that at least one of the objects 101, 102, and 103 is positioned at a predetermined location within the image data (detection range). The processor 31 also controls the arm 20 using at least the rotation information of the sensor 60 around the rotation axis 60a.
[0043] In the first embodiment, the processor 31 can detect the X-axis position of the objects 101, 102, and 103 based on the rotation information detected by the detection device 63a, and use the detection result to make the tip of the arm 20 follow the object 101. The resolution of the detection device 63a, such as an encoder, may be several thousand or more per rotation. In this way, it is possible to make the resolution of the detection device 63a higher than the resolution (number of pixels) of a sensor 60 such as a two-dimensional camera in one direction at a lower cost. Furthermore, in the first embodiment, the sensor 60 rotates only around one rotation axis 60a, and is less affected by the deflection of the support member or support mechanism supporting the sensor 60, and errors with the design shape of the support member or support mechanism. Thus, this configuration is advantageous for improving the detection accuracy of the X-axis position of the objects 101, 102, and 103 detected based on the rotation position or amount of rotation detected by the detection device 63a.
[0044] Note that the processor 31 may cause the display devices 32 and 34A to display a teaching screen for associating the rotation information detected by the detection device 63a with the positions of the objects 101, 102, and 103 in the X-axis direction. In this case, for example, the object 103 is placed at several positions on the transport device 2, and at each position, the processor 31 controls the drive unit 62 so that the object 103 is arranged at the predetermined position in the image data. Then, the user uses the input device 34 or the like to input the actual position of the object 103 at each position. The user may input the actual position of the object 103 at each position, for example, by bringing the distal end portion of the robot into contact with the corresponding position of the article 100.
[0045] Note that in the first embodiment, the positions of the objects 101, 102, and 103 in the image data may vary in the image Y direction orthogonal to the image X direction. In this case, it is also possible that the processor 31 performs follow-up control in the Z-axis direction in addition to follow-up control in the X-axis direction for the distal end portion in step S1-4, using the variation amount as well.
[0046] Further, in the first embodiment, the passage route of each of the objects 101, 102, and 103 may be determined in advance. For example, in the first embodiment, the sensor 60 is installed to face the horizontal direction, but the sensor 60 may be rotatably supported around a rotation axis 60a extending in the horizontal direction, and configured to face vertically downward. In this case, if the height position of the object 101 conveyed by the conveyor device 2 does not change, the height position can be used instead of the passage route information. That is, as long as the processor 31 recognizes the fixed height position, the processor 31 can obtain the position of the object 101 in the X-axis direction based on the rotation information detected by the detection device 63a. Also in this case, the same operational effects as described above are achieved.
[0047] As described above, there are many cases where the passage routes of the respective objects 101, 102, and 103 are not completely determined. In the first embodiment, the user can set the passage route of the object as described above. This configuration is useful for detecting the position of a moving object in the moving direction with a simple configuration for various aspects of the transport device 2.
[0048] Further, there may be cases where the processor 31 cannot cause the rotation direction of the sensor 60 to completely follow the target 102 or the like. Such a situation may occur when the conveyance speed of the conveyance device 2 changes unintentionally or the like. In this case, the processor 31 can obtain the position of the target 102 in the X-axis direction based on the position of the target 102 in the image X direction in the image data and the rotation information detected by the detection device 63a. The position of the target 102 in the image X direction may be the number of separated pixels, distance, or the like in the image X direction between the center of the image data and the centroid position of the target 102. Compared with the case where the target 102 is arranged at the center of the image data, the rotation direction of the sensor 60 is deviated by an angle α corresponding to the number of separated pixels or the distance. For example, for each rotational position of the sensor 60, data in which the number of separated pixels is associated with the angle α, and data in which the angle α is associated with the actual deviation amount of the target in the X-axis direction can be stored in the storage unit 33.
[0049] Thereby, even when the target 102 is not arranged at the center of the image data, the processor 31 can obtain the position of the target 102 in the X-axis direction. Even in this case, the sensor 60 only rotates around one rotation axis 60a, and the influence of deflection of the support member or support mechanism that supports the sensor 60, errors from the designed shape of the support member or support mechanism, and the like is small. Even if the conveyance speed of the conveyance device 2 changes relatively abruptly, deflection or the like is unlikely to occur in the support portion of the sensor 60 that follows the change. This configuration is useful for obtaining an accurate position of the target in the X-axis direction.
[0050] As described above, in a configuration where the sensor 60 does not perfectly follow the transport speed of the transport device 2, the transport device 2 may also transport the article 100 in a circular direction centered on the rotation axis 60a. In this case, the distance from the rotation axis 60a to the object becomes approximately constant, and the number of separated pixels in the image data and the actual amount of deviation also become constant or approximately constant regardless of the rotation position of the sensor 60. In this case, the processor 31 obtains the position of the object 101, object 102, etc. in the transport direction based at least on the rotation information of the sensor 60 around the rotation axis 60a, and uses this position to make the arm 20 follow the object 101, etc. Therefore, it becomes possible to control the position of the tip of the arm 20 to match the position of the moving object while simplifying the device configuration. The processor 31 may also use the number of separated pixels to obtain the position of the object 101, object 102, etc. in the transport direction.
[0051] The control device 30 and robot system 1 of the second embodiment will be described below with reference to Figure 8. In the second embodiment, the sensor 60 of the first embodiment is changed to a sensor 60' which is a three-dimensional camera, a three-dimensional distance sensor, etc., and the processing of each program is changed accordingly as described below. In the second embodiment, components that are not specifically described have the same functions as in the first embodiment and are denoted by the same reference numerals as in the first embodiment. Furthermore, in the second embodiment, it is possible to make the same modifications as in the first embodiment, within the limits permitted in the second embodiment.
[0052] The control device 30 of the second embodiment has the same configuration as the first embodiment. The storage unit 33 stores at least an object detection program 33C, a sensor rotation position control program 33D, an object position detection program 33E, a tracking control program 33F, and a force control program 33G. An example of processing performed by the processor 31 based on these programs will be described below with reference to the flowchart in Figure 8.
[0053] First, the processor 31 starts detecting the positions of targets 101, 102, and 103 in the 3D image data (detection range) or point cloud data (detection range) that are sequentially received from the sensor 60', based on the target detection program 33C (step S3-1). In this embodiment, the processor 31 receives point cloud data from the sensor 60'. For example, the processor 31 starts detecting the position of target 102 in the point cloud data. In this embodiment, each point in the point cloud data has distance data with respect to the rotation axis 60a of the sensor 60 or distance-related data relating to that distance data.
[0054] Next, the processor 31 starts controlling the rotational position of the drive unit 62 based on the sensor rotational position control program 33D, so that the detection position of, for example, the target 102 is always positioned at a predetermined location in the sequentially received point cloud data (step S3-2). The predetermined location in the point cloud data is, for example, the center of the field of view of the sensor 60'. As a result of step S3-2, the rotational direction of the sensor 60' follows the movement of the target in the X-axis direction.
[0055] In the second embodiment, the object 102 is always positioned at a predetermined location in the X-direction of the field of view of the point cloud data (field of view or horizontal direction of the image), and the X-direction of the field of view corresponds to the X-axis direction of the reference coordinate system 120 (Figures 1 and 2), which is the coordinate system of the arm 20. At each control cycle, the processor 31 sends a control command and / or current to the drive unit 62 to direct the position of the object 102 towards the predetermined location, according to the position of the object 102 in the image data at that time.
[0056] Next, while step S3-2 is being performed, the processor 31 performs processing based on the target position detection program 33E. That is, the processor 31 starts detecting the X-axis position of the target 102, which is positioned at a predetermined location in the image data, using the distance-related data of the point corresponding to the target 102 in the point cloud data (step S3-3). More specifically, the processor 31 calculates the X-axis position of the target 102 with respect to the sensor 60 or the rotation axis 60a based on the rotation information of the output shaft 63 and the sensor 60 around the rotation axis 60a detected by the detection device 63a, and the distance-related data. In this way, in the second embodiment, it is possible to detect the X-axis position of the target 102 without using the passage path information of the first embodiment.
[0057] Next, the processor 31 starts tracking control to make the tool 40 at the tip of the arm 20, the mounting portion 111 of the component 110 supported by the tool 40, etc., follow the target 101, similar to the first embodiment (step S3-4). In step S3-3, the processor 31 can also calculate the position of the sensor 60 and the Y-axis and Z-axis positions of the target 102 relative to the rotation axis 60a based on the rotation information and the distance data. In this case, the processor 31 can make the mounting portion 111 of the component 110 etc. follow the target 101 in the Y-axis and Z-axis directions as well.
[0058] While follow-up control is being performed, when the processor 31 starts fitting the mounting portion 111 and the target 101 according to the operation program, the processor 31 starts force control in the same manner as in the first embodiment (step S3-5).
[0059] Next, once the fitting process between the mounting portion 111 and the target 101 is complete (step S3-6), the processor 31 releases the support of the part 110 by the tool 40 and moves the arm 20 toward the part supply location to support the next part 110 (step S3-7).
[0060] In the second embodiment as well, the processor 31 can detect the X-axis position of the objects 101, 102, and 103 based on the rotation information detected by the detection device 63a, and use the detection result to make the tip of the arm 20 follow the object 101. In the second embodiment as well as in the first embodiment, it is possible to make the resolution of the detection device 63a higher than the resolution (number of pixels) in one direction of the sensor 60' such as a three-dimensional camera or three-dimensional distance sensor at a lower cost. Furthermore, this configuration is advantageous for improving the detection accuracy of the X-axis position of the objects 101, 102, and 103, as in the first embodiment.
[0061] Furthermore, in the second embodiment, the processor 31 can obtain the position of the object 102 in the X-axis direction even if the object 102 is not positioned at the center of the point cloud data. Each point in the point cloud data has distance data. The three-dimensional distance sensor emits detection light in various directions using a prism or the like, and obtains each point using the reflected waves of each detection light. Therefore, each point in the point cloud data also has angular information about how much it is offset from the optical axis of the sensor 60'. For this reason, in the second embodiment as in the first embodiment, the position of the object 102 in the X-axis direction can be obtained even if the object 102 is not positioned at the center of the field of view of the sensor 60'. Even in this case, the sensor 60 only rotates around a single rotation axis 60a, and the effects of deflection of the support member supporting the sensor 60 and errors from the design shape of the support member are minimal. Even if the transport speed of the transport device 2 changes relatively rapidly, deflection of the support part of the sensor 60 that follows it is unlikely to occur. This configuration is useful for obtaining the accurate position of the object in the X-axis direction.
[0062] The control device 30 and robot system 1 of the third embodiment will be described below with reference to Figures 11 and 12. In the third embodiment, the arm 20 performs a predetermined operation on the object to be worked on 201 in the article 200 instead of the article 100. The article 200 is a part having a top surface, for example, a rectangular parallelepiped, a cylinder, etc., and the object to be worked on 201, such as a hole, is provided on the top surface of the article 200. In the third embodiment, the belt 2b is driven by the motor 2a of the transport device 2, and the article 200 is transported together with the belt 2b. In the third embodiment, although the object to be worked on is different as described above, the processor 31 makes the tool 40 at the tip of the arm 20 follow the object 201 in the same process as in the first embodiment. In the third embodiment, components that are not specifically described have the same function as in the first embodiment and are denoted by the same reference numerals as in the first embodiment. Furthermore, in the third embodiment as well, it is possible to make the same modifications as in the first embodiment, within the limits permitted in the third embodiment.
[0063] The control device 30 of the third embodiment also has the same configuration as the first embodiment. An example of the processing performed by the processor 31 will be described below with reference to the flowchart in Figure 12.
[0064] First, the processor 31 starts detecting the positions of objects 201 and 202 in the image data (detection range) sequentially received from the sensor 60, similar to the first embodiment (step S5-1). In the third embodiment, object 202 is the top surface of article 200. In the third embodiment as well, object 202 is a detection target used to detect the position and / or orientation of the work target 201, and object 202 moves together with the work target 201 and its relative position to object 201 does not change.
[0065] Next, the processor 31 starts controlling the rotational position of the drive unit 62 so that the detection position of the target 202 is always positioned at a predetermined position in the sequentially received image data, similar to the first embodiment (step S5-2). Instead of target 202, target 201 may always be positioned at the predetermined position. In the third embodiment, target 202 is always positioned at a predetermined position in the image X direction in the image data, and the image X direction corresponds to the X-axis direction of the reference coordinate system 120 of the arm 20. In step S5-2, the rotational direction of the sensor 60 follows the movement of target 102 in the X-axis direction, and during this tracking, the sensor 60 rotates from the position shown by the dashed line to the position shown by the solid line in the example of Figure 11.
[0066] Next, the processor 31, in the same manner as in the first embodiment, starts detecting the X-axis position of the object 202, which is positioned at a predetermined location in the image data, while step S5-2 is being performed (step S5-3). In other words, as in the first embodiment, the X-axis position of the object 202 is detected sequentially while the tracking is being performed. Note that in the third embodiment, if the height position that the object 202 passes through is determined, the user does not need to set the passage path information.
[0067] Next, while the sensor 60 is tracking, the processor 31 starts tracking control to make the tool 40 at the tip of the arm 20 follow the target 101, similar to the first embodiment (step S5-4).
[0068] While follow-up control is in operation, the processor 31 initiates the predetermined operation on the target 201 using the tool 40 via the operation program 33A (step S5-5). In the third embodiment, force control may be performed in the same manner as in the first embodiment. Also, in the first and second embodiments, force control may not be performed.
[0069] Next, when the predetermined operation on the target 201 by the tool 40 is completed (step S5-6), the processor 31 moves the arm 20 to move the tip to a waiting position for operation on the next item 200 (step S5-7). At this time, if the processor 31 has not received a signal indicating the completion of the operation (step S5-8), the processor 31 controls the drive unit 62 to rotate the sensor 60 to a rotation position for detecting the next item 200 (step S5-9). In the example in Figure 11, the sensor 60 rotates from the position of the solid line to the position of the dashed line. Then, the process returns to step S5-1 and the detection of the position of the new target begins.
[0070] In the third embodiment, the sensor 60' of the second embodiment may be used instead of the sensor 60. In this case, the processing of the third embodiment can be appropriately modified to follow the difference in processing between the second and first embodiments, so that the tip of the arm 20 can follow the target 201.
[0071] Furthermore, in the third embodiment, the object 201 can be various, and the tool 40 can perform various tasks. If the tool 40 is a welding tool, processing tool, assembly tool, cleaning tool, coating tool, injection tool, etc., the tool 40 will perform tasks on the object 201 using these tools. Also, if the tool 40 is capable of supporting an article 200 such as a hand, the entire article 200 becomes the object of work, and the tool 40 will perform tasks such as detecting the object 202 while gripping the article 200. The type of tool 40 and the type of work are not limited to these. Also, depending on the conditions, the article 200 may temporarily stop when the tip of the arm 20 performs work on the work target of the article 200. In this case, the tip of the arm 20 will perform the predetermined task while stopping at the position of the work target.
[0072] In the first, second, and third embodiments, the processor 31 uses the position of the target 102 in the X-axis direction, obtained based on the rotation information of the sensor 60, to perform tracking control, causing the tip of the arm 20 to follow the target 101. Alternatively, the processor 31 can use the position of the target 102 in the X-axis direction, obtained based on the rotation information of the sensor 60, to perform approach control, etc., bringing the tip of the arm 20 closer to the target 101. Furthermore, the processor 31 can use the position of the target 102 in the X-axis direction, obtained based on the rotation information of the sensor 60, to perform other controls besides approach control. In this case, the processor 31 can also use the output of another sensor attached to the tip of the arm 20 to cause the tip of the arm 20 to follow the target 101.
[0073] In addition, in the first, second, and third embodiments, the processor 31 can adjust the position of the tip of the arm 20 in the X-axis direction so as to synchronize with the rotation information, thereby causing the tip to follow the target 101. For example, in the first embodiment, a path for the target 102 is set, and the position of the target 102 in the X-axis direction corresponding to each rotation angle position of the sensor 60 is determined. In this case, step S1-3 can be omitted, and in step S1-4, the processor 31 can perform position correction in the X-axis direction of the operation program 33A or move the tracking coordinate system according to the rotation information.
[0074] In the first, second, and third embodiments, a sensor control device 70 may be provided on the sensor 60, as shown in Figure 9. The sensor control device 70 includes a processor 71 such as a CPU, a microcomputer processor, or a PLC processor, and a known display device 72 such as a liquid crystal display device. The sensor control device 70 also includes a memory unit 73 having non-volatile storage, RAM, etc., and examples of non-volatile storage include hard disks, flash memory, ROM, etc. In some cases, the CPU, microcomputer, PLC, or storage device of an external computer may function as part of the memory unit 73. The sensor control device 70 also includes an input device 74 including one of a computer such as a keyboard, touch panel, control panel, or tablet computer, and a transmitting / receiving unit 75 for transmitting and receiving signals. The sensor control device 70 also includes a controller 77 connected to the drive unit 62.
[0075] The memory unit 73 stores a path teaching program 33B, an object detection program 33C, a sensor rotation position control program 33D, and an object position detection program 33E, similar to those in the first embodiment. An example of processing performed by the processor 71 based on these programs will be described below with reference to the flowchart in Figure 9. The following example uses a two-dimensional camera, similar to the first embodiment.
[0076] First, the processor 71 starts detecting the positions of targets 101, 102, and 103 within the image data (detection range) sequentially received from the sensor 60, based on the target detection program 33C (step S4-1).
[0077] Next, the processor 71 starts controlling the rotational position of the drive unit 62 based on the sensor rotational position control program 33D so that, for example, the detection position of the target 102 is always positioned at a predetermined location in the sequentially received image data (step S4-2). In step S4-2, the rotational direction of the sensor 60 follows the movement of the target 102 in the X-axis direction.
[0078] Next, based on the target position detection program 33E, the processor 71 starts detecting the X-axis position of the target 102, which is positioned at a predetermined location in the image data, while step S1-2 is being performed (step S4-3). More specifically, the processor 71 obtains the X-axis position of the target 102 relative to the sensor 60 or the rotation axis 60a based on the rotation information of the output shaft 63 and the sensor 60 around the rotation axis 60a detected by the detection device 63a, and the path information of the target 102.
[0079] Next, the processor 71 transmits the position information of the target 102 in the X-axis direction (information based on rotation information) obtained sequentially in step S4-3 to the control device 30 of the robot 10 (step S4-4). Upon receiving the position information, the control device 30 can make the tip of the arm 20 follow the target 101 based on the position information.
[0080] Similar to the first embodiment, the processor 71 can display a route setting screen on the display devices 72 and 74A based on the route teaching program 33B, and the user can set the route in the same manner as in the first embodiment.
[0081] Alternatively, the processor 71 may, without performing step S4-3, sequentially transmit information about the rotation of the sensor 60 in step S4-4 when the rotation direction of the sensor 60 follows the movement of the object 102 in the X-axis direction.
[0082] In each of the above embodiments, if the robot 10 is a collaborative robot, a person will be present near the robot 10. In each of the above embodiments, the sensors 60 and 60' simply rotate around the rotation axis 60a. This configuration is useful for reducing the possibility of the sensors 60 and 60' hitting a person or other object, and for reducing the impact force.
[0083] Furthermore, in each of the above embodiments, the position of the rotation axis 60a may move slightly in the X-axis direction and / or the Z-axis direction while the sensor 60 rotates around the rotation axis 60a. Even in this case, as described above, it is possible to detect the position of the object using the rotation information of the sensor 60.
[0084] Furthermore, in each of the above embodiments, another robot may move the object instead of the transport device 2. In this case as well, the same effects as described above can be achieved. Moreover, if the object is an automobile or the like, the vehicle body on which the object on which the predetermined work is performed may be moved by its engine, wheels, etc. In these cases, the other robot, engine, wheels, etc. function as means of movement. Alternatively, instead of the transport device 2, the object may be moved by a chute on which it slides, rolls, or falls due to gravity. In this case, it is also possible to vibrate the inclined chute with a vibration device to facilitate the movement of the object on the chute. In these cases, the chute, vibration device, etc. function as means of movement. The means of movement are not limited to these.
[0085] Furthermore, in each of the above embodiments,
[0086] While embodiments of this disclosure have been described in detail, this disclosure is not limited to the individual embodiments described above. These embodiments can be added, replaced, modified, partially deleted, etc., in any way that does not depart from the gist of this disclosure or from the idea and intent of this disclosure derived from the claims and their equivalents. For example, in the embodiments described above, some configurations or steps can be omitted or added depending on the circumstances, without being bound by the above examples. The same applies when numerical values or mathematical formulas are used in the description of the embodiments above.
[0087] [Note 1] A control device capable of controlling at least a drive unit that rotates a sensor about a rotation axis extending in a direction intersecting the direction of movement of a moving object, comprising at least one processor, wherein the at least one processor is configured to perform: a sensor rotation process that controls the drive unit so that an object on the moving object is positioned within the detection range of the sensor; and an arm control process that controls the arm of a robot using at least information of the rotation of the sensor about the rotation axis. [Note 2] The control device according to Note 1, wherein in the arm control process, the at least one processor causes the arm to follow the object using at least the rotation information and the detection position of the object within the detection range of the sensor. [Note 3] The control device according to Note 1 or 2, wherein the at least one processor is configured to cause the rotation direction of the sensor to follow the moving object by controlling the drive unit based on the detection position of the object within the detection range of the sensor. [Note 4] The control device according to Note 1 or 2, wherein at least one processor controls the arm in the arm control process using information on the path of the moving object and information on its rotation. [Note 5] The control device according to Note 4, wherein at least one processor is configured to display a setting screen on a display device, the setting screen being for the user to set the path of the object. [Note 6] A robot system comprising: a drive unit that rotates a sensor about a rotation axis extending in a direction intersecting the direction of movement of a moving article; a robot arm; and a control device that controls the arm, wherein at least one processor of the control device is configured to perform: a sensor rotation process that controls the drive unit based on the output of the sensor so that an object in the moving article is placed within the detection range of the sensor; and an arm control process that controls the arm of the robot using at least information on the rotation of the sensor about the rotation axis.[Note 7] The robot system according to Note 6, wherein the at least one processor, in the arm control processing, uses at least the rotation information and the detected position of the object within the detection range of the sensor to cause the arm to follow the object. [Note 8] The robot system according to Note 6 or 7, wherein the at least one processor is configured to cause the rotation direction of the sensor to follow the moving object by controlling the drive unit based on the detected position of the object within the detection range of the sensor. [Note 9] The robot system according to Note 6 or 7, wherein the at least one processor, in the arm control processing, controls the arm using the passage path information of the moving object and the rotation information. [Note 10] A sensor device comprising: a sensor; a drive unit for rotating the sensor about a rotation axis extending in a direction intersecting the direction of movement of a moving object; and a sensor control device capable of controlling the drive unit, wherein the sensor control device is configured to perform: a sensor rotation process for controlling the drive unit so that an object on the moving object is positioned at a predetermined location within the detection range of the sensor; and a rotation information transmission process for transmitting information about the rotation of the sensor about the rotation axis to the robot's control device in order to cause the robot's arm to follow the moving object.
[0088] 1: Robot system 2: Transport device 10: Robot 20: Arm 30: Control device 31: Processor 32: Display device 33: Memory unit 33A: Operation program 33B: Path teaching program 33C: Target detection program 33D: Sensor rotation position control program 33E: Target position detection program 33F: Follow control program 33G: Force control program 34: Input device 34A: Display device 35: Transmit / receive unit 36: Servo controller 37: Controller 40: Tool 41: Force sensor 60, 60': Sensor 60a: Rotation axis 61: Support member 62: Drive unit 63: Output shaft 63a: Detection device 70: Sensor control device 71: Processor 73: Memory unit 100: Item 101 :Target 102 :Target 103 :Target 110 :Part 111 :Mounting part 120 :Reference coordinate system 200 :Item 201 :Target 202 :Target
Claims
1. A control device capable of controlling at least a drive unit that rotates a sensor around a rotation axis extending in a direction intersecting the direction of movement of a moving object, comprising at least one processor, wherein the at least one processor is configured to perform: a sensor rotation process that controls the drive unit so that an object on the moving object is positioned within the detection range of the sensor; and an arm control process that controls the arm of a robot using at least information on the rotation of the sensor around the rotation axis.
2. The control device according to claim 1, wherein the at least one processor, in the arm control processing, uses at least the rotation information and the detected position of the object within the detection range of the sensor to cause the arm to follow the object.
3. The control device according to claim 1 or 2, wherein the at least one processor is configured to control the drive unit based on the detection position of the object within the detection range of the sensor, thereby causing the rotation direction of the sensor to follow the moving object.
4. The control device according to claim 1 or 2, wherein the at least one processor controls the arm in the arm control process using information about the path of the moving object and information about its rotation.
5. The control device according to claim 4, wherein at least one processor is configured to display a settings screen on a display device, the settings screen being for a user to set the route information.
6. A robot system comprising: a drive unit that rotates a sensor about a rotation axis extending in a direction intersecting the direction of movement of a moving object; a robot arm; and a control device that controls the arm, wherein at least one processor of the control device is configured to perform: a sensor rotation process that controls the drive unit based on the output of the sensor so that an object in the moving object is placed within the detection range of the sensor; and an arm control process that controls the arm of the robot using at least information of the rotation of the sensor about the rotation axis.
7. The robot system according to claim 6, wherein the at least one processor, in the arm control processing, uses at least the rotation information and the detected position of the object within the detection range of the sensor to cause the arm to follow the object.
8. The robot system according to claim 6 or 7, wherein the at least one processor is configured to control the drive unit based on the detection position of the object within the detection range of the sensor, thereby causing the rotation direction of the sensor to follow the moving object.
9. The robot system according to claim 6 or 7, wherein the at least one processor controls the arm in the arm control process using information about the path of the moving object and information about its rotation.
10. A sensor device comprising: a sensor; a drive unit for rotating the sensor about a rotation axis extending in a direction intersecting the direction of movement of a moving object; and a sensor control device capable of controlling the drive unit, wherein the sensor control device is configured to perform: a sensor rotation process for controlling the drive unit so that an object on the moving object is positioned at a predetermined location within the detection range of the sensor; and a rotation information transmission process for transmitting information about the rotation of the sensor about the rotation axis to the robot's control device in order to cause the robot's arm to follow the moving object.