Follow-up robot

Abstract

A follower robot (10) comprises: an arm (10a); one or more visual sensors (50) provided on the arm; a feature quantity storage unit storing a first feature quantity related to at least the position and posture of a follower object as target data for causing the visual sensor to follow the follower object (102); a feature quantity detection unit detecting a second feature quantity related to at least the current position and posture of the follower object using an image obtained by the visual sensor; a movement quantity calculation unit calculating a movement instruction of the arm based on a difference between the second feature quantity and the first feature quantity and adjusting the movement instruction using at least feedforward control; a movement instruction unit moving the arm according to the movement instruction; and an input value storage unit storing a signal obtained when a specific action of the follower object starts in correspondence with an input value of the feedforward control, the input value of the feedforward control being used to cause the arm to follow the trajectory of the follower object in the specific action.

Description

Technical Field

[0001] The present invention relates to a follower robot. Background Art

[0002] Conventionally, there is known a production line including a robot, a transport device that transports articles, a track provided along the transport device, and a moving device that moves the robot along the track (for example, see Patent Document 1).

[0003] In this production line, when utilizing the conveying device to carry the article, the robot performs defect inspection and grinding of the article. In addition, when performing defect inspection and grinding, the moving device makes the robot move along the track at the same speed as the conveying device to carry the article.

[0004] Furthermore, there is known a technique for accurately aligning the position and posture of the front end portion of a robot to a fixed target position (for example, see Patent Document 2).

[0005] Prior art literature

[0006] Patent Literature

[0007] Patent Document 1: Japanese Patent Application Laid-Open No. 08-72764

[0008] Patent Document 2: Japanese Patent Application Publication No. 2017-170599 Summary of the Invention

[0009] Problems to be solved by the invention

[0010] In this production line, only defect inspection and grinding are performed. In contrast, when performing operations where robots and objects may interfere, for example, measures are needed to prevent damage to the robots, handling devices, and objects. However, this damage prevention is difficult to achieve because objects moved by handling devices may vibrate and behave in unpredictable ways.

[0011] Therefore, it is desirable to enable the robot's tool to accurately follow the object.

[0012] Solutions for solving problems

[0013] One embodiment of the present invention is a follower robot, the follower robot comprising: a movable arm; at least one visual sensor provided on the arm; a feature quantity storage unit, which stores a first feature quantity related to at least the position and posture of a follower object as target data for causing the visual sensor provided on the arm to follow the follower object; a feature quantity detection unit, which detects a second feature quantity related to at least the current position and posture of the follower object using an image obtained by the visual sensor; a movement quantity calculation unit, which calculates a movement instruction for the arm based on a difference between the second feature quantity and the first feature quantity, and adjusts the movement instruction using at least feedforward control; a movement instruction unit, which moves the arm based on the movement instruction; and an input value storage unit, which stores the feature quantity of the follower object in the input value storage unit. The signal acquired at the start of a specific action is stored in correspondence with the input value of the feedforward control, and the input value of the feedforward control is used to make the arm follow the trajectory of the follow-up object in the specific action. The movement amount calculation unit and the movement instruction unit repeat the calculation of the movement instruction and the movement of the arm based on the movement instruction while making the visual sensor follow the follow-up object. The movement instruction is used to reduce or eliminate the difference between at least the position and posture of the follow-up object as the second feature quantity and at least the position and posture of the follow-up object as the first feature quantity. The movement amount calculation unit uses the feedforward control based on the input value, and the input value is stored in the input value storage unit in correspondence with the signal acquired at the start of the specific action.

[0014] According to the above configuration, during actual work using the robot, the robot can follow the moving object with high accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 It is a schematic diagram of the structure of a working robot system including a follower robot according to one embodiment of the present invention.

[0016] Figure 2 yes Figure 1 Block diagram of the control device of the working robot system.

[0017] Figure 3 is Figure 1 An example of image data captured by the vision sensor of a working robot system.

[0018] Figure 4 Yes Figure 1 Flowchart of the operation of the control unit of the working robot system.

[0019] Figure 5 Yes Figure 1A schematic perspective view of an example of generation of an input value for feedforward control in a specific motion of a following object of a working robot system.

[0020] Figure 6 It is an explanation Figure 5 Flowchart of the input value generation process.

[0021] Figure 7 It means using Figure 6 Schematic perspective view of an example of the operation of a robot that performs feedforward control on input values ​​generated by the input value generation process.

[0022] Figure 8 Is explained through Figure 6 Flowchart of a method for fine-tuning an input value generated by an input value generation process.

[0023] Figure 9 It is a schematic diagram of the structure of a working robot system according to a modified example of this embodiment.

[0024] Figure 10 This is a flowchart illustrating a time constant generation process according to a modification of the present embodiment.

[0025] Figure 11 Is explained through Figure 10 Flowchart of a method for fine-tuning a calculated time constant during a time constant generation process.

[0026] Figure 12 This is a block diagram of a management system including a control device according to this embodiment.

[0027] Figure 13 This is a block diagram of a system including the control device according to this embodiment.

[0028] Description of the reference numerals is as follows.

[0029] 10: Robot (follow-up robot)

[0030] 10a: Arm

[0031] 23: Storage unit (feature quantity storage unit, input value storage unit)

[0032] 23c: Follow-up control program (movement instruction unit)

[0033] 23e: Feature Quantity Detection Program (Feature Quantity Detection Unit)

[0034] 23f: Movement amount calculation program (movement amount calculation unit)

[0035] 50: Vision Sensor

[0036] 102: Follower Object DETAILED DESCRIPTION

[0037] Hereinafter, a working robot system 1 according to an embodiment of the present invention will be described with reference to the drawings.

[0038] like Figure 1 As shown, the working robot system 1 of this embodiment includes: a transport device 2 that transports an object 100 as a work target; and a robot (follower robot) 10 that performs a predetermined work on a work target portion 101 of the object 100 transported by the transport device 2. Furthermore, the working robot system 1 includes: a control device 20 that controls the robot 10; a detection device 40 as a detection unit; and a visual sensor 50 attached to the robot 10.

[0039] The detection device 40 detects whether the article 100 has been transported to the predetermined location. Any device having such a function can be used as the detection device 40. In this embodiment, the detection device 40 is a photoelectric sensor, but a visual sensor 50 can also be used to detect whether the article 100 has been transported to the predetermined location.

[0040] The article 100 is not limited to a specific type. In this embodiment, the article 100 is a vehicle body as an example. The conveying device 2 is used to convey the article 100 by driving several of the plurality of rollers 3 using a motor 2a. In this embodiment, the conveying device 2 is directed toward Figure 1 Carrying items 100 on the right side.

[0041] The work target portion 101 is the portion of the object 100 on which the robot 10 performs a predetermined operation. In this embodiment, as the predetermined operation, the hand (tool) 30 of the robot 10 lifts the component 110, and the robot 10 attaches the mounting portion 111 of the component 110 to the work target portion 101. Thus, for example, a shaft 111a extending downward from the mounting portion 111 of the component 110 engages with a hole 101a provided in the work target portion 101 of the object 100.

[0042] While the article 100 is being moved by the transport device 2 , the robot 10 attaches the mounting portion 111 of the component 110 to the work target portion 101 .

[0043] The robot 10 is not limited to a specific type, but the robot 10 of this embodiment includes a plurality of servo motors 11 (see FIG. 1 ) that respectively drive a plurality of movable arms 10a. Figure 2 Each servo motor 11 has a working position detection device for detecting its working position. The working position detection device is, for example, an encoder. The detection value of the working position detection device is sent to the control device 20.

[0044] The hand 30 is attached to the front end of the arm 10a. The hand 30 of this embodiment supports the component 110 by gripping with a plurality of claws, but a hand that supports the component 110 by using magnetic force, air attraction, or the like may also be used.

[0045] The hand 30 includes a servo motor 31 for driving the claw (see Figure 2 The servo motor 31 has a working position detection device for detecting its working position. The working position detection device is, for example, an encoder. The detection value of the working position detection device is sent to the control device 20.

[0046] As the servo motors 11 and 31 , various servo motors such as a rotary motor and a linear motor can be used.

[0047] A force sensor 32 is mounted on the front end of the robot 10. The force sensor 32 measures, for example, Figure 3 The X-axis direction, Y-axis direction, and Z-axis direction, and the forces or moments around the X-axis, Y-axis, and Z-axis are shown.

[0048] The force sensor 32 only needs to be able to detect the direction and magnitude of the force applied to the hand 30 or the component 110 grasped by the hand 30. Therefore, in this embodiment, the force sensor 32 is provided between the robot 10 and the hand 30, but the force sensor 32 may also be provided inside the hand 30.

[0049] A visual sensor 50 is mounted on the front end of the arm 10a. In one example, the visual sensor 50 is mounted on the wrist flange of the robot 10 using a frame 50a. In this embodiment, the visual sensor 50 is a two-dimensional camera. The visual sensor 50 of this embodiment sequentially acquires information such as the following object 102 whose position and posture relative to the work object 101 do not change, so that the following object 102 enters a predetermined range of the field of view. Figure 3 The image data shown.

[0050] In this embodiment, the follower object 102 is Figure 3 Although the upper surface portion is indicated by hatching, other portions whose position and posture relative to the work target portion 101 do not change may be used.

[0051] The vision sensor 50 may also be attached to a tool such as the hand 30. In addition, the vision sensor 50 may also be attached to another portion of the robot 10 whose position and posture do not change with respect to the tool such as the hand 30.

[0052] The visual sensor 50 sequentially transmits image data to the control device 20. The image data is used to determine the position and posture of the moving object 102. A detector other than the control device 20 may also be used to process the image data, and the position and posture of the moving object 102 may be determined based on the processed data.

[0053] The moving object 102 is a portion of the article 100 having a predetermined shape, a portion provided with a predetermined mark, etc. In these cases, the image data is data that can determine the position and posture of the above-mentioned portion on the image.

[0054] In the image-based example, if the following object 102 is positioned at the target position, posture, and size in the image data (detection range) relative to the vision sensor 50, the position and posture of the hand 30 attached to the arm 10a will be the position and posture required for the predetermined operation on the object 100. In the position-based example, the position and posture of the hand 30 attached to the arm 10a are calibrated to correspond to the position and posture of the vision sensor 50. In this case, the control device 20 can recognize the position and posture of the following object 102 in the coordinate system of the robot 10 based on the image data, and the control device 20 can move the hand 30 attached to the arm 10a to the position and posture required for the predetermined operation.

[0055] In the present embodiment, a state is established in which the shaft 111 a of the mounting portion 111 of the component 110 can be fitted into the hole 101 a provided in the work target portion 101 of the article 100 .

[0056] The object 100 may sometimes wobble on the transport device 2. For example, if the plurality of rollers 3 of the transport device 2 are not arranged on a completely flat surface, the object 100 may wobble. If the object 100 is large, even slight wobbling of the lower end of the object 100 may cause significant wobbling of the workpiece 101. Therefore, it is important to adjust the posture of the hand 30 provided on the arm 10a.

[0057] In the image-based example, the position, posture, and size change of the following object 102 in the image data of the vision sensor 50 are previously associated with the position and posture change of the coordinate system of the robot 10 in the control device 20 .

[0058] like Figure 2As shown, the control device 20 includes a control unit 21 having a CPU, RAM, and other components; a display device 22; and a storage unit (feature value storage unit, input value storage unit) 23 having nonvolatile memory, ROM, and other components. Furthermore, the control device 20 includes a plurality of servo controllers 24, each corresponding to the servo motors 11 of the robot 10; a servo controller 25 corresponding to the servo motors 31 of the hand 30; and an input unit 26 connected to the control device 20. In one example, the input unit 26 is an input device such as an operating panel carried by the operator. The input unit 26 may also wirelessly communicate with the control device 20.

[0059] The storage unit 23 stores a system program 23a, which carries out the basic functions of the control device 20. The storage unit 23 also stores an action program 23b. Furthermore, the storage unit 23 stores a follow-up control program (movement instruction unit) 23c, a force control program 23d, a feature quantity detection program (feature quantity detection unit) 23e, and a movement amount calculation program (movement amount calculation unit) 23f.

[0060] Furthermore, the storage unit (input value storage unit) 23 stores, in a corresponding manner, a signal acquired when a specific movement of the following object 102 starts and an input value for feedforward control.

[0061] The specific motions of the follower object 102 are assumed to be unstable motions during the operation of the conveying device 2 for conveying the article 100, such as stops due to other work processes, restarts from stops, or emergency stops. When these specific motions begin, the control device 20 acquires a signal identifying each specific motion.

[0062] The control unit 21 sends a movement command for performing a predetermined operation on the object 100 to each servo controller 24, 25 according to these programs. As a result, the robot 10 and the hand 30 perform the predetermined operation on the object 100. Figure 4 The operation of the control unit 21 at this time is described in the flowchart of FIG.

[0063] First, when the detection device 40 detects the object 100 (step S1-1), the control unit 21 begins sending pre-operation movement instructions to the robot 10 and hand 30 according to the action program 23b (step S1-2). As a result, the robot 10 moves the shaft 111a of the component 110 held by the hand 30 close to the hole 101a of the work target portion 101. At this time, the control unit 21 may also use data such as the transport speed of the transport device 2 and the position of the work target portion 101 within the object 100. Furthermore, after step S1-4 described below, the shaft 111a of the component 110 is engaged with the hole 101a of the object 100 according to the action program 23b. Furthermore, in step S1-1, the visual sensor 50 may be used instead of the detection device 40 to detect the object 100.

[0064] Through the control of the robot 10 in step S1-2, the component 110 reaches the preparation position and posture for the predetermined operation (fitting). Consequently, if the following object 102 is present within the visual sensor 50's field of view (detection range) or within a predetermined range of the field of view (step S1-3), the control unit 21 begins control based on the following control program 23c, the feature quantity detection program 23e, and the movement amount calculation program 23f (step S1-4). In step S1-4, for example, the following control is performed. Furthermore, in the following control, at least the position and posture of the following object 102 are detected based on the image data from the visual sensor 50. Based on the detected position and posture, the control unit 21 causes the position and posture of the visual sensor 50 mounted on the arm 10a to track the following object 102. Since the position and posture of the visual sensor 50 relative to the hand 30 are fixed, the hand 30 of the robot 10 tracks the object 100 while the following object 102 is always positioned at the target position and posture according to the image data from the visual sensor 50.

[0065] This control is achieved, for example, by the following control.

[0066] In this control, the target position, target posture, and target size of the moving object 102 to be placed in the image data are stored as the first feature value in the storage unit 23. The target size is, for example, the size of the outline when the feature is an outline.

[0067] The control unit 21 detects the position, posture, and size of the moving object 102 on the image data sequentially obtained by the visual sensor 50 as the second feature quantity according to the feature quantity detection program 23 e .

[0068] For example, the control unit 21 performs projective transformation on the model of the follower object 102 stored in the storage unit 23 and simultaneously performs a matching search between the projected model and the follower object 102 in the image data, thereby detecting the position and posture of the follower object 102. This model can be created using CAD data or an actual object. Because the relative position and posture of the work object 101 and the follower object 102 are fixed, the control unit 21 can determine the relative position and posture of the tip of the arm 10a and the follower object 102 based on the position and posture of the follower object 102.

[0069] The control unit 21 calculates a movement command for making the position, posture, and size of the following object 102 in the image data consistent with the first feature amount based on the movement amount calculation program 23 f.

[0070] The calculated movement command is used to eliminate or reduce the difference between the position, posture, and size of the moving object 102 in the image data and the first feature value. The calculated movement command is used, for example, to change the position of the hand 30 attached to the arm 10a in the X-axis direction, the Y-axis direction, and the Z-axis direction, and to change the posture of the hand 30 around the X-axis, the Y-axis, and the Z-axis.

[0071] Furthermore, in the above-described control, the control unit 21 may further adjust the calculated movement command based on parameters determined by the mechanical properties of the arm 10a. For example, the mechanical properties include the torque applied to the arm 10a due to the overall or local rigidity of the arm 10a, the rigidity of each movable part, the weight of the hand 30, the weight of the component 110, and the weight of both the hand 30 and the component 110. Furthermore, since the amount and direction of deflection of the arm 10a change depending on the angles of the joints that are the movable parts of the arm 10a, the state of each movable part of the arm 10a is also included in the mechanical properties.

[0072] In other words, when the posture of arm 10a changes in response to a movement command, the moment applied to arm 10a by the weight of hand 30 and component 110, as well as the state of each movable part of arm 10a, change in accordance with this posture change. Therefore, by adjusting the movement command in consideration of these mechanical characteristics, hand 30 can more accurately follow object 100.

[0073] The control unit 21 can use the continuous plurality of image data to obtain a change trend of the second feature quantity. For example, when the position, posture, and size of the following object 102 in the image data of the visual sensor 50 gradually approach the first feature quantity as the target data, the change trend of the relative position and posture of the visual sensor 50 with respect to the following object 102 is captured from the continuous plurality of image data.

[0074] If there is a tendency for the relative position and relative posture to change, the control unit 21 can also adjust the movement command using feedforward control based on the tendency according to the movement amount calculation program 23f. For example, the average speed can be calculated based on the change in the movement amount and used as the basic speed for feedforward control.

[0075] Feedforward control allows feedback control of deviations while maintaining a certain relative velocity relative to the object. Without feedforward control, the robot's movement velocity can drop to zero for a moment when image features match. This can cause frequent deceleration and acceleration, but feedforward control prevents this.

[0076] The correction data being fed forward is preferably subjected to known filtering or smoothing processes such as moving average. This allows the control unit 21 to detect changes in the position and posture of the article 100 due to external disturbances, sway of the article 100 due to the accuracy of the conveyor 2, the possibility of overshoot, electrical noise, and the like. The control unit 21 can then appropriately address changes due to external disturbances, sway due to the accuracy of the conveyor 2, reduce overshoot, and eliminate electrical noise.

[0077] Input values ​​such as the basic speed provided for feedforward control can also be arbitrarily input by the user based on the results measured by external measuring instruments.

[0078] Furthermore, a robot model that takes into account the deflection (torsion) of the speed reducer can be created, and the vibration of the arm can be estimated and fed back to reduce the vibration of the arm.

[0079] In this embodiment, feedforward control is performed assuming that the following object 102 performs a specific motion as a situation where the relative position and relative posture tend to change. Specifically, when a signal is obtained indicating that the following object 102 has started the specific motion, an input value corresponding to the signal and stored in the storage unit 23 is read, and feedforward control is performed based on the input value.

[0080] The input value of the feedforward control is obtained in advance by the following method.

[0081] First, if Figure 5 As shown, a visual sensor 50 is prepared that is fixed relative to the ground on which the transport device 2 is fixed.

[0082] In this state, if Figure 6As shown, the following object 102 is positioned within the visual field of the visual sensor 50, and the visual sensor 50 is activated (step S10). Next, the aforementioned specific action is executed (step S11). For example, if the transport device 2 is to be emergency stopped, an emergency stop signal (specific signal) that can confirm the emergency stop is obtained by the control device 20 (step S12).

[0083] Thus, the following object 102 is Figure 5 To record the trajectory, the visual sensor 50 sequentially acquires image data containing the moving object 102 at minute time intervals, and processes the acquired image data to sequentially detect the position of the moving object 102 (step S13).

[0084] Next, the detected position change of the following object 102 is determined (step S14). If the position change exceeds a predetermined threshold, the process starting from step S13 is repeated. If the position change falls below the predetermined threshold in step S14, the trajectory of the following object 102 in the robot coordinate system is calculated based on the position information of the following object 102 acquired in time series (step S15).

[0085] According to the trajectory of the follower object 102, the input value of the feedforward control is calculated (step S16). Figure 7 As shown, the input value of the feedforward control is calculated as a command signal for making the motion trajectory of the robot 10 coincide with the trajectory of the following object 102 .

[0086] The calculated input value is a command signal that changes within a finite time interval and is stored in correspondence with the acquired signal that identifies the specific action (step S17). The feedforward control input value is calculated in the same manner for other specific actions besides the emergency stop and stored in the storage unit 23 in correspondence with the signal that identifies each specific action.

[0087] Next, a method of finely adjusting the input value in a state where the input value of the feedforward control is stored in this manner will be described.

[0088] like Figure 8 As shown, while the robot 10 is in operation, the follower object 102 is caused to perform a specific motion (step S20). When the specific motion is started, a specific signal that can identify the specific motion is obtained in the control device 20 (step S21).

[0089] The control device 20 reads the input value stored in the storage unit 23 corresponding to the acquired specific signal (step S22) and begins feedforward control by sequentially adding the input value to the command signal at predetermined short time intervals (step S23). At this point, the robot 10 uses the visual sensor 50 to acquire an image containing the moving object 102 (step S24). The acquired image is processed to detect the position of the moving object 102 on the image (step S25).

[0090] In this state, it is sequentially determined whether the detected position of the following object 102 is within a predetermined range (step S26). If the position of the following object 102 is deviated beyond the predetermined range, the direction and amount of the position deviation of the following object 102 are stored in correspondence with the time of the input value (step S27). If the position deviation of the following object 102 is within the predetermined range and the position deviation is stored, it is determined whether the input value has ended (step S28), and the process starting from step S25 is repeated until the input value has ended.

[0091] After the input values ​​are read, the input values ​​for the feedforward control stored in the storage unit 23 are corrected based on the stored positional offset of the follower object 102 (step S29). The input values ​​can be fine-tuned by repeating the steps S20 to S29 until the positional offset of the follower object 102 falls within a predetermined range relative to the input values ​​over the entire period.

[0092] This has the advantage that, when the robot 10 of this embodiment is used for actual work, even if the transport device 2 is in unstable motion, the tool 30 of the robot 10 can accurately follow the object 100. Furthermore, when the visual sensor 50 included in the robot 10 is used, the input value for the feedforward control can be easily set.

[0093] Alternatively, the vision sensor 50 may be prepared separately from the vision sensor 50 mounted on the robot 10. In this case, the vision sensor 50 fixed to the floor is pre-calibrated with respect to the coordinate system of the robot 10. When the vision sensor 50 mounted on the robot 10 is used, the robot 10 is kept stationary. In this case, the vision sensor 50 is calibrated with respect to the coordinate system of the robot 10.

[0094] Furthermore, the control unit 21 can interpolate the detection results of the second feature quantity using the tendency of changes in relative position and relative posture, etc. Therefore, even if the acquisition period of the second feature quantity becomes longer, similar to the imaging period of the visual sensor 50, the interpolated detection results can be used to estimate the second feature quantity during the acquisition period or to estimate the second feature quantity in the future.

[0095] The control unit 21 controls the hand 30 of the arm 10a to follow the work target part 101. As a result, the position and posture of the shaft 111a of the mounting portion 111 of the component 110 match the position and posture of the hole 101a of the work target part 101.

[0096] As described above, changes in the position, posture, and size of the tracking object 102 in the image data from the vision sensor 50 are correlated within the control unit 20 with changes in the position and posture of the robot 10's coordinate system. Therefore, as the vision sensor 50 tracks the tracking object 102, the robot 10's coordinate system moves in the conveying direction of the conveying device 2, allowing the position and posture of the coordinate system to align with the movement of the article 10 being conveyed by the conveying device 2. In this situation, the work target portion 101 of the article 100 is being moved by the conveying device 2, but as viewed from the control unit 21, the work target portion 101 appears to be nearly stationary within the coordinate system.

[0097] In this controlled state, the control unit 21 starts force control based on the force control program 23d (step S1-5). A known force control method can be used as the force control. In this embodiment, the robot 10 moves the component 110 in a direction to escape the force detected by the force sensor 32. The amount of movement is determined by the control unit 21 based on the detection value of the force sensor 32.

[0098] For example, when the shaft 111a of the component 110 held by the hand 30 begins to engage with the hole 101a of the article 100, the force sensor 32 detects a force in a direction opposite to the conveying direction of the conveying device 2, and the control unit 21 causes the component 110 to move slightly in a direction opposite to the conveying direction to escape the detected force.

[0099] Next, when the second characteristic quantity detected sequentially based on the image data from the visual sensor 50 fluctuates beyond a predetermined threshold (step S1-6), the control unit 21 performs a first abnormality response operation (step S1-7). Fluctuations exceeding the predetermined threshold include, for example, significant movement of the follower object 102 within the image data or movement of the follower object 102 faster than a predetermined speed within the image data. The rotational speed of the motor 2a may also fluctuate significantly, for example, due to unstable power supply, which can cause a sudden drop in the rotational speed of the motor 2a. In these cases, the position of the follower object 102 relative to the tip of the arm 10a fluctuates beyond the predetermined threshold.

[0100] As a first abnormality response operation, the control unit 21 performs an operation to shorten the force control control cycle, increase sensitivity, stop mating, abort the mating operation, retreat in the direction opposite to the mating direction, stop transport, or a combination of these operations. Shortening the force control control cycle or increasing sensitivity allows the robot 10 to move more sensitively to the force applied to the component 110. In this embodiment, the control unit 21 performs an operation to abort the mating operation, retreat in the direction opposite to the mating direction, stop the transport device 2, or a combination of these operations.

[0101] In addition, when the second characteristic value is below the predetermined reference value in step S1-6 and the detection value of the force sensor 32 exceeds the predetermined reference value (step S1-8), the control unit 21 performs a second abnormality response operation (step S1-9). When the detection value of the force sensor 32 exceeds the predetermined reference value, there is a high possibility that an abnormal force is applied to the component 110, the article 100, etc. Therefore, the control unit 21 performs the following operation as the second abnormality response operation. That is, an operation is performed to stop the robot 10, an operation is performed to move the robot 10 in a direction to escape the direction of the force detected by the force sensor 32, an operation is performed to stop the conveying device 2, an operation is performed to retreat in a direction opposite to the mating direction, an operation is performed to stop the conveying, or an operation is performed in combination with these. In this embodiment, the control unit 21 performs an operation to stop the robot 10.

[0102] On the other hand, if the detection value of the force sensor 32 in step S1-8 is below the predetermined reference value, the control unit 21 determines whether the fitting operation is completed (for example, whether the distance traveled in the Z direction exceeds a predetermined value) (step S1-10). If the fitting operation is completed, the control unit 21 sends a predetermined movement instruction and work instruction to the arm 10a and hand 30 (step S1-11). As a result, the hand 30 releases the component 110 and moves away from the component 110. The hand 30 uses the arm 10a to move to a waiting position or a location where the next component 110 is stored. If it is determined in step S1-10 that the fitting operation is not completed, the process starting from step S1-6 is repeated.

[0103] Furthermore, in the above embodiment, the control unit 21 may further perform wide-range detection processing to detect the second feature quantity in a first range within the image data according to the feature quantity detection program 23e, and then perform narrow-range detection processing in a second range within the obtained image data. The narrow-range detection processing is processing to detect the second feature quantity in a second range narrower than the first range.

[0104] For example, when the difference between the first and second feature quantities is large, wide-range detection processing is performed, and when the difference between the first and second feature quantities falls below a predetermined value, narrow-range detection processing is performed. This can improve processing speed and accuracy as the difference between the first and second feature quantities decreases.

[0105] Separately or concurrently with this processing, the control unit 21 may also set the area containing the detected moving object 102 within the image data as the detection range for the second feature quantity according to the feature quantity detection program 23e. For example, the detection range can be set by setting a circumscribed rectangle tangent to the detected contour of the moving object 102 and enlarging the circumscribed rectangle at a predetermined magnification.

[0106] Furthermore, the magnification factor can be changed based on the size (dimensions) of the following object 102 within the image data, the distance between the visual sensor 50 and the following object 102, and other factors. For example, as the visual sensor 50 approaches the following object 102, the amount of movement of the following object 102 within the image data increases, and therefore the magnification factor can be increased. This allows for efficient and accurate detection of the position and posture of the following object 102.

[0107] In addition, if Figure 9 As shown, the hand 30 serving as a tool may also be attached to another robot, a working robot 60. In this case, the arm 60a and the hand 30 of the working robot 60 are controlled by a control device 70. In one example, the control device 70 has the same structure as the control device 20, and the arm 60a also has the same structure as the arm 10a.

[0108] The position and orientation of the coordinate system of the vision sensor 50 are associated with the position and orientation of the coordinate system of the robot 60 within the control device 70. While the control unit 21 is causing the vision sensor 50 to follow the tracking object 102, the control device 70 operates the robot 60 within the coordinate system of the robot 60. Since the position and orientation of the coordinate system of the robot 60 change based on the position and orientation of the coordinate system of the vision sensor 50, the control device 70 can perform operations using the motion program 23b set based on the coordinate system of the robot 60.

[0109] Even in this case, as described above, when the control device 20 causes the position and posture of the visual sensor 50 to track the following object 102, the position and posture of the coordinate system of the robot 60 can be caused to track the work object 101 based on information about the movement command and information about the difference between the second detection amount and the first detection amount. Therefore, when the robot 60 performs the operation of fitting the shaft 111a of the component 110 into the hole 101a of the object 100 according to the operation program 23b, the hand 30 of the robot 60 tracks the object 100.

[0110] The control device 20 and the control device 70 may also be connected to a higher-level control system such as a production management system, and information may be transferred between the control device 20 and the control device 70 via the higher-level control system.

[0111] Alternatively, a robot including a track provided above the transport device 2 and along the transport device 2, and a movable arm movably mounted on the track can be used in place of the robot 60. In this case, the vision sensor 50 is mounted on the tip of the movable arm, and the posture of the movable arm and the vision sensor 50 can be changed, for example, about the X-axis and about the Y-axis.

[0112] The movable arm is preferably capable of moving the tip end portion and the position of the vision sensor 50 in the Y-axis direction, but may be unable to freely move the tip end portion and the position of the vision sensor 50 in the Y-axis direction.

[0113] Even in this case, the position and posture of the vision sensor 50 attached to the movable arm can be made to follow the moving object 102 .

[0114] Even when the front end of the movable arm is not freely movable in the Y-axis direction, the position in the X-axis direction and the posture around the X-axis and Y-axis of the visual sensor 50 attached to the movable arm can be caused to track the following object 102 based on the difference between the second feature quantity and the first feature quantity. If this tracking is possible, even if the following object 102 moves in the Y-axis direction in the image data, its movement amount can be detected, achieving the same operational effect as described above.

[0115] Furthermore, the shape of the following object 102, etc., may be additionally detected as the second feature quantity. In this case, the first feature quantity related to the shape of the following object 102, etc., is stored in the storage unit 23. Since the shape of the following object 102 changes in accordance with the distance and angle between the arm 10a and the following object 102, more accurate following control can be performed.

[0116] Alternatively, multiple vision sensors 50 may be used, each tracking multiple moving objects 102. In this case, when the moving object 102 is located at each predetermined position in the plurality of image data obtained by the multiple vision sensors 50, it can be determined that the hand 30 attached to the arm 10a is located at a predetermined position and posture relative to the work target portion 101 of the object 100.

[0117] Thus, the robot 10 of this embodiment includes: at least one visual sensor 50 mounted on the arm 10a; and a storage unit 23 that stores a first feature value as target data for causing the visual sensor 50 mounted on the arm 10a to track the following object 102. Furthermore, in this embodiment, the second feature value related to at least the current position and posture of the following object 102 is detected using images obtained by the visual sensor 50.

[0118] The movement command for the arm 10a is then calculated based on the difference between the second feature quantity and the first feature quantity. Furthermore, while the visual sensor 50 is tracking the moving object 102, the calculation of the movement command and the movement of the arm based on the movement command are repeated. This allows the relative position and posture of the hand 30 with respect to the object 100 being transported by the transport device 2 to gradually approach the target data. This is useful for ensuring that the movement of the arm 10a of the robot 10 accurately tracks the object 100 being transported by the transport device 2.

[0119] In this embodiment, the first feature quantity includes a model of the moving object 102. When the feature portion of the object 100 is the moving object 102, the control unit 21 performs a matching search between the feature portion in the image data obtained by the visual sensor 50 and the model after projective transformation, thereby obtaining the position and posture (second feature quantity) of the feature portion in the image data.

[0120] This structure is useful for accurately approximating the relative position and posture of the visual sensor 50 to the following object 102 of the article 100 conveyed by the conveying device 2 to target data. The characteristic portion may also be a pattern provided on the surface of the article 100.

[0121] Furthermore, in this embodiment, at least the movement command is adjusted using feedforward control. In this configuration, feedforward control is used to perform control that takes into account the movement tendency of the article 100 being conveyed by the conveying device 2. This is useful for quickly and accurately approximating the relative position and posture of the visual sensor 50 relative to the moving object 102 of the article 100 to target data.

[0122] Furthermore, in this embodiment, before detecting the second characteristic quantity, the control unit 21 uses data obtained by the vision sensor 50 or other sensors 40 to calculate a pre-operation movement command for bringing the following object 102 into the detection range of the vision sensor 50. Therefore, before performing following control of the arm 10a, the vision sensor 50 is quickly positioned at the position required for following.

[0123] Furthermore, the working robot system of this embodiment includes a transport device 2 and a robot 10. While the visual sensor 50 provided on the robot 10 is tracking the following object 102, the robot 10 performs a predetermined operation on the object 100. Alternatively, the working robot system of this embodiment uses information regarding a movement instruction or information used for calculating a movement instruction that causes the visual sensor 50 provided on the robot 10 to track the following object 102, while the working robot 60 performs a predetermined operation on the object 100.

[0124] When the working robot 60 is used, it is possible to perform a predetermined work on the article 100 at a location far from the visual sensor 50. Alternatively, a plurality of working robots 60 may perform a predetermined work on the article 100 using the above information.

[0125] The working robot system of this embodiment further includes a force sensor 32. The force sensor 32 detects a force generated by contact between the component 110 or the hand 30 supported by the robot 10 and the object 100, or a force generated by contact between the component 110 or the hand 30 supported by the working robot 60 and the object 100.

[0126] Furthermore, when performing a predetermined task, the control devices 20 and 70 of the robot 10 or the working robot 60 also use the detection value of the force sensor 32 to cause the hand 30 provided on the robot 10 or the working robot 60 to follow the object 100 .

[0127] By also using the detection value of the force sensor 32 for follow-up control, the accuracy of the follow-up control can be further improved. While it can sometimes be difficult to correlate the relative posture of the hand 30 with the detection value of the force sensor 32 relative to the object 100, this relative posture is corrected in this embodiment, effectively improving the accuracy of the follow-up control.

[0128] Furthermore, in the working robot system of this embodiment, when the second characteristic value fluctuates beyond a predetermined reference, at least one of the control devices 20 and 70 of the robot 10 or working robot 60 performing the predetermined task, and the transport device 2, performs an abnormality response operation. This effectively prevents damage to the robot 10 or 60, the object 100, and the component 110 during follow-up control.

[0129] In the working robot system of this embodiment, the following object 102 is a part of the object 100. In this configuration, the position of the following object 102 in the object 100 is fixed, which is useful for further improving the accuracy of the following control.

[0130] Alternatively, a processing tool may be supported at the front end of the robot 10 or the working robot 60, and the robot 10 or the working robot 60 may process the object 100 being transported by the transport device 2 as a predetermined operation. In this case, the processing tool may be a drill, a milling cutter, a drilling and tapping tool, a deburring tool, or other tools.

[0131] Even in this case, the same effects as above can be achieved by bringing the machining tool close to the work object 101 in step S1-2, performing the above-mentioned follow-up control, and performing force control based on the contact between the machining tool and the work object 101. Alternatively, the machining tool may be a welding gun, a welding torch, or the like.

[0132] Alternatively, the conveying device 2 may be a conveying device that conveys the article 100 along a curved route or a conveying device that conveys the article 100 along a zigzag route. Even in these cases, the control unit 21 can use the detection results of the vision sensor 50 to cause the front end of the robot 10 or the working robot 60 to follow the work target portion 101.

[0133] Furthermore, if the position of the work object 101 relative to the robot 10 changes beyond a predetermined reference in step S1-6, the control unit 21 can perform the first abnormality handling operation in step S1-7.

[0134] Alternatively, another robot or AGV (Automated Guided Vehicle) can be used instead of the transport device 2 to move the object 100. Even in this case, the same effects as described above can be achieved. Furthermore, if the object 100 is a car or its frame, the object 100 can be moved while performing a predetermined task using its engine, wheels, etc. In these cases, the other robot, engine, wheels, etc. function as the transport device.

[0135] Alternatively, instead of the transport device 2, the article 100 can be transported using a chute, where the article 100 slides, rolls, or falls due to gravity. In this case, the tilted chute can be vibrated using a vibration device, thereby also smoothing the movement of the article 100 on the chute. In these cases, the chute, vibration device, etc. function as the transport device, and the article 100 transported by the chute is removed by a tool attached to the robot 10.

[0136] In this embodiment, the force sensor 32 is mounted on the front end of the robot 10 or the working robot 60. Alternatively, the force sensor 32 may be disposed between the transport device 2 and the object 100, or within the object 100. Even in this case, force control can be performed based on the detection value of the force sensor 32, achieving the same effect as described above.

[0137] In addition, the vision sensor 50 may be mounted on a portion other than the wrist flange of the robot 10 or the working robot 60 .

[0138] Furthermore, the visual sensor 50 may be a stereo camera. In this case, a pair of cameras can be used to obtain distance image data of the moving object 102, and the position and posture of the moving object 102 can be determined using the image data and the corresponding stereo model.

[0139] In this embodiment, the object followed by the vision sensor 50 is different from the work object of the robot 10. However, the object followed by the vision sensor 50 may be the same as the work object of the robot 10. For example, if a slight positional offset between the hand 30 serving as a tool of the robot 10 and the work object is permitted, or if the vision sensor 50 always sees the object followed when performing work using the hand 30 serving as a tool, the object followed and the work object may be set to be the same.

[0140] Furthermore, in the above-described embodiment, the position, posture, and size of the following object 102 are arranged at a target position on the image data from the vision sensor 50, thereby adjusting the position and posture of the hand 30, which serves as a tool, to the position and posture required for performing a predetermined operation on the object 100. Alternatively, the position and posture of the following object 102 can be arranged at a target position on the image data from the vision sensor 50, thereby adjusting the position and posture of the tool attached to the robot 10 to the position and posture required for a predetermined operation.

[0141] For example, in the case of operations such as laser welding, laser processing, and sealant coating in which the distance between the tool and the article 100 hardly changes, and in the case of operations that can be performed even if the distance between the tool and the article 100 changes, it is also possible not to use the information on the size as the first characteristic quantity and the information on the size as the second characteristic quantity.

[0142] In addition, in this embodiment, an example is given of calculating the trajectory of the follower object 102 in the robot coordinate system, but it is also possible to calculate only the elapsed time of a specific action of the follower object 102 in the robot coordinate system (for example, a stop action, or an action restarting from a stop state) as the time constant of the specific action.

[0143] In this case, if Figure 10 As shown, in step S12, after the control device 20 receives the emergency stop signal, the time when the emergency stop signal was received is stored in the storage unit 23 as the specific action start time. (Step S31) Then, in step S14, if the position change becomes less than a predetermined threshold, the time when the follower object 102 stops is stored in the storage unit 23 as the specific action stop time (Step S32).

[0144] Next, the elapsed time is calculated as the difference between the stored specific action start time and the specific action stop time (step S33), and the calculated elapsed time is stored in the storage unit 23 as a time constant corresponding to the signal that can determine each specific action of the follower object 102 (step S34).

[0145] In addition, the end of a specific action (for example, when the action reaches a certain speed after completely stopping or restarting from a stopped state) is determined by detecting the position and posture of the follower object 102 based on the image data obtained by the visual sensor 50. However, instead, when a specific action stops, the operator can visually determine the stop of the follower object 102 without using the visual sensor 50 and use a measuring device such as a stopwatch to measure the elapsed time.

[0146] In addition, when the time constant can be determined by the setting of the converter of the conveying device 2, the time constant determined by the setting may be used as it is.

[0147] Next, a method of fine-tuning the time constant in a state where the elapsed time calculated in this way is stored as the time constant will be described. Figure 11 As shown, while the robot 10 is in operation, the follower object 102 is caused to perform a specific action (step S40). When the specific action is started, the control device 20 acquires a specific signal that can identify the specific action (step S41), stores the time when the specific signal was acquired in the storage unit 23 as the specific action start time (step S42), reads the time constant set by the control device 20 from the storage unit 23 (step S43), and starts feedforward control according to the set time constant (step S44).

[0148] Then, the initial value of the maximum position deviation of the following object 102 is set to 0 (step S45). Then, the visual sensor 50 provided on the arm 10a of the robot 10 acquires an image containing the following object 102 (step S46). The acquired image is processed and the position of the following object 102 on the detection image is adjusted (step S47).

[0149] In this state, it is determined whether the absolute value of the detected position offset of the following object 102 is greater than the absolute value of the stored maximum position offset (step S48). If the absolute value of the position offset of the following object 102 is greater than the absolute value of the maximum position offset, the maximum position offset is updated and stored in the storage unit 23 (step S49).

[0150] If the absolute value of the position offset of the follower object 102 is less than or equal to the absolute value of the maximum position offset, and the maximum position offset is updated, a determination is made as to whether a time period longer than a set time constant has elapsed and whether the change in the position offset (the difference between the previous detected position and the current detected position of the follower object 102 in the robot coordinate system) has become constant (step S50). If a time period longer than the set time constant has not elapsed or the change in the position offset is not constant, the process from step S47 onwards is repeated.

[0151] For example, when a specific action stops, the position offset change becomes 0 because the follower object 102 and the robot 10 are stopped. Furthermore, when a specific action is restarted from a stop, the follower object 102 and the robot 10 maintain a constant speed. If the speeds are the same, the position offset change becomes 0. However, if there is a speed difference, the temporal change in the position offset change becomes a fixed value that is not zero.

[0152] After a time period longer than the set time constant has elapsed and the temporal variation in the positional deviation has become constant, it is determined whether the absolute value of the maximum positional deviation has become larger than a predetermined threshold (step S51 ).

[0153] Next, when the absolute value of the maximum position deviation is less than a predetermined threshold, whether the time constant should be increased or decreased is determined based on the sign of the maximum position deviation, and an amount corresponding to the absolute value of the maximum position deviation is added or subtracted from the time constant, thereby updating the time constant of the feedforward control stored in the storage unit 23 (step S52). Alternatively, the amount of increase or decrease ΔT corresponding to the absolute value of the maximum position deviation can be calculated, for example, according to the following formula (1).

[0154] ΔT=D / V (1)

[0155] Wherein, D is the maximum position offset, and V is the fixed speed of the following object 102 .

[0156] Then, in step S51 , the process is repeated from step S40 until the absolute value of the maximum positional deviation amount becomes equal to or smaller than a predetermined threshold value. If the absolute value of the maximum positional deviation amount becomes equal to or smaller than the predetermined threshold value, the process is terminated.

[0157] In addition, the example in which the follower object 102 is accelerated or decelerated immediately upon receiving a specific signal that identifies a specific action to thereby perform a specific action has been described. However, alternatively, the follower object 102 may be accelerated or decelerated in accordance with the specific signal at predetermined time intervals starting from the time the specific signal is received. In this case, an arbitrary time is set as the predetermined time interval.

[0158] In addition, in this embodiment, the input value of the feedforward control obtained by the above method can also be used for other robots operating on the same conveying device as the conveying device 2 for conveying the item 100 (a conveying device with the same object, or other conveying devices with the same specifications and settings).

[0159] In addition, in this embodiment, Figure 12 As shown, multiple control devices 20 may also be connected to a higher-level control system 100. For example, the higher-level control system 100 may be a computer connected to the multiple control devices 20 by wire, or a computer located on the same premises as the multiple control devices 20. The higher-level control system 100 is sometimes referred to as a fog computer. The higher-level control system 100 may also be a production management system, a shipping management system, a robot management system, a department management system, or the like.

[0160] The multiple upper-level control systems 100 may also be connected to other upper-level control systems 200. Other upper-level control systems 200 are, for example, cloud servers that are wired or wirelessly connected to the multiple upper-level control systems 100. The multiple control devices 20 and the upper-level control systems 100 form, for example, a management system.

[0161] The upper control system 100 includes a control unit including a processor, a display device, a storage unit including a nonvolatile memory, ROM, RAM, and the like, and input devices such as a keyboard, a touch panel, and an operation panel.

[0162] like Figure 13 As shown, such a system may also include, for example, multiple edge computers 8, multiple higher-level control systems 100, and one or more other higher-level control systems 200. In such a system, the control device 20 and the robot 10 may be edge computers. Parts of the control device 20 and the robot 10 may also be higher-level control systems. Such a system may include a wired or wireless network.

Claims

1. A follower robot, characterized in that: have: movable arms; at least one vision sensor disposed on the arm; a feature quantity storage unit that stores a first feature quantity related to at least a position and a posture of a following object as target data for causing the visual sensor provided on the arm to follow the following object; a feature quantity detecting unit configured to detect a second feature quantity related to at least a current position and posture of the moving object using an image obtained by the visual sensor; a movement amount calculation unit that calculates a movement instruction of the arm based on a difference between the second feature amount and the first feature amount using feedback control, and adjusts the movement instruction using at least feedforward control; a movement instruction unit that moves the arm according to the movement instruction; as well as an input value storage unit for storing, in correspondence with a signal acquired when a specific movement of the follower object begins, an input value for the feedforward control, the input value for the feedforward control being used to cause the arm to follow the trajectory of the follower object in the specific movement; The movement amount calculation unit and the movement instruction unit repeat calculation of the movement instruction and movement of the arm based on the movement instruction while causing the visual sensor to follow the moving object. The movement instruction is used to reduce or eliminate the difference between at least the position and posture of the following object as the second feature value and at least the position and posture of the following object as the first feature value, The movement amount calculation unit utilizes the feedforward control based on the input value stored in the input value storage unit in correspondence with the signal acquired when the specific action starts.

2. The follower robot according to claim 1, characterized in that: The input value is calculated so that the trajectory of the arm approximates the trajectory of the moving object detected by the visual sensor in a state where the arm is kept stationary.

3. The follower robot according to claim 1, characterized in that: The input values ​​are calculated so as to make the trajectory of the arm approximate the trajectory of the moving object detected by other fixedly arranged vision sensors.

Images