Robot system

Abstract

Robot system (1) comprising: a conveyor (2) that conveys an object; a robot (3) that performs a process on the object conveyed by the conveyor (2); a visual sensor (4) that detects visual information about the object being conveyed by the conveyor (2); a high-frequency processing unit (9) that processes the visual information detected by the visual sensor (4) at a first frequency; and a low-frequency processing unit (10) that detects a position of the object by processing, at a second frequency lower than the first frequency, the visual information detected by the visual sensor (4); wherein the robot system (1) further comprises a control unit (6) which controls the robot (3) on the basis of the conveying speed and / or the position of the object detected after the position is detected by the low-frequency processing unit (10) and by processing the visual information at the first frequency, and on the basis of the position of the object detected by the low-frequency processing unit (10).

Description

[0001] The present invention relates to a robot system.

[0002] According to the prior art, a robot system is known in which an object is conveyed by a conveyor or a mark is imaged on the conveyor, the moving speed of the object conveyed by the conveyor is detected on the basis of the obtained image, and the position of a robot hand is controlled in synchronization with the movement of the object, thereby detecting the moving object with the robot hand (see, for example, JP 2009 - 028 818 A).

[0003] There is also known a robot system in which a workpiece on a conveyor is identified based on an image detected by a visual sensor in synchronization with a signal from an encoder that detects an amount by which the conveyor is moved in a conveying route, and the workpiece is handled by adjusting the position of the hand to the identified workpiece, the hand being attached to a distal end of a robot (see, for example, JP 2012 - 192 466 A).

[0004] Document DE 10 2016 115 987 A1 discloses a device for adjusting a conveyor coordinate system in a robot system. The device comprises markings on a moving part of a conveyor and a sensor on a robot that detects the positions of these markings. These positions are then used to determine two coordinates in a base coordinate system of the robot and to determine the conveyor coordinate system.

[0005] Document DE 10 2015 219 544 A1 discloses a computer-aided method and system for tracking optical codes. This method involves receiving and identifying positions of barcode fragments in different images and predicting future positions based on a tracking model. The goal is to efficiently track and decode barcodes, even if they are located in different fields of view and at different times.

[0006] Document DE 10 2015 015 638 A1 discloses a system and method for transferring articles using a robot that tracks and picks up the articles on a conveyor. It includes an image processing section for detecting and tracking articles at various stages of transport, allowing the precise position and movement of the articles to be determined for efficient handling and transfer.

[0007] Document AT 507 339 B1 discloses a method and system for sequentially picking up and / or processing objects located on a continuously moving conveyor. The system uses an image acquisition system and an object recognition system to enable the positioning, identification, and selection of objects. This system enables efficient handling and processing of objects with high performance relative to the hardware requirements.

[0008] Document EP 2 538 373 A2 discloses a picking system comprising a conveyor, a robot, a main camera, and a controller. The conveyor transports workpieces, while the robot performs holding and moving operations. The main camera records the conveyor's transport path, and the controller detects the workpieces based on the images captured by the main camera and instructs the robot to perform the holding operation on the detected workpieces. If the workpieces overlap, the controller instructs the robot to also hold the overlapping workpieces.

[0009] US 2007 / 0 073 439 A1 discloses a machine vision system, method, and article useful in the field of robotics. One embodiment generates signals that emulate the output of an encoder based on captured images of an object that may be moving. Another embodiment provides digital data directly to a robot controller without the use of an intermediary transceiver such as an encoder interface card. Furthermore, another embodiment can predict or determine the occurrence of occlusion and move a camera and / or the object accordingly.

[0010] However, in order to detect the amount of movement of the conveyor based on the image captured by the visual sensor with a resolution comparable to that of the encoder, it is necessary to detect the position of the object at a high frequency, approximately equivalent to 1 millisecond. On the other hand, a tremendous amount of computation is required for high-load processing, such as pattern matching, to detect the shape and orientation of the object being conveyed by the conveyor with high precision, and therefore, it is a problem that it is not possible to perform high-frequency processing.

[0011] The present invention has been conceived in view of the circumstances described above, and one of its objects is to provide a robot system capable of detecting, on the basis of visual information detected by a visual sensor, the speed at which an object is conveyed by a conveyor and / or a position of the object, and of precisely detecting the position of the object, thereby making it possible to perform, by means of a robot, appropriate work related to the object conveyed by the conveyor.

[0012] To achieve the above-described object, the present invention provides the following solutions. One aspect of the present invention provides a robot system comprising: a conveyor that conveys an object; a robot that performs a process on the object conveyed by the conveyor; a visual sensor that acquires visual information about the object conveyed by the conveyor; a high-frequency processing unit that first processes the visual information acquired by the visual sensor at a first frequency; a low-frequency processing unit that acquires a position of the object by processing the visual information acquired by the visual sensor at a second frequency lower than the first frequency;and a control unit that controls the robot based on the conveying speed and / or the position of the object detected after the position is detected by the low-frequency processing unit (10) and by processing the visual information at the first frequency, and on the position of the object detected by the low-frequency processing unit.;

[0013] In this aspect, when a plurality of objects are conveyed by the conveyor, the visual sensor acquires visual information about the objects, and the plurality of acquired pieces of visual information are transmitted to the high-frequency processing unit and the low-frequency processing unit. In the high-frequency processing unit, by processing at the first frequency the visual information acquired by the visual sensor, either the speeds of the objects, which can be calculated with a relatively low processing load, or the positions of the objects are acquired at the first frequency, which is a high frequency. This makes it possible to acquire either the conveying speeds or the positions of the objects with a resolution equivalent to that of the encoder.

[0014] On the other hand, in the low frequency processing unit, because the visual information is processed at the second frequency which is lower than the first frequency, it is possible to perform processing that requires a huge amount of calculation, and therefore it is possible to detect the position of the object with high precision.

[0015] Thereby, it is possible to precisely apply a process to the object conveyed by the conveyor by means of the robot based on the position of the object detected by the low frequency processing unit by causing the robot to follow the object conveyed by the conveyor by moving the robot based on the conveying speed or the position of the object detected by the high frequency processing unit.

[0016] In the aspect described above, the high frequency processing unit may process a portion of the visual information detected by the visual sensor at the first frequency.

[0017] This makes it possible to facilitate the detection of the object's movement speed at high frequency by reducing the amount of visual information processed by the high frequency processing unit.

[0018] In the aspect described above, the visual sensor can capture an image of the object on the conveyor as the visual information.

[0019] Thereby, it is possible to detect the conveying speed of the object by processing the images captured by the visual sensor at different times at the first frequency, and by applying, at the second frequency, processing that requires a huge amount of calculation, such as pattern matching or the like, to the images of the object captured by the visual sensor, it is possible to detect the position and orientation of the object with high precision.

[0020] In the aspect described above, the conveyor may be provided with a tag that moves at a speed the same as that of the object.

[0021] Therefore, even in a state where the object is not conveyed by the conveyor, it is possible to accurately detect the conveying speed of the object by detecting the visual information of the label moved by the conveyor.

[0022] In the aspect described above, the high-frequency processing unit can detect the conveying speed of the object, its approximate position and orientation.

[0023] As a result, the high-frequency processing unit detects the approximate position and orientation of the object in addition to its conveying speed. Although it is difficult to detect the position and orientation for processing visual information with high precision at a high frequency, the high-frequency processing unit allows the approximate position and orientation to be detected, and thus, it is possible to improve operating efficiency by using this information as information for initial operation of the robot that applies processing to the object.

[0024] In the aspect described above, the robot can perform work by following the object conveyed by the conveyor.

[0025] By causing the robot to move to follow the conveyed object by using the conveying speed of the object detected by the high-frequency processing unit and by using the position and orientation of the object precisely detected by the low-frequency processing unit, it is thereby possible to remove the object conveyed by the conveyor without error.

[0026] In the aspect described above, the visual sensor may output the detected visual information immediately before or after receiving the trigger based on a trigger provided externally thereto.

[0027] As a result, based on the trigger from the high-frequency processing unit, the low-frequency processing unit or other equipment located outside the visual sensor, the visual information is output as needed, and it is possible to use the information to detect the position and orientation with high precision, adjust the visual sensor, check its working status, etc.

[0028] The present invention offers an advantage in that it is possible to detect at least the conveying speed at which an object is conveyed by a conveying device and / or the position of the object based on visual information detected by a visual sensor, and to detect the position of the object precisely, so that it is possible to perform appropriate work on the object conveyed by the conveying device by means of a robot. Fig. 1 is an overall configuration diagram showing a robot system according to an embodiment of the present invention. Fig. 2 is a block diagram showing the robot system in Fig. 1 shows. Fig. 3 is a graph showing changes over time in images taken by a camera of the robot system in Fig. 1 and shows the conveyor speed. Fig. 4 is a diagram similar Fig. 3 for a case where images of a plurality of objects in the same field of view in the robot system in Fig. 1 can be recorded. Fig. 5 is an overall configuration diagram showing a first modification of the robot system in Fig. 1 shows. Fig. 6 is an overall configuration diagram showing a second modification of the robot system in Fig. 1 shows. Fig. 7 is an overall configuration diagram showing a third modification of the robot system in Fig. 1 shows.

[0029] A robot system 1 according to an embodiment of the present invention will be described below with reference to the drawings.

[0030] As in Fig. 1, the robot system 1 according to this embodiment is provided with: a conveyor (conveyor) 2 that conveys an article O as an object; a robot 3 installed near the conveyor 2; a two-dimensional camera (visual sensor) 4 installed facing downward above the conveyor 2 at a position located on an upstream side of the robot in a conveying direction; an image processing unit 5 that processes the images captured by the two-dimensional camera; and a control unit 6 that controls the robot 3 based on images processed by the image processing unit 5.

[0031] The conveyor 2 is, for example, a belt conveyor and is provided with a belt 7 on which the article O is placed and conveyed in one direction. The belt 7 is driven by a motor (not shown).

[0032] Although the robot 3 may be of any type, such as a floor-mounted type, a suspended type, or the like, the robot 3 has, for example, a robot hand 8 that can grasp the article O at a distal end of its wrist.

[0033] The two-dimensional camera 4 has a field of view defined within a specific range of the conveyor 2 in the conveying direction and captures two-dimensional images of the article O conveyed on the conveyor 2. The two-dimensional camera 4 is configured, for example, to capture two-dimensional images (visual information) at a high frequency (first frequency) of 1 millisecond and output the two-dimensional images to the image processing unit 5, and also output the two-dimensional images in response to externally input triggers.

[0034] As in Fig. 2, the image processing unit 5 is provided with: a high-frequency processing unit 9 that applies image processing to portions of the acquired two-dimensional images at the first frequency; and a low-frequency processing unit 10 that applies image processing to the acquired two-dimensional images at a second frequency lower than the first frequency.

[0035] The high-frequency processing unit 9 calculates the positions of the centers of gravity of the articles O included in the respective two-dimensional images transmitted to it at the first frequency, and calculates the speed at which the articles O are conveyed by the conveyor 2 based on the displacement of the positions of the centers of gravity of the articles O in the two-dimensional images acquired at different times.

[0036] The low-frequency processing unit 10 applies image processing such as pattern matching or the like to the two-dimensional images output from the two-dimensional camera 4 and detects the positions and orientations of the articles O included in the two-dimensional images.

[0037] The control unit 6 generates drive signals (drive instructions) for controlling the robot 3 based on the speeds at which the articles O are conveyed by the conveyor 2, which are calculated based on the positions of an identical article O detected by the high-frequency processing unit 9 based on the images captured at different times, and the positions and orientations of the articles O detected by the low-frequency processing unit 10.

[0038] For example, the control unit 6 determines the current position of an article O by successively accumulating every 1 millisecond the amount by which the conveyor 2 is moved using the speed at which the article O is conveyed by the conveyor 2, which is calculated by the high-frequency processing unit 9 from the moment the low-frequency processing unit 10 detects the position and orientation of the article O. The control unit 6 therefore outputs the drive signals in accordance with the current position of the article O to the robot 3. If a case occurs where the article O is temporarily not detected, the amount by which the conveyor 2 is moved is calculated using the speed at which the article O was conveyed by the conveyor 2 immediately before.

[0039] The control unit 6 controls the robot 3 to operate in accordance with an operation program that it learns in advance, and controls the robot 3 to perform a process of detecting the article O conveyed by the conveyor 2 and picking up the article O from the conveyor 2 by using the robot hand 8, by performing tracking that follows the article O on the conveyor 2 based on the conveying speed calculated by the high-frequency processing unit 9, and by changing the orientation of the robot hand 8 in accordance with the position and orientation of the article O detected by the low-frequency processing unit 10.

[0040] Specifically, for example, as in Fig. 3, when three images are taken in the same field of view at different times t1, t2, and t3 separated from each other by a first frequency time interval Δt, the high frequency processing unit 9 detects the position of the article O by performing simple image processing having a low processing load (for example, processing for detecting the position of the centers of gravity and cluster detection processing), and calculates the coordinate positions d1, d2, and d3 of the centers of gravity of the detected article O. In the figure, the speed V is the speed at which the article O is conveyed.

[0041] The high-frequency processing unit 9 also recognizes, as an identical article O, the articles O that have the centers of gravity positioned on the same coordinate in a direction orthogonal to the conveying direction in the images sequentially acquired in the time axis direction, and calculates the conveying speed by dividing the differences between the coordinate values of the centers of gravity of the respective articles O in the conveying direction by the time interval Δt for capturing the images. When the conveying speed is calculated multiple times for the identical article O, an average thereof or a value adjusted by the least square method or the like is output as the conveying speed.

[0042] In this case, in the processing for detecting the position of the center of gravity of the articles O, which is repeatedly executed at the first frequency, the high-frequency processing unit 9 defines a partial area including one of the articles O, and sequentially detects the positions of the center of gravity by performing image processing on the portions of the images identified by the partial area, rather than the entire images transmitted from the two-dimensional camera 4. It is of course possible to use the entire images. However, in this case, the processing load increases compared to the case where portions of the images are used.

[0043] On the other hand, the low-frequency processing unit 10 performs image processing with a high processing load (for example, processing for detecting the position and orientation of the article O by means of pattern matching based on the outline of the article O) on the images transmitted to it from the two-dimensional camera 4, and outputs the detected position and orientation of the article O to the control unit 6.

[0044] The operation of the thus configured robot system according to this embodiment will be described below.

[0045] In the robot system 1 according to this embodiment, when the article O is conveyed by the conveyor 2, the two-dimensional camera 4 captures the images of the article O.

[0046] The images captured by image pickup are sequentially transmitted to the high-frequency processing unit 9 of the image processing unit 5, and in accordance with the triggers output from the low-frequency processing unit 10 at the second frequency, the images captured immediately before (or immediately after) receiving the triggers are also transmitted to the low-frequency processing unit 10.

[0047] In the high-frequency processing unit 9, by applying image processing with a relatively low processing load to the respective images transmitted thereto from the two-dimensional camera 4, the article O is recognized therein, and the coordinate positions d1, d2 and d3 of the center of gravity of the article O are detected on the basis of partial images of surfaces having the article O. Then, as in Fig. 3, based on the coordinate positions d1, d2, and d3 of the centers of gravity of the identical article O acquired based on the images acquired at different times t1, t2, and t3 separated by the predetermined time intervals Δt, the speed at which the article O is conveyed by the conveyor 2 at the first frequency is calculated and input to the control unit. Since the two-dimensional camera 4 and the robot 3 are arranged at positions separated by a predetermined distance, the article O moves within an operation area of the robot 3 when the amount of time obtained by dividing this distance by the conveying speed has elapsed.

[0048] In the low-frequency processing unit 10, by applying image processing with a high processing load, such as pattern matching or the like, at the second frequency lower than the first frequency to the respective images transmitted thereto from the two-dimensional camera 4 in accordance with the triggers, the position and orientation of the article O are detected on the basis of the complete image in a highly precise manner and input to the control unit 6.

[0049] The control unit 6 sets a tracking coordinate system TF by detecting the position and orientation of the article O at one of the time points at which the article O exists in the images, and calculates the amounts by which the position of the center of gravity of the article O is moved between the moment the article O is detected and the current time by determining the amount by which the conveyor 2 is moved based on the speed at which the article O is moved by the conveyor 2 at each timing of the first frequency (for example, 1 millisecond), and calculating the obtained movement amounts. Then, a current tracking coordinate system TF' is calculated by multiplying the tracking coordinate system TF by a coordinate conversion matrix having the movement amounts as its elements. TF'=T⋅TF

[0050] The control unit 6 can therefore cause the robot hand 8 to be moved so as to follow the article O conveyed by the conveyor 2 based on the calculated tracking coordinate system TF', and can also adjust the position and orientation of the robot hand 8 in accordance with the position and orientation of the detected article O, therefore causing the robot hand 8 to grasp the article O and lift the article O from the conveyor 2.

[0051] In this case, when the robot 3 is driven such that the robot hand 8 follows the article O on the conveyor 2, because the two-dimensional camera 4 captures images of subsequent articles O and the new conveying speed is calculated by the high-frequency processing unit 9, the control unit controls the robot 3 using the newly calculated conveying speed. This makes it possible to correctly lift the article O even if the conveying speed of the conveyor 2 fluctuates.

[0052] As described above, in the robot system 1 according to this embodiment, it is possible to detect, with a resolution equivalent to that of the encoder, the speed at which the article O is moved by means of processing with a relatively low processing load at the first frequency, which is a higher frequency, and it is possible to cause the robot 3 to perform the follow-up operation for the article O being conveyed.As for detecting the position and orientation of the article O, which requires processing with a relatively high processing load, high-precision detection is possible by performing the processing at the second frequency lower than the first frequency, and therefore, it is an advantage that it is possible to cause the robot hand 8 to grasp the article O more reliably by adjusting the orientation of the robot hand 8 to precisely align it with the orientation of the article O.

[0053] Since the moving speed, position, and orientation of the article O are detected in this way by using the single two-dimensional camera 4, there is also an advantage that it is possible to achieve cost reduction.

[0054] By using, in the high-frequency processing unit 9, portions of the images transmitted thereto from the two-dimensional camera 4 to detect the moving speed of the article O, it is possible to simplify the high-frequency processing by reducing the processing load.

[0055] As a result, as indicated by brackets in Fig. 2, it is possible to detect the approximate position and orientation of the article O in the high-frequency processing unit 9. For example, it is possible to detect a longitudinal direction of the article O, an approximate shape of a cluster, etc. By inputting the approximate position and orientation of the article O detected in this way to the control unit 6, the control unit 6 can start the operation of the robot 3 based on the information about the approximate position and orientation of the article O transmitted to it at the first frequency from the high-frequency processing unit 9, without waiting for the high-precision information from the low-frequency processing unit 10, so there is an advantage in that it is possible to further facilitate tracking.

[0056] Alternatively, although the partial areas having the article O are used as the portions of the images transmitted from the two-dimensional camera 4, the amount of information may be reduced by thinning the pixels from the images transmitted from the two-dimensional camera 4.

[0057] Although in this embodiment, the two-dimensional camera has been described as an example of the visual sensor that detects the visual information, there is no limitation thereto, and a three-dimensional camera or other types of visual sensors may be employed.

[0058] Although in this embodiment, the conveying speed of the conveyor 2 is calculated based on changes in the positions of the article O conveyed by the conveyor 2, alternatively, marks may be provided on the surface of the conveyor 2 at appropriate intervals, and the conveying speed may be calculated based on changes in the positions of the marks by recognizing the marks included in the images captured by the two-dimensional camera 4.

[0059] In this case, the labels are useful for calculating the movement speed because the labels are continuously delivered into the field of view of the two-dimensional camera 4 in a more reliable manner.

[0060] Although in this embodiment the case where the single article O is detected in the images has been described, it is alternatively possible, as shown in Fig. 4, it is possible to apply the present invention to a case where a plurality of articles a1, a2 and a3 are simultaneously placed in the field of view of the two-dimensional camera 4.

[0061] In other words, in the case where the plurality of articles a1, a2, and a3 are detected in the images, for the respective articles a1, a2, and a3, the identical correspondence with respect to the articles a1, a2, and a3 in the images captured at different times can be determined, and the conveying speed can be calculated by averaging the speeds V1, V2, and V3 calculated separately based on the moving distances among the articles a1, a2, and a3 determined to be the same.

[0062] In this case, the moving distances of the same articles a1, a2 and a3 are respectively determined based on differences in the coordinate positions d11, d12 and d13, coordinate positions d22, d23 and d24 and coordinate positions d33 and d34 of the centers of gravity of the same articles a1, a2 and a3, which are calculated based on the images taken at different times t1, t2 and t3 separated by the predetermined time interval Δt.

[0063] Although this embodiment has been described as an example in which the image processing unit 5, which processes the images output from the two-dimensional camera 4, is provided as a separate component, alternatively, as shown in Fig. 5, the high frequency processing unit 9 may be arranged in the two-dimensional camera 4, and the low frequency processing unit 10 may be arranged in the control unit 6.

[0064] By incorporating the high-frequency processing unit 9 into the two-dimensional camera 4, it is possible to more reliably detect the speed at which the article O is conveyed by the conveyor at a high frequency by preventing communication delays. Since the low-frequency processing unit 10 does not require high-speed processing, it is preferable that the low-frequency processing unit 10 be incorporated into the control unit 6. In the case where the control frequency of the control unit 6 is lower than the first frequency of the high-frequency processing unit 9, it is preferable that the information about the positions of the center of gravity of the article O be transmitted from the high-frequency processing unit 9 in units standardized by the control frequency of the control unit 6.For example, in the case where the first frequency corresponds to 1 millisecond and the control frequency of the control unit 6 corresponds to 8 milliseconds, eight pieces of information about the position of the center of gravity corresponding to 8 milliseconds in the high-frequency processing unit 9 can all be transmitted simultaneously to the control unit 6 at a frequency corresponding to 8 milliseconds.

[0065] Although in this embodiment an example has been described in which the single robot 3 is controlled, alternatively, as shown in Fig. 6, a plurality of robots 3 may be arranged along the conveying direction of the conveyor 2, and the control unit 6 may thus be connected to a high-level cell control device 11.

[0066] In the case where several robots 3 are used to perform work on the articles O conveyed on the same conveyor 2, it is possible to manage at one point the conveying speed of the conveyor 2, which is calculated on the basis of the images acquired by the single two-dimensional camera 4.

[0067] In the case where the conveying speed of the conveyor 2 is managed by the control units 6 of the respective robots 3, it is necessary to synchronize their management among the control units 6, and an error may occur due to the influence of communication delays, etc.; however, by performing management by means of the cell control devices 11, it is possible to prevent the occurrence of such a problem. Although an example has been described in which, in order to further suppress the influence of communication delays, the high-frequency processing unit 9 is arranged in the two-dimensional camera 4 and the low-frequency processing unit 10 is arranged in the cell control device 11, there is no limitation in this regard. The high-frequency processing unit 9 may also be arranged in the cell control device 11, for example.

[0068] Although in this embodiment, the case where the article O conveyed by the conveyor 2 is grasped and picked up has been described as an example, the invention can alternatively be applied to a case where any other processing is applied to the conveyed article O.

[0069] As in Fig. 7, it is permissible to use a configuration in which the control unit 6 outputs the triggers to the two-dimensional camera 4 as needed, the two-dimensional camera 4 outputs the images to the control unit 6 in accordance with the triggers, and the images can be displayed on the display unit 12 in the control device 6, whereby it is possible to check the images thereon.

[0070] Note that the control unit 6 may sequentially calculate the positions of the article O on the conveyor 2 based on the positions of the article O detected by the high-frequency processing unit 9 at a high frequency as described above, and the control unit 6 may then output drive signals to the robot 3 in response to the positions of the article O obtained by the calculation. Tracking to follow the article O on the conveyor 2 may also be performed based on the positions of the article O by the above-mentioned calculation. In this case, an effect the same as or similar to that mentioned above can also be realized.

[0071] Furthermore, the two-dimensional camera 4 may be mounted on a distal end portion of the robot 3. In this case, in the control unit 6, the reference coordinate system, which is the coordinate system used to control the robot 3, and the position and orientation (sensor coordinate system) of the two-dimensional camera 4 are made to coincide with each other. With this configuration, the control unit 6 can know the position and orientation of the two-dimensional camera 4 even when the position of the distal end portion of the robot 3 is changed. It is therefore possible to accurately convert the detection results of the position and orientation of the article O based on the images captured by the two-dimensional camera 4 into the position and orientation of the article O when viewed from the reference coordinate system.

[0072] When the two-dimensional camera 4 is attached to the distal end portion of the robot 3 as described above, the article O followed by the robot 3, articles O around the article O being followed, and the like exist in the field of view of the two-dimensional camera 4. In this case, an effect the same as or similar to the above-mentioned effect can also be realized because the position of the article O can be detected at the high frequency by the high-frequency processing unit 9.

[0073] Furthermore, the two-dimensional camera 4 can be mounted near the robot 3 using a frame, a stand, or the like, and the article O followed by the robot 3, the article O around the article O being followed, and the like can exist in the field of view of the two-dimensional camera 4. In this case, the reference coordinate system of the robot 3 and the position and orientation (sensor coordinate system) of the two-dimensional camera 4 are made consistent with each other in the control unit 6, and the control unit 6 can accurately convert the detection results of the position and orientation of the article O based on the images captured by the two-dimensional camera 4 into the position and orientation of the article O when viewed from the reference coordinate system.In this case, an effect which is the same as or similar to the above-mentioned effect can also be realized because the position of the article O can be detected at the high frequency by the high frequency processing unit 9.

[0074] Although the above-mentioned embodiment has a single two-dimensional camera 4 for detecting the article O, in the embodiment, other two-dimensional cameras for inspecting articles O, arrival detection, and the like may be provided, and it is possible to perform the high-frequency processing and the low-frequency processing in a case where a plurality of three-dimensional cameras 4 are used for detecting the articles O.

[0075] It is also possible to convey the article O by using a conveyor that moves the article O in an X-axis direction and also in a Y-axis direction. Note that the X-axis and the Y-axis extend horizontally, and the X-axis intersects the Y-axis at right angles. In this case, the high-frequency processing unit 9 can detect the position of the article O at a high frequency, and the high-frequency processing unit 9 can calculate, with the high frequency, a speed at which the article O is conveyed in the X-axis direction and a speed at which the article O is conveyed in the Y-axis direction. It is therefore possible to achieve an effect that is the same as or similar to the above-mentioned effect. Note that a case where the article O is moved in a Z-axis direction is the same as or similar to this case.

[0076] It is also possible to convey the article O by using another robot instead of using the conveyor 2. Furthermore, when the article O to be conveyed is a vehicle body of a motor vehicle or the like, the article O may be conveyed by a machine, wheels, and the like. It is also possible to convey the article O by using a chute along which the articles O slide down, roll down, or fall due to gravity. In these cases, the other robot, the machine, the wheels, the chute, or the like function as the conveying device. List of reference symbols 1 robot system 2 conveyors (conveyor) 3 robots 4 two-dimensional camera (visual sensor) 6 Control unit 9 High-frequency processing unit 10 Low frequency processing unit O, a1, a2, a3 articles

Claims

[1] Robot system (1) comprising: a conveyor (2) that conveys an object; a robot (3) that performs a process on the object conveyed by the conveyor (2); a visual sensor (4) that detects visual information about the object being conveyed by the conveyor (2); a high-frequency processing unit (9) that processes the visual information detected by the visual sensor (4) at a first frequency; and a low-frequency processing unit (10) that detects a position of the object by processing, at a second frequency lower than the first frequency, the visual information detected by the visual sensor (4); wherein the robot system (1) further comprises a control unit (6) which controls the robot (3) on the basis of the conveying speed and / or the position of the object detected after the position is detected by the low-frequency processing unit (10) and by processing the visual information at the first frequency, and on the basis of the position of the object detected by the low-frequency processing unit (10). [2] Robot system (1) according to claim 1, wherein the high-frequency processing unit (9) processes a portion of the visual information detected by the visual sensor (4) at the first frequency. [3] The robot system (1) according to claim 1 or 2, wherein the visual sensor (4) detects an image of the object on the conveyor (2) as the visual information. [4] Robot system (1) according to one of claims 1 to 3, wherein the conveyor (2) is provided with a marking which is moved at a speed which is the same as that of the object. [5] Robot system (1) according to one of claims 1 to 4, wherein the robot (3) performs work by following the object conveyed by the conveyor (2). [6] The robot system (1) according to any one of claims 1 to 5, wherein the visual sensor (4) outputs, based on a trigger provided externally thereto, the visual information detected immediately before or immediately after receiving the trigger.

Images