Sensor fusion for line tracking

Abstract

To provide a method for determining a position of an object moved along a conveyor belt.SOLUTION: A method includes measuring a position of a conveyor belt using a motor encoder while the conveyor belt is moved, and providing a position signal obtained by measuring a position of an object based on the measured position of the conveyor belt. The method includes determining stopping of the conveyor belt, providing a CAD model of the object, and generating a point group expression of the object using a 3D vision system. Then, the method matches the model with the point group and determines the position of the object, provides a model position signal at the position of the object based on the matched model and point group, and corrects an error of the position signal measured and generated as a result of stopping the conveyor belt using the model position signal.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present disclosure relates generally to robotic systems and methods for determining the position of an object moving along a conveyor belt, and more particularly, to robotic systems and methods for determining the position of an object moving along a conveyor belt, the methods including matching a CAD model of the object with a point cloud of the object from a 3D vision sensor to determine the object's position and correcting for errors from motor encoder measurements due to conveyor belt backlash when the conveyor belt is stopped. [Background technology]

[0002] The use of industrial robots to perform a variety of manufacturing, assembly, and material transfer operations is well known. In many robot workspace environments, obstacles are present and may be in the path of the robot's movement. Obstacles may be permanent structures, such as machinery or fixtures, or they may be temporary or movable. Because robots must operate in or around objects while performing operations such as welding, the objects being operated by the robot may themselves be obstacles. Therefore, various types of collision avoidance and interference checking processes are performed during robot operation.

[0003] For example, a robot may perform certain production operations, such as screwing, welding, or painting, on an object as it moves along a conveyor belt. The position of the object on the conveyor belt must be known to prevent collisions between the robot and the object and to effectively perform the operation on the object. Currently, motor encoders are often used to identify the conveyor belt position, e.g., the object's location. Here, a motor encoder is a rotary encoder attached to an electric motor that provides a closed-loop feedback signal by tracking the speed and / or position of the motor shaft. However, typical conveyor belts for these types of production operations often stop and start during operation for various reasons, causing the conveyor belt to suddenly jiggle or rebound, which in turn introduces errors into the position measurements from the encoder and therefore makes tracking the object on the conveyor belt difficult.

[0004] In one known robotic system that uses a motor encoder to determine the position of an object on a conveyor belt as described above, the system also uses a camera to provide images capturing features corresponding to the object moving on the conveyor belt, and the system tracks the movement of the feature based on the difference in position between successive images. From this tracked movement of the object, an emulated output signal is generated corresponding to the signal generated by the motor encoder, and the emulated signal is transmitted to a robot controller to manage the robot's operation. However, the visual information consists of 2D images, and the image features must be detected, where the tracking function relies solely on the output of the vision system. Furthermore, reference points are used to define the position and / or orientation of the object on the conveyor belt. By synchronizing the moving reference points with fixed reference points with known positions, a processing system can computationally determine the object's position within the known object geometry.

[0005] In another known robotic system that uses motor encoders to determine the position of an object on a conveyor belt as described, the system also approximates the shape of the object with a simple shape such as a box, sphere, or capsule. In the example of a car body moving on a conveyor belt, the car body is approximated with two boxes, thereby preventing operations such as screwing, welding, or interior painting from being performed. Summary of the Invention

[0006] The following discussion discloses and describes a robotic system and method for determining the position of an object moving along a conveyor belt. The method includes measuring the position of the conveyor belt while it is moving using a motor encoder and providing a measured position signal for the object's position based on the measured position of the conveyor belt. The method also includes determining when the conveyor belt has stopped, providing a CAD model of the object, and generating a point cloud representation of the object using a 3D vision system, where the point cloud includes points that identify the locations of features on the object. The method then matches the CAD model of the object to the point cloud to determine the object's position, provides a model position signal for the object's position based on the matched model and point cloud, and uses the model position signal to correct errors in the measured position signal that occur as a result of the conveyor belt stopping.

[0007] Additional features of the present disclosure will become apparent from the following description and appended claims, taken in conjunction with the accompanying drawings. [Brief explanation of the drawings]

[0008] [Figure 1] Figure 1 is an illustration of a robot system that includes a robot that paints a vehicle body moving along a conveyor belt. [Figure 2] Figure 2 is a schematic block diagram of an object location system for determining object position to correct for conveyor belt backlash errors in a robotic system. DETAILED DESCRIPTION OF THE INVENTION

[0009] The following discussion of embodiments of the present disclosure for a robotic system and method for determining the position of an object moving along a conveyor belt that compensates for backlash errors when the conveyor belt is stopped is merely exemplary in nature and is not intended to limit the invention or its applications or uses in any way.

[0010] FIG. 1 is an exemplary illustration of a robotic system 10 including a robot 12 with a paint nozzle 14 painting a car body 16 as the car body moves along a conveyor belt 18. System 10 is intended to represent any type of robotic system that may benefit from the discussion herein, where robot 12 may be any robot suitable for that purpose. Furthermore, the painting operation and car body 16 are for illustrative purposes only, where car body 16 is intended to represent any suitable object, and painting is intended to represent any suitable robotic operation, including welding and fastening. In order for robot 12 to effectively paint car body 16 and prevent collisions between robot 12 and car body 16, robot 12 needs to know the exact position of car body 16 as it moves along conveyor belt 18. To accomplish this, conveyor belt motor encoder 20 is located near conveyor belt 18 and provides a signal to robot controller 24 indicative of the speed at which belt 18 is moving. The system 10 also includes one or more 3D cameras 22 positioned at desired locations relative to the conveyor belt 18 and robot 12 to provide point cloud data to a robot controller 24 that controls the robot 12 to move the paint nozzle 14. A point cloud is a collection of data points in space defined by a particular coordinate system, with each point in the point cloud having an x, y, and z value. A laser sensor 26 also provides a signal to the controller 24 indicating when tracking of the vehicle body 16 should begin.

[0011] While the conveyor belt 18 is moving, the position of the car body 16 is continuously updated using information from the encoder 20. If the conveyor belt 18 stops, backlash in the belt 18 will cause errors in the measurements from the encoder 20 that must be corrected. While the conveyor belt 18 is stopped, the 3D camera 22 generates a point cloud that is matched or compared to a CAD model of the car body 16 stored in the controller 24 to correct for missing points and determine the exact position of the car body 16. The combination of high-frequency object position data from the encoder 20 while the belt 18 is moving and low-frequency object position data by matching the point cloud from the 3D camera 22 while the belt 18 is stopped with the CAD model of the car body 16 allows for the correction of measurements from the encoder 20 due to belt backlash, thus enabling accurate tracking of the car body 16 on the conveyor belt 18.

[0012] FIG. 2 is a schematic block diagram of an object location system 30 that determines the position of a car body 16 moving along a conveyor belt 18 and corrects for conveyor belt backlash errors, as described above. The system 30 includes a CAD model 32 of the car body 16 and a 3D vision system 34 that provides a point cloud of the car body 16, where the vision system 34 may include one or more 3D cameras or other 3D optical detectors. The CAD model 32 and the point cloud are matched in a point cloud matching processor 36, which operates any suitable point cloud matching algorithm to correct for missing cloud points and determine the precise location of the car body 16. One suitable algorithm is known as an iterative nearest neighbor algorithm, familiar to those skilled in the art, which rotates and translates the mesh shape of the CAD model to match or align with points in the point cloud, where the matched CAD model indicates the orientation and position of the car body 16. The position is then sent to an error correction processor 38, which also receives measurements from a conveyor belt motor encoder 40 representing encoder 20, corrects the measurements, and provides a position signal on line 42 identifying the exact position of body 16, which can be used to precisely control robot 12.

[0013] The point cloud matching processor 36 provides low-frequency position data of the car body 16 obtained when the conveyor belt 18 is stopped, and the measurements from the encoder 40 provide high-frequency position data of the car body 16 while the conveyor belt 18 is moving. Thus, when the conveyor belt 18 is moving, no data is provided to the error correction processor 38 from the matching processor 36, and only the encoder measurements provide the position of the car body 16 on the conveyor belt 18. When the conveyor belt 18 stops, which can be identified by the controller 24 in any suitable manner, the final position of the conveyor belt 18 provided by the encoder measurements will be inaccurate due to the sudden tilt of the belt 18 when it stops. The point cloud matching process is performed to correct the measurements from the encoder 40, so that when the belt 18 begins moving again, the measurements from the encoder 40 will be accurate. In this way, objects on the conveyor belt 18 are represented by their complex shapes and are not approximated by simple shapes, so that operations such as interior painting, welding, or screwing can be performed accurately.

[0014] The foregoing discussion discloses and describes merely exemplary embodiments of the present disclosure. Those skilled in the art will readily recognize from such discussion and the accompanying drawings and claims that various changes, modifications, and variations can be made without departing from the spirit and scope of the disclosure, as defined in the following claims.

Claims

1. 1. A method for identifying the location of an object moving along a conveyor belt, comprising: measuring the position of the conveyor belt while the conveyor belt is moving; providing a measured position signal of the position of the object based on the measured position of the conveyor belt, the position signal being high frequency position data; determining that the conveyor belt has stopped; providing a model of said object; generating a point cloud representation of the object using a vision system, the point cloud including points that identify the locations of features on the object; matching the model of the object with the point cloud to determine the location of the object; providing a model position signal of the position of the object based on the matched model and the point cloud, the model position signal being low frequency position data; and correcting an error in the position signal due to oscillations resulting from the conveyor belt stopping by comparing the position signal when the conveyor belt stopped with the model position signal.

2. The method of claim 1 , wherein measuring the position of the conveyor belt while the conveyor belt is moving includes using a motor encoder.

3. The method of claim 1 , wherein providing a model of the object comprises providing a CAD model.

4. The method of claim 1 , wherein generating the point cloud representation of the object includes using a 3D vision system.

5. The method of claim 4 , wherein the 3D vision system comprises at least one 3D camera.

6. The method of claim 5 , wherein the at least one 3D camera is a plurality of 3D cameras.

7. The method of claim 1 , wherein matching the model of the object with the point cloud comprises using a point cloud matching algorithm.

8. The method of claim 7 , wherein the point cloud matching algorithm is an iterative nearest neighbor algorithm.

9. The method of claim 1 , wherein matching the model of the object with the point cloud includes translating and rotating the model to match feature points in the point cloud.

10. The method of claim 1 , wherein the method is performed in a robotic system.

11. 1. A method for identifying the location of an object moving along a conveyor belt, said method being performed by a robotic system; measuring the position of the conveyor belt while it is moving using a motor encoder; providing a measured position signal of the position of the object based on the measured position of the conveyor belt, the position signal being high frequency position data; determining that the conveyor belt has stopped; providing a CAD model of the object; generating a point cloud representation of the object using a 3D vision system, the point cloud including points that identify the locations of features on the object; determining the position of the object by matching the model of the object with the point cloud and translating and rotating the model to match feature points in the point cloud; providing a model position signal of the position of the object based on the matched model and the point cloud, the model position signal being low frequency position data; and correcting an error in the position signal due to oscillations resulting from the conveyor belt stopping using the position signal when the conveyor belt stopped and the model position signal.

12. The method of claim 11 , wherein matching the model of the object with the point cloud comprises using an iterative nearest neighbor algorithm.

13. 1. A system for identifying the location of an object moving along a conveyor belt, comprising: means for measuring the position of the conveyor belt while the conveyor belt is moving; and means for providing a measured position signal of the position of the object based on the measured position of the conveyor belt, the position signal being high frequency position data; moreover, means for determining when the conveyor belt has stopped; means for providing a model of said object; means for generating a point cloud representation of the object using a vision system, the point cloud including points identifying the locations of features on the object; means for matching the model of the object with the point cloud to determine the position of the object; means for providing a model position signal of the position of the object based on the matched model and the point cloud, the model position signal being low frequency position data; moreover, means for correcting an error in the position signal due to oscillations occurring as a result of the conveyor belt being stopped using the position signal when the conveyor belt was stopped and the model position signal.

14. 14. The system of claim 13, wherein the means for measuring the position of the conveyor belt while the conveyor belt is moving includes using a motor encoder.

15. The system of claim 13 , wherein the means for providing a model of the object provides a CAD model.

16. The system of claim 13 , wherein the means for generating a point cloud representation of the object using a vision system uses a 3D vision system.

17. The system of claim 16 , wherein the 3D vision system comprises at least one 3D camera.

18. The system of claim 17 , wherein the at least one 3D camera is a plurality of 3D cameras.

19. The system of claim 13 , wherein the means for matching the model of the object with the point cloud uses an iterative nearest neighbor algorithm.

20. 14. The system of claim 13, wherein the means for matching the model of the object with the point cloud translates and rotates the model to match feature points in the point cloud.

Images