Collision prediction method, collision prediction device, and welding system

Abstract

To provide a collision prediction method having higher accuracy including an accessory of a robot.SOLUTION: A collision prediction method for performing collision prediction of a multi-joint robot and a peripheral object includes: a first acquisition step of acquiring configuration information and attitude information of the robot; a second acquisition step of acquiring point group data including the robot and the peripheral object; a classification step of classifying the point group data acquired by the second acquisition step into point group data corresponding to the robot and point group data corresponding to the peripheral object, on the basis of the configuration information and the attitude information of the robot; and a prediction step of performing collision prediction of the robot and the peripheral object, on the basis of a classification result by the classification step.SELECTED DRAWING: Figure 9

Description

[Technical field]

[0001] The present invention relates to a collision prediction method, a collision prediction device, and a welding system. [Background technology]

[0002] Conventionally, when a robot performs an operation on an object to be operated, the operation is controlled so as not to collide with peripheral devices located nearby. Collision prediction can be used not only during the operation of the robot, but also when generating an operation program for controlling the operation. Examples of peripheral devices for a robot include a positioner for holding the object to be operated and an imaging device for imaging the object to be operated. In addition to the peripheral devices, cables and other accessories of the robot are also assumed to collide with the robot during its operation. There is a demand for collision prediction for such accessories as well and for controlling the operation of the robot.

[0003] For example, Patent Document 1 discloses a method for predicting collisions with a robot, in which a three-dimensional sensor is used to measure position data of objects around the robot, and the interfering coordinates are stored to avoid collisions. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Patent No. 6816070 Summary of the Invention [Problem to be solved by the invention]

[0005] The method of Patent Document 1 targets collision avoidance only at the tip of the robot, and is therefore unable to predict collisions with parts of the robot other than the tip or with accessories. In addition, accurate collision prediction is required even when the work site environment changes, such as when the positioner's posture changes, but conventional methods can result in reduced processing efficiency and erroneous collision determinations.

[0006] In view of the above problems, an object of the present invention is to provide a method for predicting collisions with higher accuracy, including with robot accessories. [Means for solving the problem]

[0007] In order to solve the above problems, the present invention has the following configuration. That is, a collision prediction method for predicting a collision between an articulated robot and a surrounding object, comprising: a first acquisition step of acquiring configuration information and posture information of the robot; a second acquisition step of acquiring point cloud data including the robot and the peripheral objects; a classification step of classifying the point cloud data acquired in the second acquisition step into point cloud data corresponding to the robot and point cloud data corresponding to the peripheral objects based on configuration information and posture information of the robot; a prediction step of predicting a collision between the robot and the surrounding object based on a classification result by the classification step; The collision prediction method includes:

[0008] According to another aspect of the present invention, there is provided a collision prediction device for predicting a collision between a multi-joint robot and a surrounding object, comprising: A first acquisition means for acquiring configuration information and posture information of the robot; A second acquisition means for acquiring point cloud data including the robot and the peripheral objects; A classification means for classifying the point cloud data acquired by the second acquisition means into point cloud data corresponding to the robot and point cloud data corresponding to the peripheral object based on configuration information and posture information of the robot; a prediction means for predicting a collision between the robot and the surrounding object based on a classification result by the classification means; A collision prediction device having the following.

[0009] According to another aspect of the present invention, there is provided a welding system including a collision prediction device for predicting a collision between a multi-joint robot and a surrounding object, and a welding robot as the robot, The collision prediction device includes: A first acquisition means for acquiring configuration information and posture information of the robot; A second acquisition means for acquiring point cloud data including the robot and the peripheral objects; A classification means for classifying the point cloud data acquired by the second acquisition means into point cloud data corresponding to the robot and point cloud data corresponding to the peripheral object based on configuration information and posture information of the robot; a prediction means for predicting a collision between the robot and the surrounding object based on a classification result by the classification means; A welding system having Effect of the Invention

[0010] According to the present invention, it is possible to more accurately predict collisions between a robot, including robot appendages, and surrounding objects. [Brief description of the drawings]

[0011] [Figure 1] 1 is a schematic diagram showing an example of a system configuration according to an embodiment of the present invention; [Diagram 2] FIG. 1 is a block diagram showing an example of the configuration of a robot control device according to an embodiment of the present invention. [Diagram 3] FIG. 2 is a block diagram showing an example of a functional configuration of a robot control device according to an embodiment of the present invention. [Figure 4] FIG. 2 is a conceptual diagram for explaining a movable range of a robot according to an embodiment of the present invention. [Figure 5A] 1 is an example diagram for explaining a method for classifying point cloud data according to an embodiment of the present invention; [Figure 5B] 1 is an example diagram for explaining a method for classifying point cloud data according to an embodiment of the present invention; [Figure 5C] 1 is an example diagram for explaining a method for classifying point cloud data according to an embodiment of the present invention; [Figure 5D] 1 is an example diagram for explaining a method for classifying point cloud data according to an embodiment of the present invention; [Figure 5E] 1 is an example diagram for explaining a method for classifying point cloud data according to an embodiment of the present invention; [Figure 6A] 1 is an example diagram for explaining a method for classifying point cloud data according to an embodiment of the present invention; [Figure 6B] 1 is an example diagram for explaining a method for classifying point cloud data according to an embodiment of the present invention; [Figure 6C] 1 is an example diagram for explaining a method for classifying point cloud data according to an embodiment of the present invention; [Figure 7A] 1 is an example diagram for explaining a method for classifying point cloud data according to an embodiment of the present invention; [Figure 7B] 1 is an example diagram for explaining a method for classifying point cloud data according to an embodiment of the present invention; [Figure 7C] 1 is an example diagram for explaining a method for classifying point cloud data according to an embodiment of the present invention; [Figure 8A] 1 is an example diagram for explaining a method of collision prediction according to an embodiment of the present invention; [Figure 8B] 1 is an example diagram for explaining a method of collision prediction according to an embodiment of the present invention; [Figure 8C] 1 is an example diagram for explaining a method of collision prediction according to an embodiment of the present invention; [Figure 9] 4 is a flowchart of a collision prediction process according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] Hereinafter, an embodiment for carrying out the present invention will be described with reference to the drawings. Note that the embodiment described below is one embodiment for explaining the present invention, and is not intended to be interpreted as limiting the present invention, and all configurations described in each embodiment are not necessarily essential configurations for solving the problems of the present invention. In addition, in each drawing, the same components are given the same reference numbers to indicate their correspondence.

[0013] <First embodiment> Hereinafter, an embodiment of the present invention will be described with reference to the drawings. In this embodiment, an example of a welding system in which a multi-joint robot is used as a welding robot will be described, but the present invention is not limited to this. For example, the present invention is applicable to any robot that performs operations such as gripping an object.

[0014] [Welding system configuration] FIG. 1 shows an example of the configuration of a welding system 1 according to the present embodiment. The welding system 1 shown in FIG. 1 includes a welding robot 10, a robot control device 20, a power supply device 30, a camera 40, and a positioner 50. The welding robot 10 is a six-axis articulated robot, and a welding torch 11 is attached to its tip. A welding wire 13 is supplied to the welding torch 11 from a wire feeder 12. The welding wire 13 is fed from the tip of the welding torch 11 toward a welding point. The power supply device 30 supplies power to the welding wire 13. The robot control device 20 is configured to be able to communicate with the welding robot 10 and its peripheral devices, and controls various devices related to welding. The power supply device 30 is provided with a current sensor (not shown) that detects a welding current flowing from the welding wire 13 to a work (not shown) held by the positioner 50 during welding, and a voltage sensor (not shown) that detects an arc voltage between the welding wire 13 and the work. Additionally, the welding robot 10 is provided with accessories such as a cable 16 related to a control signal and a cable 17 for the welding wire 13 .

[0015] The power supply device 30 has a processing unit and a storage unit, both of which are not shown. The processing unit is, for example, configured with a CPU (Central Processing Unit). The storage unit is, for example, configured with a volatile or non-volatile memory, such as a HDD (Hard Disk Drive), a ROM (Read Only Memory), or a RAM (Random Access Memory). The processing unit controls the power applied to the welding wire 13 by executing a computer program for power supply control stored in the storage unit. The power supply device 30 is also connected to the wire feeder 12, and the processing unit controls the feed speed and feed amount of the welding wire 13.

[0016] The camera 40 is installed, for example, around the welding robot 10 and is used to capture the surroundings of the welding robot 10. The camera 40 is configured to be capable of acquiring point cloud data as three-dimensional data. For example, a ToF (Time of Flight) camera, a stereo camera, or a LiDAR (Light Detection and Ranging) sensor can be used as the camera 40. Since each of the above sensors has different characteristics, they may be used according to the measurement environment and the measurement target. In this embodiment, the camera 40 is installed above the welding robot 10 and configured to be able to capture the welding robot 10 located below and its surroundings. Note that the installation position and capture range of the camera 40 are not limited to this, and may be set arbitrarily so as to include the movable range of the welding robot 10 described later.

[0017] A ToF camera that can be used as camera 40 irradiates a laser light onto a measurement target and measures the reflected laser light with an imaging element to calculate the distance for each pixel. The measurable distance of a ToF camera is about tens of centimeters to several meters. A stereo camera uses multiple images taken by multiple cameras (for example, two cameras) to calculate the distance from their parallax. The measurable distance of a stereo camera is about tens of centimeters to several meters. A LiDAR calculates the distance by irradiating a laser light onto the surroundings and measuring the reflected laser light. The measurable distance of a LiDAR is about tens of centimeters to several meters.

[0018] Note that multiple cameras 40 may be installed, and point cloud data captured by each may be used separately to perform collision prediction, which will be described later. Alternatively, the point cloud data captured by each of the multiple cameras 40 may be unified and integrated by performing coordinate conversion processing, and then collision prediction may be performed using the integrated point cloud data. By using multiple cameras 40, blind spots can be reduced, enabling more accurate collision prediction.

[0019] Each part constituting the welding system 1 is communicatively connected by various wired / wireless communication methods. The communication method here is not limited to one, and a combination of multiple communication methods may be used for connection.

[0020] [Robot control device configuration] FIG. 2 shows an example of the configuration of a robot control device 20 that controls the operation of the welding robot 10. The robot control device 20 includes a CPU 201 that controls the entire device, a memory 202 that stores data, an operation panel 203 including a plurality of switches, a communication unit 204, a robot connection unit 205, and a positioner connection unit 206. The memory 202 is composed of a volatile or non-volatile storage device such as a ROM, a RAM, or a HDD. A control program 202A used to control the welding robot 10 is stored in the memory 202. The CPU 201 executes the control program 202A to control various operations by the welding robot 10 and the positioner 50. At this time, the robot control device 20 predicts a collision using a collision prediction method described later, and controls the posture, movement trajectory, etc. of the welding robot 10 based on the prediction result. When controlling the welding robot 10, the robot control device 20 can identify posture information such as the position, coordinates, and angle of each part of the welding robot 10 at that time.

[0021] The communication unit 204 includes a communication module for wired or wireless communication. The communication unit 204 is used for data and signal communication with the power supply device 30, the camera 40, and the like. The communication method and standard used in the communication unit 204 are not particularly limited, and a plurality of methods may be combined, or may be different for each connected device. For example, the power supply device 30 provides the current value of the welding current detected by a current sensor (not shown) and the voltage value of the arc voltage detected by a voltage sensor (not shown) to the CPU 201 via the communication unit 204. Also, point cloud data as three-dimensional data as described later is acquired from the camera 40.

[0022] A drive circuit of the welding robot 10 is connected to the robot connection unit 205. The CPU 201 outputs a control signal based on the control program 202A via the robot connection unit 205 to a drive circuit (not shown) provided in the welding robot 10.

[0023] A drive circuit of the positioner 50 is connected to the positioner connection unit 206. The CPU 201 outputs a control signal based on the control program 202A via the positioner connection unit 206 to a drive circuit (not shown) included in the positioner 50.

[0024] [Function configuration] 3 is a block diagram showing an example of a functional configuration of the robot control device 20. Each function may be realized, for example, by the CPU 201 reading and executing the control program 202A. Note that the functional configuration shown here is an example, and other functions may also be included.

[0025] The data acquisition unit 301 acquires point cloud data captured by the camera 40. The data acquisition unit 301 may notify the camera 40 of the timing of capturing the point cloud data, the capture settings, and the like. Furthermore, the data acquisition unit 301 acquires posture information such as the position, coordinates, and angle of each part constituting the welding robot 10 at that time corresponding to the timing of capturing the point cloud data. The setting reception unit 302 receives various settings related to the collision prediction according to the present embodiment. For example, the setting may be configured to be able to receive the capture settings by the camera 40. The data management unit 303 holds and manages the point cloud data acquired by the data acquisition unit 301 and the various settings received by the setting reception unit 302. The data management unit 303 also manages device information such as a three-dimensional model showing the shape of the welding robot 10 and the device configuration. The device information managed by the data management unit 303 may include configuration information on the specifications, such as the shape and structure of the welding robot 10, which are specified in advance, as well as posture information such as the position, coordinates, and angle of each part that may change with the operation.

[0026] The preprocessing unit 304 performs preprocessing on the acquired point cloud data. The preprocessing may vary depending on the point cloud data to be used, and may include, for example, filtering, outlier removal, clustering, and coordinate conversion. Therefore, depending on the point cloud data, preprocessing may be omitted. The data classification unit 305 performs classification processing on the point cloud data. Details of the classification processing will be described later.

[0027] The collision prediction unit 306 performs collision prediction using the data classified by the data classification unit 305. Details of collision prediction will be described later. The prediction result output unit 307 outputs the prediction result obtained by the collision prediction unit 306. The prediction result may be output so as to be used for control within the system, or may be output so as to be identifiable by the worker. The robot control unit 308 controls the operation of the welding robot 10 based on the prediction result by the collision prediction unit 306.

[0028] [Robot movement range] Fig. 4 is a conceptual diagram for explaining the movable range of welding robot 10. With multi-axis welding robot 10, the area occupied by welding robot 10 changes depending on its posture, etc. In Fig. 4, area 400 on the XY plane is the shooting range of camera 40. Within area 400, hatched area 402 indicates the movable range in which welding robot 10 can be positioned in any posture. Therefore, area 401 indicating the range other than area 402 indicates the outside of the movable range in which welding robot 10 cannot be positioned in any posture.

[0029] The positional relationship between the camera 40 and the welding robot 10 changes depending on the installation position and orientation of the camera 40. Therefore, the movable range of the welding robot 10 as seen from the camera 40 is appropriately determined depending on the system configuration, etc.

[0030] In the present embodiment, in the pre-processing, at least a part of the point cloud data located in the area 401 outside the movable range is removed from the point cloud data acquired by the camera 40. In other words, the area outside the movable range can be treated as a place where no collision occurs between the welding robot 10 and surrounding objects. This reduces the processing load and prevents errors in collision prediction.

[0031] [Point Cloud Data Classification] Hereinafter, classification of point cloud data according to this embodiment will be described. In this embodiment, in order to make it possible to distinguish between the welding robot 10 and other objects located around it, the point cloud data captured by the camera 40 is classified into point cloud data corresponding to the welding robot 10 and point cloud data corresponding to other objects. Then, collision prediction is performed based on these classifications. Note that the "other objects" here are not limited to peripheral devices such as the positioner 50, but may include general peripheral objects with which a collision may occur, such as floors and shelves.

[0032] In this embodiment, a plurality of classification methods for point cloud data can be implemented. Since the accuracy and processing time differ depending on the classification method, it is possible to switch between them depending on the purpose. In this embodiment, an example using four classification methods is shown, which are classification by a clustering method (also referred to as a "first classification method"), classification by time-series point cloud difference (also referred to as a "second classification method"), classification targeting only the tip of the welding robot 10 (also referred to as a "third classification method"), and classification by distance difference from the point cloud at the previous time (also referred to as a "fourth classification method").

[0033] (First classification method) The flow of the first classification method will be described with reference to Figures 5A to 5E. Figure 5A shows an example of point cloud data 500 including point cloud data 501 captured at a certain point in time by camera 40. Clustering is performed on the point cloud data 501 shown in Figure 5A using, for example, DBSCAN (Density-Based Spatial Clustering of Applications with Noise) clustering, which is a known method.

[0034] FIG. 5B shows an example of point cloud data 500 after the clustering process. The point cloud data 500 is classified into a plurality of points, such as point cloud data 511, 512, and 513, and is shown identifiable. Furthermore, for the processing result of FIG. 5B, information corresponding to the structural position, such as the position of the joint of the welding robot 10, is set. The position of the joint can be specified from information such as the specifications and posture of the welding robot 10, and information on a three-dimensional model. FIG. 5C is a conceptual diagram showing a state in which information on the position of the joint of the welding robot 10 is superimposed. Here, an example is shown in which four pieces of information, 521a, 521b, 521c, and 521d, are set. Furthermore, a point cloud located within a predetermined range is specified from the information 521a, 521b, 521c, and 521d. FIG. 5D shows an example in which predetermined radius ranges 531a, 531b, 531c, and 531d are set for each of the information 521a, 521b, 521c, and 521d.

[0035] Then, all clusters including point cloud data included in ranges 531a, 531b, 531c, and 531d are labeled as point cloud data indicating welding robot 10. In addition, other point cloud data is labeled as objects (other objects) other than welding robot 10. As a result, as shown in Fig. 5E, the point cloud data is classified into point cloud data 541 corresponding to welding robot 10 and point cloud data 542 corresponding to other objects.

[0036] In the above description, as shown in FIG. 5C, an example is shown in which point cloud data is specified using information on the positions of the joints of the welding robot 10. In addition to this, any other point cloud (hereinafter referred to as a "simulated point cloud") may be set, and these may be included in the point cloud data of the welding robot 10. In other words, the welding robot 10 is assumed to occupy a larger area than the actual area, and a collision prediction is performed in the subsequent stage. For example, after the positions of a plurality of joints are set, these may be connected by straight lines, and any number of simulated point clouds may be set at predetermined intervals in the linear fashion around the joints. For example, the straight lines may correspond to the links of the welding robot 10.

[0037] Also, an arbitrary simulated point cloud may be set around the point cloud data corresponding to the welding robot 10, and this may be included in the point cloud data of the welding robot 10. Such a simulated point cloud may be set in a predetermined shape corresponding to, for example, a cable located behind the arm according to the posture of the welding robot 10. In this case, the size and shape of the set simulated point cloud may be set based on the cable specifications, the posture of the arm, etc., which are specified in advance.

[0038] Also, an arbitrary simulated point cloud may be set at a position at a predefined distance based on the position of the point cloud data corresponding to a specific part of the welding robot 10, and this may be included in the point cloud data of the welding robot 10. Such a simulated point cloud may correspond to, for example, a cable located below the arm of the welding robot 10. In this case, the size and shape of the simulated point cloud may be set based on the predefined specifications of the cable.

[0039] (Second classification method) The second classification method will be described with reference to Figures 6A to 6C. First, as an initial state, point cloud data around the welding robot 10 is acquired, and the point cloud data corresponding to the welding robot 10 and point cloud data corresponding to other objects are clustered by the first classification method. Figure 6A shows the result of clustering point cloud data 600 by the first classification method, and point cloud data 601 corresponding to the welding robot 10 and point cloud data 602 corresponding to other objects are obtained. This is the initial setting.

[0040] Next, when processing the point cloud data at the time when collision prediction is performed (for convenience, referred to as "current time"), it is determined whether or not another object (here, the positioner 50) that may move has moved since the time when it was in the initial state. For example, it may be determined whether or not the positioner 50 has moved based on a control signal or the like. If the other object has moved, the first classification method is used to classify the point cloud data at that time. On the other hand, if the positioner 50 has not moved, the following processing is performed.

[0041] The shortest distance between each point in the point cloud included in the point cloud data of the initial setting and the point cloud included in the point cloud data of the current time is detected, and the point cloud data whose shortest distance is equal to or less than a threshold is deleted, thereby extracting only the changed point cloud data. In FIG. 6B, point cloud data 611 indicates point cloud data corresponding to the welding robot 10 that is determined to have not changed from the initial setting. Point cloud data 613 indicates point cloud data corresponding to the welding robot 10 that is determined to have changed from the initial setting. Point cloud data 612 is point cloud data corresponding to another object, and is assumed to have not changed from the initial setting here. FIG. 6C shows a state in which the point cloud data 611 determined to have not changed is deleted from the point cloud data shown in FIG. 6B. Using this point cloud data, a collision prediction is performed in the latter stage.

[0042] The time point of the initial state is not particularly limited. For example, the time point when the welding system 1 starts operating may be set as the initial state, or the reference initial state may be updated at regular intervals.

[0043] (Third classification method) The third classification method will be described with reference to Figs. 7A to 7C. First, point cloud data of the entire welding system 1 as shown in Fig. 5A is acquired. In such point cloud data, the tip of welding robot 10 and the joint closest to the tip are identified based on the configuration information of welding robot 10. Fig. 7A shows data 700 of the acquired point cloud data focusing on the area around the tip of welding robot 10. Here, point 702a corresponding to the tip of welding robot 10 and point 702b corresponding to the joint are identified.

[0044] Furthermore, a part of a predefined three-dimensional model of the welding robot is placed based on the positional relationship between points 702a and 702b. Fig. 7B shows an example of placing the three-dimensional model, and region 711 indicates the range in which the three-dimensional model is placed. Meanwhile, point cloud data 712 corresponds to point cloud data not included in region 711. Note that the three-dimensional model placed here may be a simple model instead of a detailed three-dimensional model.

[0045] 7C shows a state where the point cloud data included in the area 711 shown in FIG. 7B has been deleted. Here, among the point cloud data included in the area 711, point cloud data 722 corresponding to the position of the tip of the welding robot 10 is left. Then, point cloud data 721a, 721b, and 721c outside the area 711 are all treated as point clouds corresponding to other objects. Then, using the point cloud data 722 corresponding to the tip of the welding robot 10 and the other point cloud data, a collision prediction is performed in the subsequent stage. Note that the range (amount of point cloud data) treated as the tip of the welding robot 10 (corresponding to point cloud data 722) is not particularly limited, and may be determined based on the size of the welding robot 10, the processing load, etc.

[0046] (Fourth classification method) The fourth classification method will be described. As in the second classification method, first, as an initial state, point cloud data around the welding robot 10 is acquired, and the point cloud data corresponding to the welding robot 10 and the point cloud data corresponding to other objects are clustered by the first classification method. This is set as the initial setting. Next, the newly acquired current point cloud data is referenced, and it is determined whether it is closer to the point cloud data corresponding to the welding robot 10 in the initial setting or the point cloud data of the other object. Then, each of the current point cloud data is classified as closer to the welding robot 10 or the other object. At this time, a threshold value for distance is set, and if the distance between the current point cloud data and the point cloud data of the welding robot 10 in the initial setting exceeds the threshold value, the current point cloud data is treated as the point cloud data of the other object. After the determination of the point cloud data is completed, the initial setting is updated with the point cloud data of the current time, and this is set as the point cloud data of the previous time. After that, when new point cloud data is acquired, it is compared with the point cloud data of the previous time and the classification is repeated.

[0047] In the fourth classification method, the above-mentioned simulated point cloud may be included in the initial setting and compared with the current point cloud data. In this case, the simulated point cloud may also be included when setting the point cloud data of the previous time.

[0048] In the first classification method, the body of the welding robot 10 (including the tip and joints) and accessories (including cables) correspond to parts where a collision is predictable. Furthermore, the distance between objects that can be classified is the longest among the three classification methods. Furthermore, the execution time is also the longest among the four classification methods. On the other hand, in the first classification method, classification is possible even if the point cloud corresponding to the welding robot 10 is broken due to the presence of an obstacle or the like.

[0049] In the second classification method, the parts where a collision is predictable correspond to the main body of the welding robot 10 (including the tip and joints) and the accessories (including cables). The distance between objects that can be classified is intermediate between the first and third classification methods. The execution time is also intermediate between the first and third classification methods. In the second classification method, the execution time is longer when an object other than the welding robot 10 moves, but when no object other than the welding robot 10 moves, clustering is not used and therefore the execution time is shorter than that of the first classification method.

[0050] In the third classification method, the body (tip) of the welding robot 10 corresponds to a part where a collision is predictable. In addition, the distance between objects that can be classified is the shortest among the four classification methods. In addition, the execution time is also the shortest among the four classification methods.

[0051] The fourth classification method refers to the distance between the point cloud data of the welding robot 10 in the previous state and the point cloud data at the current state, and is therefore less susceptible to the intrusion of people into the work area, the movement of the positioner, etc. Also, the fourth classification method requires a shorter execution time than the second classification method.

[0052] [Method of collision prediction] In this embodiment, collision prediction is performed using point cloud data classified using the above-mentioned multiple classification methods. In this embodiment, three prediction methods are used: a collision prediction method based on the shortest distance between point clouds (hereinafter also referred to as a "first prediction method"), a collision prediction method using voxel overlap (hereinafter also referred to as a "second prediction method"), and a collision prediction method based on a time series (hereinafter also referred to as a "third prediction method").

[0053] These collision prediction methods may be switched according to the above-mentioned classification methods, or may be configured so that an operator can specify in advance which method to use. For example, the first prediction method can use the classification results of any of the first to fourth classification methods. The second prediction method can use the classification results of the first, second, and fourth classification methods. The third prediction method can use the classification results of the first classification method.

[0054] (First prediction method) 8A is a diagram for explaining a method of collision prediction based on the shortest distance between point clouds as a first prediction method. Here, points 801a, 801b, and 801c included in the point cloud data corresponding to the welding robot 10 and points 802a, 802b, and 802c included in the point cloud data corresponding to other objects are shown as the results of the classification process.

[0055] In the first prediction method, first, a threshold value for the distance for determining that a collision is about to occur is set. The threshold value here may be predefined based on the operating speed of the welding robot 10, the processing interval of the collision prediction, and the like. Next, the shortest distance between points is derived in the point cloud data. In the case of the example of FIG. 8A, the distance between a point 801c included in the point cloud data of the welding robot 10 and a point 802c included in the point cloud data of the other object is derived as the shortest distance 803. The shortest distance here is a distance in a three-dimensional space. The method for deriving the shortest distance is not particularly limited, and for example, a known kd tree method may be used. Then, if the shortest distance 803 is equal to or less than the threshold, it is determined that a collision is about to occur.

[0056] The distance between the point groups may not be a single shortest distance, but may be, for example, an average of the top pairs of points with shortest distances between the point groups. Also, the shortest distance may be derived after removing outliers between the point groups.

[0057] As described above, the shortest distance may be derived by including the simulated point cloud in the point cloud data corresponding to the welding robot 10. In this case, the processing speed may be increased by reducing a point cloud in an arbitrary range from the point cloud data corresponding to the welding robot 10 including the simulated point cloud.

[0058] Also, the point cloud for deriving the shortest distance may be selected based on the moving direction of the welding robot 10. The moving direction of the welding robot 10 can be specified according to the welding direction, posture, etc. For example, the range for deriving the shortest distance between the point cloud data of the welding robot 10 and the point cloud data of another object may be specified according to the following flow.

[0059] (i) Calculate the movement vector of the tip position (e.g., TCP (Tool Center Point)), joint positions, and link point cloud (including simulated point cloud) of the welding robot 10. For example, the movement direction of the welding robot 10 may be acquired by referring to teaching data. Alternatively, the movement vector may be calculated by calculating the difference between the current position vector and the past position vector. Note that the reference position of the welding robot 10 when calculating the movement vector is not limited to the above, and may further include other parts.

[0060] (ii) A three-dimensional shape of a predefined size and shape is placed along the position vector calculated in (i). For example, a cylinder, which is a predefined three-dimensional shape, may be placed in the direction of the movement vector with the current TCP as the center of the bottom surface. Alternatively, a rectangular parallelepiped, which is a predefined three-dimensional shape, may be placed parallel to the link with one point of the simulated point cloud of the current link as the center of the bottom surface. The three-dimensional shape used here is not limited to a cylinder or a cube, and any shape may be used. Also, the configuration of the three-dimensional shape may be changed depending on the part of the welding robot 10 to be placed.

[0061] (iii) From the point cloud data corresponding to other objects, extract point cloud data located within the three-dimensional shape arranged in (ii), and use the extracted point cloud data as the subject of deriving the shortest distance to the point cloud data corresponding to the welding robot 10.

[0062] By using the above-described method, it is possible to reduce the amount of point cloud data used for collision prediction, thereby enabling faster processing.

[0063] (Second prediction method) 8B is a diagram for explaining a collision prediction method using voxel overlap as a second prediction method. Here, points 811a, 811b, and 811c included in the point cloud data corresponding to the welding robot 10 are shown as a result of the classification process. Also shown is a voxel 812 defined based on the point cloud data corresponding to another object.

[0064] In the second prediction method, first, a threshold value for the number of points of the welding robot 10 and a voxel size based on the point cloud data of the other object are set in order to determine that a collision is about to occur. The threshold value here may be determined in advance based on the operating speed of the welding robot 10 and the processing interval of the collision prediction. The voxel size may also be determined in advance depending on the safety of the collision prediction. The larger the voxel size, the more likely it is that a collision is about to occur even if the welding robot 10 is farther away from the other object.

[0065] Next, the point cloud data of the other object is converted into voxels based on the set size. This results in a state like voxel 812 shown in Fig. 8B. Then, the number of points that overlap with voxel 812 and correspond to welding robot 10 is counted. In the example of Fig. 8B, point 811c overlaps, and the overlap number is "1". If the overlap number is equal to or greater than the set threshold, it is determined that a collision is about to occur.

[0066] Note that the voxelization is not limited to the above, and the point cloud data corresponding to the welding robot 10 may be voxelized, or both the point cloud data corresponding to the welding robot 10 and the point cloud data corresponding to other objects may be voxelized.

[0067] 8C shows an example in which both the point cloud data corresponding to the welding robot 10 and the point cloud data corresponding to another object are converted into voxels. Points 822a, 822b, and 822c are points included in the point cloud data corresponding to the welding robot 10, and thus voxel 821a is set. Points 823a, 823b, and 823c are points included in the point cloud data corresponding to the other object, and thus voxel 821b is set.

[0068] As shown in the example of FIG. 8C, when a collision prediction is performed using a voxel of point cloud data corresponding to the welding robot 10 and a voxel of point cloud data corresponding to another object, first, it is determined whether or not the voxels overlap. If they overlap, the number of points included in each voxel is counted, and an evaluation value is derived based on the number. If the evaluation value is equal to or greater than a preset threshold, it may be determined that a collision is about to occur. The method of deriving the evaluation value is not particularly limited, and may be, for example, the product of the number of points included in each voxel. In the example of FIG. 8C, it may be derived as (number of points included in voxel 821a: 3) × (number of points included in voxel 821b: 3) = (evaluation value: 9).

[0069] (Third prediction method) A third prediction method, a collision prediction method based on a time series, will be described. First, a threshold value for the point cloud of the welding robot 10 for determining that a collision is about to occur is set. The threshold value here may be predefined based on the operating speed of the welding robot 10, the processing interval of the collision prediction, etc.

[0070] Also, the number of points in the point cloud data corresponding to the welding robot 10 that corresponds to the initial posture of the welding robot 10 is stored. Next, the number of points in the point cloud data corresponding to the welding robot 10 at the time when the collision prediction is performed (for convenience, referred to as the "current time") is acquired. Then, the difference between the number of points included in the point cloud data corresponding to the welding robot 10 in the initial posture and the number of points included in the point cloud data corresponding to the welding robot 10 at the current time is calculated. If the difference is equal to or greater than a preset threshold, it is determined that a collision is about to occur. After the determination, the number of points in the stored point cloud data is updated with the number of points in the point cloud data corresponding to the welding robot 10 at the current time. In the subsequent collision prediction process, the updated value is used and repeated.

[0071] The first prediction method makes it easy to confirm the predicted collision position, but the processing time tends to become longer as the number of point cloud data corresponding to the welding robot 10 increases. The second prediction method makes it less easy to confirm the predicted collision position than the first prediction method, but makes it possible to reduce the processing time.

[0072] The third prediction method makes it easy to confirm the predicted collision position, and can reduce the processing execution time. The third prediction method also has the characteristic of being able to handle even minor misclassification of point clouds. Therefore, in the third prediction method, even if there is an erroneous judgment in the first classification method, the increase or decrease in the number of points is small, so it is possible to suppress the influence of classification accuracy on the judgment result. It is preferable to apply the third prediction method when the other object is equal to or larger than a predetermined size. It is also expected that the number of points in the point cloud data corresponding to the welding robot 10 will increase or decrease significantly depending on the change in the posture of the welding robot 10, so it is preferable to switch whether or not to apply the third prediction method depending on the change in posture.

[0073] [Processing flow] 9 shows a flowchart of the collision prediction process of the welding robot 10 according to this embodiment. This process flow may be realized, for example, by the CPU 201 of the robot control device 20 reading and executing the control program 202A. This process flow may be started simultaneously with the start of operation of the welding system 1, and the welding robot 10 is assumed to be continuously operating based on an arbitrary operation program in parallel with this process flow.

[0074] In S901, the robot control device 20 performs initial settings for the time series data. In this process, the initial settings used in the above-mentioned second classification method and the like are performed. More specifically, the classification results of the point cloud data of the welding robot 10 in the basic posture and other objects are set. Note that this process may be omitted depending on the method used in the subsequent classification process.

[0075] In S902, the robot control device 20 acquires device information. For example, information on a preset three-dimensional model of the welding robot 10, information on the device configuration, real-time posture information and position information of the welding robot 10, position information of the positioner 50, etc. may be acquired.

[0076] In S903, the robot control device 20 acquires point cloud data captured by the camera 40.

[0077] In S904, the robot control device 20 applies pre-processing to the point cloud data acquired in S903. Examples of the pre-processing here include coordinate conversion processing, downsampling processing, out-of-range point cloud removal processing, and outlier removal processing. In the coordinate conversion processing, for example, coordinate conversion between different coordinate systems such as a camera coordinate system and a system coordinate system is performed. In addition, when point cloud data captured using a plurality of cameras 40 are integrated and used, coordinate conversion may be performed so that these correspond to each other. In the downsampling processing, the acquired point cloud data is reduced to a desired resolution according to the processing load, etc. In the out-of-range point cloud removal processing, for example, as shown in FIG. 4, since it is not necessary to perform collision prediction for the area 401 outside the range in which the welding robot 10 operates, the point cloud data of these areas is removed from the acquired point cloud data. In the outlier removal processing, points having values ​​exceeding a predetermined threshold value are removed from the point cloud data. Note that the types of pre-processing are merely examples, and other types of processing may be performed. In addition, depending on the point cloud data acquired, all or part of the pre-processing may be omitted.

[0078] In S905, the robot controller 20 performs classification processing on the target point cloud data. As described above, in this embodiment, three classification methods are available, and which method is used may be a classification method arbitrarily set by the worker, or may be selectively switched on the robot controller 20 side depending on the processing load, the operation speed and operation content of the welding robot 10, and the like.

[0079] In S906, the robot control device 20 performs collision prediction based on the classification result performed in S905. As described above, in this embodiment, three prediction methods are available, and which method is used may be selectively switched depending on the classification method used in S905. At this time, a classification method arbitrarily set by the worker may be used, or the robot control device 20 may switch depending on the processing load, the operation speed and operation content of the welding robot 10, etc.

[0080] In S907, the robot controller 20 judges whether or not a collision is about to occur based on the prediction result in S906. If a collision is about to occur (S907: YES), the process of the robot controller 20 proceeds to S910. On the other hand, if a collision is not about to occur (S907: NO), the process of the robot controller 20 proceeds to S908.

[0081] In S908, the robot control device 20 updates the time-series data based on the processing results of S905 and S906. Note that, depending on the classification processing to be used, this step may be omitted, as in the case of S901. After that, the processing of the robot control device 20 proceeds to S909.

[0082] In S909, the robot controller 20 outputs the prediction result in S906. The prediction result may include, for example, in addition to the fact that a collision is not imminent, the distance between the welding robot 10 and other objects, and information on the processed point cloud data. The output method is not particularly limited, and the result may be notified to the user via the operation panel 203 or the like, or may be stored as history information in the memory 202. After that, the process of the robot controller 20 returns to S902, and the process is repeated.

[0083] In S910, the robot controller 20 determines that the welding robot 10 is about to collide with the object, and controls the welding robot 10 to stop its operation. After that, the process of the robot controller 20 proceeds to S911.

[0084] In S911, the robot control device 20 outputs the prediction result in S906 and a message indicating that the welding robot 10 has been stopped based on the prediction result. The prediction result may include, for example, in addition to the fact that a collision was about to occur, the distance between the welding robot 10 and other objects, and information on the processed point cloud data. The output method is not particularly limited, and the user may be notified via the operation panel 203 or the like, or the result may be stored as history information in the memory 202. Then, this processing flow ends.

[0085] It should be noted that even during the above process flow, if the operation of the welding robot 10 is completed, the process flow may be terminated.

[0086] As described above, this embodiment makes it possible to more accurately predict a collision between a robot, including an accessory of the robot, and a surrounding object.

[0087] <Other embodiments> In the present invention, a program or application for realizing the functions of one or more of the above-mentioned embodiments can be supplied to a system or device via a network or a storage medium, etc., and one or more processors in a computer of the system or device can read and execute the program, thereby making it possible to realize the present invention.

[0088] Alternatively, the function may be realized by a circuit that realizes one or more functions, such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array).

[0089] As described above, the present specification discloses the following: (1) A collision prediction method for predicting a collision between an articulated robot (e.g., 10) and a surrounding object (e.g., 50), comprising: a first acquisition step of acquiring configuration information and posture information of the robot; a second acquisition step of acquiring point cloud data including the robot and the peripheral objects; a classification step of classifying the point cloud data acquired in the second acquisition step into point cloud data corresponding to the robot and point cloud data corresponding to the peripheral objects based on configuration information and posture information of the robot; a prediction step of predicting a collision between the robot and the surrounding object based on a classification result by the classification step; The collision prediction method includes: This configuration enables more accurate collision prediction between the robot and surrounding objects, including the robot's accessories.

[0090] (2) In the classification step, a first classification method for clustering the point cloud data acquired in the second acquisition step, and classifying the point cloud data into point cloud data corresponding to the robot and point cloud data corresponding to the peripheral objects based on a result of the clustering and a position of the robot specified by configuration information and posture information acquired in the first acquisition step; a second classification method for acquiring first point cloud data corresponding to the robot from the point cloud data acquired in the second acquisition step using the first classification method, comparing the first point cloud data with second point cloud data stored in advance, and removing from the first point cloud data a point cloud whose shortest distance is equal to or less than a predetermined threshold, thereby classifying the first point cloud data as point cloud data corresponding to the robot; a third classification method for predicting a position of a tip of the robot in the point cloud data acquired in the second acquisition step based on the configuration information and the posture information, and classifying only a point cloud corresponding to the tip of the robot among points that overlap with a three-dimensional model of the robot when the three-dimensional model is placed at the predicted position as point cloud data corresponding to the robot; a fourth classification method for acquiring first point cloud data corresponding to the robot using the first classification method from the point cloud data acquired in the second acquisition step, and classifying the first point cloud data as the same as the closer point cloud data compared with past point cloud data corresponding to the robot and past point cloud data corresponding to the peripheral object; The collision prediction method according to (1), further comprising: According to this configuration, by making it possible to classify point cloud data using a plurality of types of classification methods, it becomes possible to realize more appropriate collision prediction according to the processing load and processing accuracy.

[0091] (3) further comprising a determination step of determining whether the peripheral object has moved; The collision prediction method described in (2), wherein, in the classification process, when it is determined that the surrounding object has moved, the first classification method is used, and when it is determined that the surrounding object has not moved, the second classification method is used. According to this configuration, it is possible to switch the classification method and perform collision prediction depending on the operation of a peripheral device located around the robot.

[0092] (4) In the prediction step, a first prediction method for predicting a collision by comparing a shortest distance between point cloud data corresponding to the robot and point cloud data corresponding to the peripheral object with a predetermined threshold; a second prediction method for predicting a collision by using at least one of the point cloud data corresponding to the robot and the point cloud data corresponding to the peripheral object as voxels and determining overlap based on the voxels; a third prediction method for predicting a collision based on a difference in the number of points between current point cloud data corresponding to the robot and point cloud data corresponding to the robot prior to the current point cloud data; The collision prediction method according to (2) or (3), further comprising: According to this configuration, by making it possible to predict collisions using multiple types of prediction methods for point cloud data classified using multiple classification methods, it is possible to achieve more appropriate collision prediction depending on the processing load and processing accuracy.

[0093] (5) Further comprising a pre-processing step of performing a predetermined pre-processing on the point cloud data acquired in the second acquisition step, the predetermined pre-processing includes a process of removing point cloud data located outside a movable range of the robot, The collision prediction method according to any one of (1) to (4), wherein the classification step classifies point cloud data that has been preprocessed in the preprocessing step. According to this configuration, it is possible to perform collision prediction with greater accuracy and with reduced processing load by focusing on the range in which a robot collision may occur.

[0094] (6) The method further includes a setting step of setting a simulated point cloud for the point cloud data classified as the point cloud data corresponding to the robot in the classification step, In the prediction step, a collision between the robot and the surrounding object is predicted using point cloud data including the simulated point cloud installed in the installation step; The collision prediction method according to any one of (1) to (5), wherein the simulated point cloud is set in a straight line connecting the position of the tip of the robot and the position of the joint of the robot, or in a predetermined shape based on a position corresponding to a predetermined part of the robot. According to this configuration, it is possible to perform collision prediction by taking into consideration the area corresponding to the desired member or part in addition to the point cloud data corresponding to the robot.

[0095] (7) acquiring information on a moving direction of a predetermined position of the robot; an extraction step of setting a predefined range along a moving direction of the robot with the predetermined position as a reference for the point cloud data corresponding to the robot, and extracting point cloud data corresponding to the peripheral objects included in the range; and The collision prediction method according to any one of (1) to (6), wherein in the prediction step, a collision between the robot and the peripheral object is predicted using point cloud data corresponding to the peripheral object extracted in the extraction step and point cloud data corresponding to the robot. With this configuration, it is possible to limit the point cloud data to be used for prediction by taking into account the direction of movement of the robot, thereby reducing the amount of point cloud data and speeding up processing.

[0096] (8) A collision prediction method according to any one of (1) to (7), wherein the posture information includes at least one of a position and angle of a tip of the robot and a position and angle of a joint of the robot. According to this configuration, it is possible to perform collision prediction with high accuracy based on the robot specifications and the real-time state of the robot.

[0097] (9) A collision prediction method according to any one of (1) to (8), wherein the configuration information includes a three-dimensional model showing the shape of the robot. According to this configuration, it is possible to perform collision prediction with high accuracy by using a three-dimensional model corresponding to the specifications of the robot.

[0098] (10) A collision prediction device (e.g., 20) that predicts a collision between an articulated robot (e.g., 10) and a surrounding object (e.g., 50), comprising: A first acquisition means (e.g., 301) for acquiring configuration information and posture information of the robot; A second acquisition means (e.g., 301) for acquiring point cloud data including the robot and the peripheral objects; A classification means (e.g., 305) for classifying the point cloud data acquired by the second acquisition means into point cloud data corresponding to the robot and point cloud data corresponding to the peripheral objects based on configuration information and posture information of the robot; A prediction means (e.g., 306) for predicting a collision between the robot and the surrounding object based on the classification result by the classification means; A collision prediction device having the following. This configuration enables more accurate collision prediction between the robot and surrounding objects, including the robot's accessories.

[0099] (11) A welding system (e.g., 1) including a collision prediction device (e.g., 20) according to (10) and a welding robot (e.g., 10) as the robot. According to this configuration, for a welding robot, it is possible to more accurately predict a collision between the welding robot and surrounding objects, including attachments of the welding robot. [Explanation of symbols]

[0100] 1 Welding system 10. Welding robot 11 Welding torch 12 Wire feeder 13 Welding wire 16,17 Cable 20 Robot control device 201 CPU 202 Memory 202A Control Program 203 Operation Panel 204 Communications Department 205 Robot Connection 206 Positioner connection part 30 Power supply 40 Camera 50 Positioner

Claims

1. A collision prediction method for predicting a collision between an articulated robot and a surrounding object, comprising the steps of: a first acquisition step of acquiring configuration information and posture information of the robot; a second acquisition step of acquiring point cloud data including the robot and the peripheral objects; a classification step of classifying the point cloud data acquired in the second acquisition step into point cloud data corresponding to the robot and point cloud data corresponding to the peripheral objects based on configuration information and posture information of the robot; a prediction step of predicting a collision between the robot and the peripheral object based on a classification result obtained by the classification step; The collision prediction method includes:

2. In the classification step, a first classification method for clustering the point cloud data acquired in the second acquisition step, and classifying the point cloud data into point cloud data corresponding to the robot and point cloud data corresponding to the peripheral objects based on a result of the clustering and a position of the robot specified by configuration information and posture information acquired in the first acquisition step; a second classification method for acquiring first point cloud data corresponding to the robot from the point cloud data acquired in the second acquisition step using the first classification method, comparing the first point cloud data with second point cloud data stored in advance, and removing from the first point cloud data a point cloud whose shortest distance is equal to or less than a predetermined threshold, thereby classifying the first point cloud data as point cloud data corresponding to the robot; a third classification method for predicting a position of a tip of the robot in the point cloud data acquired in the second acquisition step based on the configuration information and the posture information, and classifying only a point cloud corresponding to the tip of the robot among points that overlap with a three-dimensional model of the robot when the three-dimensional model is placed at the predicted position as point cloud data corresponding to the robot; a fourth classification method for acquiring first point cloud data corresponding to the robot using the first classification method from the point cloud data acquired in the second acquisition step, and classifying the first point cloud data in the same category as a closer point cloud data compared with past point cloud data corresponding to the robot and past point cloud data corresponding to the peripheral object; The method of claim 1 , further comprising:

3. The method further includes a determination step of determining whether the peripheral object has moved, 3. The collision prediction method according to claim 2, wherein in the classification step, when it is determined that the peripheral object has moved, the first classification method is used, and when it is determined that the peripheral object has not moved, the second classification method is used.

4. In the prediction step, a first prediction method for predicting a collision by comparing a shortest distance between point cloud data corresponding to the robot and point cloud data corresponding to the peripheral object with a predetermined threshold; a second prediction method for predicting a collision by determining overlap based on voxels of at least one of the point cloud data corresponding to the robot and the point cloud data corresponding to the peripheral object; a third prediction method for predicting a collision based on a difference in the number of points between current point cloud data corresponding to the robot and point cloud data corresponding to the robot prior to the current point cloud data; The method of claim 2 , further comprising:

5. The method further includes a pre-processing step of performing a predetermined pre-processing on the point cloud data acquired in the second acquisition step, the predetermined pre-processing includes a process of removing point cloud data located outside a movable range of the robot, The collision prediction method according to claim 1 , wherein the classification step performs classification on the point cloud data that has been preprocessed in the preprocessing step.

6. The method further includes a setting step of setting a simulated point cloud for the point cloud data classified as the point cloud data corresponding to the robot in the classification step, In the prediction step, a collision between the robot and the surrounding object is predicted using point cloud data including the simulated point cloud installed in the installation step; 2. The collision prediction method according to claim 1, wherein the simulated point cloud is set in a straight line connecting the position of the tip of the robot and the position of the joint of the robot, or in a predetermined shape based on a position corresponding to a predetermined part of the robot.

7. acquiring information on a moving direction of a predetermined position of the robot; an extraction step of setting a predefined range along a moving direction of the robot with the predetermined position as a reference for the point cloud data corresponding to the robot, and extracting point cloud data corresponding to the peripheral objects included in the range; Further comprising:

2. The collision prediction method according to claim 1, wherein in the prediction step, a collision between the robot and the peripheral object is predicted using point cloud data corresponding to the peripheral object extracted in the extraction step and point cloud data corresponding to the robot.

8. The collision prediction method according to claim 1 , wherein the posture information includes at least one of a position and an angle of a tip portion of the robot and a position and an angle of a joint portion of the robot.

9. The collision prediction method according to claim 1 , wherein the configuration information includes a three-dimensional model showing a shape of the robot.

10. A collision prediction device that predicts a collision between an articulated robot and a surrounding object, A first acquisition means for acquiring configuration information and posture information of the robot; A second acquisition means for acquiring point cloud data including the robot and the peripheral objects; a classification means for classifying the point cloud data acquired by the second acquisition means into point cloud data corresponding to the robot and point cloud data corresponding to the peripheral object based on configuration information and posture information of the robot; a prediction means for predicting a collision between the robot and the surrounding object based on a classification result by the classification means; A collision prediction device having the following.

11. A welding system comprising: the collision prediction device according to claim 10; and a welding robot as the robot.

Images