Welding position detection device and robot teaching system

Abstract

To provide a welding position detection device and a robot teaching system which can easily and properly detect a welding position.SOLUTION: A welding position detection device 220 comprises: a reference surface detection unit 221 which detects a reference surface being the reference on the basis of point group data acquired by a 3D camera; a specific point group data extraction unit 222 which extracts specific point group data at a position apart from the reference surface in the point group data constituting a member arranged so as to be in contact with the reference surface; a specific point group data projection unit 223 which projects the specific point group data to the reference surface; and a welding position determination unit 224 which determines a position of the specific point group data projected to the reference surface as a welding position.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention relates to a welding position detection device and a robot teaching system. [Background technology]

[0002] In recent years, many robots have become widespread in the industrial world. These robots are used, for example, for assembling, welding, and transporting electronic and mechanical parts, thereby improving the efficiency and automation of factory production lines.

[0003] For example, in the case of a welding robot, a program for performing a desired operation must be created and stored in advance as so-called teaching data. The teaching data is generated by an operator using a teaching pendant in cooperation with a robot control device to operate the actual robot and record the operations.

[0004] Creating teaching data while an operator actually operates a robot using a teaching pendant is highly dependent on the operator's skill and can take a long time. For this reason, technology has been disclosed for a welding device that detects the welding position, determines the welding conditions, and the position and posture of the welding torch, and performs automatic welding (for example, Patent Document 1).

[0005] In the technology disclosed in Patent Document 1, shape data representing the outline of the welding object is extracted using a shape sensor configured by combining an irradiation device and an imaging device, and further, boundaries of multiple surfaces (for example, top surface, groove surface, and bead surface) corresponding to change points in the shape data are extracted.The bead surface and groove surface are then identified, and welding conditions, welding target position, etc. are determined to perform automatic welding. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] Patent No. 5847697 Summary of the Invention [Problem to be solved by the invention]

[0007] However, in the technology disclosed in Patent Document 1, the welding position is detected based on shape data representing the contour of the welding object measured by a shape sensor, but the shape sensor is configured by combining an irradiation device and an imaging device. Therefore, in order to extract the boundary (welding position) between multiple surfaces, the shape sensor must be placed at an appropriate position and distance from the welding position, and since measurements are required at multiple locations, it takes time to detect the welding position. In other words, the shape sensor must be placed close to the welding position and measure the area around the welding position with high accuracy to detect the change point where the angle of the contour of the welding object included in the shape data suddenly changes.

[0008] Another possible method is to use a 3D camera to detect the entire workpiece and obtain the welding position from there, but in this case, the shape of the workpiece and the installed jig are also obtained, so points where the shape changes suddenly will be detected even outside the welding position. Furthermore, depending on the granularity of the 3D camera, measurement data at positions where the shape changes significantly in butt shapes such as horizontal fillets may be smoothed, making it impossible to detect the angle change points.

[0009] Therefore, an object of the present invention is to provide a welding position detection device that can easily and appropriately detect a welding position, and a robot teaching system using the same. [Means for solving the problem]

[0010] A welding position detection device according to one embodiment of the present invention includes a reference plane detection unit that detects a reference plane based on point cloud data acquired by a 3D camera, a specific point cloud data extraction unit that extracts specific point cloud data at a position away from the reference plane from the point cloud data constituting a component that is arranged in contact with the reference plane, a specific point cloud data projection unit that projects the specific point cloud data onto the reference plane, and a welding position determination unit that determines the position of the specific point cloud data projected onto the reference plane as the welding position.

[0011] According to this aspect, the reference plane detection unit detects a reference plane based on point cloud data acquired by the 3D camera, and the specific point cloud data extraction unit extracts specific point cloud data at positions away from the reference plane from point cloud data constituting a component placed in contact with the reference plane. The point cloud data away from the reference plane extracted by the specific point cloud data extraction unit can be appropriately extracted without imposing strict conditions on the performance or installation position of the 3D camera. The specific point cloud data projection unit then projects the specific point cloud data onto the reference plane, and the welding position determination unit determines the position on the reference plane of the specific point cloud data projected onto the reference plane as the welding position. This allows for easy and appropriate detection of the welding position.

[0012] In the above aspect, the specific point cloud data projection unit may project the specific point cloud data onto the reference surface by moving the specific point cloud data along the member in the reference surface direction.

[0013] According to this aspect, the specific point cloud data projection unit projects the specific point cloud data onto the reference surface by moving the specific point cloud data along the member in the reference surface direction, so that even if the member has a curved shape such as a cylinder, the specific point cloud data can be appropriately projected onto the reference surface. As a result, the welding position can be appropriately detected regardless of the shape of the member.

[0014] In the above aspect, the specific point cloud data projection unit may project the specific point cloud data onto the reference plane by moving the specific point cloud data in a direction perpendicular to the reference plane.

[0015] According to this aspect, the specific point cloud data projection unit projects the specific point cloud data onto the reference plane by moving the specific point cloud data in a direction perpendicular to the reference plane, so that, for example, it is possible to more appropriately detect the welding position for a member arranged to stand upright on a flat plate. Also, in some cases, it is possible to simplify the calculation process.

[0016] In the above aspect, the welding position determination unit may further include a welding area estimation unit that estimates the welding area based on the shape of the component, and the welding position determination unit may determine the position of the specific point cloud data projected onto the reference plane as the welding position, and may complement the entire welding position from the determined welding position based on the estimated welding area.

[0017] According to this aspect, the welding area estimation unit estimates the welding area based on the shape of the member, and the welding position determination unit determines the position on the reference plane of the specific point cloud data projected onto the reference plane as the welding position, and interpolates the entire welding position from the determined welding position based on the estimated welding area. This makes it possible to properly detect the welding position even in an area that could not be photographed due to the positional relationship between the imaging unit and the workpiece to be welded.

[0018] A robot teaching system according to one embodiment of the present invention includes an imaging unit having a 3D camera that captures images including an object to be welded; a reference plane detection unit that detects a reference plane based on point cloud data acquired by the imaging unit; a specific point cloud data extraction unit that extracts specific point cloud data at a position away from the reference plane from the point cloud data constituting a component that is arranged in contact with the reference plane; a specific point cloud data projection unit that projects the specific point cloud data onto the reference plane; a welding position determination unit that determines the position of the specific point cloud data projected onto the reference plane as a welding position; and a program generation unit that generates a work program for operating a manipulator to arc weld at the determined welding position.

[0019] According to this aspect, the photographing unit has a 3D camera that photographs an image including the welding object, the reference plane detection unit detects the reference plane based on the point cloud data acquired by the photographing unit, and the specific point cloud data extraction unit extracts specific point cloud data at a position away from the reference plane from the point cloud data constituting the member disposed in contact with the reference plane. The specific point cloud data projection unit projects the specific point cloud data onto the reference plane, and the welding position determination unit determines the position on the reference plane of the specific point cloud data projected onto the reference plane as the welding position. The program generation unit generates an operation program for operating the manipulator to arc-weld to the determined welding position. This allows for easy and appropriate detection of the welding position, resulting in more appropriate arc welding. [Effects of the Invention]

[0020] According to the present invention, it is possible to provide a welding position detection device that can easily and appropriately detect a welding position, and a robot teaching system using the same. [Brief explanation of the drawings]

[0021] [Figure 1] 1 is a diagram illustrating an example of the configuration of a welding robot system 100 including a robot teaching system according to an embodiment of the present invention. [Figure 2] FIG. 1 is a diagram illustrating an example of a functional configuration of a robot teaching system 200 according to an embodiment of the present invention. [Figure 3A] 10 is a diagram showing a specific example of how a welding position is detected by a welding position detection device 220 when the reference surface is a flat surface. FIG. [Figure 3B] 10 is a diagram showing another specific example in which the welding position is detected by the welding position detection device 220 when the reference surface is a flat surface. FIG. [Figure 4A] 10 is a diagram showing a specific example of how a welding position is detected by a welding position detection device 220 when the reference surface is a curved surface. FIG. [Figure 4B] 10 is a diagram showing another specific example of how the welding position is detected by the welding position detection device 220 when the reference surface is a curved surface. FIG. [Figure 5] 10 is a flowchart showing the flow of processing in a welding position detection method M100 executed by a welding position detection device 220 according to one embodiment of the present invention. [Figure 6] 10 is a diagram showing a specific example of how a welding position is detected by a welding position detection device 220 when a vertically standing surface is used as a reference surface. FIG. [Figure 7] FIG. 10 is a diagram showing a specific example of how a welding position is detected by a welding position detection device 220 when a surface with a small area is used as a reference surface. [Figure 8] 10 is a diagram showing a specific example in which a plurality of welding positions are detected by the welding position detection device 220 when the reference surface is a flat surface. FIG. DETAILED DESCRIPTION OF THE INVENTION

[0022] Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings. Note that the embodiment described below is merely a specific example for carrying out the present invention and is not intended to limit the present invention. Furthermore, to facilitate understanding of the description, the same components in each drawing will be assigned the same reference numerals as much as possible, and duplicate descriptions may be omitted.

[0023] <One embodiment> [Basic configuration of welding robot system] FIG. 1 is a diagram illustrating the configuration of a welding robot system 100 including a robot teaching system according to one embodiment of the present invention. As shown in FIG. 1, the welding robot system 100 includes, for example, an image capturing terminal 1, a robot control device 2, and a manipulator 3. The image capturing terminal 1 and the robot control device 2 are connected, for example, via a network N, and the robot control device 2 and the manipulator 3 are connected, for example, via a communication cable C. The network N may be wired (including a communication cable) or wireless. The welding robot system 100 may also include a teaching pendant. The teaching pendant is an operating device that allows an operator to teach the manipulator 3 how to operate.

[0024] The manipulator 3 is a welding robot (industrial robot) that performs arc welding in accordance with working conditions set in the robot control device 2. The manipulator 3 has, for example, an articulated arm 31 provided on a base member fixed to the floor of a factory or the like, and a welding torch 32 (end effector) connected to the tip of the articulated arm 31.

[0025] The robot control device 2 is a control unit that controls the operation of the manipulator 3, and includes, for example, a control unit 21, a storage unit 22, a communication unit 23, and a welding power supply unit 24.

[0026] The control unit 21 controls the manipulator 3 and the welding power supply unit 24 by, for example, causing a processor to execute an operation program stored in the storage unit 22.

[0027] The communication unit 23 controls communication with the photographing terminal 1 connected via the network N, and controls communication with the manipulator 3 connected via the communication cable C.

[0028] Welding power supply unit 24 supplies welding current, welding voltage, and the like to manipulator 3 in accordance with predetermined welding conditions, for example, to generate an arc between the tip of the welding wire and the workpiece. The welding conditions include data items such as welding conditions, welding start position, welding end position, arc discharge time, welding distance, welding torch attitude, and welding torch movement speed. Welding power supply unit 24 may be provided separately from robot control device 2.

[0029] The photographing terminal 1 is a 3D camera that acquires coordinate data corresponding to a photographed object and uses point cloud data to grasp the shape of the photographed object. For example, the coordinate data corresponding to the photographed object may be acquired by calculating it based on a plurality of images of the photographed object taken from a plurality of different positions. In this case, a three-dimensional measurement method using a known stereo method may be used. Alternatively, a distance measurement sensor such as a LiDAR (Light Detection and Ranging) sensor, a millimeter wave sensor, or an ultrasonic sensor may be used, and the shape of the photographed object may be grasped by irradiating the photographed object with laser light and acquiring point cloud data based on the reflected light.

[0030] The 3D camera may be a portable terminal equipped with a 3D camera. Portable terminals include, for example, tablet terminals, smartphones, personal digital assistants (PDAs), notebook PCs (personal computers), and other portable terminals. The photographing terminal 1 includes, for example, a control unit 11, a photographing unit 12, a communication unit 13, and a display unit 14.

[0031] The control unit 11 controls each unit of the photographing terminal 1 by the processor executing a predetermined program stored in the memory.

[0032] As described above, the photographing unit 12 acquires, as point cloud data, coordinate data acquired using a three-dimensional measurement technique such as a known stereo method, or coordinate data acquired by 3D scanning using a distance measurement sensor, etc. Note that the point cloud data may include, for example, position (distance) information represented by three-dimensional coordinate values ​​(X, Y, Z) and color information represented by (R, G, B).

[0033] The communication unit 13 controls communication with the robot control device 2 connected via the network N.

[0034] The display unit 14 is, for example, a display having a touch panel, and displays the shape of the subject based on the point cloud data acquired by the photographing unit 12, and accepts input of operation instructions and the like from the worker. The display unit 14 may be provided separately from the photographing terminal 1, for example, as a display device having a touch panel.

[0035] [Robot teaching system configuration] 2 is a diagram illustrating a functional configuration of a robot teaching system 200 according to one embodiment of the present invention. As shown in FIG. 2, the robot teaching system 200 has, as its functional configuration, for example, an imaging unit 211 and a program generation unit 212, and further includes a welding position detection device 220 for detecting a welding position based on point cloud data acquired by the imaging unit 211. The welding position detection device 220 includes a reference plane detection unit 221, a specific point cloud data extraction unit 222, a specific point cloud data projection unit 223, and a welding position determination unit 224.

[0036] Of these functions, the photographing unit 211 is a function possessed by the photographing terminal 1. On the other hand, the program generating unit 212 and each unit in the welding position detecting device 220 may be entirely provided by either the photographing terminal 1 or the robot control device 2, or each function may be distributed and provided in the photographing terminal 1 and the robot control device 2. Furthermore, some or all of the above functions may be provided by devices other than the photographing terminal 1 and the robot control device 2.

[0037] The photographing unit 211 is the same as the photographing unit 12 of the photographing terminal 1, and acquires point cloud data as a function of a 3D camera. The photographing unit 211 acquires point cloud data that includes at least the workpieces to be welded.

[0038] The program generating unit 212 generates an operation program for operating the manipulator to perform arc welding at the welding position detected by the welding position detecting device 220 .

[0039] The welding position detection device 220 detects the welding position based on the point cloud data acquired by the imaging unit 211. The function of each unit in the welding position detection device 220 will be described below.

[0040] The reference surface detection unit 221 detects a reference surface to be used as a reference based on the point cloud data acquired by the imaging unit 211 (3D camera). For example, the reference surface may be a flat surface or a curved surface, and may be detected by having the user select it. The user may designate any one of the surfaces of the workpiece to be welded as the reference surface, for example, the surface with the largest area, or a surface that includes or is close to the center of the field of view.

[0041] Furthermore, the reference plane detection unit 221 may automatically detect the plane with the largest area, or the plane that includes the center of the field of view or is close to the center of the field of view, based on point cloud data acquired by a 3D camera.

[0042] Note that the surface detected as the reference surface by the reference surface detection unit 221 and the means for detection are not limited to these, and for example, a surface on which multiple parts are arranged may be detected as the reference surface, or data may be accumulated and an appropriate reference surface may be detected using AI or the like.

[0043] The specific point cloud data extraction unit 222 extracts specific point cloud data at positions away from the reference plane from among the point cloud data of components placed in contact with the reference plane. For example, the specific point cloud data extraction unit 222 extracts point cloud data of components placed in contact with the reference plane detected by the reference plane detection unit 221 as specific point cloud data. The specific point cloud data extraction unit 222 may extract point cloud data of the components that are located approximately 5 mm to 15 mm away from the reference plane. If an attempt is made to directly extract a change point (such as a boundary between multiple surfaces) where the shape changes abruptly from the point cloud data as the welding position, for example, a part of a base or jig unrelated to the welding position may be detected. Furthermore, depending on the roughness (resolution, number of pixels) of the point cloud data and the direction of photography by the imaging unit 211, the change point may not be detected. On the other hand, the point cloud data approximately 5 mm to 15 mm away from the reference plane extracted by the specific point cloud data extraction unit 222 can be appropriately extracted without imposing strict conditions on the performance or installation position of the 3D camera.

[0044] Note that the closer the extracted specific point cloud data is to the reference plane, the less likely an error will occur when determining the welding position in projection, which will be described later, but point cloud data configured near the reference plane may make it difficult to distinguish between the reference plane and the component. Therefore, the extent to which point cloud data that is far from the reference plane should be extracted as specific point cloud data can be set depending on the shape of the workpiece to be welded, the coarseness (resolution, number of pixels) of the point cloud data acquired by the imaging unit 211, etc.

[0045] Here, the point cloud data extracted as the specific point cloud data may be point cloud data at a position a certain distance (for example, 10 mm) away from the reference plane. Also, the point cloud data may be extracted so that the distance from the reference plane has a predetermined range depending on the number of point cloud data to be extracted, depending on the roughness (resolution, number of pixels) of the point cloud data acquired by the imaging unit 211. Specifically, for example, point cloud data at a position 9 mm to 11 mm away from the reference plane may be extracted.

[0046] The specific point cloud data projection unit 223 projects the specific point cloud data extracted by the specific point cloud data extraction unit 222 onto the reference plane detected by the reference plane detection unit 221. Specifically, the specific point cloud data projection unit 223 moves each piece of specific point cloud data, which is point cloud data constituting a component placed in contact with the reference plane, along the component in the reference plane direction, and projects it to a position intersecting with the reference plane.

[0047] Typically, the specific point cloud data projection unit 223 moves each piece of specific point cloud data in the direction of the reference plane along the side of the rectangular parallelepiped if the component is a rectangular parallelepiped, or along the side of the cylinder if the component is a cylinder. For a component arranged to stand perpendicular to the plate, if the reference plane is a flat surface, the specific point cloud data projection unit 223 projects the specific point cloud data onto the reference plane by moving the specific point cloud data in the direction perpendicular to the reference plane.

[0048] The welding position determination unit 224 determines, as the welding position, the position on the reference plane of the specific point cloud data projected onto the reference plane by the specific point cloud data projection unit 223. For example, the welding position determination unit 224 may connect the positions on the reference plane of each of the specific point cloud data projected onto the reference plane to form a welding line. Here, the welding line may be generated so as to be a straight line or a curve using an approximation formula (e.g., the least squares method, etc.) for the positions on the reference plane of each of the specific point cloud data projected onto the reference plane, or the welding line may be determined based on the shape of the members.

[0049] Furthermore, a welding area estimation unit may be provided that estimates the welding area in an area that could not be photographed by the photographing unit 211 due to the positional relationship between the photographing unit 211 and the workpieces to be welded. For example, if the photographing unit 211 photographs the workpieces to be welded from one direction and acquires point cloud data, the welding position on the photographing unit 211 side can be determined as described above, but point cloud data cannot be acquired for the back side of the workpiece as seen from the photographing unit 211. In other words, specific point cloud data cannot be extracted for the back side of the workpiece as described above, and the welding position cannot be determined by projecting it onto a reference plane. Therefore, the welding area estimation unit may estimate the welding area based on the welding position determined by the welding position determination unit 224 described above, the positional relationship between the photographing unit 211 and the workpieces to be welded, the shape of the components, etc.

[0050] The welding position determination unit 224 determines the position on the reference plane of the specific point cloud data projected onto the reference plane by the specific point cloud data projection unit 223 as the welding position, and complements the entire welding position from the determined welding position based on the welding range estimated by the welding range estimation unit.

[0051] Typically, if the welding objects are a flat plate (reference surface) and a rectangular parallelepiped (component), the specific point cloud data on the reference surface projected onto the reference surface by the specific point cloud data projection unit 223 represents a portion of the welding position (rectangle), and the welding area estimation unit estimates the area that could not be imaged by the imaging unit 211 (the back side of the component) based on the shape of the component (rectangle). Also, if the welding objects are a flat plate (reference surface) and a cylinder (component), the specific point cloud data on the reference surface projected onto the reference surface by the specific point cloud data projection unit 223 represents a portion of the welding position (circle), and the welding area estimation unit estimates the area that could not be imaged by the imaging unit 211 (the back side of the component) based on the shape of the component (cylinder). Then, the welding position determination unit 224 may determine the entire welding position based on the position of the specific point cloud data on the reference surface projected onto the reference surface by the specific point cloud data projection unit 223 and the welding area estimated by the welding area estimation unit.

[0052] [Example of detecting welding positions] Furthermore, the manner in which the welding position is detected by the welding position detection device 220 will be described in detail below with reference to a specific example.

[0053] 3A is a diagram showing a specific example in which, when the reference surface is a plane, the welding position is detected by the welding position detection device 220. As shown in FIG. 3A, the workpieces to be welded are a plate-shaped flat plate and a rectangular parallelepiped arranged approximately perpendicular to the plate, and these are to be welded.

[0054] In FIG. 3A(a), point cloud data PG1 is acquired by the imaging unit 211 (3D camera).

[0055] The reference plane detection unit 221 detects a reference plane BS1 based on the point cloud data PG1 (FIG. 3A(b)). Here, the reference plane detection unit 221 automatically detects the top surface of a horizontally placed flat plate that has the largest area as the reference plane BS1.

[0056] The specific point cloud data extraction unit 222 extracts specific point cloud data PG11 at a position away from the reference plane BS1 from the point cloud data PG10 constituting the component arranged in contact with the reference plane BS1 (FIG. 3A(c)).

[0057] 3A(c) shows the specific point cloud data PG11 in more detail. The specific point cloud data extraction unit 222 may extract point cloud data located on a straight line a certain distance away from the reference plane BS1 as the specific point cloud data. However, here, the specific point cloud data PG11 is extracted as point cloud data located at a distance from the reference plane BS1 with a predetermined range of width. The width of the predetermined range may be set according to the coarseness (resolution, number of pixels) of the point cloud data acquired by the imaging unit 211.

[0058] The specific point cloud data projection unit 223 moves the specific point cloud data PG11 along the side surface of the point cloud data PG10 constituting the component in the direction of the reference plane BS1, and projects it onto a position PG12 where it intersects with the reference plane BS1 (FIG. 3A(d)). Here, the specific point cloud data projection unit 223 moves the specific point cloud data PG11 in a direction perpendicular to the reference plane BS1, thereby projecting the specific point cloud data PG11 onto a position PG12 on the reference plane BS1.

[0059] Here, the position PG12 on the reference plane BS1 is shown in more detail in the enlarged view of portion d1 in FIG. 3A(d). As shown in the enlarged view of portion c1 above, the specific point cloud data projection unit 223 projects each of the specific point cloud data PG11 located at a distance from the reference plane BS1 with a predetermined range of width onto the reference plane BS1, which is a coplanar surface, as shown in the enlarged view of portion d1. In other words, each of the specific point cloud data PG11 is moved toward the reference plane BS1, but not all by the same distance. Instead, each of the specific point cloud data PG11 is moved until it intersects with the reference plane BS1. This results in the specific point cloud data PG11 being more densely arranged at position PG12 on the reference plane BS1.

[0060] The welding position determining unit 224 determines a welding line WL12 based on a position PG12 on the reference plane BS1 obtained by projecting the specific point cloud data PG11 onto the reference plane BS1 (FIG. 3A(e)).

[0061] 3A(e), the weld line WL12 on the reference plane BS1 is shown in more detail in the enlarged view of portion e1. As shown in the enlarged view of portion d1, the welding position determination unit 224 determines the position PG12 on the reference plane BS1 by using, for example, an approximation formula, so that the weld line WL12 is a straight line, as shown in the enlarged view of portion e1.

[0062] Furthermore, the welding area estimation unit estimates a welding line WL12' on the back side of the workpiece to be welded, based on the welding line WL12 determined by the welding position determination unit 224, the positional relationship between the imaging unit 211 and the workpiece to be welded, the shape of the components, etc. Then, the welding position determination unit 224 determines the entire welding position based on the welding line WL12 and the welding line WL12' (FIG. 3A(f)).

[0063] 3B is a diagram showing another specific example in which the welding position is detected by the welding position detection device 220 when the reference surface is a plane. As shown in FIG. 3B, the workpieces to be welded are a plate-shaped flat plate and a cylinder arranged approximately perpendicular to the plate, and these are to be welded. FIG. 3B differs from FIG. 3A in that the member arranged approximately perpendicular to the plate is a cylinder.

[0064] In FIG. 3B(a), point cloud data PG2 is acquired by the imaging unit 211 (3D camera).

[0065] The reference plane detection unit 221 detects a reference plane BS2 based on the point cloud data PG2 (FIG. 3B(b)).

[0066] The specific point cloud data extraction unit 222 extracts specific point cloud data PG21 at a position away from the reference plane BS2 from the point cloud data PG20 constituting the member placed in contact with the reference plane BS1 (FIG. 3B(c)). Note that since the member is a cylinder with curved sides, the specific point cloud data PG21 extracted by the specific point cloud data extraction unit 222 is extracted along the shape of the side (curved surface) of the cylinder.

[0067] The specific point cloud data projection unit 223 moves the specific point cloud data PG21 along the point cloud data PG20 constituting the component in the direction of the reference plane BS2, and projects it onto a position PG22 where it intersects with the reference plane BS2 (FIG. 3B(d)).

[0068] As described above, the specific point group data PG21 is extracted along the side (curved) shape of the cylinder, so the shape of the position PG22 on the reference plane BS2 projected onto the reference plane BS2 is roughly a part of a circle (arc shape).

[0069] The welding position determination unit 224 determines the welding line WL22 based on the position PG22 on the reference plane BS2 obtained by projecting the specific point cloud data PG21 onto the reference plane BS2 (FIG. 3B(e)). For example, the welding position determination unit 224 may determine the welding line WL22 so as to be an arc using an approximation formula based on the shape of the position PG22 on the reference plane BS2 projected onto the reference plane BS2.

[0070] Furthermore, the welding area estimation unit estimates a welding line WL22' on the back side of the workpiece to be welded, and the welding position determination unit 224 determines the entire welding position based on the welding line WL22 and the welding line WL22' (FIG. 3B(f)).

[0071] Next, a case will be described in which the welding surface of the workpiece to be welded is a curved surface and the curved surface is detected as the reference surface described above.

[0072] Fig. 4A is a diagram showing a specific example in which the welding position is detected by the welding position detection device 220 when the reference surface is a curved surface. As shown in Fig. 4A, the workpieces to be welded are a lying cylinder and a cylinder arranged so as to stand upright and approximately perpendicular to the side surface of the lying cylinder, and these are to be welded. Fig. 4A differs from Fig. 3B in that the cylinder is not a flat plate but is arranged so as to stand upright and approximately perpendicular to the side surface of the lying cylinder.

[0073] In FIG. 4A(a), point cloud data PG3 is acquired by the imaging unit 211 (3D camera).

[0074] The reference plane detection unit 221 detects a reference plane BS3 based on the point cloud data PG3 (FIG. 4A(b)). Here, the reference plane BS3 is not a flat surface as in FIGS. 3A and 3B, but is a curved surface, that is, the side surface of a cylinder.

[0075] The specific point cloud data extraction unit 222 extracts specific point cloud data PG31 at a position away from the reference plane BS3 from the point cloud data PG30 constituting the member placed in contact with the reference plane BS3 (FIG. 4A(c)). Note that since the member is a cylinder with curved sides, the specific point cloud data PG31 extracted by the specific point cloud data extraction unit 222 is extracted along the shape of the side (curved surface) of the cylinder.

[0076] The specific point cloud data projection unit 223 moves the specific point cloud data PG31 along the point cloud data PG30 constituting the component in the direction of the reference plane BS3, and projects it onto a position PG32 intersecting with the reference plane BS3 (FIG. 4A(d)).

[0077] As described above, the specific point group data PG31 is extracted along the side (curved) shape of the cylinder, so that a portion of the roughly circular shape (arc shape) is further projected onto the curved reference surface BS3 to form a position PG32 on the reference surface BS3.

[0078] The welding position determination unit 224 determines the welding line WL32 based on the position PG32 on the reference plane BS3 obtained by projecting the specific point cloud data PG31 onto the reference plane BS3 (FIG. 4A(e)). For example, the welding position determination unit 224 may use an approximation formula based on the shape of the position PG32 on the reference plane BS3 projected onto the reference plane BS3 to determine the welding line WL32 so that it has a shape formed by projecting an arc onto the curved reference plane BS3.

[0079] Furthermore, the welding area estimation unit estimates a welding line WL32' on the back side of the workpiece to be welded, and the welding position determination unit 224 determines the entire welding position based on the welding line WL32 and the welding line WL32' (FIG. 4A(f)).

[0080] 4A(d), the specific point cloud data projection unit 223 moves the specific point cloud data PG31 in the direction of the reference plane BS3 along the point cloud data PG30 constituting the component, but it may also be projected by moving it in a direction perpendicular to the reference plane BS3. This simplifies the projection (movement direction) of the specific point cloud data PG31 and reduces the amount of calculation processing. Because the reference plane BS3 is a curved surface, the direction along the point cloud data PG30 constituting the component differs from the direction perpendicular to the reference plane BS3. However, if the distance from the reference plane BS3 is small, the error due to the deviation in the movement direction is also small. Therefore, for example, the projection (movement direction) processing may be selected depending on the required accuracy, etc.

[0081] 4B is a diagram showing another specific example in which the welding position is detected by the welding position detection device 220 when the reference surface is a curved surface. As shown in FIG. 4B, the workpieces to be welded are a sphere or a portion of a sphere and a cylinder arranged so as to stand upright and approximately perpendicular to the surface of the sphere, and these are to be welded. FIG. 4B differs from FIG. 4A in that the cylinder is arranged so as to stand upright and approximately perpendicular to the surface of the sphere or a portion of a sphere, rather than on the side of a lying cylinder.

[0082] In FIG. 4B(a), point cloud data PG4 is acquired by the imaging unit 211 (3D camera).

[0083] The reference plane detection unit 221 detects a reference plane BS4 based on the point cloud data PG4 (FIG. 4B(b)). Here, the reference plane BS4 is not a flat surface as in FIGS. 3A and 3B, but is detected as a curved spherical surface.

[0084] The specific point cloud data extraction unit 222 extracts specific point cloud data PG41 at a position away from the reference plane BS4 from the point cloud data PG40 constituting the member placed in contact with the reference plane BS4 (FIG. 4B(c)). Note that since the member is a cylinder with curved sides, the specific point cloud data PG41 extracted by the specific point cloud data extraction unit 222 is extracted along the shape of the side (curved surface) of the cylinder.

[0085] The specific point cloud data projection unit 223 moves the specific point cloud data PG41 along the point cloud data PG40 constituting the component in the direction of the reference plane BS4, and projects it onto a position PG42 where it intersects with the reference plane BS4 (FIG. 4B(d)). Alternatively, as described in FIG. 4A, the specific point cloud data PG41 may be projected by moving it in a direction perpendicular to the reference plane BS4.

[0086] As described above, the specific point group data PG41 is extracted along the side (curved) shape of the cylinder, so that a portion of the roughly circular shape (arc shape) is further projected onto the reference plane BS4, which is the surface of a sphere, to form a position PG42 on the reference plane BS4.

[0087] The welding position determination unit 224 determines the welding line WL42 based on the position PG42 on the reference plane BS4 obtained by projecting the specific point cloud data PG41 onto the reference plane BS4 (FIG. 4B(e)). For example, the welding position determination unit 224 may use an approximation formula based on the shape of the position PG42 on the reference plane BS4 projected onto the reference plane BS4 to determine the welding line WL42 so that the welding line WL42 has a shape formed by projecting an arc onto the reference plane BS4, which is the surface of a sphere.

[0088] Furthermore, the welding area estimation unit estimates a welding line WL42' on the back side of the workpiece to be welded, and the welding position determination unit 224 determines the entire welding position based on the welding line WL42 and the welding line WL42' (FIG. 4B(f)).

[0089] As described using Figures 3A, 3B, 4A and 4B, the welding position detection device 220 can appropriately detect the welding position even if the workpiece to be welded is a flat plate, a rectangular parallelepiped, a cylinder, a sphere, or a combination thereof, and the reference surface to be detected is a flat surface or a curved surface.

[0090] The workpieces to be welded are not limited to flat plates, rectangular parallelepipeds, cylinders, spheres, and combinations thereof, but may also be, for example, cubes, other polygonal prisms, cones, and polygonal pyramids.

[0091] [Welding position detection method] Next, a method for detecting a welding position by the welding position detecting device 220 according to one embodiment of the present invention will be specifically described in detail.

[0092] 5 is a flowchart showing the flow of processing of a welding position detection method M100 executed by the welding position detection device 220 according to one embodiment of the present invention. As shown in FIG. 5, the welding position detection method M100 includes steps S110 to S140, and each step is executed by a processor included in the welding position detection device 220.

[0093] In step S110, the welding position detection device 220 detects a reference plane to be used as a reference based on the point cloud data acquired by the photographing unit 211 (3D camera) (reference plane detection step). As a specific example, the reference plane detection unit 221 in the welding position detection device 220 detects, as the reference plane, a plane having the largest area or including or close to the center of the field of view, either by having the user select it or automatically.

[0094] In step S120, the welding position detection device 220 extracts specific point cloud data at positions away from the reference surface from the point cloud data constituting the member disposed so as to abut against the reference surface detected in step S110 (specific point cloud data extraction step). As a specific example, the specific point cloud data extraction unit 222 in the welding position detection device 220 may extract, from the point cloud data constituting the member, point cloud data that is away from the reference surface by about 5 mm to 15 mm, so as to have a width in a predetermined range, as the specific point cloud data.

[0095] In step S130, the welding position detection device 220 projects the specific point cloud data extracted in step S120 onto the reference plane detected in step S110 (specific point cloud data projection step). As a specific example, the specific point cloud data projection unit 223 in the welding position detection device 220 moves each piece of specific point cloud data, which is point cloud data constituting a member arranged to abut against the reference plane, along the side surface of the member toward the reference plane, and projects it to a position intersecting with the reference plane.

[0096] In step S140, welding position detection device 220 determines the position on the reference plane of the specific point cloud data projected onto the reference plane in step S130 as the welding position (welding position determination step). As a specific example, welding position determination unit 224 in welding position detection device 220 may determine a welding line to be a straight line or a curve using an approximation equation based on the position on the reference plane of each of the specific point cloud data projected onto the reference plane. Note that the welding area estimation unit may estimate the welding area, such as the back side of the workpiece, based on the welding line, the positional relationship between imaging unit 211 and the workpiece to be welded, the shape of the component, etc., and welding position determination unit 224 may complement the entire welding position based on these.

[0097] As described above, in the welding position detecting device 220 and welding position detecting method M100 according to one embodiment of the present invention, the reference plane detecting unit 221 detects the reference planes BS1-BS4 based on the point cloud data PG1-PG4 acquired by the 3D camera, and the specific point cloud data extracting unit 222 extracts the specific point cloud data PG11-PG41 at positions away from the reference planes BS1-BS4 from the point cloud data PG10-PG40 constituting the members arranged to abut on the reference planes BS1-BS4. The specific point cloud data projecting unit 223 then projects the specific point cloud data PG11-PG41 onto the reference planes BS1-BS4, and the welding position determining unit 224 determines the weld lines WL12-WL42 based on the positions PG12-PG42 on the reference planes BS1-BS4 of the specific point cloud data PG11-PG41 projected onto the reference planes BS1-BS4. Furthermore, the welding position determination unit 224 determines the entire welding position by complementing the welding lines WL12' to WL42' estimated by the welding range estimation unit, thereby making it possible to easily and appropriately detect the welding position.

[0098] Furthermore, according to the robot teaching system 200 using the welding position detecting device 220 according to one embodiment of the present invention, the program generating unit 212 generates an operation program for operating the manipulator to perform arc welding at the welding position, based on the welding position appropriately detected by the welding position detecting device 220. This allows the welding position to be detected easily and appropriately, and more appropriate arc welding to be performed.

[0099] In this embodiment, the manner in which the welding position detection device 220 detects the welding position has been described in detail using specific examples using Figures 3A, 3B, 4A, and 4B. However, in addition to these, for example, there may be cases in which specific point cloud data is projected horizontally or upward.

[0100] Fig. 6 is a diagram showing a specific example in which a welding position is detected by the welding position detection device 220 when a vertically standing surface is used as the reference surface. As shown in Fig. 6, the workpiece to be welded is made up of three plate-shaped flat plates arranged approximately perpendicular to each other, and these are to be welded.

[0101] In FIG. 6(a), point cloud data PG5 is acquired by the imaging unit 211 (3D camera).

[0102] The reference plane detection unit 221 detects a reference plane BS5 based on the point cloud data PG5 (FIG. 6(b)). Three flat plates with the same area are arranged, and in this case, the inner surface of a flat plate standing upright in the vertical direction is detected as the reference plane BS5 by the user's selection.

[0103] The specific point cloud data extraction unit 222 extracts specific point cloud data PG51A and PG51B at positions away from the reference plane BS5 from the point cloud data PG50A and PG50B constituting two components arranged in contact with the reference plane BS5 (Figure 6(c)).

[0104] The specific point cloud data projection unit 223 moves the specific point cloud data PG51A and PG51B along the point cloud data PG50A and PG50B that make up the two components in the direction of the reference plane BS5, and projects them onto positions PG52A and PG52B where they intersect with the reference plane BS5 (FIG. 6(d)). Here, the specific point cloud data projection unit 223 moves the specific point cloud data PG51A and PG51B in a direction perpendicular to the reference plane BS5, thereby projecting the specific point cloud data PG51A and PG51B onto positions PG52A and PG52B on the reference plane BS5.

[0105] The welding position determination unit 224 determines the welding lines WL52A and WL52B to be straight lines, for example, using an approximation formula, based on positions PG52A and PG52B on the reference plane BS5 obtained by projecting the specific point cloud data PG51A and PG51B onto the reference plane BS5 (Figure 6(e)).

[0106] Fig. 7 is a diagram showing a specific example in which a welding position is detected by the welding position detection device 220 when a surface with a small area is used as the reference surface. As shown in Fig. 7, the workpieces to be welded are a cylinder and a circular member placed on the top surface of the cylinder, and a case will be described in which these are welded.

[0107] In FIG. 7(a), point cloud data PG6 is acquired by the imaging unit 211 (3D camera).

[0108] The reference plane detection unit 221 detects a reference plane BS6 based on the point cloud data PG6 (FIG. 7(b)). A flat plate with the largest area is placed, but in this case, a cylinder and a circular member placed on the top surface of the cylinder are to be welded together, so the surface of the circular member placed on the top surface of the cylinder is detected as the reference plane BS6 by selection by the user.

[0109] The specific point cloud data extraction unit 222 extracts specific point cloud data PG61 at a position away from the reference plane BS6 from the point cloud data PG60 constituting the member placed in contact with the reference plane BS6 (FIG. 7(c)). Note that since the member is a cylinder with curved sides, the specific point cloud data PG61 extracted by the specific point cloud data extraction unit 222 is extracted along the shape of the side (curved surface) of the cylinder.

[0110] The specific point cloud data projection unit 223 moves the specific point cloud data PG61 along the point cloud data PG60 constituting the component in the direction of the reference plane BS6, and projects it onto a position PG62 where it intersects with the reference plane BS6 (FIG. 7(d)). In this case, since the reference plane BS6 is positioned above the component (cylinder), the specific point cloud data projection unit 223 moves the specific point cloud data PG61 upward and projects it onto a position PG62 on the reference plane BS6.

[0111] The welding position determining unit 224 determines the welding line WL62 to be an arc, for example, using an approximation formula based on the position PG62 on the reference plane BS6 obtained by projecting the specific point cloud data PG61 onto the reference plane BS6 (FIG. 7(e)).

[0112] Furthermore, the welding area estimation unit estimates a welding line WL62' on the back side of the workpiece to be welded, and the welding position determination unit 224 determines the entire welding position based on the welding line WL62 and the welding line WL62' (FIG. 7(f)).

[0113] Next, a case where a plurality of welding positions are detected will be described in detail with a specific example.

[0114] Fig. 8 is a diagram showing a specific example in which, when the reference surface is a plane, a plurality of welding positions are detected by the welding position detection device 220. As shown in Fig. 8, the workpieces to be welded are a plate-shaped flat plate, a rectangular parallelepiped arranged approximately perpendicular to the flat plate, a cylinder, and two connected cylinders (hereinafter referred to as connected cylinders), and these are to be welded.

[0115] In FIG. 8(a), point cloud data PG7 is acquired by the imaging unit 211 (3D camera).

[0116] Then, the reference plane detection unit 221 detects the top surface of the flat plate as the reference plane BS7, and the specific point cloud data extraction unit 222 and the specific point cloud data projection unit 223 project the specific point cloud data for the rectangular prism, cylinder, and connected cylinder onto positions on the reference plane BS7 (Figure 8(b)).

[0117] Here, PG72A, PG72B, and PG72C projected onto positions on reference plane BS7 are determined to be weld lines corresponding to individual workpieces to be welded, based on the shapes of the workpieces to be welded (rectangular parallelepiped, cylinder, and connected cylinder). That is, PG72A, PG72B, and PG72C projected onto positions on reference plane BS7 are grouped (clustered) without being connected. This grouping may be performed automatically based on the shapes of the workpieces to be welded, or may be specified by the user. Alternatively, data may be accumulated and the grouping may be performed using AI or the like.

[0118] Then, the welding position determination unit 224 determines the welding lines WL72A, WL72B, and WL72C based on the grouped PG72A, PG72B, and PG72C projected onto positions on the reference plane BS7, and further complements the welding lines WL72A', WL72B', and WL72C' estimated by the welding range estimation unit to determine the entire welding position (Figure 8(c)).

[0119] The above-described embodiments are intended to facilitate understanding of the present invention and are not intended to limit the present invention. The elements of the embodiments, as well as their arrangement, materials, conditions, shapes, sizes, etc., are not limited to those illustrated and can be modified as appropriate. Furthermore, configurations shown in different embodiments can be partially substituted or combined with each other. [Explanation of symbols]

[0120] 1...photography terminal, 2...robot control device, 3...manipulator, 11...control unit, 12...photography unit, 13...communication unit, 14...display unit, 21...control unit, 22...storage unit, 23...communication unit, 24...welding power supply unit, 31...articulated arm, 32...welding torch, C...communication cable, N...network, 100...welding robot system, 200...robot teaching system, 211...photography unit, 212...program generation unit, 220...welding position detection device, 221...reference plane detection unit, 222...specific point cloud data extraction unit, 223...specific point cloud data projection unit, 224...welding position determination unit, BS1 to BS7...reference plane, PG1 to PG7...point cloud data, PG10 to PG40, PG50A, PG50B, PG60...point cloud data, PG11 to PG41, PG51A, PG51B, PG61...specific point cloud data, PG12 to PG42, PG52A, PG52B, PG62...position on the reference surface, WL12 to WL42, WL52A, WL52B, WL62, WL72A, WL72B, W, L72C...weld lines, WL12' to WL42', WL62', WL72A', WL72B', WL72C'...weld lines, M100...weld position detection method, S110 to S140...each step of welding position detection method M100

Claims

1. a reference plane detection unit that detects a reference plane based on point cloud data acquired by the 3D camera; a specific point cloud data extraction unit that extracts specific point cloud data at a position away from the reference surface on a surface of a member that is placed in contact with the reference surface, from point cloud data that constitutes the member that is placed in contact with the reference surface; a specific point cloud data projection unit that projects the specific point cloud data onto the reference surface by moving the specific point cloud data along the member in the reference surface direction; a welding position determination unit that determines the position of the specific point cloud data projected onto the reference plane as a welding position, Welding position detection device.

2. the specific point cloud data projection unit projects the specific point cloud data onto the reference plane by moving the specific point cloud data in a direction perpendicular to the reference plane. The welding position detection device according to claim 1 .

3. a welding area estimation unit that estimates a welding area based on the shape of the member; the welding position determination unit determines a position of the specific point cloud data projected onto the reference plane as a welding position, and interpolates the entire welding position from the determined welding position based on the estimated welding range. The welding position detecting device according to claim 1 or 2.

4. an imaging unit having a 3D camera that captures an image including an object to be welded; a reference plane detection unit that detects a reference plane serving as a reference based on the point cloud data acquired by the photographing unit; a specific point cloud data extraction unit that extracts specific point cloud data at a position away from the reference surface on a surface of a member that is placed in contact with the reference surface, from point cloud data that constitutes the member that is placed in contact with the reference surface; a specific point cloud data projection unit that projects the specific point cloud data onto the reference surface by moving the specific point cloud data along the member in the reference surface direction; a welding position determination unit that determines the position of the specific point cloud data projected onto the reference plane as a welding position; a program generating unit that generates an operation program for operating the manipulator to perform arc welding at the determined welding position, Robot teaching system.

Images