Weld line detection system

Abstract

This invention provides a welding line detection system that improves the efficiency of teaching welding robot actions and enhances welding quality. It comprises: an imaging unit (211) for capturing an image of the welding object; a coordinate system setting unit (212) for setting a user coordinate system based on markers contained in the captured image; a point group data drawing unit (213) for detecting specific positions of markers in the image, setting the detected specific positions onto point group data obtained by a distance measurement sensor measuring the distance to the welding object, and drawing the point group data, assigned coordinates to the user coordinate system with the set specific positions as the origin, onto the user coordinate system; and a welding line detection unit (214) for detecting the welding line of the welding object based on the point group data drawn on the user coordinate system.

Description

Technical Field

[0001] This invention relates to a welding line inspection system. Background Technology

[0002] Patent Document 1 discloses a motion teaching system for teaching actions to a welding robot. In this system, during motion teaching, the welding torch located at the fingertips of the robotic arm is replaced by a camera. Furthermore, based on the position information of the welding area generated from the image captured by the camera, and the position and posture information of the robotic arm when the image was captured, a trajectory of the robotic arm's position and posture required for welding the welding area is generated. Teaching data is generated by synthesizing the trajectories from each welding area.

[0003] Existing technical documents

[0004] Patent documents

[0005] Patent Document 1: JP 2019-150930

[0006] However, in the motion teaching system of Patent Document 1, before motion teaching, a line (welding line) representing the welding part of the object set as the generation trajectory during motion teaching is drawn onto the object to be welded. Furthermore, this task is performed by the operator drawing the welding line onto the object to be welded with a pen. Therefore, not only is the task time-consuming, but sometimes the line drawn with a pen also remains on the surface of the object to be welded. Summary of the Invention

[0007] Therefore, the purpose of this invention is to provide a welding line inspection system that can improve the efficiency of teaching welding robot actions and enhance welding quality.

[0008] One aspect of the present invention relates to a weld line detection system comprising: an image capture terminal for capturing an image of a welding object; a coordinate system setting unit for setting a user coordinate system based on a marker contained in the captured image; a point group data plotting unit for detecting a specific position of a marker in the image, setting the detected specific position on point group data obtained by a distance measurement sensor measuring the distance to the welding object, and plotting the point group data, to which coordinates of the user coordinate system with the set specific position as the origin, onto the user coordinate system; and a weld line detection unit for detecting the weld line of the welding object based on the point group data plotted on the user coordinate system.

[0009] According to this method, a specific position of the mark can be detected based on an image obtained by photographing the welding object and the mark. The specific position of the detected mark is set on point group data, and the point group data, which is assigned coordinates of a user coordinate system with the set specific position of the mark as the origin, is drawn onto the image. The weld line of the welding object is detected based on this point group data. Therefore, the weld line of the welding object can be recognized even if the operator does not draw the weld line on the welding object.

[0010] In the above method, the welding line detection unit can also recognize multiple surfaces corresponding to the welding object based on point group data, and detect the intersection of two surfaces contained in the multiple surfaces as the welding line.

[0011] According to this method, the intersection of two surfaces represented by point group data drawn in the user coordinate system can be detected as a welding line, which can improve the detection accuracy of welding lines.

[0012] In the above method, the welding line detection unit may also detect at least a portion of the intersection line of the two surfaces as a welding line.

[0013] According to this method, at least a portion of the intersection line of two surfaces can be detected as a weld line, and no candidate weld lines can be missed.

[0014] In the above method, it may also include: a program creation unit that creates a welding operation program based on the welding line detected by the welding line detection unit.

[0015] According to this method, since a welding procedure can be created to weld the detected welding lines, welding can be performed efficiently along the welding lines specified by the procedure during welding.

[0016] In the above methods, the logo can also be an AR logo.

[0017] According to this method, when recognizing an AR sign, a program can be launched that aligns the user's coordinate system, with the location of the AR sign as the origin, with the actual image for display.

[0018] In the above-described manner, the shooting terminal may also include: an image sensor for capturing images; and a distance measurement sensor.

[0019] This method can fix the positional relationship between the image sensor and the distance measurement sensor, and can synchronize the timing of the data acquired by each sensor.

[0020] In the above-described manner, the shooting terminal may also include: a display unit that displays an image; and a control unit that controls the content displayed by the display unit, wherein the control unit causes the welding line detected by the welding line detection unit to coincide with the image and display it on the display unit.

[0021] According to this method, the operator of the camera terminal can easily determine whether there are any undetected solder lines while looking at the display of the camera terminal.

[0022] The effects of the invention

[0023] According to the present invention, a welding line inspection system can be provided that can improve the efficiency of teaching actions of welding robots and improve welding quality. Attached Figure Description

[0024] Figure 1 This is a diagram illustrating the structure of a welding robot system that includes the welding line inspection system according to the embodiments.

[0025] Figure 2 This is a diagram illustrating the functional structure of a welding line inspection system.

[0026] Figure 3 This is a diagram showing an example of a welding object.

[0027] Figure 4 This is a diagram illustrating an example of a user coordinate system with the location of the marker as its origin.

[0028] Figure 5 This is a diagram showing an example of a welding object.

[0029] Figure 6 This is a diagram representing an example of point group data plotted in a user coordinate system.

[0030] Figure 7 This is a flowchart illustrating an example of the actions taken when creating a work procedure in a welding line inspection system.

[0031] Explanation of reference numerals in the attached figures

[0032] 1...Shooting terminal, 2...Robot control device, 3...Robot arm, 11...Control unit, 12...Shooting unit, 13...Communication unit, 14...Display unit, 21...Control unit, 22...Storage unit, 23...Communication unit, 24...Welding power supply unit, 31...Multi-joint arm, 32...Welding torch, 100...Welding robot system, 211...Shooting unit, 212...Coordinate system setting unit, 213...Point group data plotting unit, 214...Welding line detection unit, 215...Programming unit, C...Communication cable, M...Marker, N...Network Detailed Implementation

[0033] Suitable embodiments of the invention will be described with reference to the accompanying drawings. Furthermore, elements labeled with the same reference numerals in the drawings have the same or identical structure. Also, since the drawings are schematic, the dimensions and proportions of the constituent elements differ from actual dimensions.

[0034] Figure 1 This diagram illustrates the structure of a welding robot system that includes the welding line inspection system according to the embodiments. The welding robot system 100 includes, for example, a camera terminal 1, a robot control device 2, and a robotic arm 3. The camera terminal 1 and the robot control device 2 are connected via, for example, a network N, and the robot control device 2 and the robotic arm 3 are connected via, for example, a communication cable C. The network N can be wired (including the communication cable) or wireless. Additionally, the welding robot system 100 may include a teach pendant. The teach pendant is an operating device used by the operator to teach the robotic arm 3 its movements.

[0035] The robotic arm 3 is a welding robot that performs arc welding according to the construction conditions set in the robot control device 2. The robotic arm 3 has, for example, a multi-joint arm 31 mounted on a base member fixed to the ground or the like in the factory; and a welding torch 32 connected to the front end of the multi-joint arm 31.

[0036] The robot control device 2 is a control component that controls the movement of the robotic arm 3, and includes, for example, a control unit 21, a storage unit 22, a communication unit 23, and a welding power supply unit 24.

[0037] The control unit 21 controls the robot arm 3 and the welding power supply unit 24 by executing the work program stored in the storage unit 22 through the processor, for example.

[0038] The communication unit 23 controls communication with the shooting terminal 1 connected via network N, or controls communication with the robotic arm 3 connected via communication cable C.

[0039] The welding power supply unit 24, for example, supplies welding current and welding voltage to the robot arm 3 according to predetermined welding conditions to generate an electric arc between the tip of the welding wire and the workpiece. The welding conditions include data such as welding conditions, welding start position, welding end position, arc discharge time, welding distance, torch posture, and torch movement speed. The welding power supply unit 24 can be separate from the robot control device 2.

[0040] The shooting terminal 1 is, for example, a digital camera, but it can also be a portable terminal with a built-in digital camera. Portable terminals include, for example, tablet computers, smartphones, portable digital assistants (PDAs), and laptop computers (PCs) that can be carried and moved. The shooting terminal 1 includes, for example, a control unit 11, a shooting unit 12, a communication unit 13, and a display unit 14.

[0041] The control unit 11 controls each part of the shooting terminal 1 by executing a given program stored in the memory through the processor.

[0042] The imaging unit 12 includes, for example, a lens and an image sensor (image sensor), which converts the light from the subject received by the lens into electrical signals (digital image data).

[0043] The communication unit 13 controls communication with the robot control device 2 connected via network N.

[0044] The display unit 14 is, for example, a display with a touch panel, which displays the image of the subject obtained by the shooting unit 12 and accepts input such as operation instructions from the operator. The display unit 14, for example, can be a display device with a touch panel and is provided separately from the shooting terminal 1.

[0045] Figure 2 This diagram illustrates the functional structure of the weld line detection system according to the present invention. The weld line detection system, as a functional structure, includes, for example, an imaging unit 211, a coordinate system setting unit 212, a point group data plotting unit 213, a weld line detection unit 214, and a program generation unit 215. The imaging unit 211 is a function possessed by the imaging terminal 1. On the other hand, the coordinate system setting unit 212, the point group data plotting unit 213, the weld line detection unit 214, and the program generation unit 215 can all be provided by either the imaging terminal 1 or the robot control device 2, or each function can be distributed among the imaging terminal 1 and the robot control device 2. Furthermore, other devices besides the imaging terminal 1 and the robot control device 2 can possess some or all of the above functions.

[0046] The imaging unit 211 is the same as the imaging unit 12 of the imaging terminal 1 described above. In this embodiment, the imaging unit 211, for example, takes a structure composed of multiple iron plate components (workpieces) that are the object of arc welding as the welding object for imaging. Figure 3 An example of a welding object is shown. In this figure, a structure consisting of one workpiece Wa serving as the base plate, two workpieces Wb and Wc serving as side plates, and one workpiece Wd serving as the back plate is shown as the welding object. The symbol M, which will be described later, is placed within the space formed by this structure.

[0047] Figure 2 The coordinate system setting unit 212 shown sets a user coordinate system with the position of the marker contained in the image of the shooting unit 211 as the origin. Figure 4 An example of a user coordinate system with the location of the marker as the origin is shown. In this figure, the location of the marker M is set as the origin O, and a three-dimensional orthogonal coordinate system based on the X-axis, Y-axis, and Z-axis that are orthogonal to each other at the origin O is displayed as the user coordinate system.

[0048] Furthermore, the origin of the user coordinate system only needs to be set based on a marker (such as a corner or center of the marker). The reason for setting the origin of the user coordinate system based on the marker, rather than on the camera terminal 1, is as follows: Since the camera terminal 1 is moved by the operator during shooting, it is difficult to determine its position in the robot coordinate system. In contrast, since the marker is fixed, it is easier to determine its position in the robot coordinate system. Therefore, setting the user coordinate system based on the marker makes it easier to calibrate the positional relationship between the user coordinate system and the robot coordinate system compared to setting it based on the camera terminal 1.

[0049] Here, the mark M need only be an identifier that enables the imaging unit 211 to recognize that it has been placed in the space. As a mark, for example, an AR mark is preferred. By using an AR mark, when the AR mark placed in the space is recognized, it is easy to make the user coordinate system with the AR mark as the origin coincide with the actual image for display.

[0050] A user coordinate system based on a sign can be set by moving the origin of the camera coordinate system (e.g., the center of the lens) to a specific position of the sign, as described later. Such a user coordinate system can be set, for example, using known techniques for setting the coordinate system of an AR sign.

[0051] Figure 2 The point group data drawing unit 213 shown obtains the coordinate data (point group data) corresponding to the welding object and draws the obtained coordinate data into the user coordinate system.

[0052] The specific details are explained below. The dot group data drawing unit 213 detects the specific position of a marker (e.g., a corner or center of the marker) based on the image captured by the imaging unit 211. It sets this detected specific position on the dot group data acquired by the distance measurement sensor (described later). The dot group data, assigned coordinates to a user coordinate system with the set specific position of the marker as its origin, is then drawn onto the user coordinate system. The specific position of the marker set on the dot group data can be automatically determined through data parsing, or it can be specified by the operator through instructions.

[0053] The coordinate data corresponding to the object being welded can be obtained, for example, by a distance measuring sensor. Any sensor capable of determining the distance to the object being welded can be used. Examples of distance measuring sensors include LiDAR (Light Detection and Ranging) sensors, millimeter-wave sensors, and ultrasonic sensors. Alternatively, the coordinate data corresponding to the object being welded can be estimated based on multiple images obtained from different locations. In this case, a three-dimensional measurement method based on known stereo methods can be used.

[0054] Here, a distance measurement sensor can be included in the imaging terminal 1. This allows for a fixed positional relationship between the image sensor and the distance measurement sensor, and enables timed alignment of data acquired by each sensor. Therefore, the processing efficiency of setting the specific position of the aforementioned marker on the point group data is improved. Furthermore, by incorporating both an image sensor and a distance measurement sensor into the imaging terminal 1, the operator of the imaging terminal 1 can freely move to any position where they can simultaneously photograph the welding line and the marker of the welding object, thus improving work efficiency.

[0055] Furthermore, a sensor that combines the functions of a sensor for capturing images and a sensor for measuring distance can be included in the imaging terminal 1. As a result, since an image containing the welding object and the distance to the welding object can be acquired from the same location and at the same time, the processing efficiency of setting the specific position of the aforementioned marker on the point group data can be further improved.

[0056] refer to Figure 5 as well as Figure 6 This explains the concept of plotting the coordinate data corresponding to the welding object as a group of points onto the user coordinate system.

[0057] Figure 5 This is a diagram illustrating an example of a welding object. The diagram illustrates a welding object consisting of a workpiece We placed on a workbench T and a workpiece Wf placed approximately perpendicular to workpiece We. The symbol M is placed next to the welding object.

[0058] Figure 6 This is a diagram representing an example of point group data plotted to a user coordinate system. In this diagram, [the data will be] compared to... Figure 5 The coordinate data Wec, Wfc, and Wf corresponding to workpieces We and Wf are respectively. Figure 5 The coordinate data Tc corresponding to the workbench T is used as point group data and plotted onto the user coordinate system.

[0059] Figure 2The weld line detection unit 214 shown detects the weld line of the welding object based on point group data drawn in the user coordinate system. Specifically, the weld line detection unit 214 recognizes multiple planes corresponding to the welding object based on the point group data, and detects the intersection of two planes contained in these multiple planes as weld lines. When there are multiple combinations of two planes, the weld line is detected for each of each combination. For example, in Figure 6 In this process, the plane represented by the coordinate data Wec corresponding to workpiece We and the plane represented by the coordinate data Wfc corresponding to workpiece Wf are identified, and the intersection of these two planes is detected as the welding line. Thus, as... Figure 5 As shown, the weld line L is detected as the intersection of workpiece Wf and workpiece We, and is displayed in a way that overlaps with the actual image. Furthermore, it is preferable to detect the weld line end-to-end as the end-to-end intersection of the two planes. The endpoints for detecting the weld line can be set under any conditions.

[0060] Here, the weld line is not limited to the intersection of two planes. For example, the intersection of two planes can be detected as a weld line.

[0061] In addition, when inspecting weld lines, one or both of the following (1) and (2) can be added to the conditions for inspecting weld lines.

[0062] (1) If the detected weld line is less than a certain length, the weld line is not used (ignore the weld line).

[0063] (2) Exclude a given length portion located at both ends of the detected weld line from the weld line (ignore a portion of the weld line).

[0064] Figure 2 The program creation unit 215 shown generates a work program for arc welding based on the welding line detected by the welding line detection unit 214. The program creation unit 215 stores the generated work program in the storage unit 22 of the robot control device 2. Thus, when the robot arm 3 performs arc welding, the control unit 21 of the robot control device 2 can read the work program and control the robot arm 3 according to the welding steps specified in the work program, so that welding is performed along the welding line specified in the work program.

[0065] Here, when the program creation unit 215 creates the work program, for example, it is preferable to perform calibration to make the robot coordinate system and the user coordinate system consistent with the robot 3 based on the position of the robot 3, or to specify the position information of the robot 3 on the user coordinate system, so as to establish a correspondence between the position on the user coordinate system and the position on the robot coordinate system.

[0066] refer to Figure 7This is an example of the actions taken by the program creation unit 215 when it creates the work program.

[0067] Initially, the program creation unit 215 determines the welding line in the user coordinate system based on the welding line detected by the welding line detection unit 214 (step S101).

[0068] Next, the program creation unit 215 sets the torch's attitude, such as the torch's advance angle, retreat angle, and target angle, for the welding line determined in step S101 (step S102).

[0069] Next, the program creation unit 215 determines the trajectory of the posture of each part of the robot arm 3 in the user coordinate system (step S103).

[0070] Next, the program creation unit 215 transforms the trajectory in the user coordinate system determined in step S103 into the trajectory in the robot coordinate system, and creates a work program (step S104). Then the operation ends.

[0071] Here, each step of the above actions can be performed in either the shooting terminal 1 or the robot control device 2, or a part of each step can be performed in the shooting terminal 1 and the remaining part can be performed in the robot control device 2.

[0072] As described above, the weld line detection system according to the embodiment can establish a user coordinate system with a specific position of the marker as the origin based on an image obtained by photographing the welding object and a marker placed in the space formed by the welding object. Point group data corresponding to the welding object is plotted in this user coordinate system, and the weld line of the welding object is detected based on this point group data. Therefore, the weld line of the welding object can be recognized even if the operator does not draw the weld line on the welding object.

[0073] Therefore, the welding line detection system according to the implementation method can improve the work efficiency of teaching the robot arm 3 and improve the welding quality.

[0074] Furthermore, since the imaging terminal 1 described in the aforementioned embodiment is a portable and movable terminal, the operator can photograph the welding object from any location, thus enabling the detection of weld lines located at the operator's desired position. When the position of the imaging terminal 1 is fixed, weld lines located in areas or ranges that cannot be photographed from that fixed position cannot be detected. However, welding operations sometimes involve drilling under the welding object. Therefore, by using the portable and movable imaging terminal 1, for example, the operator can move to a position where they can photograph weld lines located under the welding object, thereby detecting weld lines that the imaging terminal 1, located in a fixed position, cannot detect.

[0075] [Variation Example]

[0076] Furthermore, the present invention is not limited to the foregoing embodiments and can be implemented in various other forms without departing from the spirit of the invention. Therefore, the above embodiments are merely illustrative in all respects and are not intended to be limiting.

[0077] For example, the welding line detection unit 214 according to the aforementioned embodiment detects the welding line of the object to be welded, but it can also detect candidate welding lines. In this case, it is preferable to further include a welding line selection unit, which allows the operator to select the welding line to be actually welded from the detected candidate welding lines. In addition, it is preferable that the program creation unit 215 creates a work program for performing arc welding based on the welding line selected by the welding line selection unit.

[0078] Alternatively, the control unit 11 of the imaging terminal 1 according to the aforementioned embodiment can display the weld lines detected by the weld line detection unit 214 on the display unit 14, coinciding with the image captured by the imaging unit 12. This allows the operator of the imaging terminal 1 to easily determine whether there are any undetected weld lines while looking at the display unit 14. In other words, effective information for determining whether to re-encode the image using the imaging terminal 1 can be provided to the operator.

Claims

1. A welding line inspection system, characterized in that, have: The imaging terminal captures images of the object being welded. The coordinate system setting unit sets a user coordinate system based on the markers contained in the captured image; The dot group data drawing unit detects a specific position of the mark based on the image, sets the detected specific position on the dot group data obtained by a distance measurement sensor that measures the distance to the welding object, and draws the dot group data, which is assigned coordinates of the user coordinate system with the set specific position as the origin, onto the user coordinate system. The welding line detection unit detects the welding line of the welding object based on the point group data plotted in the user coordinate system.

2. The welding line inspection system according to claim 1, characterized in that, The welding line detection unit recognizes multiple surfaces corresponding to the welding object based on the point group data, and detects the intersection line of two surfaces contained in the multiple surfaces as the welding line.

3. The welding line inspection system according to claim 2, characterized in that, The welding line detection unit detects at least a portion of the intersection line of the two surfaces as the welding line.

4. The welding line inspection system according to claim 1 or 2, characterized in that, The welding line detection system also has the following features: The program creation unit creates a welding operation program based on the welding line detected by the welding line detection unit.

5. The welding line inspection system according to claim 1 or 2, characterized in that, The logo is an AR logo.

6. The welding line inspection system according to claim 1 or 2, characterized in that, The shooting terminal includes: an image sensor for capturing the image; and a distance measurement sensor.

7. The welding line detection system according to claim 6, characterized in that, The shooting terminal also has: A display unit that displays the image; and The control unit controls the content displayed on the display unit. The control unit displays the weld line detected by the weld line detection unit on the display unit so that it coincides with the image.

Images