Sign detection device and robot teaching system

Abstract

The present application provides a mark detection device capable of easily detecting the position of a mark, and a robot teaching system using the same. The mark detection device (220) includes: a three-dimensional plane detection unit (221) that detects a plane in a three-dimensional space, i.e., a three-dimensional plane, based on point cloud data acquired by a 3D camera; a two-dimensional plane image generation unit (222) that generates a two-dimensional plane image by projecting the point cloud data constituting the detected three-dimensional plane in a vertical direction with respect to the three-dimensional plane; a mark detection unit (223) that detects a mark from the generated two-dimensional plane image; and a mark position calculation unit (224) that calculates a three-dimensional coordinate in the three-dimensional plane for the mark contained in the detected two-dimensional plane image.

Description

Technical Field

[0001] This invention relates to a sign detection device and a robot teaching system. Background Technology

[0002] In recent years, robots have become increasingly prevalent in industry. These robots are used, for example, in the assembly, welding, and transportation of electronic and mechanical components, aiming to improve the efficiency and automation of factory production lines.

[0003] For example, in welding robots, a program needs to be created to perform the desired actions, and this program is pre-stored as so-called teach data. The operator uses a teach pendant and works with the robot control unit to operate the actual robot, recording its actions to generate the teach data.

[0004] In order to enable operators to actually make the robot move while generating teaching data using the teach pendant, the skill of the operator is highly dependent and sometimes takes a long time. In order to reduce the burden on the operator, a robot system that uses AR devices and markers to recognize the robot's position has been disclosed (e.g., Patent Document 1).

[0005] In the technology disclosed in Patent Document 1, an AR device is used to simultaneously detect two reference markers, thereby recognizing the positional relationship between the markers and the robot coordinate system, thus suppressing deviations in the display position of the AR graphic and recognizing the robot's position and orientation. In such a robot system, in order to accurately recognize the position and orientation of the workpiece and the robot, it is necessary to appropriately detect the reference markers.

[0006] Existing technical documents

[0007] Patent documents

[0008] Patent Document 1: JP2021-62463

[0009] However, in the technology disclosed in Patent Document 1, in the detection of marks, for example when using a two-dimensional camera, in order to calculate the three-dimensional position from the two-dimensional image, it is necessary to pre-set the camera parameters that determine the three-dimensional coordinates corresponding to each pixel of the two-dimensional image, thus requiring complicated pre-preparation. Summary of the Invention

[0010] Therefore, the object of the present invention is to provide a sign detection device that can easily detect the position of a sign and a robot teaching system using the same.

[0011] One aspect of the present invention relates to a sign detection apparatus comprising: a three-dimensional plane detection unit that detects a plane, i.e., a three-dimensional plane, in three-dimensional space based on point group data acquired by a 3D camera; a two-dimensional plane image generation unit that generates a two-dimensional plane image by projecting the point group data constituting the detected three-dimensional plane onto a vertical direction based on the three-dimensional plane; a sign detection unit that detects a sign from the generated two-dimensional plane image; and a sign position calculation unit that calculates the three-dimensional coordinates in the three-dimensional plane for the sign contained in the detected two-dimensional plane image.

[0012] According to this method, the three-dimensional plane detection unit detects a three-dimensional plane based on point group data acquired by a 3D camera, and the two-dimensional plane image generation unit generates a two-dimensional plane image by projecting the point group data constituting the three-dimensional plane onto a vertical direction based on the three-dimensional plane. Furthermore, the sign detection unit detects signs from the two-dimensional plane image, and the sign position calculation unit calculates the three-dimensional coordinates (X, Y, Z) in the three-dimensional plane for the signs contained in the two-dimensional plane image. Thus, the position of signs set on a plane can be accurately detected. Moreover, the position of signs set on a plane can be easily detected without complicated camera parameter settings, thus contributing to product cost reduction.

[0013] In the above method, it is also possible that the sign detection unit detects a sign on each of the three-dimensional planes when the three-dimensional plane detection unit detects multiple three-dimensional planes.

[0014] According to this method, the sign detection unit can detect signs on each of the multiple three-dimensional planes, thus enabling efficient and reliable sign detection.

[0015] In the above method, the sign detection unit can also detect signs in descending order of area across multiple three-dimensional planes.

[0016] According to this method, since the sign detection unit detects signs in descending order of area across multiple three-dimensional planes, it can process signs in descending order of perceived likelihood of them being set, and can expect to detect signs at an early stage. This helps to reduce computational processing and shorten processing time.

[0017] In the above method, the sign detection unit can also detect signs in order of distance from the center of the field of view to the farthest point in multiple three-dimensional planes.

[0018] According to this method, since the sign detection unit detects signs in multiple three-dimensional planes in order of distance from the center of the field of view from near to far, it processes signs in order of probability from high to low, and can expect to detect signs at an early stage. This helps to reduce computational processing and shorten processing time.

[0019] One aspect of the present invention relates to a robot teaching system comprising: a camera unit having a 3D camera for capturing images including a workpiece and a mark; a three-dimensional plane detection unit for detecting a plane in three-dimensional space, i.e., a three-dimensional plane, based on point group data acquired by the 3D camera; a two-dimensional plane image generation unit for generating a two-dimensional plane image by projecting the point group data constituting the detected three-dimensional plane onto a vertical direction based on the three-dimensional plane; a mark detection unit for detecting marks from the generated two-dimensional plane image; a mark position calculation unit for calculating the three-dimensional coordinates in the three-dimensional plane for the marks contained in the detected two-dimensional plane image; a camera coordinate system setting unit for setting a camera coordinate system based on the calculated three-dimensional coordinates of the marks; and a program generation unit for transforming the motion path of the robot arm in the camera coordinate system into a robot coordinate system set in a robot control device, and generating a work program for making the robot arm move.

[0020] According to this method, the 3D plane detection unit detects a 3D plane based on point group data acquired by a 3D camera, and the 2D plane image generation unit generates a 2D plane image by projecting the point group data constituting the 3D plane onto a vertical direction based on the 3D plane. Furthermore, the sign detection unit detects signs from the 2D plane image, and the sign position calculation unit calculates the 3D coordinates (X, Y, Z) in the 3D plane for the signs contained in the 2D plane image. The camera coordinate system setting unit sets the camera coordinate system based on the 3D coordinates of the signs calculated by the sign position calculation unit, and the program generation unit transforms the robot's motion path in the camera coordinate system into the robot coordinate system set in the robot control device, and generates a work program for moving the robot. As a result, since the camera coordinate system is set based on appropriately detected 3D coordinates of the signs, and the work program is generated, the robot can be moved more easily and appropriately.

[0021] The effects of the invention

[0022] According to the present invention, a sign detection device that can easily detect the position of a sign and a robot teaching system utilizing the same are provided. Attached Figure Description

[0023] Figure 1 This is a diagram illustrating the structure of a welding robot system 100 that includes a robot teaching system according to an embodiment of the present invention.

[0024] Figure 2 This is a diagram illustrating the functional structure of a robot teaching system 200 according to an embodiment of the present invention.

[0025] Figure 3This diagram illustrates the situation where the marker M is set on the workpiece W, which is the object of welding, and the camera coordinate system is set with the position of the marker M as the origin O.

[0026] Figure 4 This diagram illustrates the specific processing in the mark detection device 220 that detects the position of the mark M set on the workpiece W.

[0027] Figure 5 This is a flowchart illustrating the processing flow of the mark detection method M100 performed by the mark detection device 220 according to an embodiment of the present invention.

[0028] Explanation of reference numerals in the attached figures

[0029] 1...Shooting terminal, 2...Robot control device, 3...Manipulator, 11...Control unit, 12...Shooting unit, 13...Communication unit, 14...Display unit, 21...Control unit, 22...Storage unit, 23...Communication unit, 24...Welding power supply unit, 31...Multi-joint arm, 32...Welding torch, 100...Welding robot system, 200...Robot teaching system, 211...Shooting unit, 212...Camera coordinate system setting unit, 213...Program generation unit, 220...Mark detection device, 221...3D plane detection unit, 222...2D plane image generation unit, 223...Mark detection unit, 224...Mark position calculation unit, C...Communication cable, N...Network, M...Mark, W...Workpiece, L1~L3...Welding position, Wa, Wb...Plane, Wa2...Plane image, M100...Mark detection method, S110~S140...Steps of Mark Detection Method M100 Detailed Implementation

[0030] The embodiments of the present invention will be specifically described below with reference to the accompanying drawings. Furthermore, the embodiments described below are merely specific examples for carrying out the present invention and are not intended to limit the scope of the invention. In addition, to facilitate understanding, the same reference numerals are used as much as possible to denote the same constituent elements in the drawings, and sometimes repeated descriptions are omitted.

[0031] <One Implementation Method>

[0032] [Basic Structure of Welding Robot System]

[0033] Figure 1 This is a diagram illustrating the structure of a welding robot system 100 that includes a robot teaching system according to an embodiment of the present invention. Figure 1As shown, the welding robot system 100 includes, for example, a camera terminal 1, a robot control device 2, and a robotic arm 3. The camera terminal 1 and the robot control device 2 are connected, for example, via a network N, and the robot control device 2 and the robotic arm 3 are connected, for example, via a communication cable C. The network N can be wired (including the communication cable) or wireless. Additionally, the welding robot system 100 may include a teach pendant. The teach pendant is an operating device used by the operator to teach the robotic arm 3 its movements.

[0034] The robotic arm 3 is a welding robot (industrial robot) that performs arc welding according to the construction conditions set in the robot control device 2. The robotic arm 3 has, for example, a multi-joint arm 31 mounted on a base member fixed to the ground or the like in the factory; and a welding torch 32 (end effector) connected to the front end of the multi-joint arm 31.

[0035] The robot control device 2 is a control component that controls the movement of the robotic arm 3, and includes, for example, a control unit 21, a storage unit 22, a communication unit 23, and a welding power supply unit 24.

[0036] The control unit 21 controls the robot arm 3 and the welding power supply unit 24 by executing the work program stored in the storage unit 22 through the storage processor, for example.

[0037] The communication unit 23 controls communication with the shooting terminal 1 connected via network N, and controls communication with the robotic arm 3 connected via communication cable C.

[0038] The welding power supply unit 24, for example, supplies welding current and welding voltage to the robot arm 3 according to predetermined welding conditions to generate an electric arc between the tip of the welding wire and the workpiece. These welding conditions include data such as welding conditions, welding start position, welding end position, arc discharge time, welding distance, torch posture, and torch movement speed. The welding power supply unit 24 can be separately mounted from the robot control unit 2.

[0039] The imaging terminal 1 is a 3D camera that acquires coordinate data corresponding to the object being photographed and uses point group data to determine the shape of the object. For example, coordinate data corresponding to the object can be estimated based on multiple images taken from different locations. In this case, a 3D measurement method based on known stereo methods can be used. Alternatively, distance measurement sensors such as LiDAR (Light Detection and Ranging) sensors, millimeter-wave sensors, and ultrasonic sensors can be used. Alternatively, a laser can be irradiated onto the object, and point group data can be obtained based on the reflected light to determine the shape of the object.

[0040] Furthermore, the 3D camera can be a portable terminal equipped with a 3D camera. Portable terminals include, for example, tablet computers, smartphones, portable information terminals (PDAs), and laptop PCs (personal computers) that can be carried and moved. The shooting terminal 1 includes, for example, a control unit 11, a shooting unit 12, a communication unit 13, and a display unit 14.

[0041] The control unit 11 controls each part of the shooting terminal 1 by executing a given program stored in the memory through the processor.

[0042] As described above, the imaging unit 12 acquires coordinate data, for example, coordinate data obtained using a known stereoscopic method for three-dimensional measurement, or coordinate data obtained by 3D scanning using a distance measurement sensor, etc., and uses this data as point group data. Furthermore, the point group data may include, for example, position (distance) information expressed in three-dimensional coordinate values ​​(X, Y, Z) and color information expressed in (R, G, B).

[0043] The communication unit 13 controls communication with the robot control device 2 connected via network N.

[0044] The display unit 14 is, for example, a display with a touch panel, which displays the shape of the subject based on the dot group data acquired by the imaging unit 12, and accepts input such as operation instructions from the operator. The display unit 14 may be, for example, a display device with a touch panel, and may be separately provided from the imaging terminal 1.

[0045] [Structure of a Robot Teaching System]

[0046] Figure 2 This is a diagram illustrating the functional structure of a robot teaching system 200 according to an embodiment of the present invention. For example... Figure 2 As shown, the robot teaching system 200, as a functional structure, includes, for example, a camera unit 211, a camera coordinate system setting unit 212, and a program generation unit 213, and further includes a sign detection device 220 for detecting signs that serve as references for setting the camera coordinate system. Furthermore, the sign detection device 220 includes a three-dimensional plane detection unit 221, a two-dimensional plane image generation unit 222, a sign detection unit 223, and a sign position calculation unit 224.

[0047] The imaging unit 211, one of these functions, is a function possessed by the imaging terminal 1. On the other hand, each of the camera coordinate system setting unit 212, program generation unit 213, and marker detection device 220 can be fully implemented in either the imaging terminal 1 or the robot control device 2, or each function can be distributed between the imaging terminal 1 and the robot control device 2. Furthermore, some or all of the above functions can be implemented by other devices besides the imaging terminal 1 and the robot control device 2.

[0048] The imaging unit 211 is the same as the imaging unit 12 of the imaging terminal 1, and functions as a 3D camera to acquire point group data. In addition, the imaging unit 211 acquires point group data that includes at least the mark and the workpiece to be welded.

[0049] The camera coordinate system setting unit 212 sets the camera coordinate system based on the markers recognized by the point group data obtained by the shooting unit 211. For example, the camera coordinate system setting unit 212 sets the position of the markers recognized by the point group data obtained by the shooting unit 211 as the origin, and sets a three-dimensional orthogonal coordinate system composed of the X-axis, Y-axis and Z-axis that are orthogonal to each other at the origin, as the camera coordinate system.

[0050] Figure 3 This diagram illustrates a scenario where the marker M is set on the workpiece W, which is the object of welding, and the camera coordinate system is established with the position of marker M as the origin O. For example... Figure 3 As shown, the marker M is set on the base plate of the workpiece W. The camera coordinate system setting unit 212 uses the position of the marker M, which is recognized by the point group data obtained by the shooting unit 211, as the origin O. It sets a three-dimensional orthogonal coordinate system composed of the X-axis, Y-axis, and Z-axis, which are orthogonal to each other at the origin O, as the camera coordinate system. In addition, the origin O can be any point among the markers M. For example, a predetermined point, a center point, or a selected point among the markers M can be set as the origin O.

[0051] The marker M can be any identifier that the camera unit 211 can recognize as being placed in space. For example, an AR marker is preferred. When an AR marker is recognized in space, it is easy to display the actual point group data as a camera coordinate system with that AR marker as the origin. Details regarding the processing of the marker detection device 220 for detecting the marker M will be described later.

[0052] In the camera coordinate system set by the camera coordinate system setting unit 212, the motion path of the robot arm 3 is set based on the welding positions L1, L2, and L3 of the workpiece W, which are known from the point group data obtained by the imaging unit 211. For example, the welding positions L1, L2, and L3 are known from the point group data obtained by the imaging unit 211, and the user manually selects the motion path of the robot arm 3 (welding torch 32), or calculates the distance between the welding positions L1, L2, and L3 and the welding torch 32, so that the welding torch 32 mounted at the front end of the robot arm 3 moves along the welding positions L1, L2, and L3 of the workpiece W.

[0053] On the other hand, the robot control device 2 obtains the position and orientation (robot coordinate system) of the robot arm 3 from information related to the angles of each axis controlled by the robot control device 2. Then, based on the setting position of the marker M and the position and orientation (setting position) of the robot arm 3 relative to the setting position of the marker M, calibration can be performed to align the camera coordinate system and the robot coordinate system. This calibration can be performed, for example, by aligning the tip of the robot arm 3 (welding torch 32) with the setting position of the marker M.

[0054] The program generation unit 213 transforms the motion path of the robot arm 3 set in the camera coordinate system into the robot coordinate system set in the robot control device 2, and generates a work program for making the robot arm 3 move.

[0055] Thus, in the robot teaching system 200, a camera coordinate system is set up based on the marker M recognized from the point group data obtained from the camera unit 211. The welding positions L1, L2, L3 and the motion path of the robot arm 3 in the camera coordinate system are transformed into the robot coordinate system, and a work program is generated that allows the robot arm 3 to perform welding appropriately at the welding positions L1, L2, L3. Then, the work program is stored in the storage unit of the robot control device 2, which controls the movement of the robot arm 3 based on the work program. As a result, the welding robot system 100 can perform welding appropriately at the welding positions L1, L2, L3 of the workpiece W.

[0056] [Details of the processing in the sign detection device 220]

[0057] As described above, since the camera coordinate system is set with the position of the marker M as the origin O, it is important to properly detect the position of the marker M. The following details the processing of the marker detection device 220 for detecting the position of the marker M.

[0058] Figure 4 This diagram illustrates the specific processing steps in the mark detection device 220, which detects the position of the mark M set on the workpiece W. (See diagram for example.) Figure 4 As shown, the sign detection device 220 detects the position of the sign M based on point group data obtained by a 3D camera.

[0059] like Figure 4 As shown in (a), the point group data acquired by the 3D camera is depicted. For example, the imaging unit 211 acquires coordinate data obtained using a three-dimensional measurement method based on a known stereo method, or coordinate data obtained by 3D scanning using a distance measurement sensor, etc., as point group data.

[0060] The 3D plane detection unit 221 detects planes in 3D space, i.e., 3D planes, based on point group data acquired by a 3D camera. Figure 4 (b)). For example, the three-dimensional plane detection unit 221, based on the point group data obtained by the imaging unit 211, sets a plane with a given area as a three-dimensional plane to detect the bottom plate, i.e., plane Wa, and the back plate, i.e., plane Wb, constituting the workpiece W.

[0061] The two-dimensional planar image generation unit 222 generates a two-dimensional planar image by projecting the point group data constituting the three-dimensional plane detected by the three-dimensional planar detection unit 221 onto the two-dimensional plane. Figure 4 (c) and (d)). For example, regarding the plane Wa detected by the three-dimensional plane detection unit 221, the so-called projection onto a two-dimensional plane includes: moving the point group data, whose positions are indicated by three-dimensional coordinate values ​​(X, Y, Z), in a vertical direction based on the plane Wa to a position intersecting the plane Wa, thereby generating a two-dimensional plane image. The three-dimensional coordinate values ​​(X, Y, Z) of each point group data and the color information indicated by (R, G, B) are used to establish pixel information (XY coordinate values ​​and color information) correspondingly transformed into a two-dimensional plane image based on the plane equation of the plane Wa. Thus, as... Figure 4 As shown in (d), the two-dimensional planar image generation unit 222 generates a two-dimensional planar image Wa2. At this time, when the granularity of the point group data is coarse, the two-dimensional planar image generation unit 222 can generate the two-dimensional planar image Wa2 by performing a given image processing (e.g., smoothing processing such as Gaussian blur, pixel interpolation processing, etc.).

[0062] The marker detection unit 223 detects the marker M from the two-dimensional planar image Wa2 generated by the two-dimensional planar image generation unit 222. Figure 4 (e)). For example, the sign detection unit 223 detects the sign M from the two-dimensional planar image Wa2 generated by the two-dimensional planar image generation unit 222 by performing pattern matching with the image of the sign that has been stored in advance.

[0063] The marker position calculation unit 224 calculates the three-dimensional coordinates of the marker M contained in the two-dimensional planar image Wa2 detected by the marker detection unit 223 in the plane Wa, which is detected as a three-dimensional plane by the three-dimensional planar detection unit 221. Figure 4 (f)). For example, using Figure 4 As explained in (c) and (d), the three-dimensional coordinate values ​​(X, Y, Z) of the point group data constituting plane Wa are moved in the vertical direction relative to plane Wa to a position intersecting plane Wa to generate a two-dimensional planar image Wa2, but the reverse process is performed. That is, the positional information of the marker M contained in the two-dimensional planar image Wa2 is transformed into three-dimensional coordinate values ​​(X, Y, Z) on plane Wa, which is a three-dimensional plane.

[0064] In this way, the sign detection device 220 appropriately detects the position of the sign M set on the plane Wa.

[0065] [Mark Detection Method]

[0066] Next, a method for detecting marks according to an embodiment of the present invention will be described in detail.

[0067] Figure 5 This is a flowchart illustrating the processing flow of the mark detection method M100 performed by the mark detection device 220 according to an embodiment of the present invention. For example... Figure 5 As shown, the mark detection method M100 includes steps S110 to S140, each step being executed by the processor contained in the mark detection device 220.

[0068] In step S110, the mark detection device 220 detects a plane in three-dimensional space, i.e., a three-dimensional plane, based on point group data obtained by a 3D camera (three-dimensional plane detection step). As a specific example, the three-dimensional plane detection unit 221 in the mark detection device 220 detects planes Wa and Wb, etc., that constitute the workpiece W, as planes with a given area or more.

[0069] In step S120, the sign detection device 220 generates a two-dimensional plane image (two-dimensional plane image generation step) by projecting the point group data constituting the three-dimensional plane detected in step S110 onto a two-dimensional plane. As a specific example, the two-dimensional plane image generation unit 222 in the sign detection device 220 generates a two-dimensional plane image Wa2 by moving the point group data, which are shown in terms of the position of the three-dimensional coordinate values ​​(X, Y, Z), in the vertical direction based on the plane Wa to a position intersecting the plane Wa.

[0070] In step S130, the sign detection device 220 detects the sign M from the two-dimensional planar image generated in step S120 (sign detection step). As a specific example, the sign detection unit 223 in the sign detection device 220 detects the sign M from the two-dimensional planar image Wa2 by performing pattern matching with a pre-stored image of the sign.

[0071] In step S140, the sign detection device 220 calculates the three-dimensional coordinates (three-dimensional coordinate calculation step) of the sign M contained in the two-dimensional planar image detected in step S130, which is a three-dimensional plane. As a specific example, the sign position calculation unit 224 in the sign detection device 220 transforms the position information of the sign M contained in the two-dimensional planar image Wa2 into three-dimensional coordinate values ​​(X, Y, Z) on the plane Wa, which is a three-dimensional plane.

[0072] As described above, according to an embodiment of the present invention, the sign detection apparatus 220 and sign detection method M100 involve a three-dimensional plane detection unit 221 detecting planes Wa and Wb in three-dimensional space based on point group data acquired by a 3D camera, and a two-dimensional plane image generation unit 222 generating a two-dimensional plane image Wa2 by projecting the point group data constituting the plane Wa in a vertical direction based on the plane Wa. Then, the sign detection unit 223 detects a sign M from the two-dimensional plane image Wa2, and the sign position calculation unit 224 calculates the three-dimensional coordinates (X, Y, Z) in the three-dimensional plane Wa for the sign M contained in the two-dimensional plane image Wa2. Thus, the position of a sign M set on a plane can be appropriately detected, and the position of a sign M set on a plane can be easily detected even without complex camera parameter settings.

[0073] Furthermore, according to the robot teaching system 200 utilizing the mark detection device 220 according to an embodiment of the present invention, the camera coordinate system setting unit 212 sets the camera coordinate system with the position of the mark M appropriately detected by the mark detection device 220 as the origin O. The program generation unit 213 transforms the motion path of the robot arm 3 set in the camera coordinate system into the robot coordinate system set in the robot control device 2 based on the setting position of the mark M, and generates a work program for moving the robot arm 3. As a result, welding can be performed more easily at a suitable welding position.

[0074] Furthermore, in this embodiment, the marker detection unit 223 detects a marker M on plane Wa, one of the three-dimensional planes Wa and Wb detected by the three-dimensional plane detection unit 221. Alternatively, the user can select which of the multiple planes contains the marker, and the marker can be detected on each of the multiple planes.

[0075] In fact, as long as the user knows the plane to set the sign M, they can make the sign detection unit 223 detect the sign M efficiently and well by selecting that plane.

[0076] On the other hand, for example, if a marker is detected on each of the multiple planes, and reliable advance processing is performed, then marker M can be detected automatically. Here, the order in which the markers are detected on the multiple planes can be, for example, in order of area from largest to smallest, in order of distance from the center of the field of view from closest to furthest, or in a combination thereof.

[0077] Considering the high probability that the plane with the largest area and the plane closest to the center of the field of view among the multiple planes detected by the three-dimensional plane detection unit 221 will be marked with a flag M, it is expected that the flag will be detected at an early stage during the process of detecting the flag on each of the multiple planes. This helps to reduce the computational processing and shorten the processing time.

[0078] Furthermore, regarding the order in which the markers are detected on multiple planes, any order that efficiently detects the markers is acceptable, and there are no restrictions on this order.

[0079] As a specific example, a marker composed of a given color (e.g., red) or a marker containing a greater proportion of the given color can be used. Furthermore, when multiple planes are detected by the three-dimensional plane detection unit 221, the planes can be processed in descending order of the amount of the given color they contain. For each plane, color information (R, G, B) is included in the point group data constituting each plane, and the proportion of the given color information contained in each plane can be determined based on this color information.

[0080] In this way, by using a given color to form the mark, even when multiple planes are detected by the three-dimensional plane detection unit 221, the plane containing the mark can be detected efficiently. This helps to reduce computational processing and shorten processing time.

[0081] Furthermore, in this embodiment, the mark detection device 220 used in the robot teaching system 200 of the welding robot system 100 has been described as an example, but industrial robots using the mark detection device 220 are not limited to the welding robot system 100. For example, the mark detection device 220 can also be used in assembly robot systems, transport robot systems, etc., that assemble and transport electronic components and mechanical components.

[0082] The embodiments described above are intended to facilitate understanding of the present invention and are not intended to limit the scope of the invention. The elements, configurations, materials, conditions, shapes, and dimensions of the embodiments are not limited to the examples and can be appropriately modified. Furthermore, the structures shown in different embodiments can be partially substituted for or combined with each other.

Claims

1. A mark detection device, characterized in that, have: The three-dimensional plane detection unit detects planes in three-dimensional space, i.e., three-dimensional planes, based on point group data obtained by a 3D camera. The two-dimensional planar image generation unit generates a two-dimensional planar image by projecting the point group data constituting the detected three-dimensional plane in a vertical direction based on the three-dimensional plane. A sign detection unit that detects signs from the generated two-dimensional planar image; and The marker position calculation unit calculates the three-dimensional coordinates in the three-dimensional plane for the markers contained in the detected two-dimensional planar image. The mark is placed on the workpiece.

2. The mark detection device according to claim 1, characterized in that, When the three-dimensional plane detection unit detects multiple three-dimensional planes, the sign detection unit detects a sign for each plane.

3. The mark detection device according to claim 2, characterized in that, The sign detection unit detects signs in descending order of area among the plurality of three-dimensional planes.

4. The mark detection device according to claim 2, characterized in that, The sign detection unit detects signs in the order of distance from the center of the field of view to the farthest point in the plurality of three-dimensional planes.

5. A robot teaching system, characterized in that, have: The imaging unit is equipped with 3D cameras that capture images of the workpiece and its markings. The three-dimensional plane detection unit detects planes in three-dimensional space, i.e., three-dimensional planes, based on point group data obtained by the 3D camera. The two-dimensional planar image generation unit generates a two-dimensional planar image by projecting the point group data constituting the detected three-dimensional plane in a vertical direction based on the three-dimensional plane. A sign detection unit detects signs from the generated two-dimensional planar image; The marker position calculation unit calculates the three-dimensional coordinates in the three-dimensional plane for the marker contained in the detected two-dimensional plane image; The camera coordinate system setting unit sets the camera coordinate system based on the calculated three-dimensional coordinates of the marker; The program generation unit transforms the movement path of the robot arm in the camera coordinate system into the robot coordinate system set in the robot control device, and simultaneously generates a work program to make the robot arm move. The mark is placed on the workpiece.

Images