Robot system and robot teaching method
The robot system and method address the challenge of recognizing teaching points on workpieces with varying colors and materials by adjusting light conditions for accurate detection, ensuring reliable robot operation across different workpiece configurations.
Patent Information
- Application Number
- PCT/JP2023/020612
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2023-06-02
- Publication Date
- 2024-12-05
AI Technical Summary
Existing methods for teaching robots using light on workpieces face challenges due to difficulties in recognizing light on workpieces with varying colors and materials, leading to inconsistent recognition of teaching points.
A robot system and method that includes a light irradiation unit to indicate teaching points and a detection unit to recognize the light reflected by the workpiece, with a control section adjusting conditions for accurate recognition based on the workpiece's configuration, using techniques such as adjusting light wavelength and threshold settings for binarization.
Enables reliable recognition of teaching points on workpieces regardless of their configuration, ensuring accurate robot operation by adapting light conditions for effective teaching point indication and recognition.
Smart Images

Figure JP2023020612_05122024_PF_FP_ABST
Abstract
Description
Robot system and robot teaching method
[0001] The present invention relates to a technique for teaching a robot a location to be worked on based on a teaching point indicated on a workpiece by irradiating the workpiece with light.
[0002] Patent Document 1 describes a technology in which an image is captured of a position indicated by a worker's fingertip or the like, and instructions are given to a robot based on this image. Patent Document 2 describes a technology in which an image including an object to be operated is captured by a two-dimensional image sensor, the position of the object to be operated is identified in the image, and instructions are given to a robot based on the identified position. Furthermore, Patent Document 3 describes a technology in which a laser sensor detects the work position on the work line of a workpiece and generates teaching points for the robot.
[0003] JP 2022-030207 A JP 2001-347478 A WO2022 / 186054 A
[0004] Incidentally, by shining light onto a workpiece using a laser pointer or the like, a teaching point can be indicated on the workpiece, and the teaching point can be recognized by a camera or the like, and the teaching point can be taught to a robot based on the results. Furthermore, a worker can confirm the teaching point by visually checking the light irradiated onto the workpiece. However, depending on the configuration of the workpiece (e.g., color or material), it may be difficult for the camera to recognize the light irradiated onto the workpiece, or for the worker to visually check it.
[0005] This invention has been made in consideration of the above-mentioned problems, and aims to enable appropriate responses to differences in the configuration of a workpiece when indicating teaching points on the workpiece by irradiating the workpiece with light.
[0006] The robot system of the present invention comprises a robot, a light irradiation unit that irradiates a workpiece with teaching point indication light, which is light that indicates a teaching point for the robot, a teaching point recognition unit that recognizes the position of the teaching point indication light irradiated on the workpiece as a teaching point, a light detection unit that detects light reflected by the workpiece, and a control unit that controls at least one of the light irradiation unit and the teaching point recognition unit based on the detection result of the light detection unit.
[0007] The robot teaching method of the present invention includes the steps of irradiating a workpiece with teaching point indication light, which is light that indicates a teaching point for the robot, from a light irradiation unit; detecting the light reflected by the workpiece using a light detection unit; controlling at least one of the light irradiation unit and the teaching point recognition unit based on the detection result of the light detection unit; and recognizing the position of the teaching point indication light irradiated onto the workpiece as a teaching point by the teaching point recognition unit.
[0008] In the present invention (robot system and robot teaching method) configured as described above, the light irradiation unit irradiates a workpiece with a teaching point indicator light that indicates a teaching point for the robot, and the teaching point recognition unit recognizes the position of the teaching point indicator light irradiated on the workpiece as the teaching point. A light detection unit is also provided that detects light reflected by the workpiece, and at least one of the light irradiation unit and the teaching point recognition unit is controlled based on the detection result of the light detection unit. As a result, when irradiating the workpiece with light to indicate a teaching point on the workpiece, appropriate responses can be made according to differences in the configuration of the workpiece.
[0009] The robot system may also be configured so that the control unit adjusts the conditions for irradiating the light irradiating unit with teaching point indicating light and causing the teaching point recognizing unit to recognize the teaching point based on the light detected by the light detecting unit. With this configuration, it is possible to perform an operation such as irradiating a workpiece with light indicating the teaching point and recognizing the teaching point under appropriate conditions according to the configuration of the workpiece. As a result, the teaching point irradiated on the workpiece can be reliably recognized regardless of the configuration of the workpiece.
[0010] The robot system may also be configured so that the light detection unit detects a first light reflected from a position on the workpiece where the teaching point indicating light is irradiated and a second light reflected from a position on the workpiece where the teaching point indicating light is not irradiated, and the control unit adjusts conditions for causing the light irradiating unit to irradiate the teaching point indicating light and for the teaching point recognition unit to recognize the teaching point based on the first light and the second light detected by the light detection unit. In this configuration, the first light reflected from a position on the workpiece where the teaching point indicating light is irradiated and the second light reflected from a position on the workpiece where the teaching point indicating light is not irradiated are detected. Then, conditions for causing the light irradiating unit to irradiate the teaching point indicating light and for the teaching point recognition unit to recognize the teaching point are adjusted based on the first light and the second light. Therefore, an operation such as irradiating the workpiece with the teaching point indicating light and recognizing the teaching point can be performed under appropriate conditions according to the configuration of the workpiece. As a result, it is possible to reliably recognize the teaching point irradiated on the workpiece regardless of the configuration of the workpiece.
[0011] Various specific configurations for detecting the first light and the second light are conceivable. For example, the robot system may be configured so that, when the light irradiating unit irradiates the workpiece with teaching point indicator light, the light detecting unit detects light reflected from the teaching point as the first light and detects light reflected from a position different from the teaching point as the second light. Alternatively, the robot system may be configured so that, when the light irradiating unit irradiates the workpiece with teaching point indicator light, the light detecting unit detects light reflected from the teaching point as the first light and, when the light irradiating unit does not irradiate the workpiece with teaching point indicator light, the light detecting unit detects light reflected from the workpiece as the second light.
[0012] The robot system may also be configured such that the teaching point recognition unit captures an image of the workpiece and recognizes the teaching points shown in the image, and the control unit adjusts the conditions for the teaching point recognition unit to recognize the teaching points. In this configuration, the teaching point recognition unit can recognize the teaching points under appropriate conditions according to the configuration of the workpiece.
[0013] The robot system may be configured so that the teaching point recognition unit recognizes teaching points shown in the image by binarizing the image with a predetermined threshold value, and the control unit adjusts the threshold value. In this configuration, the teaching point recognition unit can perform binarization of the image to recognize the teaching points with an appropriate threshold value according to the configuration of the workpiece.
[0014] The robot system may also be configured such that the control unit adjusts the conditions under which the light irradiation unit irradiates the workpiece with light. In this configuration, the light irradiation unit can irradiate the workpiece with light under appropriate conditions according to the configuration of the workpiece.
[0015] The robot system may also be configured such that the control unit adjusts the wavelength of the light that the light irradiation unit irradiates onto the workpiece. In this configuration, the light irradiation unit can irradiate the workpiece with light of an appropriate wavelength depending on the configuration of the workpiece.
[0016] The robot system may also be configured so that the control unit adjusts the wavelength of the light irradiated onto the workpiece by the light irradiator based on the RGB values indicated by the light detection result of the light detector. In this configuration, the light irradiator can irradiate the workpiece with light at an appropriate wavelength according to the configuration of the workpiece.
[0017] The robot system may also be configured so that the control unit adjusts the wavelength of the light irradiated onto the workpiece by the light irradiator based on the hue, brightness, or saturation of the light detected by the light detector. In this configuration, the light irradiator can irradiate the workpiece with light at an appropriate wavelength depending on the configuration of the workpiece.
[0018] The robot system may also be configured such that the light irradiation unit irradiates the workpiece with recognition light, which is light used by the teaching point recognition unit to recognize the teaching point, and visibility light, which is used by the worker to visually recognize the teaching point, and the recognition light and visibility light have different wavelengths. In other words, there have been cases where light that is easy for the teaching point recognition unit, which is a machine, to recognize is different from light that is easy for the worker to visually recognize. In contrast, with the above configuration, the teaching point recognition unit can reliably recognize the teaching point indicated by the recognition light, and the worker can reliably visually recognize the teaching point indicated by the visibility light.
[0019] The robot system may also be configured such that the control unit adjusts the conditions under which the light irradiating unit irradiates the workpiece with visible light based on the detection results of the light detecting unit. In this configuration, light that is easily recognizable by the worker can be irradiated onto the workpiece as visible light, depending on the configuration of the workpiece.
[0020] The robot system may also be configured so that the light detection unit detects light reflected from a position on the workpiece adjacent to the teaching point as the second light. In this configuration, the operation of irradiating the workpiece with light indicating the teaching point and recognizing the teaching point can be performed under appropriate conditions according to the configuration of the workpiece at the position adjacent to the teaching point. As a result, it is possible to reliably recognize the teaching point irradiated on the workpiece regardless of the configuration of the workpiece.
[0021] The robot system may further include a housing that positions the light emitting unit and the light detecting unit relative to each other and holds them together. In this configuration, the light reflected from the workpiece can be reliably detected by the light detecting unit.
[0022] The robot system may also be configured to further include a communication unit held in the housing that transmits the results of light detection by the light detection unit to the control unit. In this configuration, the worker can carry the housing that holds the light irradiation unit and the light detection unit separately from the control unit to perform work.
[0023] The robot system may also be configured to further include an optical element that is held in the housing so as to be positioned between the workpiece and the light irradiating unit and controls the passage of light, wherein the light irradiated from the light irradiating unit passes through the optical element and is irradiated onto the teaching point on the workpiece, and the light reflected by the workpiece is reflected by the optical element toward the light detecting unit, and the light detecting unit detects the light reflected by the optical element. With this configuration, the light reflected from the workpiece can be reliably detected by the light detecting unit.
[0024] The robot system may also be configured such that the control unit causes the light irradiation unit to irradiate light having a pattern corresponding to the work location where the robot will be working. In this configuration, light having an appropriate pattern is irradiated onto the teaching point, thereby accurately teaching the operation to be performed on the workpiece.
[0025] The robot system may also be configured to further include an angle detector that detects the angle at which the light irradiation unit irradiates light, and the teaching point recognition unit captures an image including the position of the teaching point estimated based on the angle detected by the angle detector, and recognizes the teaching point based on the image. With this configuration, it is possible to reliably capture an image of the teaching point and recognize the teaching point.
[0026] According to the present invention, when indicating teaching points on a workpiece by irradiating the workpiece with light, it is possible to appropriately respond to differences in the configuration of the workpiece.
[0027] 1 is a diagram schematically showing a first example of a robot system according to the present invention. FIG. 2 is a plan view schematically showing the field of view of an adjustment camera. FIG. 3 is a flowchart showing an example of robot control executed by an information processing device. FIG. 4 is a flowchart showing a first example of parameter optimization executed in the robot control of FIG. 3. FIG. 5 is a diagram schematically showing an example of calculation executed according to the flowchart of FIG. 4. FIG. 6 is a flowchart showing a second example of parameter optimization executed in the robot control of FIG. 3. FIG. 7 is a flowchart showing a modified example of teaching point recognition. FIG. 8 is a diagram schematically showing a second example of a robot system according to the present invention. FIG. 9 is a flowchart showing a third example of parameter optimization executed in the robot control of FIG. 3. FIG. 10 is a diagram showing an example of teaching point indication control using a pointer light for recognition and a pointer light for visibility. FIG. 11 is a diagram schematically showing the relationship between the configuration of a working location of a robot and the pointer light. FIG. 12 is a diagram schematically showing a method of recognizing a teaching point in three dimensions.
[0028] 1 is a diagram schematically illustrating a first example of a robot system according to the present invention. The robot system 1 includes a robot 2 that performs work on a workpiece 9, a recognition camera 3 that is used to recognize the workpiece 9, a teaching point designation terminal 4 that indicates teaching points Pt to the robot 2, and an information processing device 5 that controls the robot 2, the recognition camera 3, and the teaching point designation terminal 4.
[0029] The robot 2 includes a robot arm 21 and an end effector 23 attached to a tip 211 of the robot arm 21. The robot arm 21 has a plurality of rotatable joints 212, and the tip 211 of the robot arm 21 moves in accordance with the rotation of the joints 212, thereby moving the end effector 23. Various specific configurations of the end effector 23 are assumed depending on the work to be performed by the robot 2. For example, work performed by the end effector 23 may include gripping, suction, welding, screwing, or polishing an object.
[0030] The recognition camera 3 is attached to the robot arm 21 of the robot 2, and moves along with the robot arm 21, which is displaced in accordance with the rotation of the joint 212. The recognition camera 3 captures an image of the workpiece 9 with a solid-state image sensor, thereby acquiring recognition image data D3 indicative of the workpiece 9.
[0031] The teaching point designation terminal 4 includes a housing 41, a laser pointer 42, a half mirror 43, an adjustment camera 44, an operation button 45, a communication unit 46, and a control unit 47. The laser pointer 42, the half mirror 43, the adjustment camera 44, the communication unit 46, and the control unit 47 are housed within the housing 41. The housing 41 holds the laser pointer 42, the half mirror 43, the adjustment camera 44, the communication unit 46, and the control unit 47 while positioning them relative to one another. Note that FIG. 1 shows the interior of the housing 41 in order to illustrate these components. An operator can carry the teaching point designation terminal 4 by grasping the housing 41. In other words, the teaching point designation terminal 4 is a portable terminal. The operation button 45 is attached to the outside of the housing 41, and the operator can operate the operation button 45. Note that the operation button 45 includes a light button for turning on the laser pointer 42 and a start button for issuing an instruction to start robot control, which will be described later.
[0032] The laser pointer 42 emits pointer light Lp, which is laser light having a predetermined cross-sectional shape (for example, a circle). The pointer light Lp emitted from the laser pointer 42 is irradiated onto the workpiece 9 from the front end 411 of the housing 41. Here, a point on the workpiece 9 facing the laser pointer 42 becomes the teaching point Pt. Therefore, the pointer light Lp emitted from the laser pointer 42 is irradiated onto the teaching point Pt on the workpiece 9.
[0033] The half mirror 43 is disposed between the laser pointer 42 and the front end 411 of the housing 41, and the pointer light Lp emitted from the laser pointer 42 passes through the half mirror 43 before being irradiated onto the workpiece 9. In other words, the half mirror 43 allows the pointer light Lp to pass from the laser pointer 42 toward the front end 411 of the housing 41. On the other hand, the half mirror 43 reflects the reflected light Lr that is reflected by the workpiece 9 and enters from the front end 411 toward the adjustment camera 44.
[0034] The adjustment camera 44 acquires adjustment image data D44 representing the workpiece 9 by capturing reflected light Lr from the workpiece 9 using a solid-state imaging element. The adjustment camera 44 has a field of view as shown in FIG. 2 . Here, FIG. 2 is a plan view schematically illustrating the field of view of the adjustment camera. That is, the teaching point designating terminal 4 is held in the housing 41 while being positioned relative to the laser pointer 42 so as to capture an image of the teaching point Pt and its vicinity (the teaching point vicinity Pn) ( FIG. 4 ). In this example, the teaching point vicinity Pn is a range surrounding and adjacent to the teaching point Pt. Therefore, the adjustment camera 44 acquires adjustment image data D44 by capturing reflected light Lrt from the teaching point Pt and reflected light Lrn from the teaching point vicinity Pn. This adjustment image data D44 includes images of the teaching point Pt and the teaching point vicinity Pn on the workpiece 9.
[0035] The communication unit 46 wirelessly communicates with the information processing device 5. The communication unit 46 transmits adjustment image data D44 acquired by the adjustment camera 44 to the information processing device 5 and receives commands from the information processing device 5.
[0036] The control unit 47 is, for example, a processor. The laser pointer 42 irradiates the workpiece 9 with pointer light Lp under the control of the control unit 47. The adjustment camera 44 acquires adjustment image data D44 under the control of the control unit 47. The communication unit 46 communicates with the information processing device 5 under the control of the control unit 47. The control unit 47 controls the irradiation of the pointer light Lp from the laser pointer 42 and the acquisition of the adjustment image data D44 by the adjustment camera 44, depending on the result of confirming the operation of the operation button 45 by the worker. The control unit 47 also transmits the pressing of the operation button 45 by the worker to the information processing device 5 via the communication unit 46.
[0037] The information processing device 5 is a computer and includes a robot control unit 51, a camera communication unit 52, a portable terminal communication unit 53, a recognition image processing unit 54, a recognition parameter adjustment unit 55, an adjustment image processing unit 56, and a pointer parameter adjustment unit 57. These functional units 51 to 57 can be configured in a processor, for example, by the processor executing predetermined software.
[0038] The robot control unit 51 controls the robot 2 to cause the robot 2 to perform a predetermined task. The camera communication unit 52 communicates with the recognition camera 3 via a wired connection and receives recognition image data D3 acquired by the recognition camera 3. The portable terminal communication unit 53 communicates wirelessly with the communication unit 46 of the teaching point designation terminal 4 and transmits requests to the communication unit 46 for the teaching point designation terminal 4 and receives responses from the teaching point designation terminal 4 from the communication unit 46. The portable terminal communication unit 53 also receives adjustment image data D44 acquired by the adjustment camera 44 from the communication unit 46.
[0039] The recognition image processing unit 54 performs image processing on the recognition image data D3 received by the camera communication unit 52, and recognizes the teaching point Pt based on the recognition image data D3. The recognition parameter adjustment unit 55 adjusts the recognition parameters used by the recognition image processing unit 54 to recognize the teaching point Pt.
[0040] The adjustment image processing unit 56 performs image processing on the adjustment image data D44 received by the portable terminal communication unit 53. Based on the result of the image processing by the adjustment image processing unit 56, the pointer parameter adjustment unit 57 adjusts the pointer parameters that control the pointer light Lp that the teaching point indication terminal 4 irradiates onto the workpiece 9. The operations of the adjustment image processing unit 56 and the pointer parameter adjustment unit 57 will be described in detail later.
[0041] Fig. 3 is a flowchart showing an example of robot control executed by an information processing device, Fig. 4 is a flowchart showing a first example of parameter optimization executed in the robot control of Fig. 3, and Fig. 5 is a diagram schematically showing an example of calculation executed according to the flowchart of Fig. 4. The flowcharts of Fig. 3 and Fig. 4 are executed under the control of the information processing device 5.
[0042] When the portable terminal communication unit 53 confirms that the start button (operation button 45) of the teaching point indicating terminal 4 has been continuously pressed for a predetermined time by the operator ("YES" in steps S101 and S102), it executes parameter optimization in step S103. This parameter optimization is a process of optimizing the recognition parameters (binarization threshold) and the pointer parameters (color of the pointer light Lp).
[0043] In step S201 of parameter optimization ( FIG. 4 ), the portable terminal communication unit 53 acquires adjustment image data D44 (RGB values) from the teaching point designation terminal 4 (step S201), the adjustment camera 44 capturing an image of reflected light Lr from the workpiece 9. This adjustment image data D44 includes images of the teaching point Pt and the teaching point vicinity Pn of the teaching point Pt ( FIG. 5 ). Note that in step S201, the laser pointer 42 is not lit, and the pointer light Lp is not irradiated onto the teaching point Pt. Furthermore, in step S201, the adjustment image processing unit 56 acquires the RGB values of the teaching point vicinity Pn indicated by the adjustment image data D44.
[0044] In step S202, the pointer parameter adjustment unit 57 calculates the average values of the RGB values. Specifically, the average value of the R (red) gradation values indicated by each pixel included in the teaching point neighborhood Pn in the adjustment image data D44 is calculated. A similar calculation is performed for G (green) and B (blue), thereby obtaining the average value Ar of R, the average value Ag of G, and the average value Ab of B in the teaching point neighborhood Pn. In step S203, the pointer parameter adjustment unit 57 sets these average values Ar, Ag, and Ab as parameter A of an RGB adjustment function, which will be described later.
[0045] In step S204, the pointer parameter adjustment unit 57 transmits a command to turn on the laser pointer 42 to the teaching point designating terminal 4 via the portable terminal communication unit 53. The control unit 47 of the teaching point designating terminal 4 turns on the laser pointer 42 in response to the lighting command. As a result, the pointer light Lp is irradiated onto the teaching point Pt on the workpiece 9.
[0046] The lighting command also includes a command to change the color of the pointer light Lp among R, G, and B. In response to this, the control unit 47 of the teaching point designating terminal 4 starts changing the color of the pointer light Lp. As a result, the color of the pointer light Lp irradiated onto the teaching point Pt changes, for example, in the order of R, G, and B (step S205).
[0047] In step S206, the pointer parameter adjustment unit 57 acquires the adjustment image data D44 (RGB values) output by the adjustment camera 44 from the teaching point designating terminal 4 via the portable terminal communication unit 53. Specifically, the adjustment image data D44(R) acquired by the adjustment camera 44 during the period when the red (R) pointer light Lp(R) is irradiated, the adjustment image data D44(G) acquired by the adjustment camera 44 during the period when the green (G) pointer light Lp(G) is irradiated, and the adjustment image data D44(B) acquired by the adjustment camera 44 during the period when the blue (B) pointer light Lp(B) is irradiated are acquired.
[0048] In step S207, the adjustment image processing unit 56 calculates the output value Vo of the RGB adjustment function as follows. According to this RGB adjustment function, when the input value Vi is equal to or greater than the central value (=128) of the 256 gradations, the output value Vo is calculated by: Output value Vo = ((Vi - A) / (255 - A)) × 128 + 127; and when the input value Vi is less than the central value (=128) of the 256 gradations, the output value Vo is calculated by: Output value Vo = (Vi / A) × 128. Here, when adjusting the R gradation value, the average value Ar of R is set as the parameter A; when adjusting the G gradation value, the average value Ag of G is set as the parameter A; and when adjusting the B gradation value, the average value Ab of B is set as the parameter A.
[0049] That is, the RGB values indicated by the adjustment image data D44(R) are used as input values Vi to calculate the output value Vo of the RGB adjustment function, and the adjustment image data D44(R) is converted. Similarly, the RGB values indicated by the adjustment image data D44(G) are used as input values Vi to calculate the output value Vo of the RGB adjustment function, and the adjustment image data D44(G) is converted. The RGB values indicated by the adjustment image data D44(B) are used as input values Vi to calculate the output value Vo of the RGB adjustment function, and the adjustment image data D44(B) is converted.
[0050] In step S208, the adjustment image processing unit 56 acquires the RGB values at the teaching point Pt and the RGB values at the teaching point neighborhood Pn contained in the converted adjustment image data D44. Acquisition of these RGB values is performed for each color of adjustment image data D44(R), D44(G), and D44(B). The acquired RGB value may be the value of a predetermined pixel, or may be the average value of the values of each pixel contained in the target range (teaching point Pt and teaching point neighborhood Pn).
[0051] In step S209, the adjustment image processing unit 56 calculates the difference between the RGB values at the teaching point Pt and the RGB values at the teaching point Pn vicinity as an evaluation value. Note that, as shown in FIG. 5 , in the adjustment image data D44(R) acquired while irradiating the teaching point Pt with a red (R) pointer light Lp, the values of the colors (G, B) other than the color (R) of the pointer light Lp remain the same between the teaching point Pt and the teaching point vicinity Pn. Therefore, the evaluation value calculated based on the converted adjustment image data D44(R) only needs to be calculated for red (R), which is the color of the pointer light Lp irradiated at the teaching point Pt to acquire the adjustment image data D44(R). The same applies to the other colors (G, B). Therefore, in the example of FIG. 5, the evaluation value for the adjustment image data D44(R) is calculated as 55 (=255-200), and the evaluation value for the adjustment image data D44(B) is calculated as 245 (=255-10).
[0052] In step S210, the pointer parameter adjustment unit 57 sets the color of the pointer light Lp to the color that has the maximum evaluation value among R, G, and B. In the example of Fig. 5, the color of the pointer light Lp is set to blue (B).
[0053] In step S211, the recognition parameter adjustment unit 55 sets a threshold (recognition parameter) that the recognition image processing unit 54 uses to binarize the recognition image data D3 based on the RGB values acquired in step S208. Specifically, the threshold is set to a value between the value (255) at the teaching point Pt and the value (10) at the teaching point vicinity Pn indicated by the adjustment image data D44 (after conversion) acquired for the color (B) of the pointer light Lp set in step S210. The threshold may be the average value of the value (255) at the teaching point Pt and the value (10) at the teaching point vicinity Pn, or a value greater or less than the average value by a predetermined value. Alternatively, optimal thresholds for combinations of the values at the teaching point Pt and the values at the teaching point vicinity Pn may be stored in a lookup table, and the threshold may be set based on this lookup table.
[0054] Once the parameter optimization (step S103) is completed, step S104 in FIG. 3 is executed. In step S104, the camera communication unit 52 acquires, from the recognition camera 3, recognition image data D3 including the teaching point Pt on the workpiece 9 illuminated with the pointer light Lp(B) of the color (B) set in step S210. The recognition image processing unit 54 binarizes the recognition image data D3 acquired in step S104 using the threshold set in step S211 (step S105), and recognizes the teaching point Pt based on the binarized recognition image data D3 (step S106). In step S107, the robot control unit 51 moves the end effector 23 toward the position of the teaching point Pt recognized by the recognition image processing unit 54.
[0055] In the embodiment described above, the laser pointer 42 (light irradiation unit) irradiates the workpiece 9 with pointer light Lp (teach point indication light) that indicates a teaching point Pt for the robot 2, and the recognition camera 3 and the recognition image processing unit 54 (teach point recognition unit) recognize the position of the pointer light Lp irradiated on the workpiece 9 as the teaching point Pt. In addition, an adjustment camera 44 (light detection unit) is provided that detects reflected light Lr reflected by the workpiece 9, and at least one of the laser pointer 42 and the recognition image processing unit 54 is controlled based on the detection result of the adjustment camera 44 (parameter optimization). As a result, when irradiating the workpiece 9 with light to indicate the teaching point Pt on the workpiece 9, appropriate responses can be made according to differences in the configuration of the workpiece 9.
[0056] Furthermore, the pointer parameter adjustment unit 57 (control unit) adjusts the conditions (recognition parameters and pointer parameters) for irradiating the laser pointer 42 with the pointer light Lp and causing the recognition image processing unit 54 to recognize the teaching point Pt based on the reflected light Lr detected by the adjustment camera 44 (parameter optimization). With this configuration, it is possible to perform an operation such as irradiating the workpiece 9 with the pointer light Lp indicating the teaching point Pt and recognizing the teaching point Pt under appropriate conditions according to the configuration of the workpiece 9. As a result, the teaching point Pt irradiated on the workpiece 9 can be reliably recognized regardless of the configuration of the workpiece 9.
[0057] Furthermore, reflected light Lrt (first light) reflected from the teaching point Pt on the workpiece 9 where the pointer light Lp is irradiated, and reflected light Lrn (second light) reflected from the vicinity of the teaching point Pn on the workpiece 9 where the pointer light Lp is not irradiated, are detected (step S206). Then, the conditions (recognition parameters and pointer parameters) for irradiating the laser pointer 42 (light irradiating unit) with the pointer light Lp and causing the recognition image processing unit 54 (teaching point recognition unit) to recognize the teaching point Pt are adjusted based on the reflected light Lrt and Lrn (steps S207 to S210). Therefore, the operation of irradiating the workpiece 9 with the pointer light Lp indicating the teaching point Pt and recognizing the teaching point Pt can be performed under appropriate conditions according to the configuration of the workpiece 9. As a result, the teaching point Pt irradiated on the workpiece 9 can be reliably recognized regardless of the configuration of the workpiece 9.
[0058] Furthermore, the recognition camera 3 (teaching point recognition unit) captures an image of the workpiece 9 to obtain recognition image data D3 (step S104), and the recognition image processing unit 54 (teaching point recognition unit) recognizes the teaching points Pt indicated in the recognition image data D3 (steps S105 to S106). In response to this, the recognition parameter adjustment unit 55 (control unit) adjusts the threshold value (recognition parameter) used by the recognition image processing unit 54 to recognize the teaching points Pt (step S211). In this configuration, the recognition image processing unit 54 can recognize the teaching points Pt using an appropriate threshold value according to the configuration of the workpiece 9.
[0059] The recognition image processing unit 54 also binarizes the recognition image data D3 using a predetermined threshold value to recognize the teaching points Pt indicated in the recognition image data D3 (steps S105 to S106). In response to this, the recognition parameter adjustment unit 55 adjusts the threshold value (step S211). In this configuration, the recognition image processing unit 54 can binarize the recognition image data D3 for recognizing the teaching points Pt using an appropriate threshold value according to the configuration of the workpiece 9.
[0060] Furthermore, the pointer parameter adjustment unit 57 (control unit) adjusts the color (pointer parameters) of the pointer light Lp that the laser pointer 42 irradiates onto the workpiece 9 (steps S204 to S210). With this configuration, the laser pointer 42 can irradiate the workpiece 9 with the pointer light Lp in an appropriate color (wavelength) according to the configuration of the workpiece 9.
[0061] Furthermore, the pointer parameter adjustment unit 57 adjusts the color (wavelength of light) of the pointer light Lp that the laser pointer 42 irradiates onto the workpiece 9 based on the RGB values indicated by the results (adjustment image data D44) of the reflected light Lrt and reflected light Lrn detected by the adjustment camera 44 (steps S204 to S210). With this configuration, the laser pointer 42 can irradiate the workpiece 9 with the pointer light Lp in an appropriate color (wavelength) according to the configuration of the workpiece 9.
[0062] The adjustment camera 44 also detects reflected light Lrn reflected from a teaching point vicinity Pn adjacent to the teaching point Pt on the workpiece 9. With this configuration, an operation of irradiating the workpiece 9 with pointer light Lp to recognize the teaching point Pt can be performed under appropriate conditions (recognition parameters and pointer parameters) according to the configuration of the workpiece 9 at a position adjacent to the teaching point Pt. As a result, it is possible to reliably recognize the teaching point Pt irradiated on the workpiece 9 regardless of the configuration of the workpiece 9.
[0063] Also provided is a housing 41 that positions the laser pointer 42 and the adjustment camera 44 relative to each other and holds them together. With this configuration, the reflected light Lrt and reflected light Lrn reflected from the workpiece 9 can be reliably detected by the adjustment camera 44. Furthermore, since only the reflected light Lr incident from the front end 411 (incident port) of the housing 41 reaches the adjustment camera 44, external disturbance light is prevented from entering the adjustment camera 44.
[0064] The teaching point indicating terminal 4 is also provided with a communication unit 46 held in the housing 41. This communication unit 46 transmits the results (adjustment image data D44) of the reflected light Lrt and the reflected light Lrn detected by the adjustment camera 44 to the portable terminal communication unit 53 (control unit). In this configuration, the worker can carry around the housing 41 that holds the laser pointer 42 and the adjustment camera 44 separately from the information processing device 5 to perform work.
[0065] Also provided is a half mirror 43 (optical element) that is held in the housing 41 so as to be positioned between the workpiece 9 and the laser pointer 42 and controls the passage of light. Pointer light Lp emitted from the laser pointer 42 passes through the half mirror 43 and is irradiated onto the teaching point Pt of the workpiece 9, and reflected light Lrt and reflected light Lrn reflected by the workpiece 9 are reflected by the half mirror 43 toward the adjustment camera 44. The adjustment camera 44 then detects the reflected light Lrt and reflected light Lrn reflected by the half mirror 43. With this configuration, the reflected light Lrt and reflected light Lrn reflected from the workpiece 9 can be reliably detected by the adjustment camera 44.
[0066] Fig. 6 is a flowchart showing a second example of parameter optimization executed in the robot control of Fig. 3. The flowchart of Fig. 6 is executed under the control of the information processing device 5.
[0067] In step S301, the pointer parameter adjustment unit 57 transmits a lighting command to the teaching point designating terminal 4 via the portable terminal communication unit 53. The control unit 47 receives the lighting command via the communication unit 46 and causes the laser pointer 42 to emit pointer light Lp of one color (for example, R) of R, G, or B. As a result, the pointer light Lp is irradiated onto the teaching point Pt of the workpiece 9.
[0068] In step S302, the pointer parameter adjustment unit 57 acquires adjustment image data D44 (RGB values) acquired by the adjustment camera 44 from the teaching point designating terminal 4 via the portable terminal communication unit 53. This adjustment image data D44 includes an image of the teaching point Pt irradiated with the pointer light Lp.
[0069] In step S303, the pointer parameter adjustment unit 57 transmits a light-off command to the teaching point designating terminal 4 via the portable terminal communication unit 53, and the control unit 47 receives the light-off command via the communication unit 46 and causes the laser pointer 42 to stop emitting the pointer light Lp. As a result, the pointer light Lp is no longer irradiated onto the teaching point Pt of the workpiece 9.
[0070] In step S304, the pointer parameter adjustment unit 57 acquires the adjustment image data D44 (RGB values) acquired by the adjustment camera 44 from the teaching point designating terminal 4 via the portable terminal communication unit 53. This adjustment image data D44 includes an image of the teaching point Pt that is not irradiated with the pointer light Lp.
[0071] In step S305, the adjustment image processing unit 56 calculates an evaluation value given as the difference between the adjustment image data D44 acquired in step S302 and the adjustment image data D44 acquired in step S304. For the same reason as in step S209 in Fig. 4 above, the evaluation value needs to be calculated only for the color of the pointer light Lp that was irradiated onto the teaching point Pt to acquire the adjustment image data D44 in step S302.
[0072] In step S306, the pointer parameter adjustment unit 57 checks whether steps S301 to S305 have been executed for all the colors R, R, and G. If there is a color for which steps S301 to S305 have not been executed, the color is changed and steps S301 to S305 are executed. As a result, evaluation values are obtained for all the colors R, R, and G ("YES" in step S306).
[0073] In step S307, the pointer parameter adjustment unit 57 sets the color of the pointer light Lp to the color of R, G, or B that has the largest evaluation value. In step S308, the recognition parameter adjustment unit 55 sets a threshold (recognition parameter) that the recognition image processing unit 54 uses to binarize the recognition image data D3, based on the RGB values acquired in steps S302 and S304. Specifically, for the color of the pointer light Lp set in step S307, a threshold is set between the value at the teaching point Pt indicated by the adjustment image data D44 acquired in step S302 and the value at the teaching point Pt indicated by the adjustment image data D44 acquired in step S304. The specific method for setting the threshold between the two values is the same as described above.
[0074] In the embodiment described above, the laser pointer 42 (light irradiation unit) irradiates the workpiece 9 with pointer light Lp (teach point indication light) that indicates a teaching point Pt for the robot 2, and the recognition camera 3 and the recognition image processing unit 54 (teach point recognition unit) recognize the position of the pointer light Lp irradiated on the workpiece 9 as the teaching point Pt. In addition, an adjustment camera 44 (light detection unit) is provided that detects reflected light Lr reflected by the workpiece 9, and at least one of the laser pointer 42 and the recognition image processing unit 54 is controlled based on the detection result of the adjustment camera 44 (parameter optimization). As a result, when irradiating the workpiece 9 with light to indicate the teaching point Pt on the workpiece 9, appropriate responses can be made according to differences in the configuration of the workpiece 9.
[0075] Furthermore, the pointer parameter adjustment unit 57 (control unit) adjusts the conditions (recognition parameters and pointer parameters) for irradiating the laser pointer 42 with the pointer light Lp and causing the recognition image processing unit 54 to recognize the teaching point Pt based on the reflected light Lr detected by the adjustment camera 44 (parameter optimization). With this configuration, it is possible to perform an operation such as irradiating the workpiece 9 with the pointer light Lp indicating the teaching point Pt and recognizing the teaching point Pt under appropriate conditions according to the configuration of the workpiece 9. As a result, the teaching point Pt irradiated on the workpiece 9 can be reliably recognized regardless of the configuration of the workpiece 9.
[0076] Furthermore, reflected light Lrt (first light) reflected from a teaching point Pt on the workpiece 9 where the pointer light Lp is irradiated and reflected light Lrt (second light) reflected from a teaching point Pt on the workpiece 9 where the pointer light Lp is not irradiated are detected (steps S302 and S304). Then, conditions (recognition parameters and pointer parameters) for irradiating the laser pointer 42 (light irradiating unit) with the pointer light Lp and causing the recognition image processing unit 54 (teaching point recognition unit) to recognize the teaching point Pt are adjusted based on the reflected light Lrt (steps S307 and S308). Therefore, an operation such as irradiating the workpiece 9 with the pointer light Lp indicating the teaching point Pt and recognizing the teaching point Pt can be performed under appropriate conditions according to the configuration of the workpiece 9. As a result, the teaching point Pt irradiated on the workpiece 9 can be reliably recognized regardless of the configuration of the workpiece 9.
[0077] In the robot control of Fig. 3, teaching point recognition is performed in steps S104 to S106. Instead of these steps, teaching point recognition of Fig. 7 may be performed. Fig. 7 is a flowchart showing a modified example of teaching point recognition. The flowchart of Fig. 7 is executed under the control of the information processing device 5.
[0078] When the worker presses the lighting button (operation button 45) of the teaching point indicating terminal 4, the control unit 47 causes the laser pointer 42 to emit pointer light Lp and transmits a lighting notification to the portable terminal communication unit 53 of the information processing device 5 via the communication unit 46. In response to this, when the portable terminal communication unit 53 receives the lighting notification ("YES" in step S401), the camera communication unit 52 causes the recognition camera 3 to capture an image of the workpiece 9 and acquires recognition image data D3 from the recognition camera 3. This recognition image data D3 includes an image of the teaching point Pt irradiated with the pointer light Lp.
[0079] When the worker stops pressing the light button (operation button 45) of the teaching point indicating terminal 4, the control unit 47 causes the laser pointer 42 to stop emitting the pointer light Lp and transmits a light-off notification to the portable terminal communication unit 53 of the information processing device 5 via the communication unit 46. In response to this, when the portable terminal communication unit 53 receives the light-off notification ("YES" in step S403), the camera communication unit 52 causes the recognition camera 3 to capture an image of the workpiece 9 and acquires recognition image data D3 from the recognition camera 3. This recognition image data D3 includes an image of the teaching point Pt that is not irradiated with the pointer light Lp.
[0080] In step S405, the recognition image processing unit 54 calculates the difference between the recognition image data D3 acquired in step S402 and the recognition image data D3 acquired in step S404 to acquire a difference image, and binarizes this difference image using a threshold value (step S405). In step S405, the recognition image processing unit 54 recognizes the teaching point Pt based on the binarized difference image.
[0081] Figure 8 is a diagram showing a second example of a robot system according to the present invention. The difference from the first example of Figure 1 is that the robot system 1 is provided with a terminal stand 6 that holds a teaching point indication terminal 4. Here, the differences from the first example of Figure 1 will be mainly explained, and common parts will be given the same reference numerals and explanations thereof will be omitted as appropriate.
[0082] The terminal stand 6 has a stand body 61 and a holder 62 that detachably holds the teaching point indication terminal 4 relative to the stand body 61. The holder 62 is rotatable in two different angular directions θ and φ relative to the stand body 61. The terminal stand 6 also has an angle sensor 63 that detects the rotation angle of the holder 62, and transmits the rotation angle of the holder 62 detected by the angle sensor 63 to the information processing device 5. For example, the rotation angle may be transmitted to the portable terminal communication unit 53 via the communication unit 46 of the teaching point indication terminal 4, or may be transmitted to the information processing device 5 by a separately provided communication means.
[0083] This rotation angle indicates the rotation angle of the teaching point designating terminal 4 held by the holder 62, in other words, the direction in which the laser pointer 42 of the teaching point designating terminal 4 emits the pointer light Lp. Therefore, the position coordinates of the teaching point Pt can be estimated from this rotation angle. Therefore, the robot control unit 51 estimates the position coordinates of the teaching point Pt based on the rotation angle of the holder 62 received from the terminal stand 6. Then, when the teaching point Pt is to be imaged by the recognition camera 3, the robot control unit 51 controls the robot arm 21 to position the recognition camera 3 facing the position coordinates.
[0084] In this embodiment, an angle sensor 63 (angle detector) is provided that detects the angle at which the laser pointer 42 irradiates the pointer light Lp. The recognition camera 3 then captures an image including the position of the teaching point Pt estimated based on the angle detected by the angle sensor 63, obtains recognition image data D3, and recognizes the teaching point Pt based on this recognition image data D3. With this configuration, it is possible to reliably capture an image of the teaching point Pt and recognize the teaching point Pt.
[0085] Fig. 9 is a flowchart showing a third example of parameter optimization executed by the robot control of Fig. 3. The flowchart of Fig. 6 is executed under the control of the information processing device 5. In the first and second examples of parameter optimization described above, the recognition parameters (threshold values) are adjusted based on the RGB values indicated by the adjustment image data D44. In contrast, in the third example of Fig. 9, the recognition parameters (threshold values) are adjusted based on the luminance indicated by the adjustment camera 44.
[0086] In step S501, the pointer parameter adjustment unit 57 transmits a lighting command to the teaching point designating terminal 4 via the portable terminal communication unit 53, and the control unit 47 of the teaching point designating terminal 4 causes the laser pointer 42 to emit pointer light Lp in response to the lighting command received by the communication unit 46. As a result, the pointer light Lp is irradiated onto the teaching point Pt of the workpiece 9.
[0087] In step S502, the portable terminal communication unit 53 acquires the adjustment image data D44 (brightness value) acquired by the adjustment camera 44 by capturing an image of the reflected light Lr from the workpiece 9 from the teaching point designating terminal 4. In step S503, the recognition parameter adjustment unit 55 acquires the brightness value at the teaching point Pt indicated by the adjustment image data D44 and the brightness value at the vicinity Pn of the teaching point.
[0088] In step S504, the recognition parameter adjustment unit 55 sets a threshold (recognition parameter) that the recognition image processing unit 54 uses to binarize the recognition image data D3 based on the luminance value acquired in step S503. Specifically, the threshold is set to a value between the luminance value at the teaching point Pt and the luminance value at the teaching point vicinity Pn indicated by the adjustment image data D44. The threshold may be the average value of the luminance value at the teaching point Pt and the luminance value at the teaching point vicinity Pn, or a value that is greater or less than the average value by a predetermined value. Alternatively, optimal thresholds for combinations of the luminance value at the teaching point Pt and the luminance value at the teaching point vicinity Pn may be stored in a lookup table, and the threshold may be set based on this lookup table. Note that when the threshold is set based on the luminance value in this manner, steps S104 to S106 of the robot control in FIG. 3 are executed based on the luminance value.
[0089] In the embodiment described above, the laser pointer 42 (light irradiation unit) irradiates the workpiece 9 with pointer light Lp (teach point indication light) that indicates a teaching point Pt for the robot 2, and the recognition camera 3 and the recognition image processing unit 54 (teach point recognition unit) recognize the position of the pointer light Lp irradiated on the workpiece 9 as the teaching point Pt. Also, an adjustment camera 44 (light detection unit) is provided that detects reflected light Lr reflected by the workpiece 9, and the recognition image processing unit 54 is controlled based on the detection result (adjustment image data D44) of the adjustment camera 44 (setting of the threshold value in step S504). As a result, when irradiating the workpiece 9 with light to indicate the teaching point Pt on the workpiece 9, appropriate measures can be taken in accordance with differences in the configuration of the workpiece 9.
[0090] Incidentally, the color of the pointer light Lp that is easily recognized when the pointer light Lp irradiated on the teaching point Pt is captured and recognized by the recognition camera 3 may differ from the color of the pointer light Lp that is easily recognized when an operator visually recognizes the pointer light Lp irradiated on the teaching point Pt. For example, when recognition by the recognition camera 3 is performed based on brightness, a high-brightness white is advantageous, whereas if the workpiece 9 is a bright color, a white pointer light Lp is difficult for an operator to see. Therefore, the color of the pointer light Lp for operator recognition may be set to a color different from the color of the pointer light Lp used for recognition by the recognition camera 3.
[0091] 10 is a diagram showing an example of teaching point indication control using a pointer light for recognition and a pointer light for visibility. The flowchart in FIG. 10 is executed under the control of the control unit 47 of the teaching point indication terminal 4.
[0092] When the control unit 47 confirms that the worker has pressed the lighting button (operation button 45) (step S601), it causes the laser pointer 42 to emit pointer light Lp having a visible color (e.g., blue) for a predetermined time (e.g., 1 to 5 seconds) (step S602).
[0093] In the next step S603, the control unit 47 checks whether the light button has been pressed by the operator. If the control unit 47 cannot check whether the light button has been pressed (if "NO" in step S603), the teaching point instruction control of FIG. 10 is terminated.
[0094] On the other hand, if it is confirmed that the light button has been pressed ("NO" in step S603), the laser pointer 42 is caused to emit a pointer light Lp having a color for recognition (e.g., white) for a predetermined time (e.g., 1 to 5 seconds) (step S604), and the laser pointer 42 returns to step S601).
[0095] In this control, the pointer light Lp of the color for visibility (blue) and the pointer light Lp of the color for recognition (white) are repeatedly irradiated onto the workpiece 9. Therefore, the worker can visually recognize the teaching point Pt by the pointer light Lp of the color for visibility, and the recognition camera 3 and the recognition image processing unit 54 can recognize the teaching point Pt by the pointer light Lp of the color for recognition.
[0096] According to this embodiment, the laser pointer 42 (light irradiation unit) irradiates the workpiece 9 with pointer light Lp (recognition light) for the recognition camera 3 and the recognition image processing unit 54 (taught point recognition unit) to recognize the taught point Pt, and pointer light Lp (visualization light) for the worker to visually recognize the taught point Pt. At this time, the pointer light Lp for recognition and the pointer light Lp for visualization have different colors (wavelengths). In other words, there have been cases where light that is easily recognized by machines such as the recognition camera 3 and the recognition image processing unit 54 is different from light that is easily visible to the worker. Therefore, with this configuration, the recognition camera 3 and the recognition image processing unit 54 can reliably recognize the taught point Pt indicated by the recognition pointer light Lp, and the worker can reliably visually recognize the taught point Pt indicated by the visualization pointer light Lp.
[0097] In this case, the color of the pointer light Lp for visibility may be adjusted by the method of adjusting the color of the pointer light Lp by parameter optimization as shown in Figure 4 or Figure 6. In this configuration, the adjustment image processing unit 56 (control unit) adjusts the conditions (color) for the laser pointer 42 to irradiate the workpiece 9 with the pointer light Lp for visibility based on the detection results (adjustment image data D44) of the adjustment camera 44 (light detection unit) (steps S201 to S210, S301 to S307). In this configuration, the workpiece 9 can be irradiated with a pointer light Lp that is easily recognizable by the worker, depending on the configuration of the workpiece 9.
[0098] As described above, in the above embodiment, the robot system 1 corresponds to an example of a "robot system" of the present invention, the robot 2 corresponds to an example of a "robot" of the present invention, the recognition camera 3 and the recognition image processing unit 54 function as an example of a "teaching point recognition unit" of the present invention, the housing 41 corresponds to an example of a "housing" of the present invention, the laser pointer 42 corresponds to an example of a "light irradiation unit" of the present invention, the half mirror 43 corresponds to an example of an "optical element" of the present invention, the adjustment camera 44 corresponds to an example of a "light detection unit" of the present invention, at least one of the recognition parameter adjustment unit 55 and the pointer parameter adjustment unit 57 corresponds to a "control unit" of the present invention, the angle sensor 63 corresponds to an example of an "angle detector" of the present invention, the workpiece 9 corresponds to an example of a "workpiece" of the present invention, the pointer light Lp corresponds to an example of a "teaching point indication light" of the present invention, the reflected light Lrt corresponds to an example of a "first light" of the present invention, the reflected light Lrn corresponds to an example of a "second light" of the present invention, and the teaching point Pt corresponds to an example of a "teaching point" of the present invention.
[0099] The present invention is not limited to the above embodiment, and various modifications can be made to the above without departing from the spirit of the present invention. For example, the pointer light Lp may be controlled in accordance with the configuration of the work area where the robot 2 works, as follows.
[0100] 11 is a diagram showing the relationship between the configuration of the robot's working area and the pointer light. When the working area where the robot 2 is working is a normal working area (for example, a working area provided on a flat surface), the pointer parameter adjustment unit 57 causes the laser pointer 42 to irradiate pointer light Lp having a circular cross section.
[0101] On the other hand, when the work location where the robot 2 is to work is a special work location (e.g., a hole), the pointer parameter adjustment unit 57 causes the laser pointer 42 to emit a pointer light Lp having a ring-shaped cross section. For example, when the robot 2 is performing the work of inserting a screw or a nail into a hole, if the pointer light Lp having a circular cross section is emitted onto the hole to be worked on, at least a portion of the pointer light Lp will enter the hole. As a result, the recognition image processing unit 54 may fail to recognize the pointer light Lp. In contrast, by emitting a ring-shaped pointer light Lp around the hole so as to surround the hole, the recognition image processing unit 54 can successfully recognize the teaching point Pt.
[0102] In this modification, the pointer parameter adjustment unit 57 (control unit) causes the laser pointer 42 to emit a pointer light Lp ( FIG. 11 ) having a pattern (cross-sectional shape) that corresponds to the work location where the robot 2 is to work. In this configuration, the pointer light Lp having an appropriate pattern can be emitted to the teaching point Pt, thereby accurately teaching the operation to be performed on the workpiece 9.
[0103] Note that information indicating whether the work location where work is to be performed on the workpiece 9 is a normal work location or a special work location is set in advance by, for example, the worker in the pointer parameter adjustment unit 57. In this case, the adjustment image processing unit 56 transmits a lighting command created in accordance with this information to the teaching point indication terminal 4, and the control unit 47 of the teaching point indication terminal 4 causes the laser pointer 42 to irradiate the pointer light Lp having a cross-sectional shape (pointer parameter) in accordance with this lighting command.
[0104] In the above embodiment, the recognition camera 3 and the recognition image processing unit 54 recognize the position of the teaching point Pt in two dimensions (XY plane). However, they can be configured to recognize the position of the teaching point Pt in three dimensions (FIG. 12).
[0105] 12 is a diagram schematically illustrating a method for recognizing a teaching point in three dimensions. In this example, the robot control unit 51 controls the robot arm 21 to adjust the angle of the recognition camera 3, so that the recognition image processing unit 54 acquires an image of the teaching point Pt captured by the recognition camera 3 from a first angle and an image of the teaching point Pt captured by the recognition camera 3 from a second angle. The recognition image processing unit 54 then performs stereo matching on these two images to estimate the parallax, thereby recognizing the position of the teaching point Pt in three dimensions.
[0106] Parameter optimization may also be performed based on hue, lightness, or saturation rather than RGB values. In the example of FIG. 4 where parameter optimization is performed based on hue, hue is acquired instead of RGB values. Specifically, in step S205, the hue is varied, for example, in three ways. In step S206, the hue values of the adjustment image data D44 are acquired for each of the three different hues. Note that step S207 is not executed, and the process proceeds to step S208, where the hues at the teaching point Pt and the teaching point vicinity Pn are acquired. Then, an evaluation value (the magnitude of the difference between the hue at the teaching point Pt and the hue at the teaching point vicinity Pn) is calculated for each hue (step S209), and the hue of the pointer light Lp is set to the hue with the highest evaluation value. Furthermore, a binarization threshold is set based on the hue in the same manner as described above (step S211).
[0107] When performing parameter optimization based on lightness or saturation, it is possible to perform the same parameter optimization as in the case of hue. Furthermore, the parameter optimization in FIG. 6 can also be performed based on hue, lightness, or saturation. In these examples, too, the pointer light Lp from the laser pointer 42 to the workpiece 9 can be irradiated with an appropriate color (wavelength) according to the configuration of the workpiece 9.
[0108] 4, it is not necessary to use an RGB adjustment function. In this case, steps S208 to S211 can be performed based on the RGB values of the adjustment image data D44 acquired by the adjustment camera 44 without converting the RGB values of the adjustment image data D44.
[0109] The adjustment camera 44 may be provided separately from the teaching point specifying terminal 4 .
[0110] Furthermore, the relationship between the teaching point neighborhood Pn and the teaching point Pt is not limited to the above. Therefore, the adjustment image processing unit 56 may treat a range away from the teaching point Pt as the teaching point neighborhood Pn.
[0111] Furthermore, it is not essential to use both the reflected light Lrt from the teaching point Pt and the reflected light Lrn from the vicinity of the teaching point Pn for parameter optimization. In other words, optimization of the recognition parameters or the pointer parameters may be performed based on the reflected light from either one of them.
[0112] The method for setting the color of the visual pointer light Lp by the worker is not limited to the above, and the worker may set it. The timing at which the laser pointer 42 irradiates the visual pointer light Lp is also not limited to the above example, and the visual pointer light Lp may be irradiated only while the worker is pressing a predetermined button, for example.
[0113] DESCRIPTION OF SYMBOLS 1...Robot system 2...Robot 3...Recognition camera (teaching point recognition unit) 41...Housing 42...Laser pointer (light irradiation unit) 43...Half mirror (optical element) 44...Adjustment camera (light detection unit) 54...Recognition image processing unit (teaching point recognition unit) 55...Recognition parameter adjustment unit (control unit) 57...Pointer parameter adjustment unit (control unit) 63...Angle sensor (angle detector) 9...Work Lp...Pointer light (teaching point indication light) Lrt...Reflected light (first light) Lrn...Reflected light (second light) Pt...Teaching point
Claims
1. A robot system comprising: a robot; a light irradiation unit that irradiates a workpiece with teaching point indicating light that indicates a teaching point for the robot; a teaching point recognition unit that recognizes the position of the teaching point indicating light irradiated onto the workpiece as the teaching point; a light detection unit that detects light reflected by the workpiece; and a control unit that controls at least one of the light irradiation unit and the teaching point recognition unit based on the detection result of the light detection unit.
2. A robot system as described in claim 1, wherein the control unit adjusts the conditions for irradiating the teaching point indicating light to the light irradiation unit and causing the teaching point recognition unit to recognize the teaching point based on the light detected by the light detection unit.
3. The robot system of claim 2, wherein the light detection unit detects a first light reflected from a position on the workpiece where the teaching point indication light is irradiated and a second light reflected from a position on the workpiece where the teaching point indication light is not irradiated, and the control unit adjusts the conditions for irradiating the teaching point indication light to the light irradiation unit and causing the teaching point recognition unit to recognize the teaching point based on the first light and the second light detected by the light detection unit.
4. A robot system as described in claim 3, wherein when the light irradiation unit is irradiating the teaching point indicating light onto the workpiece, the light detection unit detects light reflected from the teaching point as the first light, and detects light reflected from a position different from the teaching point as the second light.
5. A robot system as described in claim 3, wherein when the light irradiation unit is irradiating the teaching point indicating light onto the workpiece, the light detection unit detects the light reflected from the teaching point as the first light, and when the light irradiation unit is not irradiating the teaching point indicating light onto the workpiece, the light detection unit detects the light reflected from the workpiece as the second light.
6. A robot system as described in any one of claims 1 to 5, wherein the teaching point recognition unit captures an image of the workpiece and recognizes the teaching point shown in the image, and the control unit adjusts the conditions for the teaching point recognition unit to recognize the teaching point.
7. The robot system according to claim 6, wherein the teaching point recognition unit recognizes the teaching point shown in the image by binarizing the image with a predetermined threshold value, and the control unit adjusts the threshold value.
8. A robot system according to any one of claims 1 to 7, wherein the control unit adjusts the conditions under which the light irradiation unit irradiates the workpiece with light.
9. A robot system according to claim 8, wherein the control unit adjusts the wavelength of the light that the light irradiation unit irradiates onto the workpiece.
10. A robot system as described in claim 9, wherein the control unit adjusts the wavelength of the light irradiated by the light irradiation unit onto the workpiece based on the RGB values indicated by the results of light detection by the light detection unit.
11. A robot system as described in claim 9, wherein the control unit adjusts the wavelength of the light that the light irradiation unit irradiates onto the workpiece based on the hue, brightness or saturation indicated by the results of light detection by the light detection unit.
12. A robot system according to any one of claims 1 to 11, wherein the light irradiation unit irradiates the workpiece with recognition light, which is light for the teaching point recognition unit to recognize the teaching point, and visibility light, which is light for an operator to visually recognize the teaching point, and the recognition light and the visibility light have different wavelengths from each other.
13. A robot system according to claim 12, wherein the control unit adjusts the conditions under which the light irradiation unit irradiates the workpiece with the visible light based on the detection result of the light detection unit.
14. A robot system according to any one of claims 3 to 5, wherein the light detection unit detects light reflected from a position on the workpiece adjacent to the teaching point as the second light.
15. A robot system according to any one of claims 1 to 14, further comprising a housing that positions the light irradiation unit and the light detection unit relative to each other and holds them integrally.
16. The robot system according to claim 15, further comprising a communication unit held in the housing for transmitting the result of the light detection by the light detection unit to the control unit.
17. A robot system as described in claim 15 or 16, further comprising an optical element held in the housing so as to be positioned between the workpiece and the light irradiation unit and controlling the passage of light, wherein light irradiated from the light irradiation unit passes through the optical element and is irradiated onto the teaching point of the workpiece, light reflected by the workpiece is reflected by the optical element towards the light detection unit, and the light detection unit detects the light reflected by the optical element.
18. A robot system according to any one of claims 1 to 17, wherein the control unit causes the light irradiation unit to irradiate light having a pattern corresponding to a work location where the robot is to perform work.
19. A robot system as described in any one of claims 1 to 18, further comprising an angle detector that detects the angle at which the light irradiation unit irradiates light, wherein the teaching point recognition unit captures an image including a position of the teaching point estimated based on the angle detected by the angle detector, and recognizes the teaching point based on the image.
20. A robot teaching method comprising the steps of: irradiating a workpiece with teaching point indicating light, which is light that indicates a teaching point for the robot, from a light irradiation unit; detecting the light reflected by the workpiece with a light detection unit; controlling at least one of the light irradiation unit and the teaching point recognition unit based on the detection result of the light detection unit; and recognizing the position of the teaching point indicating light irradiated onto the workpiece as the teaching point by the teaching point recognition unit.
Citation Information
Patent Citations
Robot hand and part supplier using this robot hand
JP1995116984A
Robot device with visual sensor
JP1997183088A
Component mounting device
JP1999126996A
Teaching method and device on object to be operated in robot, and robot
JP2001347478A
Teaching device
JP2002172575A
