Robot action design device

Abstract

This robot action design device comprises: an action generation information storage unit (121) that stores relative action position data in which one or more action positions necessary for generating an action of a robot (20) during product manufacturing are data indicated by relative positions with respect to a relevant object; a master data storage unit (122) that stores master data that is recognition result data from when a camera (30) recognizes a reference position in a state in which the camera (30) attached to the robot (20) is at a recognition position at which the reference position is recognized; an action position correction unit (133) that causes the camera (30) to recognize the reference position and corrects the relative action position data on the basis of a comparison between recognition results and the master data; and a robot action generation unit (134) that generates an action of the robot (20) on the basis of the corrected relative action position data.

Description

Robot motion design device CROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application is based on Patent Application No. 2024-104397 filed in Japan on June 27, 2024, the contents of which are incorporated by reference in their entirety.

[0002] The present invention relates to a robot motion design device that designs the motion of a robot.

[0003] In a robot system, a camera is used to recognize a workpiece, and the robot performs a predetermined process depending on the recognized position of the workpiece, etc. Patent Document 1 discloses a technology for adjusting the camera position offline. The contents of the prior art document are incorporated by reference as explanations of the technical elements in this specification.

[0004] JP 2008-21092 A

[0005] When introducing a robot into a site, it is not always possible to install the robot in a position that perfectly matches the position in the simulation environment. However, the technology disclosed in Patent Document 1 does not take into consideration the possibility of a discrepancy between the installation position of the robot in the site and the simulation environment. Therefore, when the robot is installed in the site, there is a possibility that the robot will not operate as desired.

[0006] The present disclosure has been made based on this situation, and its purpose is to provide a robot motion design device that can prevent a robot from not operating as desired on-site.

[0007] The above object is achieved by the combination of features recited in the independent claims, and the subclaims define further advantageous specific examples. The reference numerals in parentheses in the claims correspond to specific aspects described in the following embodiments as one aspect, and do not limit the technical scope of the disclosure.

[0008] One disclosure for achieving the above object is a robot motion design device comprising: a motion generation information storage unit that stores relative motion position data, which is data indicating one or more motion positions required for generating the motion of a robot during product manufacturing as relative positions with respect to a target object; a master data storage unit that stores master data, which is recognition result data when a recognition device attached to a robot recognizes a reference position while the recognition device is in a recognition position where the recognition device recognizes the reference position; a motion position correction unit that causes the recognition device to recognize the reference position and corrects the relative motion position data based on a comparison between the recognition result and the master data; and a robot motion generation unit that generates the motion of the robot based on the relative motion position data corrected by the motion position correction unit.

[0009] This robot motion design device stores relative motion position data that indicates motion positions relative to a target object. The motion position correction unit uses a recognition device to recognize a reference position, compares the recognition result with master data, and corrects the relative motion position data based on this comparison. By correcting the relative motion position data in this way, it is possible to prevent the robot from not operating as desired on-site. Furthermore, even if an installation error occurs on-site, since the motion positions are stored as relative positions, the motion positions can be corrected by correcting each motion position by the amount of the comparison result. Therefore, motion position correction can be easily performed.

[0010] One disclosure for achieving the above object is a robot motion design device that includes a motion generation information generation unit that generates relative motion position data, which is data in which one or more motion positions required to generate the motion of the robot during product manufacturing are indicated by relative positions with respect to a target object, and a product manufacturing time information storage unit that stores information required to generate the motion of the robot during product manufacturing, wherein the product manufacturing time information storage unit stores the relative motion position data generated by the motion generation information generation unit and master data that is recognition result data when a recognition device attached to the robot recognizes a reference position while the recognition device is in a recognition position where it recognizes a reference position.

[0011] This robot motion design device is equipped with a product manufacturing information storage unit that stores information required to generate robot motion during product manufacturing. This product manufacturing information storage unit stores relative motion position data generated by the motion generation information generation unit, which indicates motion positions relative to a target object, and master data, which is recognition result data when the recognition device attached to the robot recognizes a reference position while the recognition device is in a recognition position where it recognizes a reference position. Therefore, motions that cause the robot to operate as started can be generated on-site during product manufacturing, and this generation becomes easier.

[0012] 14 is a diagram showing the configuration of a robot system. FIG. 15 is a diagram showing the configuration of a robot motion design system. FIG. 16 is a diagram explaining the flow of processing up to storing motion generation information. FIG. 17 is a diagram explaining an example of a method for calculating a recognition position. FIG. 18 is a diagram showing the procedure executed after a robot is installed. FIG. 19 is a diagram showing in detail the procedures performed in S10 and S20 of FIG. 5. FIG. 20 is a diagram showing in detail the procedure performed in S30 of FIG. 5. FIG. 21 is a diagram explaining the processing of a master data generation unit in the second embodiment. FIG. 22 is a diagram showing the processing performed by a motion position correction unit in the third embodiment. FIG. 23 is a diagram explaining the processing of a master data generation unit in the fourth embodiment. FIG. 24 is a diagram showing the procedure up to determining a master image in the fifth embodiment. FIG. 25 is a diagram showing the procedure executed in the sixth embodiment. FIG. 26 is a diagram explaining the elements provided in the master data generation unit in the seventh embodiment. FIG. 27 is a diagram showing the procedure executed in the eighth embodiment. FIG. 28 is a diagram showing the procedure executed following FIG. 14.

[0013] First Embodiment An embodiment will be described below with reference to the drawings. Fig. 1 is a diagram showing the configuration of a robot system 10. The robot system 10 includes a robot 20, a camera 30, and a robot motion design device 100. The robot motion design device 100 is a part of the robot system 10. However, a portion of the robot motion design device 100 that can be set before the robot 20 actually operates may be configured separately from the robot system 10.

[0014] The robot 20 is fixed to a mounting base 40. The robot 20 comprises a base 21, an arm 22, and a tool 23. The base 21 is fixed to the mounting base 40. The arm 22 is a movable part that moves relative to the base 21. The tool 23 is attached to the tip of the arm 22 and performs operations such as gripping.

[0015] The camera 30 is an example of a recognition device, and is attached to the arm 22 to recognize a recognized part including a reference position. The reference position is set on the product 50. The product 50 is also called a workpiece. The reference position can also be set on a fixed object other than the robot 20 in the robot system 10. A fixed object is an object whose position is fixed in the robot system 10. The fixed object is, for example, the mounting table 40 or the work table 60. A mark may be formed as a reference position on the mounting table 40 or the work table 60. Furthermore, when multiple robots 20 are installed in the robot system 10, a reference position may be set on another robot 20. Multiple reference positions may be set on a single object. For example, multiple reference positions may be set on the product 50.

[0016] The product 50 is placed on the work table 60. The robot 20 performs a preset operation on the product 50 placed on the work table 60. This operation is, for example, an assembly operation in which the robot 20 grasps the product 50 and then assembles it with another product 50.

[0017] 2 shows the configuration of the robot motion design device 100. The robot motion design device 100 includes a user input unit 110, a product manufacturing information storage unit 120, and a control unit .

[0018] The user input unit 110 is a part that is operated by a user to input various pieces of information to the control unit 130 that operates the robot motion design device 100. The user input unit 110 is, for example, a keyboard, a mouse, or the like.

[0019] The product manufacturing information storage unit 120 is a storage medium from which the control unit 130 can read information and to which the control unit 130 can write information. The product manufacturing information storage unit 120 includes a motion generation information storage unit 121, a master data storage unit 122, a robot information storage unit 123, and a shape data storage unit 124. The motion generation information storage unit 121, the master data storage unit 122, the robot information storage unit 123, and the shape data storage unit 124 may be separate storage media or may be separate storage areas of a single storage medium.

[0020] The motion generation information storage unit 121 stores motion generation information. The motion generation information is information required to generate motions to be performed by the robot 20 during product manufacturing. The motion generation information includes information on positions (hereinafter referred to as motion positions) required to generate motions to be performed by the robot 20 during product manufacturing. During product manufacturing means the time when the robot 20 performs a pre-set motion on the product 50. One of the motion positions is the recognition position. The recognition position is a position where the recognition device performs a recognition operation to recognize a reference position. When the recognition device is a camera 30, the recognition position is also called the vision recognition position.

[0021] Other examples of the operating position include a gripping position where the product 50 is gripped, and an assembly position where the gripped product 50 is assembled, when the operation of the tool 23 is gripping.

[0022] The motion generation information storage unit 121 stores motion positions as relative motion position data that indicates relative positions with respect to a target object. The target object may be any object whose position is determined relative to a reference position. An object for which a reference position is set may also be the target object. For example, the target object may be a product 50 when it is first placed on the workbench 60. The relative motion position may be a (cm) from the product 50. More specifically, the relative position is a relative position based on a specific point on the target object. The relative motion position data is data in which multiple motion positions are indicated as relative positions with respect to the target object.

[0023] The master data storage unit 122 stores master data. The master data is recognition result data when the camera 30 recognizes the reference position while the camera 30 is at the recognition position. The recognition position here means a recognition position without any error. Errors include the mounting position of the camera 30, manufacturing errors of the product 50, etc. The master data can be said to be correct data of the recognition result when the camera 30 recognizes the reference position without any error.

[0024] The robot information storage unit 123 stores robot information. The robot information is various information related to the robot 20. The robot information also includes information about the camera 30 attached to the robot 20. The information about the camera 30 is information about the angle of view, attachment position, resolution, etc. of the camera 30. Another example of robot information is tool information. The tool information is information about the type of tool, size of the tool, etc. Another example of robot information is the specifications of the robot 20. The robot specifications include the relative positions of each part of the robot 20. The robot information may also include errors due to the movement of the robot 20.

[0025] The shape data storage unit 124 stores shape data of objects including a recognized portion. Because a portion of the product 50 is the recognized portion, the shape data storage unit 124 stores shape data of the product 50. The shape data of the product 50 includes data indicating the three-dimensional shape of the product 50. The shape data also includes, for example, a reference position within the product 50, the processing accuracy of the recognized portion including the reference position, and the dimensions of the recognized portion. An example of a recognized portion is a bolt hole. The shape data can include the position of a portion of the product 50 on which the tool 23 acts, such as a gripped position gripped by the tool 23. The shape data can also include the position of a portion of the product 50 that will be assembled to another product 50. If the product 50 has multiple reference positions, the shape data storage unit 124 also stores the recognition order of the multiple reference positions. Note that the reference positions may be stored in the master data storage unit 122 together with the master data.

[0026] The control unit 130 includes at least one of a processor and a circuit as a hardware configuration. For example, the control unit 130 can be realized by a computer including a processor and a memory. Alternatively, the control unit 130 may not include a processor and may include hardware circuits other than the processor, or may include a processor and hardware circuits other than the processor. The control unit 130 includes a master data generation unit 131, a motion generation information generation unit 132, a motion position correction unit 133, a robot motion generation unit 134, and a robot motion control unit 135 as functions realized by the above hardware configuration.

[0027] The master data generation unit 131 generates the master data described above. Then, the master data generation unit 131 stores the generated master data in the master data storage unit 122. The master data can be generated, for example, by CAD from shape data of the product 50. The data generated by CAD can also be corrected to suit the actual environment to create the master data. The master data may also be generated based on an image captured by the camera 30 in the actual environment. A method for generating the master data will also be described in the following embodiments.

[0028] The motion generation information generating unit 132 generates the above-mentioned motion generation information. Then, the motion generation information generating unit 132 stores the generated motion generation information in the motion generation information storage unit 121. An example of a method for generating motion generation information is as follows. For example, the user inputs motion positions such as the position of the recognized part, the gripping position, and the assembly position using absolute coordinates via the user input unit 110. The motion generation information generating unit 132 generates relative motion position data based on the coordinates input by the user and the relative positions of each part of the robot 20.

[0029] The motion position correction unit 133 uses the camera 30 to recognize the reference position and compares the recognition result with the reference position of the master data. This determines the positional deviation of the robot 20. The relative motion position data is corrected by the amount of this positional deviation. The motion position correction unit 133 will be described in detail later.

[0030] The robot motion generation unit 134 generates the motion of the robot 20 based on the relative motion position data corrected by the motion position correction unit 133. More specifically, it is assumed that the motion position correction unit 133 corrects the relative motion positions of the recognition position, the grasped position, and the assembly position. The motion of the robot 20 generated by the robot motion generation unit 134 is a series of motions in which the robot 20 is moved to the corrected grasped position, a grasping motion is performed at the grasped position, and then the robot 20 is moved to the corrected assembly position, and an assembly motion is performed at the assembly position.

[0031] FIG. 3 shows the flow of processing up to the storage of motion generation information. This FIG. 3 will be explained. First, in S1, the user inputs necessary information using the user input unit 110. The necessary information includes position information of the object on which the robot 20 will perform a motion, such as a reference position, a grasping position, and an assembly position. The position information of the object on which the robot 20 will perform a motion may be registered as an absolute position or as a relative position. If it is registered as an absolute position, relative motion position data is generated in the motion generation information generating unit 132. The necessary information may also include specifications of the robot and the camera 30. S1 does not need to be executed immediately before S2, but it is sufficient that it be executed before S2 is executed.

[0032] Steps S2 and S3 are executed by the master data generating unit 131. In step S2, master data is generated. In step S3, the master data generated in step S2 is stored in the master data storage unit 122.

[0033] Steps S4 and S5 are executed by the motion generation information generating unit 132. In step S4, motion generation information is generated based on the information registered in step S1. In step S5, the motion generation information generated in step S4 is stored in the motion generation information storage unit 121.

[0034] 3 does not need to be performed on-site. Therefore, the master data generating unit 131 and the motion generation information generating unit 132 may be configured separately from the robot system 10.

[0035] An example of a method for calculating the recognized position generated in S4 of Fig. 3 will be described using Fig. 4. Assume that the reference position is registered in S1. The (x, y) coordinates of the recognized position at which the camera 30 recognizes the reference position are the same as the (x, y) coordinates of the reference position. Therefore, it is sufficient to calculate the height coordinate z of the recognized position.

[0036] The height coordinate z is calculated by the two-dimensional coordinates (x, y) of the reference position and the angle of view θ of the camera 30. Ｖ , θ Ｈ , can be calculated based on the size of the recognized part including the reference position. In the example of Figure 4, the recognized part is a screw hole 70 in the product 50, and the radius of the screw hole 70 is r. The center of the screw hole 70 is the reference position. In Figure 4, V is the vertical length of the imaging range 80, H is the horizontal length of the imaging range 80, ΔX is the horizontal machining error of the screw hole 70, ΔY is the vertical machining error of the screw hole 70, and α is the margin required for recognition. The imaging range 80 can also be called the recognition range.

[0037] r, ΔX, and ΔY are stored in the shape data storage unit 124. Ｖ , θ Ｈ is stored in the robot information storage unit 123. α can be calculated from the resolution. Alternatively, α may be stored in the robot information storage unit 123.

[0038] The vertical length V of the photographing range 80 is 2(r+α+ΔY). Ｖ can be expressed by Equation 1. The horizontal length H of the photographing range 80 can be 2(r+α+ΔX). Using this H, the height z Ｈ can be expressed by Equation 2.

[0039] z Ｖ = V / (2 tan θ Ｖ )...Formula 1 z Ｈ = V / (2 tan θ Ｈ ) ...Equation 2 As can be seen from Equation 1 and Equation 2, the imaging range 80 of the camera 30 changes depending on the height from the recognized object. Ｖ , z Ｈ Then, z Ｖ , z ＨThe larger value, i.e., the higher value, is selected and used as the z coordinate of the recognition position of the camera 30 when recognizing the screw hole 70. In this way, the recognition position can be determined. This recognition position is (0, 0, z) relative to the screw hole 70 as the reference position. If the absolute coordinates of the screw hole 70 are (x0, y0, z0), then the absolute coordinates of the recognition position are (x0, y0, z0 + z).

[0040] Next, the procedure to be executed after the robot 20 is installed at the site will be described. Fig. 5 shows an overview of the procedure to be executed after the robot 20 is installed. In S10, the position error of the robot system 10 is corrected. This is to correct the position error of the robot 20 in the robot system 10. For example, the position of the robot 20 relative to the mounting base 40 is corrected.

[0041] In S20, the position error of the product 50 is corrected. This is to correct the position error of the product 50 relative to the robot 20. In S30, the robot 20 is operated in a state in which the position error has been corrected in S10 and S20.

[0042] The procedure performed in S10 is shown in detail in FIG. 6 . In S11, a determination is made as to whether or not to perform position correction. The determination in S11 can be made by the user. For example, if position correction has never been performed, the determination result in S11 is YES, and the process proceeds to S12. In S12, a determination is made as to whether or not to calculate the recognized position from the camera information. The determination in S12 can also be made by the user. An example of a case in which the user makes a NO determination in S12 is when the camera information differs from the information of the camera 30 attached to the robot 20. The determinations in S11 and S12 may be made by the control unit 130. For example, the product manufacturing information storage unit 120 may store either or both of whether or not position correction has been performed and the elapsed time since the last position correction, and the control unit 130 may make the determination in S11 based on the stored contents. When the control unit 130 determines S12, the control unit 130 determines S12 based on, for example, whether or not the camera information includes height information. If the camera information does not include information on the height direction, the control unit 130 determines YES in S12.

[0043] If the determination result in S12 is YES, the process proceeds to S13. In S13, the motion generation information generator 132 accesses the robot information storage unit 123 and calculates the recognition position from the reference position, information on the recognized unit, and camera information using the method described with reference to Fig. 4. However, in Fig. 6, the reference position is a fixed object other than the robot 20 in the robot system 10. For example, the reference position is a mark provided on a corner of the top surface of the workbench 60.

[0044] The steps from S14 onwards are executed by the motion position correction unit 133. In S14, a motion plan for the recognition position calculated in S13 is generated. In S15, the motion plan generated in S14 is executed to move to the recognition position. In S16, the recognized part is photographed by the camera 30. In S17, the image photographed in S16 is compared with the master data. Then, the relative position of the reference position in the photographed image is calculated with reference to the reference position in the master data. In other words, the positional deviation of the reference position in the photographed image is calculated with reference to the reference position in the master data.

[0045] In S18, the relative motion position data of the recognition position stored in the motion generation information storage unit 121 is corrected by the amount of the relative position calculated in S17. Similarly, for other motion positions, the relative motion position data is corrected by the amount of the relative position calculated in S17. In addition, for motion positions that perform motions that act on the product 50, such as the gripping position and assembly position, if the height direction length of the tool indicated by the tool information differs from the height direction length of the tool 23 that the robot 20 actually has, the relative motion position data may also be corrected based on this difference.

[0046] By the above procedure, even if the installation position of the robot 20 at the site in the robot system 10 is deviated from the position at the time of design, the deviation of the installation position can be corrected.

[0047] Next, the procedure performed in S20 will be described. The procedure performed in S20 is almost the same as S10. The difference from S10 is that the information accessed in S13 is the reference position set for the product 50, other information about the product 50, and camera information. This difference makes it possible to correct the deviation in S18 between the relative position of the robot 20 with respect to the product 50 at the time of design and the relative position of the robot 20 with respect to the product 50 on-site.

[0048] The process performed in S30 is shown in detail in Fig. 7. S30 is executed by the robot operation control unit 135. The process shown in Fig. 7 is an example in which the tool 23 performs gripping.

[0049] In S31, it is determined whether or not to grasp. If the determination result in S31 is YES, the process proceeds to S32. In S32, a grasping position, which is relative motion position data, is acquired from the motion generation information storage unit 121. In S33, movement is made to the grasping position acquired in S32. In S34, a grasping motion is performed. After S34 is executed, the process proceeds to S35. If the determination result in S31 is NO, the process also proceeds to S35.

[0050] In S35, it is determined whether or not to assemble. If the determination result in S35 is NO, the processing of FIG. 7 is terminated. If the determination result in S35 is YES, the process proceeds to S36. In S36, an assembly position, which is relative motion position data, is acquired from the motion generation information storage unit 121. In S37, movement is made to the assembly position acquired in S36. In S38, the assembly motion is executed.

[0051] In the embodiment described above, relative motion position data indicating motion positions relative to the target object is stored. The motion position correction unit 133 recognizes the recognized part using the camera 30 and compares the recognition result with master data (S17). Based on this comparison, the relative motion position data is corrected (S18). By correcting the relative motion position data in this way, it is possible to prevent the robot 20 from not operating as desired on-site. Furthermore, even if an installation error occurs on-site, since the motion positions are stored as relative positions, the motion positions can be corrected by correcting each motion position by the amount of the comparison result in S17. Therefore, motion position correction can be easily performed.

[0052] Furthermore, in this embodiment, the operating position correction unit 133 corrects the position of the robot 20 based on a reference position set on a fixed object, such as a mark provided on a corner of the top surface of the workbench 60, and then further corrects the position of the robot 20 based on a reference position set on the product 50. In this way, it is possible to correct not only the positional deviation between the product 50 and the robot 20 but also the positional deviation of the robot 20 in the robot system 10, thereby improving the accuracy of position correction.

[0053] Furthermore, the operation position correction unit 133 corrects the relative operation position data of the operation position for performing an operation acting on the product 50 by the difference between the height direction length of the tool indicated by the tool information stored in the robot information storage unit 123 and the height direction length of the tool 23 provided on the robot 20. In this way, even if the tool 23 is changed, it is possible to prevent the robot 20 from not operating as desired.

[0054] Note that at least one of the reference positions for calculating the recognition position in S10 and the reference positions for calculating the recognition position in S20 may be set multiple times. In this case, the master data storage unit 122 stores master data corresponding to the multiple reference positions.

[0055] Assume that multiple reference positions are set for calculating the recognition position in S10. If the positional deviation (i.e., the correction amount) calculated in S17 exceeds a threshold, the action position correction unit 133 may not correct the relative action position data by the correction amount, but may re-execute S13 and subsequent steps using another reference position as the recognition target.

[0056] Assume that multiple reference positions are set for calculating the recognition position in S20. If the correction amount calculated in the step corresponding to S17 exceeds a threshold in S20, the action position correction unit 133 may re-execute the steps corresponding to S13 and thereafter in S20. In this case, the process may be re-executed from S10.

[0057] Furthermore, not only when the correction amount exceeds the threshold, but also when the correction amount cannot be calculated, the reference position may be changed and the correction amount may be calculated again. Furthermore, when the correction amount exceeds the threshold or when the correction amount cannot be calculated, instead of or in addition to changing the reference position, the recognition conditions may be changed, such as by widening the recognition range.

[0058] Second Embodiment Next, a second embodiment will be described. In the following description of the second embodiment, elements having the same reference numerals as those used previously are the same as those in the previous embodiments unless otherwise specified. Furthermore, when only a portion of the configuration is described, the previously described embodiment can be applied to the other portions of the configuration.

[0059] The second embodiment relates to the master data generating unit 131. In the second embodiment and the following, the master data may be referred to as a master image. In the second embodiment, the master image of the product 50 is generated from the shape data of the product 50.

[0060] 8, the master data generation unit 131 acquires a reference position, shape data, surface information, light source information, and camera information. In this embodiment, the reference position is a specific position on the product 50. A mark may be attached to the specific position on the product 50. If a mark is attached, the reference position can be referred to as a reference mark position.

[0061] The shape data is data on the shape of the product 50. The shape data is stored in the shape data storage unit 124. The reference position can be specified by the user. Once the master data generation unit 131 acquires the shape data and the reference position, it can generate a master image using CAD.

[0062] However, the master data generation unit 131 also acquires surface information, light source information, and camera information. The surface information is information about the surface texture of the product 50. The light source information is information about the light source of the shooting environment in the real world. The light source information is information that indicates how much light is falling on the product 50 in the real world and how. The camera information is information that indicates the characteristics of the camera 30, such as the F-number and lens distortion. This information is input by the user. Furthermore, this information may be stored in the robot information storage unit 123, the shape data storage unit 124, etc. The surface information, light source information, and camera information are information about the real environment that affects the recognition results when recognizing the recognized part in the real world.

[0063] The master data generating unit 131 acquires surface information, light source information, and camera information, and generates a two-dimensional image that is close to a real image as a master image, reflecting this information.

[0064] In this way, a master image can be created before the product 50 is brought to the site, without having to photograph the product 50 on-site using the camera 30. This reduces the preparation time for product manufacturing on-site.

[0065] Furthermore, the master data generation unit 131 acquires information about the real environment that affects the recognition result when the camera 30 recognizes the recognized part in the real world. Then, it generates a master image that reflects the acquired information about the real environment. Therefore, it is possible to generate a master image that is close to reality.

[0066] Third Embodiment The third embodiment also relates to the master data generation unit 131. In the third embodiment, the master data generation unit 131 generates an intentionally blurred image as a master image. The blurred image takes into consideration the depth of field.

[0067] The camera 30 can capture a wide range by capturing an image from a higher position than the position at which the reference mark, which is the reference position, can be clearly captured. Because it can capture a wide range, even if the position of the reference mark at the work site is significantly different from the position of the reference mark in the master image, the reference mark is likely to be included in the captured image. However, because the image is captured from a higher position than the position at which the reference mark can be clearly captured, even if the reference mark can be captured, it will be blurred. Therefore, in the third embodiment, in addition to a master image corresponding to a position at which the reference mark can be clearly captured, a blurred master image captured from a higher position is also prepared.

[0068] In the third embodiment, a plurality of master images taken from different heights are prepared as master images taken from a higher position.

[0069] 9 shows the process executed by the operation position correction unit 133 in the third embodiment. In S101, the camera 30 is moved to a recognition position where an image of the reference mark is captured. This recognition position may be the recognition position calculated in S13 in the first embodiment.

[0070] In S102, the position of the reference mark is measured. Specifically, an image is taken at the recognition position by the camera 30. Then, the image is compared with a master image in which the reference mark is clearly captured, and the reference mark is detected in the captured image.

[0071] In S103, it is determined whether or not the reference mark has been found. If the determination result in S103 is YES, the process proceeds to S109, which will be described later. If the determination result in S103 is NO, that is, if the reference position has not been found, the process proceeds to S104.

[0072] In S104, the height of the camera 30 is changed to the next shooting height. Specifically, the height of the camera 30 is set to the shooting height corresponding to the master image that is assumed to be the next highest shooting height after the master images that have been used for matching so far.

[0073] In S105, the position of the reference mark is measured again. In S105, a master image corresponding to the height of the camera 30 at this time is used. This makes it possible to measure the position of the reference mark even if the height of the camera 30 is increased. In S106, it is determined again whether the reference mark has been found. If the reference mark has not been found, the process returns to S104, and the height of the camera 30 is further changed.

[0074] If it is determined in S106 that the reference mark has been found, the process proceeds to S107, where the horizontal position of the camera 30 is adjusted to the reference mark found in the process of S105, and the height is returned to the same height as in S101.

[0075] The process of S108 is the same as S102, and the position of the reference mark in the captured image is measured by comparing it with master data obtained when the reference mark is clearly photographed.

[0076] In the next step S109, the positions of the reference marks measured in step S108 are stored. When the positions of the reference marks can be measured by the camera 30 in this way, the positional deviation is calculated from the positions of the reference marks shown in the master image and the stored positions of the reference marks. Then, the relative motion position data is corrected based on the positional deviation.

[0077] <Fourth Embodiment> The fourth embodiment also relates to the master data generation unit 131. As shown in Fig. 10 , in the fourth embodiment, the master data generation unit 131 acquires prototype image data. The prototype is a prototype of the product 50. The prototype image data is image data obtained by photographing the prototype.

[0078] The master data generation unit 131 generates a two-dimensional image of the product 50 based on the prototype image data, and uses this as a master image. The prototype may not be exactly the same as the product 50, but it has a shape and surface texture similar to that of the product 50. Therefore, by generating a master image based on the prototype image data, it is possible to generate a realistic master image that is close to a master image generated by photographing the actual product 50.

[0079] Fifth Embodiment The fifth embodiment also relates to the master data generation unit 131. In the fifth embodiment, a master image generated from shape data is corrected using one or more types of correction parameters to generate multiple master images. Then, from the multiple master images, a master image that is suited to the actual environment is selected and used.

[0080] 11 shows the procedure for determining a master image in the fifth embodiment. Steps S110 and S111 are executed by the master data generation unit 131. In step S110, a two-dimensional image is generated as the original master image. This two-dimensional image is generated from shape data, such as the image generated in the second embodiment.

[0081] In S111, the two-dimensional image generated in S110 is modified using modification parameters. The modification parameters may be stored in a predetermined storage unit such as the master data storage unit 122, or may be acquired by the master data generation unit 131 based on a user input.

[0082] The correction parameters are parameters by which the captured image is corrected. The camera 30 recognizes the reference position using the image captured by the camera 30. The correction parameters by which the image is corrected are recognition parameters that affect the recognition of the reference position. Examples of the recognition parameters include contrast, scale, and light / dark reversal. The master data generation unit 131 generates multiple master images by changing one or more correction parameters. The generated multiple master images are then stored in the master data storage unit 122.

[0083] S112 is executed by the action position correction unit 133. In S112, one master image that is suited to the real environment is selected from the multiple master images stored in the master data storage unit 122. S112 may be selected based on a user operation. Alternatively, the real environment may be measured, and the action position correction unit 133 may select one master image based on the measured real environment. Alternatively, the multiple master images may be selected in a pre-set order. The action position correction unit 133 calculates the positional deviation using the selected master image.

[0084] Sixth Embodiment In the sixth embodiment, a master image of the robot system 10 is provided for a portion other than the robot 20, and a master image of the product 50 is also provided. The master image of the robot system 10 is provided with marks for easy recognition. The landmarks are, for example, circles.

[0085] Since the marks are provided, the master data of the robot system 10 is used as a landmark master image. The portion to which the landmark is provided is, for example, a corner of the top surface of the work table 60.

[0086] 12 shows the procedure executed in the sixth embodiment. In S121, a workbench 60 is created in a virtual space.

[0087] In step S122, a landmark master image and a product master image are created in the virtual space. At this time, a light source can be set to create the master images in order to approximate the real environment.

[0088] In S123, the landmark is photographed by the camera 30. Then, the photographed image is compared with the landmark master image, and the positional deviation of the robot 20 with respect to the work table 60 is corrected.

[0089] In S124, based on the image captured in S123, the height coordinate z when capturing the image is adjusted to correct for positional deviation so as to capture an image without blur. Note that S123 may be executed again after adjusting the height coordinate z. Also, in S124, the correction amount for the master image is determined. This correction amount is a correction amount for bringing the master image closer to the captured image. For example, parameters for determining the correction amount include scale and contrast.

[0090] In S125, the landmark master image and the product master image are corrected by the correction amount determined in S124. In S126, the product 50 is photographed by the camera 30 in order to correct the position of the robot 20 relative to the product 50. The photographed image is then compared with the product master image, and the positions of the camera 30 and the robot 20 to which the camera 30 is attached relative to the product are corrected.

[0091] Seventh Embodiment In the seventh embodiment, as shown in FIG. 13, the master data generation unit 131 includes a 2D image generation unit 131a, a camera-derived image generation unit 131b, and a master data switching unit 131c.

[0092] The 2D image generating unit 131a generates a master image from the shape data of the product 50. This process may be the same as that in the second embodiment, for example. The 2D image generating unit 131a then stores the generated master image in the master data storage unit 122.

[0093] After the robot 20 is installed at the site, the camera-derived image generation unit 131b generates a master image based on an image of the product 50 taken by the camera 30. Then, the camera-derived image generation unit 131b stores the generated master image in the master data storage unit 122.

[0094] After the camera-derived image generating unit 131b generates a master image and stores it in the master data storage unit 122, the master data switching unit 131c switches the master image to be used to the master image generated by the camera-derived image generating unit 131b. The timing of the switching is arbitrary. For example, the switching may be performed immediately after the camera-derived image generating unit 131b generates the master image.

[0095] Before the master image is switched by the master data switching unit 131c, the motion position correction unit 133 performs position correction using the master image generated by the 2D image generation unit 131a. After the master image is switched, the motion position correction unit 133 performs position correction using the master image generated by the camera-derived image generation unit 131b.

[0096] Eighth Embodiment In the eighth embodiment, a master image generated from shape data is used for alignment before creating a master image derived from a camera. Fig. 14 shows the procedure executed in the eighth embodiment. Note that, at the time of executing Fig. 14, the master image generated from shape data described in the previous embodiment is stored in the master data storage unit 122.

[0097] In S201, the product 50 is placed in a predetermined position. In S202, it is determined whether or not there is a camera-derived master image. If there is no camera-derived master image (S202: YES), the process proceeds to S203.

[0098] In S203, the shooting environment is made closer to a virtual space. For example, since there is no ambient light in a virtual space, it is conceivable to attach a hood to reduce the ambient light. Also, it is conceivable to exclude from the shooting range any objects that are not in the master image generated from the shape data. Furthermore, if a light source is set in the virtual space and the master image is generated, it is conceivable to add a light source in the real environment as well.

[0099] In S204, the camera 30 is moved to the position of the reference mark. In S205, the camera 30 is made to recognize part or all of the product 50, which is the recognized part. Then, the position of the reference mark is measured based on the image. In S206, the position of the reference mark is saved. Since the shooting environment is made closer to a virtual space in S203, the success rate of measuring and saving the position of the reference mark in S205 and S206 is improved.

[0100] Saving the position of the reference mark means updating the already stored position of the reference mark to the position of the newly measured reference mark. Along with updating the position of the reference mark, the x- and y-coordinates of the recognition position are also updated. The x- and y-coordinates of the recognition position are the same as the x- and y-coordinates of the reference mark.

[0101] In S207, it is determined whether the positions of all the reference marks have been confirmed. If there are multiple reference marks, the determination result in S207 may be NO. If there are multiple reference marks, the orientation of the product 50 and the orientation of the robot 20 relative to the product 50 can also be confirmed, so multiple reference marks may be provided on the product 50.

[0102] If the determination result in S207 is NO, the process returns to S204, where the camera 30 is moved to the position of a fiducial mark that has not yet been confirmed, and the fiducial mark position is measured and saved. If the determination result in S207 is YES, the process proceeds to S208. In S208, the shooting environment is returned to the real environment. In other words, the shooting environment is returned to the state before S203 was executed. Thereafter, the process proceeds to S209 in FIG. 15.

[0103] In S209, the camera 30 is moved to the position of a reference mark. In S210, an image is captured by the camera 30. This image is then used as a new master image. In the following S211, it is determined whether all reference marks have been confirmed. If the determination result in S211 is NO, the process returns to S209, where the camera 30 is moved to the position of a reference mark that has not yet been captured, and a master image is generated.

[0104] Steps S202 to S211 are the processing of the master data generation unit 131. The processing of steps S202 to S207 allows the position of the reference mark to be measured correctly based on the master image captured in virtual space. This allows the camera 30 to be moved correctly to the position of the reference mark. Then, the processing of steps S208 to S211 updates the master image to the image captured by the camera 30. Because the image captured by the camera 30 at an appropriate position can be used as the master image, the success rate of positional deviation correction in subsequent processing improves.

[0105] If the determination result in S211 is YES, the process proceeds to S212. Also, if the determination result in S202 is NO, that is, if it is determined that there is a master image from a camera when the product 50 is placed, the process proceeds to S212.

[0106] In S212, the camera 30 is moved to the position of a reference mark. In the following S213, the position of the reference mark is measured by the camera 30. In the following S214, it is determined whether all reference marks have been confirmed. If the determination result in S214 is NO, the process returns to S212, where the camera 30 is moved to the position of a reference mark that has not yet been photographed, and the position of the reference mark is measured.

[0107] If the determination result in S214 is YES, the process proceeds to S215. S215 is performed by the motion position correction unit 133 and the robot motion generation unit 134. In S215, the motion position is corrected in accordance with the positional deviation between the position of the reference mark measured in S212 to S214 and the position of the reference mark in the corresponding master image. Then, the robot motion is generated based on the corrected motion position.

[0108] Steps S216 and S217 are executed by the robot operation control unit 135. In step S216, assembly work is performed according to the robot operation generated in step S215. In step S217, the product 50 is moved. In step S218, it is determined whether assembly of all products has been completed. If the determination result in step S218 is YES, the process ends. If the determination result in step S218 is NO, the process returns to step S201, and another product 50 is assembled.

[0109] Although the embodiments have been described above, the disclosed technology is not limited to the above-described embodiments, and the following modifications are also included in the scope of the disclosure. Furthermore, various modifications other than those described below can be made without departing from the spirit of the invention.

[0110] For example, the recognition device is not limited to the camera 30. The recognition device may be any device that can recognize the recognized part including the reference position. Other examples of recognition devices include a mechanical sensor and an ultrasonic sensor. When the recognition device is one of these sensors, the master data is a sensor value.

Claims

1. A robot motion design device comprising: a motion generation information storage unit (121) that stores relative motion position data, which is data indicating one or more motion positions required to generate the motion of a robot (20) during product manufacturing as relative positions with respect to a target object; a master data storage unit (122) that stores master data, which is recognition result data when a recognition device attached to the robot recognizes a reference position while the recognition device is in a recognition position that recognizes the reference position; a motion position correction unit (133) that causes the recognition device to recognize the reference position and corrects the relative motion position data based on a comparison between the recognition result and the master data; and a robot motion generation unit (134) that generates the motion of the robot based on the relative motion position data corrected by the motion position correction unit.

2. A robot motion design device as described in claim 1, wherein the reference positions include a reference position set on a fixed object whose position does not change relative to the robot and a reference position set on a product handled by the robot, the master data storage unit stores the master data for the reference position set on the fixed object and master data for the reference position set on the product, and the motion position correction unit corrects the relative motion position data based on the reference position set on the fixed object, and then further corrects the relative motion position data based on the reference position set on the product.

3. A robot motion design device as described in claim 1, comprising a robot information storage unit (123) that stores tool information including the height direction length of a tool equipped on the robot, and the motion position correction unit corrects the relative motion position data based on the height direction length of the tool included in the tool information.

4. A robot motion design device as described in claim 1, wherein there are a plurality of reference positions, the master data storage unit includes recognition result data when the recognition device recognizes the reference positions at a plurality of recognition positions where the recognition device recognizes each of the plurality of reference positions, and the motion position correction unit corrects the relative motion position data using another of the reference positions as the recognition target based on the fact that the correction amount of the relative motion position data using one of the reference positions as the recognition target exceeds a threshold.

5. A robot motion design device comprising: a motion generation information generation unit (132) that generates relative motion position data, which is data indicating one or more motion positions required to generate the motion of a robot during product manufacturing as relative positions to a target object; and a product manufacturing information storage unit (120) that stores information required to generate the motion of the robot during product manufacturing, wherein the product manufacturing information storage unit stores the relative motion position data generated by the motion generation information generation unit and master data that is recognition result data when a recognition device attached to the robot recognizes a reference position while the recognition device is in a recognition position where the recognition device recognizes the reference position.

6. A robot motion design device as described in claim 5, wherein the motion generation information generation unit determines a range for recognizing the recognized part (70) based on information including the size of the recognized part (70) including the reference position, and at least one of the resolution (α) of the recognition device and the processing error (ΔX, ΔY) of the recognized part.

7. A robot motion design device as described in claim 5, wherein the recognition device has a recognition range (80) that changes depending on the height from the object to be recognized, and the motion generation information generation unit determines the position of the recognition device for recognizing the recognized part based on the recognition range for recognizing the recognized part, which is a range determined depending on the size of the recognized part (70) that includes the reference position, and the relationship between the recognition range and the height from the recognized part.

8. A robot motion design device as described in claim 5, comprising: a shape data storage unit (124) that stores shape data of a product handled by the robot; and a master data generation unit (131) that generates the master data based on the shape data of the product and the reference position, and stores the generated master data in the product manufacturing information storage unit.

9. A robot motion design device as described in claim 8, wherein the master data generation unit acquires information about the real environment that affects the recognition results when the recognition device recognizes the recognized part including the reference position in the real world, and generates the master data that reflects the acquired information about the real environment.

10. A robot motion design device as described in claim 9, wherein the master data generation unit generates a plurality of master data in which the height from the reference position to the recognition device is made different from each other to cause the recognition device to recognize the recognized part.

11. A robot motion design device as described in claim 8, wherein the master data generation unit generates the master data based on the shape data of the product, the reference position, and the recognition parameters of the recognition device, and generates multiple pieces of master data by changing the recognition parameters multiple times.

12. A robot motion design device as described in claim 8, wherein the master data generation unit updates the recognition position by having the recognition device recognize a recognized part that includes the reference position and measuring the reference position, and then has the recognition device recognize the recognized part at the updated recognition position, and stores the recognition result as new master data in the product manufacturing information storage unit.

Images