Simulation device, information processing method, robot system control method, robot system, method for manufacturing articles, program, and recording medium.

Abstract

To provide a technology capable of making imaging operation efficient when a plurality of positions in a workpiece are imaged.SOLUTION: A control device includes: imaging means capable of imaging a workpiece in an imaging range of a predetermined size; movement means for moving the workpiece or the imaging means; and setting means for setting a plurality of portions in the workpiece to be imaged by the imaging means. The control device moves the workpiece or the imaging means by the movement means so as to have a first positional relationship in which at least two portions can be included in the imaging range.SELECTED DRAWING: Figure 13

Description

[Technical field]

[0001] The present invention relates to a control device, a control method, a control method for a robot system, a robot system, a manufacturing method for an article, a program, and a recording medium. [Background technology]

[0002] Conventionally, as an appearance inspection system for inspecting the surface condition of a workpiece or the assembly state, a configuration is known in which a robot holds a camera or a workpiece and operates the robot to capture images of multiple positions of the workpiece with a camera. As a robot system for appearance inspection like this, a technology has been disclosed that makes it possible to configure an operation program for specifying a surface of the workpiece to be inspected and operating a robot to capture an image of the specified surface (see Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2018-132331 A Summary of the Invention [Problem to be solved by the invention]

[0004] The device described in Patent Document 1 is configured to calculate an operation program that will result in a positional relationship between the workpiece and the camera for capturing images for each location specified for inspection. Therefore, as the number of locations to be inspected in the workpiece increases, the number of times the robot operates increases, which creates a problem in that it takes longer to capture images.

[0005] Therefore, an object of the present invention is to provide a technique that can improve the efficiency of imaging operations when imaging multiple positions on a workpiece. [Means for solving the problem]

[0006] A first aspect of the present invention is a control device comprising an imaging means capable of imaging a workpiece within an imaging range of a predetermined size, a moving means for moving the workpiece or the imaging means, and a setting means for setting multiple parts on the workpiece to be imaged by the imaging means, wherein the workpiece or the imaging means is moved by the moving means so that at least two of the parts are in a first positional relationship that can be included in the imaging range.

[0007] In addition, a second aspect of the present invention is a control method comprising the steps of: imaging a workpiece using an imaging means capable of imaging the workpiece within an imaging range of a predetermined size; moving the workpiece or the imaging means using a moving means; and setting a plurality of parts on the workpiece to be imaged by the imaging means, and moving the workpiece or the imaging means using the moving means so that at least two of the parts are in a first positional relationship that can be included in the imaging range. Effect of the Invention

[0008] According to the present invention, when imaging a plurality of positions on a workpiece, the imaging operation can be made more efficient. [Brief description of the drawings]

[0009] [Figure 1] 1 is a schematic diagram of a robot system according to a first embodiment of the present invention. [Diagram 2] 1 is a diagram showing a configuration of a robot according to a first embodiment of the present invention. [Diagram 3] 1 is a diagram illustrating a simulation device according to a first embodiment of the present invention. [Figure 4] 1 is a control block diagram of an information processing device and a simulation device according to a first embodiment of the present invention. [Diagram 5] 5 is a flowchart showing a process for acquiring data on teaching points of a robot, which is executed by a CPU of the simulation device according to the first embodiment of the present invention. [Figure 6]FIG. 2 is a diagram showing an example of the configuration of a screen displayed on a display when an input of an inspection position by an operator is received in the simulation device according to the first embodiment of the present invention. [Figure 7] FIG. 2 is a diagram showing an example of the configuration of a screen displayed on a display when an input of an imaging range of a camera by an operator is accepted in the simulation device according to the first embodiment of the present invention. [Figure 8] 5 is a flowchart showing a process for acquiring data of teaching points where a plurality of inspection positions fall within an imaging range of a camera, the process being executed by a CPU according to the first embodiment of the present invention. [Figure 9] FIG. 2 is a diagram showing an example of a state in which a plurality of inspection positions are set on a workpiece in the simulation device according to the first embodiment of the present invention. [Figure 10] FIG. 1A is a diagram showing the state in which the principal component vector and the secondary component vector of an inspection position according to a first embodiment of the present invention have been calculated, and FIG. 1B is a diagram showing how a rectangle that encompasses all inspection positions in the candidate list is obtained from the values ​​calculated in FIG. [Figure 11] 5(a) to 5(f) are diagrams showing a state in which a rectangle encompassing an inspection position in a candidate list is obtained each time an inspection position is added to a candidate list according to the first embodiment of the present invention. [Figure 12] 13(a) to 13(f) are figures showing the state in which, after one piece of teaching point data has been obtained in the first embodiment of the present invention, a rectangle encompassing the inspection position in the candidate list is obtained each time an inspection position is added to the candidate list. [Figure 13] FIG. 4 is a diagram showing a state in which data of teaching points for imaging all inspection positions according to the first embodiment of the present invention has been acquired. [Figure 14] FIG. 1 is a diagram showing an example of a screen configuration when data of teaching points acquired in a simulation device according to a first embodiment of the present invention and an inspection position included in an imaging range when an image is captured by a camera based on the data of the teaching points are displayed on a display. [Figure 15]10 is a flowchart showing a process for acquiring data on teaching points of a robot, which is executed by a CPU of a simulation device according to a second embodiment of the present invention. [Figure 16] FIG. 11 is a diagram showing an example of the configuration of a screen displayed on a display when an input of an imaging range of a camera by an operator is received in the simulation device according to the second embodiment of the present invention. [Figure 17] 10 is a flowchart showing a process for acquiring data of teaching points where a plurality of inspection positions fall within an imaging range of a camera, the process being executed by a CPU according to a second embodiment of the present invention. [Figure 18] 13 is a flowchart showing a process for acquiring data on teaching points of a robot, which is executed by a CPU of a simulation device according to a third embodiment of the present invention. [Figure 19] FIG. 13 is a diagram showing an example of the configuration of a screen displayed on a display when an input of data of an inspection position by an operator is accepted in the simulation device according to the third embodiment of the present invention. [Figure 20] 13 is a flowchart showing a process for acquiring data of teaching points where a plurality of inspection positions fall within an imaging range of a camera, the process being executed by a CPU according to a third embodiment of the present invention. [Figure 21] 13 is a flowchart showing a process for acquiring data on teaching points of a robot, which is executed by a CPU of a simulation device according to a fourth embodiment of the present invention. [Figure 22] FIG. 13 is a diagram showing an example of the configuration of a screen displayed on a display when an input of a position of an imaging range by an operator is received in a simulation device according to a fourth embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] [First embodiment] Hereinafter, the embodiment of the present invention will be described in detail with reference to the drawings. Note that the configuration shown below is merely an example, and the details of the configuration can be appropriately changed by those skilled in the art without departing from the spirit of the invention. Also, the numerical values ​​used in the following description are for reference only and are merely examples.

[0011] <Robot system configuration> Fig. 1 is an explanatory diagram showing a robot system 1 of the first embodiment. In the first embodiment, a robot system that performs visual inspection of an inspection surface 30a, which is an arbitrary flat surface of a workpiece 30, using a camera 20 held by a robot 10 as shown in Fig. 1 will be described as an example. In other words, the robot system 1 of the first embodiment is a robot system in which the robot 10, the camera 20, and the workpiece 30 are set as components, and which performs visual inspection of an object to be inspected in a two-dimensional space.

[0012] The workpiece 30 is placed on a fixed base 31. The robot 10 is connected to an information processing device 40. The information processing device 40 is connected to a simulation device 50. An operator operates the simulation device 50 to obtain data on teaching points of the robot 10 required for visual inspection.

[0013] <Robot configuration> Fig. 2 is a diagram showing the configuration of a robot 10 according to the first embodiment. As shown in Fig. 2, the robot 10 is a vertical multi-joint robot having six rotational joints. The robot 10 has a base 11 as a base end, and a link 12 is connected to the base 11 via a joint J1, a link 13 is connected to the link 12 via a joint J2, and a link 14 is connected to the link 13 via a joint J3. In addition, the robot 10 has a link 15 connected to the link 14 via a joint J4, a link 16 connected to the link 15 via a joint J5, and a camera 20 connected to the link 16 via a joint J6.

[0014] Camera 20 as an imaging means is configured to be able to image workpiece 30 (see FIG. 1) within imaging range 22. Imaging range 22 of camera 20 is configured so that the size of the range can be specified by an operator, and is configured to capture an image within a range of a specified predetermined size. Note that in FIG. 2, the shape of imaging range 22 is depicted in a visible form, but this is a description to clarify imaging range 22, and in reality it is an invisible range.

[0015] The robot 10 is connected to an information processing device 40 and operates according to commands from the information processing device 40. By inputting a robot program into the information processing device 40, an operator can perform an appearance inspection by operating the robot 10 and capturing images with the camera 20. In other words, the information processing device 40 is configured to be capable of executing a process of operating the robot 10 and moving the camera 20, and a process of capturing an image of the workpiece 30 with the camera 20 that can point at the workpiece 30 within an imaging range of a predetermined size.

[0016] The robot 10 of the first embodiment is configured to move the camera 20 connected to the link 16 via the joint J6 by operating using data of teaching points that serve as position and orientation data, and is configured as a moving means for moving the camera 20.

[0017] <Simulation device> 3 is a diagram showing a simulation device 50 of the first embodiment. The simulation device 50 includes a computer 51 serving as the device main body, a display 52 for displaying, and a keyboard 54 and a mouse 53 for inputting. In the first embodiment, the simulation device 50 will be described as being a desktop PC, which is a general-purpose computer, but is not limited to this. The simulation device 50 may be, for example, a general-purpose computer such as a laptop PC, a tablet PC, or a smartphone, or a teaching pendant.

[0018] The display 52 as a display unit displays images for the operator to instruct or edit the program. The display 52 also displays the robot 10v, the workpiece 30v, etc. in a virtual space to confirm the operation of the robot 10 based on the instruction or program editing. In the following description, objects displayed in the virtual space are indicated by reference characters with a "v" suffix. For example, the robot 10 displayed in the virtual space is indicated as the robot 10v, and the workpiece 30 displayed in the virtual space is indicated as the workpiece 30v.

[0019] <Robot system control block diagram> Fig. 4 is a control block diagram showing the hardware configuration of the information processing device 40 and the control device of the simulation device 50. As shown in Fig. 4, the information processing device 40 is configured by a computer. The information processing device 40 includes a CPU (Central Processing Unit) 45, which is an example of a processor as an information processing unit of the information processing device 40, and a memory 46 and a hard disk 47 as storage units. The information processing device 40 also includes a communication interface 48 and an I / O 49. The CPU 45, the memory 46, the hard disk 47, the communication interface 48, and the I / O 49 are connected to each other via a bus so as to be able to communicate data with each other.

[0020] The memory 46 includes, for example, a read only memory (ROM) and a random access memory (RAM). The memory 46 stores basic programs related to the operation of the computer, and temporarily stores various data such as the results of calculations.

[0021] The hard disk 47 records the results of calculations performed by the CPU 45 and various data acquired from the outside, as well as programs for causing the CPU 45 to execute various processes.

[0022] The communication interface 48 is connected to a communication interface 58 of the simulation device 50 via an Ethernet (registered trademark) cable 60, thereby enabling transmission and reception of information to and from the simulation device 50.

[0023] The I / O 49 is an interface with an external device, and is composed of, for example, a recording disk drive. A removable memory 61 such as a recording disk or a storage device can be connected to the I / O 49. The I / O 49 can read various data, programs, etc. recorded in the removable memory 61.

[0024] As described above, the simulation device 50 is configured with a general-purpose computer. The simulation device 50 includes a CPU 55, which is an example of a processor, as an information processing unit of the simulation device 50, and a memory 56 and a hard disk 57 as a storage unit. The simulation device 50 also includes a communication interface 58, an I / O 59, the display 52 described above as a display unit, and a mouse 53 and a keyboard 54 as input units. The display 52, the mouse 53, the keyboard 54, the CPU 55, the memory 56, the hard disk 57, the communication interface 58, and the I / O 59 are connected to each other via a bus so as to be able to communicate data with each other.

[0025] The memory 56 has, for example, a ROM and a RAM, and stores basic programs related to the operation of the computer, and also temporarily stores various data such as the results of calculation processing.

[0026] The hard disk 57 records the results of the calculations performed by the CPU 55 and various data acquired from the outside, as well as programs for causing the CPU 55 to execute various processes.

[0027] As described above, the communication interface 58 is connected to the communication interface 48 of the information processing device 40 via an Ethernet (registered trademark) cable 60, thereby enabling transmission and reception of information to and from the information processing device 40.

[0028] The I / O 59 is an interface with external devices. In addition, a removable memory 61 such as a recording disk or a storage device can be connected to the I / O 59. The I / O 59 can read various data, programs, and the like recorded in the removable memory 61.

[0029] <Acquisition of robot teaching point data> Next, we will explain the processing performed by the simulation device 50 to obtain teaching point data that serves as the position and posture data of the robot 10 when moving the camera 20 of the robot 10 when performing an external inspection of the workpiece 30 using the robot 10.

[0030] 5 is a flowchart showing a process executed by the CPU 55 of the simulation device 50 of the first embodiment to acquire data on teaching points of the robot 10. The simulation device 50 executes the processes of steps S10 to S14 shown in FIG. 5 to acquire data on teaching points of the robot 10 used when moving the robot 10 to capture images of a plurality of inspection positions set on the workpiece 30 by the camera 20.

[0031] 5, the CPU 55 acquires data of teaching points for moving the camera 20 by the robot 10 so that the first positional relationship is achieved in which at least two inspection positions can be included in the imaging range of the camera 20. The CPU 55 that executes the processes of steps S10 to S14 constitutes an acquisition means. The data of teaching points also constitutes information used when the camera 20 is moved by the robot so that the first positional relationship is achieved.

[0032] In the process of step S10, the CPU 55 displays a virtual space including the robot 10v and the workpiece 30v on the display 52 so that the worker can visually operate them (S10).

[0033] In the process of step S11, the CPU 55 receives input of data on a plurality of inspection positions by the operator (S11).

[0034] Fig. 6 is a diagram showing an example of the configuration of a screen displayed on the display 52 when receiving an input of an inspection position by an operator. As shown in Fig. 6, the display 52 displays a position setting window 100 in which various information about the inspection position is displayed, and a robot 10v, a camera 20v, a workpiece 30v, and the like in a virtual space.

[0035] The operator can set a desired position on the workpiece 30v to be inspected by selecting any position on the workpiece 30v with the mouse 53. The desired position is displayed as a marker 110 in the virtual space. In other words, the CPU 55 is configured to set a plurality of portions of the workpiece 30v to be imaged by the camera 20 by executing a process of accepting the position selected by the operator on the workpiece 30v displayed on the display 52 as the inspection position.

[0036] With this configuration, the robot system 1 can allow the operator to select any inspection position from the workpiece 30v displayed on the display 52, thereby improving the efficiency of the operator's work when setting the inspection position.

[0037] The set inspection position is displayed in a table 100a in a position setting window 100 together with a column 101 indicating the number of the inspection position (inspection number), a column 102 indicating the x coordinate of the inspection position, and a column 103 indicating the y coordinate of the inspection position. The set inspection position is also displayed in table 100a together with information in a column 104 indicating a normal vector with respect to an inspection surface 30av, which is a plane to be visually inspected in the workpiece 30v.

[0038] In the example shown in Fig. 6, the coordinate origin Ov of the xyz coordinate system of the inspection position is a corner 32v on the inspection surface 30av of the workpiece 30v, and the unit of each coordinate axis component is millimeters. Therefore, for example, the inspection position of inspection number 1 is 50 mm in the x-axis direction and 100 mm in the y-axis direction from the coordinate origin Ov.

[0039] A Confirm button 105 and a Cancel button 106 are displayed in the position setting window 100. When the Confirm button 105 is selected, the CPU 55 advances the process from step S11 of the process of acquiring data of the teaching points of the robot 10 to the next step, step S12. When the Cancel button 106 is selected, the CPU 55 can cancel the process of acquiring data of the teaching points of the robot 10.

[0040] When the confirm button 105 is selected, the CPU 55 creates a list of inspection positions that lists data correlating each inspection position selected by the operator with the data of each inspection position displayed in the table 100a, and stores the list in the memory 56.

[0041] When the worker selects an inspection position for the workpiece 30v, the CPU 55 changes the posture of the robot 10v displayed on the display 52 to the posture when the robot 10v approaches the inspection position selected by the worker. Also, when the worker selects any information from the inspection position data displayed in the table 100a, the CPU 55 changes the posture of the robot 10v to the posture when the robot 10v approaches the inspection position corresponding to the information selected by the worker.

[0042] With this configuration, the simulation device 50 makes it possible for the operator to check the posture of the robot 10 when inspecting the inspection position selected by the operator, and whether or not there is interference between the robot 10 and surrounding objects.

[0043] The process of step S11 constitutes a process of setting the portion of the workpiece 30 to be imaged by the camera 20. Moreover, the CPU 55 that executes the process of step S11 constitutes a setting means.

[0044] In the process of step S12, the CPU 55 receives an input of the imaging range of the camera 20 from the worker (S12). In this process, the CPU 55 receives an input of the size of the imaging range 22 from the worker.

[0045] Fig. 7 is a diagram showing an example of the configuration of a screen displayed on the display 52 when the image capturing range of the camera 20 is input by the worker in the first embodiment. As shown in Fig. 7, a range setting window 120 displaying information on the image capturing range of the camera 20 and an enlarged view of the robot 10v including the camera 20v and the image capturing range 22v in the virtual space are displayed on the display 52. ​​In the example shown in Fig. 7, an oblique view and a side view are displayed as the enlarged view of the robot 10v.

[0046] The range setting window 120 displays a width input field 121 in which the width of the imaging range 22 can be input, a height input field 122 in which the height of the imaging range 22 can be input, and a depth input field 123 in which the depth of the imaging range 22 can be input. Here, the width of the imaging range 22 is the length in the left-right direction relative to the direction of the optical axis 20a of the camera 20 (optical axis direction). The height of the imaging range 22 is the length in the up-down direction relative to the optical axis direction of the camera 20. The depth of the imaging range 22 is the length in the optical axis direction of the camera 20. In the example shown in FIG. 7, the unit of the value input in each input field is millimeters.

[0047] The CPU 55 sets the size of the imaging range 22 by inputting data into a width input field 121, a height input field 122, and a depth input field 123. This enables the robot system 1 to set the size of the imaging range 22 that can satisfy the image resolution required for visual inspection.

[0048] The CPU 55 changes the width 126, height 127, and depth 128 of the imaging range 22v displayed on the display 52 in accordance with the values ​​input in the width input field 121, height input field 122, and depth input field 123. In other words, the display 52 displays the imaging range 22v in accordance with the values ​​input in the width input field 121, height input field 122, and depth input field 123.

[0049] Furthermore, the range setting window 120 displays a number input field 133 in which the upper limit value of the number of inspection positions to be included in one imaging range 22 can be input. When the number of inspection positions to be included in one imaging range 22 reaches the number set in the number input field 133, the CPU 55 calculates data of teaching points for imaging in the imaging range 22, and acquires the data of the teaching points.

[0050] With this configuration, the robot system 1 of the first embodiment can prevent a decrease in visibility of each inspection position caused by the camera 20 capturing images of many inspection positions in a single capture.

[0051] The CPU 55 configures the length in the left-right direction and the length in the up-down direction with respect to the optical axis direction of the camera 20 by inputting the width input field 121, the height input field 122, and the depth input field 123 to set the predetermined size of the imaging range 22. The CPU 55 configures the upper limit value setting means by inputting the number input field 133 to set the upper limit value of the number of inspection positions.

[0052] A Confirm button 124 and a Cancel button 125 are displayed in the range setting window 120. When the Confirm button 124 is selected, the CPU 55 advances the process from step S12 of the process of acquiring data of the teaching points of the robot 10 to the next step, step S13. When the Cancel button 125 is selected, the CPU 55 can cancel the process of acquiring data of the teaching points of the robot 10.

[0053] In the process of step S13, the CPU 55 performs a process of calculating data of teaching points such that a plurality of inspection positions fall within the range of the imaging range 22 of the camera 20 based on the input information, and acquiring the data of the teaching points (S13). In this process, the CPU 55 calculates and acquires data of teaching points such that the inspection positions are contained in a rectangle of width 126 and length 127 at a position of depth 128 of the imaging range 22.

[0054] Details of the process of step S13 will be described with reference to the flowchart shown in Fig. 8 and a case where the inspection positions i1 to i21 shown in Fig. 9 are set for the workpiece 30v. Note that the inspection positions i1 to i21 shown in Fig. 9 are set with inspection numbers 1 to 21, respectively.

[0055] FIG. 8 is a flowchart showing the process of acquiring data of teaching points where a plurality of inspection positions fall within the imaging range 22 of the camera 20, which is executed by the CPU 55 in the process of step S13.

[0056] In the process of step S21, the CPU 55 selects an arbitrary inspection position from among the multiple inspection positions input by the operator, and adds it to a list of inspection positions that are candidates for acquiring teaching point data (candidate list) (S21). In this process, the CPU 55 acquires data of the arbitrary inspection position from the list of inspection positions, and adds the acquired data of the inspection position to the candidate list.

[0057] The CPU 55 selects an inspection position using different criteria depending on whether the data of any inspection position has not been added to the candidate list or whether the data of any inspection position has been added to the candidate list. When the data of any inspection position has not been added to the candidate list, the CPU 55 selects the inspection position with the smallest inspection number in the list of inspection positions. In the example shown in Fig. 9, when none of the inspection positions i1 to i21 have been added to the candidate list, the CPU 55 selects inspection position i1, which is the inspection position with inspection number 1, which is the smallest inspection number in the list of inspection positions, as the inspection position to be added to the candidate list.

[0058] When data of any inspection position has been added to the candidate list, the CPU 55 selects data of the inspection position located closest to the previously added inspection position. In the example shown in Fig. 9, when inspection position i2 has been added to the candidate list next to inspection position i1, the CPU 55 selects inspection position i3 located closest to inspection position i2 as the inspection position to be added to the candidate list.

[0059] In the process of step S22, the CPU 55 performs a principal component analysis on the inspection position added to the candidate list to calculate the principal component vector of the inspection position (S22). In this process, the CPU 55 first calculates a variance-covariance matrix VC1 by the following formula.

number

[0060] Here, v(x) and v(y) are the variances of the respective inspection positions, and vc(x,y) and vc(y,x) are the covariances of the respective inspection positions, which are calculated by a known method. The CPU 55 calculates the eigenvalues ​​and eigenvectors of the variance-covariance matrix VC1 by principal component analysis, and calculates the eigenvector with the largest eigenvalue as the principal component vector.

[0061] In the process of step S23, the CPU 55 obtains a rectangle that includes all the inspection positions from the principal component vector, and obtains the long side and the short side of the obtained rectangle (S23). The details of the process of step S23 will be described with reference to Figs. 10(a) and (b).

[0062] 10(a) and (b) show an example in which inspection positions i1 to i3 are added to the candidate list. Fig. 10(a) shows a state in which the principal component vectors and the secondary component vectors of the inspection positions i1 to i3 are calculated. As shown in Fig. 10(a), the CPU 55 calculates a principal component vector 150 as the principal component vector of the inspection positions i1 to i3, and calculates a secondary component vector 151 perpendicular to the principal component vector 150.

[0063] The CPU 55 calculates the maximum value 160 and the minimum value 161 when each inspection position is transferred to the principal component vector 150. In the example shown in Fig. 10(a), the value of the inspection position i1 transferred to the principal component vector 150 is the maximum value 160, and the value of the inspection position i3 transferred to the principal component vector 150 is the minimum value 161.

[0064] Further, the CPU 55 calculates a maximum value 162 and a minimum value 163 when each inspection position is transferred to the subcomponent vector 151. In the example shown in Fig. 10(a), the value transferred to the subcomponent vector 151 of the inspection position i2 is the maximum value 162, and the value transferred to the subcomponent vector 151 of the inspection position i1 is the minimum value 163.

[0065] Fig. 10(b) is a diagram showing how a rectangle including all the inspection positions in the candidate list is obtained from the values ​​calculated in Fig. 10(a). As shown in Fig. 10(b), a rectangle 152 including the inspection positions i1 to i3 is obtained as a rectangle whose long side is parallel to the principal component vector 150 and whose short side is parallel to the secondary component vector 151. The length L1 of the long side of the rectangle 152 is calculated from the distance from the maximum value 160 to the minimum value 161, and the length L2 of the short side of the rectangle 152 is calculated from the distance from the maximum value 162 to the minimum value 163.

[0066] The CPU 55 obtains a center position 166 from a midpoint 164 of the long side and a midpoint 165 of the short side of the rectangle 152. In other words, the CPU 55 obtains the center position 166 from the midpoints 164, 165 which are the center values ​​of the respective directional components. This center position 166 is used when obtaining data of the teaching points of the robot 10 in a later process.

[0067] In this way, the CPU 55 performs the principal component analysis on the multiple inspection positions to obtain a rectangular area consisting of the principal component vectors and the subcomponent vectors on the xy plane. As a result, when setting an area including multiple inspection positions, the robot system 1 can set an optimal area according to the variation in the positions set as the inspection positions by calculating the principal component vectors and the subcomponent vectors by the principal component analysis and setting the area.

[0068] In the process of step S24, the CPU 55 determines whether the length L1 of the long side and the length L2 of the short side of the rectangle acquired in the process of step S23 are shorter than the width 126 and the height 127 of the imaging range 22 of the camera 20 set in the process of step S12 (S24). In this process, the CPU 55 acquires the input width 126 and height 127 of the imaging range 22, and compares the longer width with the length L1 of the long side of the rectangle, and compares the shorter width with the length L2 of the short side of the rectangle.

[0069] 7, for example, when a value of "200" is input in the width input field 121 and a value of "100" is input in the height input field 122, the CPU 55 compares the width 126 of the imaging range 22 with the length L1 of the long side of the rectangle 152. The CPU 55 also compares the height 127 of the imaging range 22 with the length L2 of the short side of the rectangle 152. In this manner, the CPU 55 determines whether or not all of the rectangles acquired in the process of step S23 are included in the imaging range 22 set in the process of step S12.

[0070] When the CPU 55 determines that all of the rectangles acquired in the process of step S23 are included in the imaging range 22 set in the process of step S12 (Yes), the CPU 55 determines whether or not the number of inspection positions added to the candidate list has reached the upper limit (S25). In this process, the CPU 55 determines whether or not the number of inspection positions added to the candidate list has reached the upper limit inputted in the number input field 133 (see FIG. 7).

[0071] When it is determined that the number of inspection positions to be included in one imaging range 22 has reached the upper limit value input in the number input field 133 (Yes), the CPU 55 advances the process to step S27. On the other hand, when it is determined that the number of inspection positions to be included in one imaging range 22 has not reached the upper limit value input in the number input field 133 (No), the CPU 55 determines that there is room to add an inspection position to the list of candidates, and returns the process to step S21. In this way, the CPU 55 repeats the processes of steps S21 to S25 as long as it determines that all of the rectangles acquired in the process of step S23 are included in the imaging range 22 set in the process of step S12 and the number of inspection positions has not reached the upper limit.

[0072] In the process of step S24, when it is determined that at least one of the long side and the short side of the rectangle acquired in the process of step S23 is longer than the width 126 or the height 127 of the imaging range 22 (No), the CPU 55 advances the process to step S26. In step S26, the CPU 55 excludes the data of the inspection position last added to the candidate list (S26).

[0073] The flow of repeating the processes of steps S21 to S24 will be described with reference to Figs. 11(a) to (f). Fig. 11(a) is a diagram showing a state in which an inspection position i1 is added to the list of candidates, and a rectangle r1a including the inspection position i1 is obtained. Fig. 11(b) is a diagram showing a state in which an inspection position i2 is added to the list of candidates, and a rectangle r1b including the inspection positions i1 and i2 is obtained. Fig. 11(c) is a diagram showing a state in which an inspection position i3 is added to the list of candidates, and a rectangle r1c including the inspection positions i1 to i3 is obtained. Fig. 11(d) is a diagram showing a state in which an inspection position i4 is added to the list of candidates, and a rectangle r1d including the inspection positions i1 to i4 is obtained. Fig. 11(e) is a diagram showing a state in which an inspection position i5 is added to the list of candidates, and a rectangle r1e including the inspection positions i1 to i5 is obtained. FIG. 11(f) shows a state in which inspection position i6 has been added to the list of candidates and a rectangle r1f including inspection positions i1 to i6 has been acquired.

[0074] The CPU 55 repeats the processes of steps S21 to S24 until it is determined that the rectangle including all the inspection positions added to the candidate list is larger than the imaging range 22, or until it is determined that the number of inspection positions added to the candidate list has reached the upper limit. In this manner, the CPU 55 acquires each rectangle in the order of rectangles r1a to r1f. Therefore, in the examples shown in Figs. 11(a) to (f), the sizes of the rectangles r1a to r1f are gradually enlarged.

[0075] 11(a) to (f), it is determined that each of the rectangles r1a to r1e is entirely contained within the imaging range 22. On the other hand, it is determined that the length L1 of the long side of the rectangle r1f is longer than the width 126 of the imaging range 22. In this case, the CPU 55 advances the process from step S24 to step S26, and deletes the inspection position i6 from the candidate list.

[0076] In the example shown in FIGS. 11(a) to (f), by executing such processing, the CPU 55 adds the inspection positions i1 to i5 to the list of candidates and acquires the rectangle r1e, and then proceeds to the process of step S27.

[0077] In the process of step S27, the CPU 55 acquires data of the teaching point of the robot 10 using the long side and short side of the acquired rectangle (S27). In this process, the CPU 55 acquires the center position from the center point of the long side of the acquired rectangle and the midpoint of the short side.

[0078] The CPU 55 uses the x and y coordinates of the center position and the principal component vector and secondary component vector used to acquire the rectangle to acquire data of the teaching point, which serves as the position and posture data of the robot 10 for imaging the inspection position added to the candidate list.

[0079] In this manner, in the positional relationship between the camera 20 and the workpiece 30 in which multiple inspection positions can be included in the imaging range, the CPU 55 calculates the teaching point data so that the center position of the rectangular area including the multiple inspection positions becomes the center of the imaging range 22.

[0080] As a result, when imaging at least two inspection positions within an imaging range 22 whose size is set, the robot system 1 can include many inspection positions within one imaging range, thereby improving the efficiency of the operation of the robot 10 and ultimately reducing the operating time of the robot 10.

[0081] Furthermore, by repeating the processes of steps S21 to S24, when the positional relationship between the camera 20 and the workpiece 30, which allows the imaging range 22 to include the entire rectangular area, is calculated, the CPU 55 adds a new inspection position outside the rectangular area to the area. The CPU 55 acquires a new area including the added inspection position, and verifies whether the positional relationship between the camera 20 and the workpiece 30, which allows the imaging range 22 to include the entire new area, is acquired.

[0082] When the CPU 55 acquires a positional relationship between the camera 20 and the workpiece 30 such that the imaging range 22 can include the entire new area as a result of the verification, the CPU 55 repeats the processing of steps S21 to S24 again. On the other hand, when the CPU 55 does not acquire a positional relationship between the camera 20 and the workpiece 30 such that the imaging range 22 can include the entire new area as a result of the verification, the CPU 55 executes the processing of step S26 and returns the area setting to a state excluding the inspection position last added to the new area. Then, the CPU 55 executes the processing of step S27 for the positional relationship between the camera 20 and the workpiece 30 such that the imaging range 22 can include the entire area excluding the inspection position last added to the new area, and acquires data of the teaching point.

[0083] With this configuration, the robot system 1 can set the maximum number of inspection positions that can be included in one imaging range 22, and can reduce the number of times the robot 10 is operated to capture images with the camera 20, thereby reducing the operating time of the robot 10.

[0084] In the process of step S28, the CPU 55 removes the inspection position added to the list of candidates from the list of inspection positions, and then determines whether or not the inspection position is not recorded in the list of inspection positions (S28). In this process, the CPU 55 removes the inspection position added to the list of candidates from the list of inspection positions because data of teaching points for imaging by the camera 20 has been acquired for the inspection positions added to the list of candidates. Then, the CPU 55 determines whether or not the inspection position is recorded in the list of inspection positions, thereby determining whether or not an inspection position for which data of teaching points for imaging by the camera 20 has not been acquired is recorded.

[0085] In the process of step S28, when it is determined that the inspection position is recorded in the list of inspection positions (No), the CPU 55 determines that there remains an inspection position for which data of teaching points for imaging by the camera 20 has not been acquired, and returns the process to step S21. On the other hand, when it is determined that the inspection position is not recorded in the list of inspection positions (Yes), the CPU 55 determines that data of teaching points for imaging by the camera 20 has been acquired for all the inspection positions recorded in the list of inspection positions. Then, the CPU 55 advances the process from step S28 to step S14.

[0086] The process flow of steps S21 to S28 for acquiring data of teaching points to be imaged by camera 20 for all inspection positions recorded in the list of inspection positions will be described with reference to Figs. 12(a) to (f) and 13.

[0087] FIG. 12(a) is a diagram showing a state in which an inspection position i6 is added to the list of candidates and a rectangle r2a including the inspection position i6 is acquired after a rectangle r1e used in acquiring data of teaching points for imaging the inspection positions i1 to i5 is acquired. FIG. 12(b) is a diagram showing a state in which an inspection position i7 is added to the list of candidates and a rectangle r2b including the inspection positions i6 and i7 is acquired. FIG. 12(c) is a diagram showing a state in which an inspection position i8 is added to the list of candidates and a rectangle r2c including the inspection positions i6 to i8 is acquired. FIG. 12(d) is a diagram showing a state in which an inspection position i9 is added to the list of candidates and a rectangle r2d including the inspection positions i6 to i9 is acquired. FIG. 12(e) is a diagram showing a state in which an inspection position i10 is added to the list of candidates and a rectangle r2e including the inspection positions i6 to i10 is acquired. FIG. 12(f) shows a state in which inspection position i11 has been added to the candidate list, and a rectangle r2f including inspection positions i6 to i11 has been acquired.

[0088] 12(a) to (f), it is determined that each of the rectangles r2a to r2e is entirely contained within the imaging range 22. On the other hand, it is determined that the length L1 of the long side of the rectangle r2f is longer than the width 126 of the imaging range 22. In this case, the CPU 55 advances the process from step S24 to step S26, and deletes the inspection position i11 from the candidate list.

[0089] In the example shown in FIGS. 12(a) to (f), by executing such processing, the CPU 55 adds the inspection positions i6 to i10 to the list of candidates, and in a state where the rectangle r2e is acquired, the process proceeds to step S27.

[0090] In the process of step S28, the CPU 55 excludes the inspection positions i6 to i10 from the list of inspection positions. The CPU 55 determines that the inspection positions i11 to i21 are recorded in the list of inspection positions, and returns the process to step S21.

[0091] 13 is a diagram showing a state in which data of teaching points for imaging the inspection positions i1 to i21 is acquired by the processing of steps S21 to S28 by the CPU 55. As shown in FIG. 13, data of teaching points for imaging the inspection positions i1 to i5 is acquired from the center position p1 of a rectangle r1 that includes the inspection positions i1 to i5. Data of teaching points for imaging the inspection positions i6 to i10 is acquired from the center position p2 of a rectangle r2 that includes the inspection positions i6 to i10. Data of teaching points for imaging the inspection positions i11 to i15 is acquired from the center position p3 of a rectangle r3 that includes the inspection positions i11 to i15. Data of teaching points for imaging the inspection positions i16 to i21 is acquired from the center position p4 of a rectangle r4 that includes the inspection positions i16 to i21.

[0092] In this way, the CPU 55 acquires data of the teaching point (data of the teaching point in the first positional relationship) for moving to one positional relationship (first positional relationship) between the camera 20 and the workpiece 30 that allows at least two inspection positions to be included in the imaging range 22. Furthermore, when there is an inspection position that is not included in the imaging range 22 in the first positional relationship, the CPU 55 acquires another positional relationship (second positional relationship) between the camera 20 and the workpiece 30 that allows the imaging range to include the inspection position that is not included in the imaging range 22. Then, in addition to the data of the teaching point in the first positional relationship (first information), the CPU 55 acquires data of the teaching point (second information) used when the camera 20 is moved by the robot 10 so that the camera 20 and the workpiece 30 have the second positional relationship.

[0093] With this configuration, the robot system 1 can obtain data of multiple teaching points corresponding to each inspection position for imaging by the camera 20, even when multiple inspection positions are set across a range wider than the imaging range 22. This allows the robot system 1 to set any number of inspection positions and positions on the workpiece 30 regardless of the size of the imaging range 22, and reduces the amount of work required to set the inspection positions in the appearance inspection of the workpiece 30.

[0094] After executing the processes of steps S21 to S28 shown in Fig. 8 and completing the process of step S13 shown in Fig. 5, the CPU 55 advances the process to step S14. In the process of step S14, the CPU 55 displays on the display 52 the acquired data of the teaching points and the inspection positions included in the imaging range 22 when the image is captured by the camera 20 based on the data of the teaching points (S14). Then, the CPU 55 ends the process of acquiring the data of the teaching points of the robot 10.

[0095] Fig. 14 is a diagram showing an example of a screen configuration when acquired teaching point data and inspection positions included in the imaging range 22 when an image is captured by the camera 20 based on the teaching point data are displayed on the display 52. ​​As shown in Fig. 14, the display 52 displays an acquisition result window 180 of the teaching point data, rectangles r1 to r5 that are areas including the respective inspection positions in the virtual space, and center positions p1 to p5 of the rectangles r1 to r5.

[0096] In the acquisition result window 180, a tree view 181 displays data of the teaching points linked to data of the inspection positions included in the imaging range when imaging is performed based on the data of the teaching points.

[0097] Further, a Confirm button 182 and a Cancel button 183 are displayed in the acquisition result window 180. When the Confirm button 182 is selected, the CPU 55 can adopt the acquired teaching point data and record it in the memory 56. When the Cancel button 183 is selected, the CPU 55 can discard the acquired teaching point data without adopting it.

[0098] By executing these steps, in the subsequent steps, the simulation device 50 can create a robot program that uses the acquired data of teaching points to perform the operation of the robot 10 and the imaging process of the camera 20. Note that the simulation device 50 may be configured to automatically acquire the robot program from the acquired data of teaching points.

[0099] <Summary of the First Embodiment> As described above, in the robot system 1 of the first embodiment, the robot 10 moves the camera 20 so that the positional relationship between the camera 20 and the workpiece 30 becomes a first positional relationship that can include at least two inspection positions in the imaging range. This allows the robot system 1 to capture images of multiple inspection positions included in the imaging range in one imaging session, and can improve the efficiency of the operation of the robot 10 and shorten the time required to capture images of the inspection positions during the appearance inspection of the workpiece 30.

[0100] [Second embodiment] Next, a robot system 1 according to a second embodiment will be described. The robot system 1 according to the second embodiment is configured to perform visual inspection of an object to be inspected in a three-dimensional space. In this respect, the robot system 1 according to the second embodiment differs from the first embodiment described above. Since the other configurations are similar to those of the first embodiment, the same reference numerals are used for the components common to the first embodiment, and the same step numbers are used for the control processes common to the first embodiment, and the description thereof will be omitted.

[0101] <Extraction of robot teaching point data> 15 is a flowchart showing a process of acquiring data of teaching points of the robot 10 executed by the CPU 55 of the simulation device 50 of the second embodiment. The simulation device 50 extracts data of teaching points of the robot 10 by executing the processes of steps S10, S11, S14, S31, and S32 shown in FIG.

[0102] After executing the process of step S11, the CPU 55 executes the process of step S31. In the process of step S31, the CPU 55 receives an input of the imaging range of the camera 20 from the worker (S31).

[0103] Fig. 16 is a diagram showing an example of the configuration of a screen displayed on the display 52 when an input of the imaging range of the camera 20 by an operator is received in the second embodiment. As shown in Fig. 16, a range setting window 130 displaying information on the imaging range of the camera 20 and an enlarged view of the robot 10v including the camera 20v and imaging range 22v in the virtual space are displayed on the display 52. ​​In the example shown in Fig. 16, an oblique view and a side view are displayed as the enlarged view of the robot 10v.

[0104] In the range setting window 130, a width input field 121, a height input field 122, a depth input field 123, and a depth input field 131 in which a depth width can be input, which is the width of the imaging range in the depth direction based on the depth 128 of the imaging range 22. Here, the depth width of the imaging range 22 is the length in the front-rear direction relative to the direction of the optical axis 20a of the camera 20 (optical axis direction), and is the length in the front-rear direction centered on the position set by the depth 128. In the example shown in Fig. 16, the unit of the value input in each input field is millimeters.

[0105] The CPU 55 changes the width 126, the height 127, the depth 128, and the depth width 132v of the imaging range 22v displayed on the display 52 in accordance with the values ​​inputted in the width input field 121, the height input field 122, the depth input field 123, and the depth width input field 131. As shown in Fig. 16, the depth width 132v is set with the position of the depth 128 as the center. For example, when a value of "300" is inputted in the depth input field 123 and a value of "10" is inputted in the depth width input field 131, the depth width 132v is set to a range of 295 to 305 millimeters away from the camera 20 in the depth direction.

[0106] In the process of step S32, the CPU 55 extracts data of teaching points such that a plurality of inspection positions fall within the imaging range 22 of the camera 20 based on the input information (S32). In this process, the CPU 55 acquires data of teaching points such that the inspection positions are contained within a rectangular parallelepiped having a depth 128 position as the center in the depth direction within the imaging range 22 and having a horizontal width 126, a vertical width 127, and a depth 132.

[0107] Details of the process of step S32 will be described with reference to the flowchart shown in Fig. 17. Fig. 17 is a flowchart showing the process of acquiring data of teaching points where a plurality of inspection positions are within the imaging range 22 of the camera 20, which is executed by the CPU 55 in the process of step S32.

[0108] In the process of step S41, the CPU 55 selects an arbitrary inspection position from among the multiple inspection positions input by the operator, and adds it to a list of inspection positions that are candidates for acquiring teaching point data (candidate list) (S41). In this process, the CPU 55 acquires data of the arbitrary inspection position from the list of inspection positions, and adds the acquired data of the inspection position to the candidate list.

[0109] The CPU 55 selects an inspection position using different criteria depending on whether data of any inspection position has not been added to the candidate list or data of any inspection position has been added to the candidate list. When data of any inspection position has not been added to the candidate list, the CPU 55 selects the inspection position with the smallest inspection number in the list of inspection positions.

[0110] When data of any inspection position has been added to the candidate list, the CPU 55 selects data of the inspection position located closest to the previously added inspection position. At this time, the CPU 55 of the second embodiment sets a condition that the angle between the normal vectors of the previously added inspection position is equal to or less than a preset threshold value in addition to the x-coordinate and y-coordinate of the previously added inspection position. This increases the possibility that the CPU 55 selects multiple inspection positions that belong to the same plane.

[0111] In the process of step S42, the CPU 55 performs a principal component analysis on the inspection position added to the candidate list to calculate the principal component vector of the inspection position (S42). In this process, the CPU 55 first calculates a variance-covariance matrix VC2 using the following formula.

number

[0112] Here, v(x), v(y), and v(z) are the variances of the respective inspection positions, and vc(x,y), vc(x,z), vc(y,x), vc(y,z), vc(z,x), and vc(z,y) are the covariances of the respective inspection positions, which are calculated by a known method. The CPU 55 calculates the eigenvalues ​​and eigenvectors of the variance-covariance matrix VC2 by principal component analysis, and calculates the eigenvector with the largest eigenvalue as the principal component vector.

[0113] In the process of step S43, the CPU 55 obtains a rectangular parallelepiped that includes all the inspection positions from the principal component vector, and obtains the long side, short side, and depth width of the obtained rectangular parallelepiped (S43). In this process, the CPU 55 obtains the rectangular parallelepiped from the principal component vector, the secondary component vector, and the normal vector. The CPU 55 calculates, as the secondary component vector, the eigenvector with the second largest eigenvalue among the eigenvectors calculated in the process of step S42. The CPU 55 also calculates, as the normal vector, the eigenvector with the third largest eigenvalue among the eigenvectors calculated in the process of step S42.

[0114] In the process of step S44, the CPU 55 determines whether or not all of the rectangular parallelepipeds acquired in the process of step S43 are contained in a rectangular parallelepiped having its center in the depth direction at the position of depth 128 within the imaging range 22 of the camera 20 set in the process of step S31 (S44). In this process, the CPU 55 determines whether or not the lengths of the long sides, the short sides, and the depth width of the rectangular parallelepipeds acquired in the process of step S43 are shorter than the width, the height, and the depth width of the rectangular parallelepiped set in the process of step S31.

[0115] When the CPU 55 determines that all of the rectangular parallelepipeds acquired in the process of step S43 are contained within the rectangular parallelepiped of the imaging range 22 set in the process of step S31 (Yes), it determines that there is room to add an inspection position to the list of candidates, and returns the process to step S41. In this way, the CPU 55 repeats the processes of steps S41 to S44 as long as it determines that all of the rectangular parallelepipeds acquired in the process of step S43 are contained within the rectangular parallelepiped of the imaging range 22 set in the process of step S31.

[0116] On the other hand, if it is determined that the part of the rectangular parallelepiped acquired in the process of step S43 is not contained in the rectangular parallelepiped of the imaging range 22 (No), the CPU 55 advances the process to step S26.

[0117] In the process of step S25, if it is determined that the number of inspection positions included in the imaging range 22 has reached the upper limit (Yes), or after executing the process of step S26, the CPU 55 advances the process to step S45. In the process of step S45, the CPU 55 acquires data of teaching points of the robot 10 using the acquired long sides, short sides, and depth width of the rectangular parallelepiped (S45). In this process, the CPU 55 acquires the center position from the center point of the long sides, the midpoint of the short sides, and the midpoint of the depth width of the acquired rectangular parallelepiped.

[0118] The CPU 55 acquires data of the teaching point for imaging the inspection position added to the candidate list using the x-coordinates, y-coordinates, and z-coordinates of the center position and the principal component vector, secondary component vector, and normal vector used when acquiring the rectangle.

[0119] <Summary of the second embodiment> As described above, in the robot system 1 of the first embodiment, the robot 10 moves the camera 20 so that the positional relationship between the camera 20 and the workpiece 30 becomes a first positional relationship that can include at least two inspection positions in the imaging range. At this time, the robot system 1 can include multiple inspection positions of the workpiece 30 that are located at different positions in the optical axis direction of the camera 20 in one imaging range by making it possible to set a depth width in the imaging range. As a result, the robot system 1 can image multiple inspection positions that are included in the imaging range and have different positions in the optical axis direction in one imaging, and can improve the efficiency of the operation of the robot 10 and shorten the time required to image the inspection positions when inspecting the appearance of the workpiece 30.

[0120] [Third embodiment] Next, a robot system 1 according to a third embodiment will be described. The robot system 1 according to the third embodiment is configured to acquire data on the acquired teaching points to determine whether or not the robot 10 can be operated. The robot system 1 according to the third embodiment is also configured to acquire the operation time of the robot 10 when the robot 10 is operated based on the acquired teaching point data. In these respects, the robot system 1 according to the third embodiment is different from the first and second embodiments described above. Since the other configurations are the same as those of the first and second embodiments, the same reference numerals are used for the components common to the first and second embodiments, and the same step numbers are used for the control processes common to the first and second embodiments, and the description thereof will be omitted.

[0121] <Acquisition of robot teaching point data> The simulation device 50 of the third embodiment is configured so that an operator can set data on the positions and three-dimensional shapes of each of the components, namely the robot 10, the camera 20, and the workpiece 30, and objects (peripheral objects) present around each of the components. The simulation device 50 is configured so that it can verify whether interference occurs between each of the components and the peripheral objects when the robot 10 is moved. This will be described in more detail later.

[0122] 18 is a flowchart showing a process of acquiring data of teaching points of the robot 10 executed by the CPU 55 of the simulation device 50 of the third embodiment. The simulation device 50 acquires data of teaching points of the robot 10 by executing the processes of steps S10, S12, S14, S51, and S52 shown in FIG.

[0123] After executing the process of step S10, the CPU 55 executes the process of step S51. In the process of step S51, the CPU 55 accepts input of data of a plurality of inspection positions by the operator (S51). In this process, the CPU 55 of the third embodiment acquires the inspection number, position, and normal vector from the data of the inspection position selected by the operator, and accepts input of weighting data indicating the priority of the inspection position.

[0124] Fig. 19 is a diagram showing an example of the configuration of a screen displayed on the display 52 when receiving data on the inspection position input by an operator. As shown in Fig. 19, the display 52 displays a position setting window 140 in which various information on the inspection position is displayed, and a robot 10v, a camera 20v, a workpiece 30v, and the like in a virtual space.

[0125] In the third embodiment, weighting input fields 107 in which weighting data for setting a priority level for an inspection position can be input are displayed in table 140a of position setting window 140. An operator can select weighting input field 107 for each inspection position and input any numerical value as the weighting data.

[0126] After executing the process of step S12, the CPU 55 executes the process of step S52. In the process of step S52, the CPU 55 acquires data of a teaching point based on the input information, and acquires the result of the operation when the robot 10 is operated based on the data of the teaching point (S52). In this process, the CPU 55 acquires data of a teaching point in which the inspection position is included in a rectangle consisting of a horizontal width 126 and a vertical width 127 in the imaging range 22. In addition, the CPU 55 acquires a determination result of whether or not the operation is possible and an operation time as a result of the operation when the robot 10 is operated based on the acquired data of the teaching point.

[0127] Details of the process of step S52 will be described with reference to the flowchart shown in Fig. 20. Fig. 20 is a flowchart showing the process of acquiring data of teaching points where a plurality of inspection positions are within the imaging range 22 of the camera 20, which is executed by the CPU 55 in the process of step S52.

[0128] First, the CPU 55 acquires data of teaching points for operating the robot 10 (S61). In this process, the CPU 55 executes the process related to acquiring the data of the teaching points shown in Fig. 8. In the process of step S61, the CPU 55 executes the process of acquiring the data of the teaching points multiple times by repeating the process until all the data of the inspection positions stored in the list of inspection positions is exhausted, and acquires data of multiple teaching points for imaging multiple inspection positions.

[0129] The CPU 55 of the third embodiment has a different criterion for selecting an inspection position from that of the first embodiment when no data of any inspection position has been added to the list of candidates. When the CPU 55 executes the process of step S61 following the process of step S12, it selects the inspection position to which the largest value is set as weighting data among the inspection positions stored in the list of inspection positions. When there are multiple inspection positions to which the largest value is set as weighting data, the CPU 55 adds one inspection position randomly selected from the multiple inspection positions using a random number to the list of candidates.

[0130] This allows the operator to control the order in which the inspection positions are selected by the simulation device 50 in the robot system 1, and allows the operator to adjust the operation of the robot 10 based on the acquired teaching point data.

[0131] If data for any inspection position has been added to the candidate list, the CPU 55 selects data for the inspection position that is located closest to the previously added inspection position.

[0132] The CPU 55 may be configured to add data of an inspection position located near the inspection position with the highest priority among the inspection positions added to the list of candidates to the list of candidates.

[0133] With this configuration, the robot system 1 can acquire data of teaching points for moving the camera 20 by the robot 10 so that an inspection position with a higher priority among the multiple inspection positions added to the candidate list is closer to the center of the image capture range 22 than the other inspection positions. This allows the robot system 1 to place the inspection position set by the operator to have a higher priority than the other inspection positions approximately in the vicinity of the captured image.

[0134] The CPU 55 is configured to change the order of acquiring data of the inspection positions when returning from the processing of steps S62, S63, and S65 to the processing of step S61, which will be described later. In other words, the CPU 55 is configured to change the combination of at least two inspection positions included in the imaging range 22 and acquire data of the teaching points when returning from the processing of steps S62, S63, and S65 to the processing of step S61.

[0135] In the process of step S62, the CPU 55 judges whether or not the robot 10 can reach the position of the acquired teaching point data (S62). In this process, the CPU 55 verifies whether or not the robot 10 can reach the position by acquiring an inverse kinematics solution for the position indicated by the acquired teaching point data. A known method suited to the configuration of the robot 10 is used as a method for acquiring the inverse kinematics solution.

[0136] If an inverse kinematics solution exists (Yes), the CPU 55 advances the process to step S63 since there are control values ​​for each of the joints J1 to J6 of the robot 10 that can reach the position indicated by the teaching point data. On the other hand, if an inverse kinematics solution does not exist (No), the CPU 55 returns the process to step S61 since it is not possible to make the robot 10 reach the position indicated by the acquired teaching point data.

[0137] In this way, based on the acquired data of the teaching points, the CPU 55 verifies whether the camera 20 can be moved by the robot 10 so as to achieve a first positional relationship that can include a plurality of inspection positions in the imaging range 22. If the result of the verification indicates that the camera 20 can be moved, the CPU 55 advances the process to step S63, and if the camera 20 cannot be moved, the CPU 55 returns the process to step S61.

[0138] This allows the robot system 1 to quickly reacquire data on a teaching point when data on a teaching point that is actually unmovable is acquired, thereby preventing data on a teaching point that is unmovable from being acquired in advance.

[0139] In the process of step S63, the CPU 55 judges whether or not each component of the robot 10, etc. will interfere with a surrounding object when the robot 10 moves to the position of the acquired teaching point data (S63). In this process, the CPU 55 uses the position and three-dimensional shape data of each component and surrounding object input in advance to judge whether or not the robot 10 or the camera 20 will interfere with the workpiece 30 when the robot 10 moves to the position of the acquired teaching point data. The CPU 55 also judges whether or not the robot 10 or the camera 20 will interfere with a surrounding object when the robot 10 moves to the position of the acquired teaching point data.

[0140] When it is determined that the movement of the robot 10 using the acquired teaching point data will not cause interference between the constituent elements or between the constituent elements and the surrounding objects (No), the CPU 55 advances the process to step S64. On the other hand, when it is determined that the movement of the robot 10 using the acquired teaching point data will cause at least one of interference between the constituent elements or between the constituent elements and the surrounding objects (Yes), the CPU 55 returns the process to step S61.

[0141] In this way, based on the acquired data of the teaching points, the CPU 55 verifies whether at least one of the robot 10 and the camera 20 will not interfere with other objects such as peripheral objects when the camera 20 is moved by the robot 10. If the CPU 55 determines as a result of the verification that no interference will occur, it advances the process to step S64, and if interference will occur, it returns the process to step S61.

[0142] As a result, when at least one of the robot 10 and the camera 20 interferes with other objects such as surrounding objects, the robot system 1 can quickly re-acquire data on the teaching point, thereby preventing data from being acquired for a teaching point that cannot be moved.

[0143] The CPU 55 that executes the processes of steps S62 and S63 constitutes a movement verification means and an interference verification means.

[0144] In the process of step S64, the CPU 55 acquires the operation time that elapses when the robot 10 is moved sequentially based on the acquired teaching point data (S64). In this process, the CPU 55 acquires the operation time that elapses when the robot 10 is moved sequentially based on the multiple teaching point data acquired for all inspection positions and travels around the positions of the multiple teaching point data.

[0145] The CPU 55 stores the acquired operation time in the memory 56 in correspondence with the acquired teaching point data, and makes it possible to notify the operator from the display 52 of the operation time when the robot 10 is moved sequentially using the teaching point data.

[0146] In this way, the simulation device 50 allows the operator to evaluate the acquired data of the teaching points using the operation time that elapses when the robot 10 is moved sequentially according to the acquired data of the teaching points. In other words, the operation time associated with the acquired data of the teaching points functions as a value (evaluation value) for evaluating the result of operating the robot 10.

[0147] In the process of step S65, the CPU 55 judges whether a preset end condition is satisfied (S65). The simulation device 50 is configured to be able to preset a target time (target time) for the operation time as an end condition for the process related to the acquisition of data of the teaching point. In the process of step S65, the CPU 55 judges whether the acquired operation time is equal to or shorter than the target time.

[0148] When it is determined that the acquired operation time exceeds the target time (No), the CPU 55 returns the process to step S61. On the other hand, when it is determined that the acquired operation time is equal to or less than the target time (Yes), the CPU 55 ends the process due to the establishment of the end condition of the process related to the acquisition of data of the teaching point where a plurality of inspection positions fall within the imaging range 22 of the camera 20, and proceeds to the process to step S14.

[0149] By executing the processing of steps S61 to S65, the simulation device 50 obtains the operating time when the robot 10 is moved sequentially using the acquired teaching point data, and can obtain data on teaching points at which the robot 10 can operate and within the target time.

[0150] <Summary of the third embodiment> As described above, the robot system 1 of the third embodiment moves the camera 20 by the robot 10 so that the positional relationship between the camera 20 and the workpiece 30 becomes a first positional relationship that can include at least two inspection positions in the imaging range. At this time, the robot system 1 verifies whether the camera 20 can be moved by the robot 10. In addition, when the robot system 1 moves the camera 20 by the robot 10, the robot system 1 verifies whether at least one of the robot 10 and the camera 20 does not interfere with other objects including peripheral objects. Then, the robot system 1 acquires data of teaching points that enable the camera 20 to be moved by the robot 10 and that enable the camera 20 to be moved without interfering with other objects including peripheral objects.

[0151] This allows the robot system 1 to use the camera 20 to capture images of the workpiece 30 at each position to which it is moved using the data of all acquired teaching points, and to perform visual inspection by capturing images of all set inspection positions without omitting any.

[0152] Furthermore, the robot system 1 allows the operator to adjust the operation of the robot 10 based on the acquired data of teaching points by allowing the operator to set priorities for the inspection positions.

[0153] Furthermore, the robot system 1 sets a target time as a termination condition, and acquires data of teaching points so that the operation time elapsed when the robot 10 is moved sequentially using the acquired teaching point data of the robot 10 is within the target time. This enables the robot system 1 to keep the time required for capturing an image of the inspection position during the appearance inspection of the workpiece 30 within the target time.

[0154] [Fourth embodiment] Next, a robot system 1 according to a fourth embodiment will be described. In the robot system 1 according to the fourth embodiment, a position for capturing an image of the workpiece 30 by the camera 20 is set by an operator, and data on a teaching point for moving the robot 10 to that position is acquired. In this respect, the robot system 1 according to the fourth embodiment differs from the first to third embodiments described above. Since the other configurations are the same as those of the first to third embodiments, the same reference numerals are used for components common to the first to third embodiments, and the same step numbers are used for control processes common to the first to third embodiments, and descriptions thereof will be omitted.

[0155] <Acquisition of robot teaching point data> 21 is a flowchart showing a process executed by the CPU 55 of the simulation device 50 of the fourth embodiment to acquire data on teaching points of the robot 10. The simulation device 50 acquires data on teaching points of the robot 10 by executing the processes of steps S10 to S12 and S71 to S74 shown in FIG.

[0156] After executing the process of step S12, the CPU 55 receives an input of the position of the imaging range 22 by the operator (S71).

[0157] Fig. 22 is a diagram showing an example of the configuration of a screen displayed on the display 52 when receiving an input by a worker of the position (imaging position) of the imaging range 22. As shown in Fig. 22, the display 52 displays an imaging position setting window 200 in which various information on the imaging range 22 is displayed, and a robot 10v, a camera 20v, a workpiece 30v, and the like in the virtual space.

[0158] 22, an imaging position setting window 200 displays an x ​​input field 201 in which the x coordinate of the imaging position can be input, a y input field 202 in which the y coordinate of the imaging position can be input, and a z input field 203 in which the z coordinate of the imaging position can be input. The imaging position setting window 200 also displays an rx input field 204 in which the rotation angle coordinate of the imaging position about the x axis can be input, and an ry input field 205 in which the rotation angle coordinate of the imaging position about the y axis can be input. The imaging position setting window 200 also displays an rz input field 206 in which the rotation angle coordinate of the imaging position about the z axis can be input.

[0159] A mouse cursor 210 and a range model 211 indicating the imaging range are displayed on the display 52, and the range model 211 can be moved by dragging and dropping it by operating the mouse cursor 210. The CPU 55 is configured to be able to change the numerical values ​​in the input fields displayed in the imaging position setting window 200 in accordance with the movement of the range model 211.

[0160] In this way, the simulation device 50 operates the mouse 53 and the keyboard 54 to input numerical values ​​into each input field displayed in the imaging position setting window 200, thereby inputting the positional relationship between the workpiece 30 and the camera 20 when the camera 20 images multiple inspection positions. Also, the simulation device 50 operates the mouse 53 to operate the range model 211 displayed on the display 52, thereby inputting the positional relationship between the workpiece 30 and the camera 20 when the camera 20 images multiple inspection positions. The mouse 53 and the keyboard 54 constitute a positional relationship input means.

[0161] A confirm button 207 and a cancel button 208 are displayed in the imaging position setting window 200. When the confirm button 207 is selected, the CPU 55 advances the process from step S71 of the process of acquiring data of the teaching points of the robot 10 to the next step, step S72. When the cancel button 208 is selected, the CPU 55 can cancel the process of acquiring data of the teaching points of the robot 10.

[0162] When the Confirm button 207 is selected, the CPU 55 stores in the memory 56 the imaging position selected by the operator using the range model 211. When the Confirm button 207 is selected, the CPU 55 also changes the display mode of the marker 110 of the inspection position included in the range model 211. For example, the CPU 55 displays the marker 110 before the imaging position is confirmed in a first shape of "X". Furthermore, for example, the CPU 55 displays the marker 110 after the Confirm button 207 is selected in a state included in the range model 211 and the imaging position is confirmed in a second shape different from the first shape of an "X" surrounded by a "◯" as a selected marker 212.

[0163] In the process of step S72, the CPU 55 acquires data of teaching points used when operating the robot 10 to move the camera 20 to an imaging position specified by the operator (S72). In this process, the CPU 55 acquires data of teaching points used when operating the robot 10 so that the camera 20 is positioned at the center position of the imaging range 22 specified by the range model 211. After executing the process of step S72, the CPU 55 advances the process to step S14.

[0164] After executing the process of step S14, the CPU 55 advances the process to step S73. In the process of step S73, the CPU 55 determines whether or not imaging positions that enable imaging have been set for all the inspection positions (S73). In this process, the CPU 55 determines whether or not imaging positions that enable imaging have been set for all the inspection positions by executing the processes of steps S71 and S72 multiple times. When it is determined that imaging positions that enable imaging have not been set for all the inspection positions (No), the CPU 55 returns the process to step S71.

[0165] As described above, the CPU 55 displays the inspection positions for which the imaging positions have been confirmed, as the selected markers 212, on the display 52 in a display mode different from that for the inspection positions for which the imaging positions have not been confirmed. This enables the simulation device 50 to notify the operator of which inspection positions the imaging positions have not been set, and to assist in inputting the imaging positions for all the inspection positions.

[0166] When it is determined that imaging positions that enable imaging have been set for all the inspection positions (Yes), the CPU 55 determines that acquisition of data of teaching points used in the operation of the robot 10 to move the camera 20 to each imaging position specified by the operator's operation has been completed. Then, the CPU 55 ends the process of acquiring data of teaching points of the robot 10.

[0167] <Summary of the Fourth Embodiment> As described above, the robot system 1 of the fourth embodiment moves the camera 20 by the robot 10 so that the positional relationship between the camera 20 and the workpiece 30 is the positional relationship input by the worker. This allows the robot system 1 to capture an image of the workpiece 30 by the camera 20 at a position that is in line with the worker's intention.

[0168] [Other embodiments] In the first to fourth embodiments, the robot system 1 is configured to move the camera 20 connected to the robot 10. However, the present invention is not limited to this. The robot system 1 may be configured such that the workpiece 30 is connected to the link 16 of the robot 10 via a joint J6, and the camera 20 is installed on a fixed base. In other words, the robot system 1 may be configured such that the robot 10 moves the workpiece 30, and the fixed camera 20 captures an image of the workpiece 30.

[0169] In the first to fourth embodiments, the CPU 55 is configured to be able to set an inspection position whose coordinates can be expressed as a point on the workpiece 30 as the inspection position whose image is captured by the camera 20, but is not limited thereto. The CPU 55 may be configured to be able to set, for example, a predetermined range whose coordinates can be expressed as a surface or space as the portion whose image is captured by the camera 20.

[0170] Furthermore, the CPU 55 in the first embodiment displays on the display 52 the acquired data of the teaching points and the inspection positions included in the imaging range 22 when an image is captured by the camera 20 based on the data of the teaching points, but is not limited to this. The CPU 55 may be configured to display on the display 52 the workpiece 30 and the camera 20 when the camera 20 is moved by the robot 10 based on the acquired data of the teaching points.

[0171] With this configuration, the robot system 1 displays the workpiece 30 and the camera 20 when capturing an image of each inspection position based on the acquired teaching point data on the display 52. ​​This enables the robot system 1 to notify the operator of the position of the camera 20 when performing the appearance inspection.

[0172] In the first to fourth embodiments, the CPU 55 is configured to acquire data of the teaching point using information on the inspection position and the imaging range 22, but is not limited thereto. The CPU 55 may be configured to use information on the distance between the camera 20 and the workpiece 30 in the optical axis direction of the camera 20 in addition to information on the inspection position and the imaging range 22 when acquiring data of the teaching point. When configured in this way, the CPU 55 may be configured to move the camera 20 by the robot 10 while keeping the distance between the camera 20 and the workpiece 30 in the optical axis direction of the camera 20 constant.

[0173] By being configured in this manner, the robot system 1 maintains the distance between the camera 20 and the workpiece 30 in the optical axis direction of the camera 20 while the camera 20 is moving, and therefore the size of the imaging range 22 can be kept constant while the camera 20 is moving.

[0174] In the first to fourth embodiments, the CPU 55 is configured to be able to set the size of the imaging range 22 by inputting numerical values ​​into the width input field 121, the height input field 122, and the depth input field 123, but is not limited to this. The CPU 55 may be configured to set the size of the imaging range 22 in response to an operator's dragging operation of a mouse over the imaging range 22 displayed on the display 52.

[0175] In the third embodiment, the CPU 55 is configured to be able to set a target time as a termination condition, but is not limited thereto. The CPU 55 may be configured to set a processing time for executing the processes of steps S61 to S65, acquire data of teaching points until the processing time is reached, and acquire operation times for sequentially moving the robot 10 based on the acquired data of teaching points.

[0176] By being configured in this manner, the robot system 1 can present to the worker the combination of the acquired teaching point data and the operation time for sequentially moving the robot 10 based on the acquired teaching point data that results in the shortest operation time.

[0177] In the fourth embodiment, the CPU 55 is configured to change the shape of the marker 110 before the imaging position is determined and the selected marker 212 after the imaging position is determined, for example, but is not limited to this. The CPU 55 may be configured to display the marker 110 before the imaging position is determined in a first color, and display the selected marker 212 after the imaging position is determined in a second color different from the first color, for example.

[0178] In the fourth embodiment, the simulation device 50 advances the process to acquiring data of the teaching point by selecting the confirm button 207 after one imaging position is set when accepting the input of the imaging position, but is not limited to this. The simulation device 50 may be configured to be able to set a plurality of imaging positions before acquiring data of the teaching point by allowing a new imaging position to be set consecutively after one imaging position is set.

[0179] When configured in this manner, the simulation device 50 can display an add button in the imaging position setting window 200 which, when selected, confirms the range model 211 currently being displayed on the display 52 and also displays a new movable range model.

[0180] Furthermore, the control method and control device of the present invention can be applied to software design and program creation for various machines and equipment, such as industrial robots, service robots, processing machines operated by computer numerical control, etc., in addition to production equipment. For example, the machines and equipment can automatically perform operations such as expansion and contraction, bending and stretching, vertical movement, horizontal movement, or rotation, or a combination of these operations, based on information in a storage device provided in the control device.

[0181] The present invention also includes a control method for a robot system that executes the above-described control method and includes a robot manipulator as a controlled object. The present invention also includes a manufacturing method for an article using a robot system that operates by executing the above-described control method. The present invention also includes a program that can execute the above-described information processing and a computer-readable recording medium that stores the program.

[0182] The disclosure of this embodiment includes the following configuration.

[0183] (Configuration 1) An imaging means capable of imaging a workpiece within an imaging range of a predetermined size; A moving means for moving the work or the imaging means; A setting means for setting a plurality of portions on the workpiece to be imaged by the imaging means, The work or the imaging means is moved by the moving means so that at least two of the parts are in a first positional relationship that can be included in the imaging range. A control device comprising:

[0184] (Configuration 2) An acquisition means for acquiring information used when the work or the imaging means is moved by the moving means so as to have the first positional relationship, 2. The control device according to configuration 1,

[0185] (Configuration 3) When there is a portion that is not included in the imaging range in the first positional relationship, the acquisition means acquires information about the first positional relationship and information used when moving the work or the imaging means by the movement means so as to have a second positional relationship that can include the portion that is not included in the imaging range in the imaging range of the imaging means. 3. The control device according to configuration 2.

[0186] (Configuration 4) The acquisition means includes: obtaining a region including the at least two of the portions; acquiring the information about the first positional relationship that allows the region to be included in the imaging range; 4. The control device according to configuration 2 or 3.

[0187] (Configuration 5) The acquisition means includes: when the first positional relationship capable of including the entire area in the imaging range is acquired, verifying whether the first positional relationship capable of including the entire area acquired by adding the portion located outside the area in the imaging range is acquired; When the first positional relationship capable of including the entire area in the imaging range is not acquired, the information is acquired regarding the first positional relationship capable of including the entire area acquired in a state excluding the part last added to the area in the imaging range. 5. The control device according to configuration 4.

[0188] (Configuration 6) The acquiring means acquires the region by performing a principal component analysis on the at least two parts. 6. The control device according to configuration 4 or 5.

[0189] (Configuration 7) The acquisition means includes: A center position of the region is obtained from the center values ​​of each directional component of the region; In the first positional relationship, the central position is the center of the imaging range. 7. The control device according to configuration 6,

[0190] (Configuration 8) The setting means is capable of setting a priority of the portion, When selecting the at least two parts from the plurality of parts, the acquisition means selects the part to which a high priority is set among the plurality of parts in preference to the other parts. 8. The control device according to any one of configurations 2 to 7.

[0191] (Configuration 9) The acquisition means acquires information used when moving the workpiece or the imaging means by the moving means so that at least two of the parts can be included in the imaging range, and the part having a higher priority among the at least two of the parts is closer to the center of the imaging range than the other parts, to be in the first positional relationship. 9. The control device according to configuration 8,

[0192] (Configuration 10) an upper limit setting means for setting an upper limit of the number of the portions to be included in the imaging range; 10. The control device according to any one of configurations 1 to 9.

[0193] (Configuration 11) A display unit for displaying the work is provided. 11. The control device according to any one of configurations 1 to 10.

[0194] (Configuration 12) The setting means sets a location selected by a worker in the work displayed on the display unit to the portion. 12. The control device according to claim 11,

[0195] (Configuration 13) The display unit displays the workpiece and the imaging means when the workpiece or the imaging means is moved by the moving means so as to have the first positional relationship. 13. The control device according to configuration 11 or 12.

[0196] (Configuration 14) The display unit displays the imaging range. 14. A control device according to any one of configurations 11 to 13.

[0197] (Configuration 15) A movement verification means for verifying whether the work or the imaging means can be moved by the moving means so as to have the first positional relationship based on the information, 15. The control device according to any one of configurations 2 to 14.

[0198] (Configuration 16) and an interference verification means for verifying whether at least one of the workpiece, the imaging means, and the moving means interferes with another object when the workpiece or the imaging means is moved by the moving means so as to have the first positional relationship based on the information. 16. The control device according to any one of configurations 2 to 15.

[0199] (Configuration 17) The acquisition means includes: Executing the process of acquiring the information a plurality of times to acquire a plurality of pieces of information for imaging a plurality of the parts; Acquire an operation time when the moving means sequentially moves the work or the imaging means based on the acquired multiple pieces of information, When the operation time exceeds a predetermined time, a combination of the at least two portions included in the imaging range is changed to acquire the information. 17. The control device according to any one of configurations 2 to 16.

[0200] (Configuration 18) a range setting unit that sets a length in a vertical direction with respect to an optical axis direction of the imaging unit and a length in a horizontal direction with respect to the optical axis direction as the setting of the predetermined size, 18. A control device according to any one of configurations 1 to 17.

[0201] (Configuration 19) the range setting means sets a length in a front-rear direction with respect to an optical axis direction of the imaging means as the setting of the predetermined size; 19. The control device according to configuration 18,

[0202] (Configuration 20) a positional relationship input means for inputting a positional relationship between the workpiece and the imaging means when the imaging means images the plurality of portions, moving the work or the imaging means by the moving means so that the work or the imaging means becomes in the positional relationship inputted from the positional relationship inputting means; 20. A control device according to any one of configurations 1 to 19.

[0203] (Configuration 21) The moving means moves the workpiece or the imaging means while keeping a constant distance between the imaging means and the workpiece in the optical axis direction of the imaging means. 21. The control device according to any one of configurations 1 to 20.

[0204] (Method 22) A process of imaging the workpiece by an imaging means capable of imaging the workpiece in an imaging range of a predetermined size; A process of moving the work or the imaging means by a moving means; A process of setting a plurality of portions on the workpiece to be imaged by the imaging means; The work or the imaging means is moved by the moving means so that at least two of the parts are in a first positional relationship that can be included in the imaging range. A control method comprising:

[0205] (Method 23) An information processing unit executes the control method according to method 22 to control the moving means; The moving means is a robot. A method for controlling a robot system comprising:

[0206] (Configuration 24) A control device according to any one of configurations 1 to 21, The moving means is a robot. A robot system comprising:

[0207] (Method 25) 25. Manufacturing an article using the robot system according to claim 24. A method for producing an article.

[0208] (Configuration 26) A program for causing a computer to execute the control method described in Method 22.

[0209] (Configuration 27) A computer-readable recording medium storing the program according to configuration 26. [Explanation of symbols]

[0210] 1...Robot system: 10...Moving means (robot): 20...Imaging means (camera): 20a...Optical axis: 22...Imaging range: 30...Work: 40...Control device (information processing device): 45...Information processing unit (CPU): 50...Control device (simulation device): 52...Display unit (display): 53...Positional relationship input means (mouse): 54...Positional relationship input means (keyboard): 55...Setting means, acquisition means, upper limit setting means, movement verification means, interference verification means, information processing unit (CPU): i1 to i21...Part (inspection position): p1 to p5...Center position: r1 to r5...Area (rectangle)

Claims

1. A simulation device that displays a workpiece in a virtual space, an imaging means capable of imaging the workpiece, and a moving means for moving the workpiece or the imaging means on a display unit, Setting means for setting multiple parts in the aforementioned workpiece, The system includes an acquisition means for acquiring first information used when moving the workpiece or the imaging means by the moving means so that at least two of the aforementioned plurality of parts are in a first positional relationship that can be included within the imaging range of the imaging means, A simulation device characterized by the following features.

2. The first information includes the teaching point of the moving means, The simulation apparatus according to feature 1.

3. The plurality of parts include a first part, a second part, and a third part, The two parts include the first part and the second part, When the third portion is not included in the imaging range in the first positional relationship, the acquisition means acquires second information used when moving the workpiece or the imaging means by the moving means so that the third portion is included in the imaging range of the imaging means in a second positional relationship. The simulation apparatus according to feature 1.

4. The acquisition means is, A first region including at least two of the above-mentioned parts is obtained, Regarding the first positional relationship that allows the first region to be included in the imaging range, first information is acquired. The simulation apparatus according to feature 1.

5. The acquisition means is, When a first positional relationship is obtained in which the entire first region can be included in the imaging range, a portion of the plurality of parts located outside the first region is added to obtain a second region including the added portion and the at least two portions, and it is verified whether the first positional relationship is obtained in which the entire second region can be included in the imaging range. When a first positional relationship that can encompass the entire second region within the imaging range is not obtained, the first information is obtained regarding the first positional relationship that can encompass the entire first region within the imaging range. The simulation apparatus according to feature 4.

6. The acquisition means performs principal component analysis on the at least two portions to acquire the first region. The simulation apparatus according to feature 4.

7. The acquisition means is, The center position of the region is obtained from the center values ​​of each directional component in the first region, In the first positional relationship, the center position is the center of the imaging range. The simulation apparatus according to feature 6.

8. The setting means can set a higher priority for one of the multiple parts than for the other parts. The acquisition means, when selecting at least two parts from the plurality of parts, selects one part in preference to the other parts. The simulation apparatus according to feature 1.

9. The acquisition means acquires the first information that a certain part is closer to the center of the imaging range than the other part. The simulation apparatus according to feature 8.

10. The system includes an upper limit setting means for setting an upper limit on the number of parts to be included in the imaging range. The simulation apparatus according to feature 1.

11. The setting means sets the portion of the workpiece selected by the operator as the plurality of parts. The simulation apparatus according to feature 1.

12. The imaging range is displayed on the display unit. The simulation apparatus according to feature 1.

13. The system includes a movement verification means for verifying whether the workpiece or the imaging means can be moved by the movement means to achieve a first positional relationship based on the first information. The simulation apparatus according to feature 1.

14. The system includes interference verification means for verifying whether at least one of the workpiece, the imaging means, and the moving means interferes with another object when the workpiece or the imaging means is moved by the moving means to achieve the first positional relationship based on the first information. The simulation apparatus according to feature 2.

15. The acquisition means is, The process of acquiring the first information is performed multiple times to acquire multiple data for imaging the multiple parts, Based on the acquired data, the operating time is obtained when the moving means sequentially moves the workpiece or the imaging means. When the aforementioned operating time exceeds a predetermined time, the combination of the at least two portions included in the imaging range is changed to acquire the first information. The simulation apparatus according to feature 2.

16. The imaging range setting means includes a range setting means for setting the length in the vertical direction with respect to the optical axis direction of the imaging means and the length in the horizontal direction with respect to the direction of the optical axis. The simulation apparatus according to feature 1.

17. The range setting means sets the length in the front-to-back direction with respect to the optical axis direction of the imaging means as setting the imaging range. The simulation apparatus according to feature 16.

18. The imaging means includes a position relationship input means for inputting the positional relationship between the workpiece and the imaging means when imaging multiple portions, The workpiece or the imaging means is moved by the moving means so that it matches the positional relationship input from the positional relationship input means. The simulation apparatus according to feature 1.

19. The moving means moves the workpiece or the imaging means while maintaining a constant distance between the imaging means and the workpiece in the optical axis direction of the imaging means. The simulation apparatus according to feature 1.

20. The information processing unit, A process for displaying a workpiece, imaging means capable of imaging the workpiece, and moving means for moving the workpiece or the imaging means, in a virtual space on a display unit. A process of setting multiple parts to the workpiece, The process involves acquiring first information used when moving the workpiece or the imaging means by the moving means so that at least two of the aforementioned parts are in a first positional relationship that can be included within the imaging range of the imaging means. An information processing method characterized by the following:

21. An information processing device connected to the simulation device according to any one of claims 1 to 19, which operates a robot corresponding to the moving means based on the first information, and causes a camera corresponding to the imaging means to image an object corresponding to the workpiece, A method for controlling a robot system characterized by the following features.

22. An information processing device that receives the first information from the simulation device according to any one of claims 1 to 19, A robot corresponding to the mobile means, connected to the information processing device, A robotic system equipped with, The information processing device operates the robot based on the first information. A robotic system characterized by the following features.

23. A robot system according to claim 22 is used to manufacture an article. A method for manufacturing an article characterized by the following:

24. A program for a computer to execute the information processing method described in claim 20.

25. A recording medium readable by a computer storing the program described in claim 24.

Images