Inspection route generation device, inspection route generation method

Abstract

To provide a test route that can reduce the time of a test and also can suppress reduction of the accuracy of a test to an appearance test device for running a test on the appearance of a test target object.SOLUTION: A test route generator 100 generates a test route for an appearance test device 200 for running a test on the appearance of a test target object on the basis of an image taken by an imaging unit 210. In the test route generator 100, a route determination unit 141 calculates a route length between each test position when the imaging unit 210 moves between a plurality of test positions and the posture change amount of the imaging unit 210 on the basis of test position information 1330 and test information configuration information 131, and determines a test route on the basis of the calculated route length and posture change amount.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to an apparatus and a method for generating an inspection path when an appearance inspection apparatus for inspecting an inspection object moves to a plurality of inspection positions.

Background Art

[0002] <00​​​​​​​​​​​​​​​​​​​​​​​​​​ Therefore, the main objective of the present invention is to provide an inspection path for an appearance inspection device that performs an appearance inspection of an object to be inspected, which can shorten the inspection time while suppressing a decrease in inspection accuracy. [Means for solving the problem]

[0006] The inspection path generation device according to the present invention generates an inspection path for an appearance inspection device that performs an appearance inspection of an object to be inspected based on an image captured by an imaging unit, and comprises: a storage unit that stores inspection position information representing a plurality of inspection positions in which the imaging unit captures images of the object to be inspected, and inspection device configuration information representing the configuration of the appearance inspection device, and a path determination unit that calculates the path length between each inspection position and the amount of change in the posture of the imaging unit when the imaging unit moves between the plurality of inspection positions based on the inspection position information and the inspection device configuration information, and determines the inspection path based on the calculated path length and the amount of change in posture, and the appearance inspection device comprises: an imaging unit and a robot unit to which the imaging unit is attached at its tip, and the robot unit comprises a plurality of joints that are rotatable along a predetermined rotation direction and between each joint The robot is configured in combination with a link, and the position and orientation of the imaging unit can be controlled by adjusting the angle of each joint, and the inspection device configuration information includes robot configuration information representing the shape of each link of the robot unit, the angle range of each joint and the limit speed information, and inspection equipment configuration information representing the arrangement of the imaging unit and the robot unit in the visual inspection device, and the path determination unit calculates the range of orientations that the robot unit can take at each inspection position based on the inspection position information and the robot configuration information and inspection equipment configuration information included in the inspection device configuration information, and calculates the range of motion of the imaging unit at each inspection position as a constraint condition for the orientation of the imaging unit based on the calculated range of orientations at each inspection position, and calculates the amount of orientation change based on the calculated constraint condition, The imaging unit generates a candidate inspection route for each of the inspection locations as it moves, thereby generating a plurality of candidate inspection routes, and in each of the generated candidate inspection routes... Based on the path length between each inspection position, the amount of change in the orientation of the imaging unit, and the direction of travel Let L be the total path length obtained by summing the aforementioned path lengths, D be the number of changes in the direction of travel, and Y be the total change in attitude obtained by summing the aforementioned changes in attitude. With α, β, and γ as predetermined weighting coefficients, the total travel cost Cc for each of the multiple candidate inspection paths is calculated using equation (1). Calculate, Cc = αL + βD + γY ...Equation (1) The above calculation Total movement cost Cc Based , one of the multiple candidate inspection routes The aforementioned inspection route to The system determines the rotation angle of the imaging unit at each inspection position and selects a combination of rotation angles for the imaging unit at each inspection position such that the amount of change in the rotation angle of the imaging unit in a predetermined direction between each inspection position in the determined inspection path is minimized. The inspection path generation method according to the present invention generates an inspection path for a visual inspection device that performs a visual inspection of an object to be inspected based on an image captured by an imaging unit, wherein the visual inspection device comprises an imaging unit and a robot unit to which the imaging unit is attached at its tip, the robot unit is configured by combining a plurality of joints that can rotate along a predetermined rotation direction and links connecting each joint, and the position and orientation of the imaging unit can be controlled by adjusting the angle of each joint, and the processing involves a computing unit acquiring inspection position information representing a plurality of inspection positions in which the imaging unit takes images of the object to be inspected, and inspection device configuration information representing the configuration of the visual inspection device, and based on the acquired inspection position information and inspection device configuration information, the processing involves determining the path length between each inspection position and the amount of change in the orientation of the imaging unit when the imaging unit moves between the plurality of inspection positions. The process involves calculating the inspection path and determining the inspection path based on the calculated path length and the amount of change in posture. The inspection device configuration information includes robot configuration information representing the shape of each link of the robot unit, the angle range of each joint, and the limit speed information, and inspection equipment configuration information representing the arrangement of the imaging unit and the robot unit in the visual inspection device. In the process of calculating the amount of change in posture, the range of postures that the robot unit can take at each inspection position is calculated based on the inspection position information and the robot configuration information and inspection equipment configuration information included in the inspection device configuration information. Based on the calculated range of postures at each inspection position, the range of motion of the imaging unit at each inspection position is calculated as a constraint condition for the posture of the imaging unit. Based on the calculated constraint condition, the amount of change in posture is calculated. In the process of determining the inspection path, The imaging unit generates a candidate inspection route for each of the inspection locations as it moves, thereby generating a plurality of candidate inspection routes, and in each of the generated candidate inspection routes... Based on the path length between each inspection position, the amount of change in the orientation of the imaging unit, and the direction of travel Let L be the total path length obtained by summing the aforementioned path lengths, D be the number of changes in the direction of travel, and Y be the total change in attitude obtained by summing the aforementioned changes in attitude. With α, β, and γ as predetermined weighting coefficients, the total travel cost Cc for each of the multiple candidate inspection paths is calculated using equation (1). Calculate, Cc = αL + βD + γY ...Equation (1) The above calculation Total movement cost Cc Based , one of the multiple candidate inspection routes The aforementioned inspection route toThe system determines the rotation angle of the imaging unit at each inspection position and selects a combination of rotation angles for the imaging unit at each inspection position such that the amount of change in the rotation angle of the imaging unit in a predetermined direction between each inspection position in the determined inspection path is minimized. [Effects of the Invention]

[0007] According to the present invention, it is possible to provide an inspection path for an appearance inspection device that performs an appearance inspection of an object to be inspected, which can shorten the inspection time while suppressing a decrease in inspection accuracy. Other issues, configurations, and effects not mentioned above will be clarified by the following description of the embodiments. [Brief explanation of the drawing]

[0008] [Figure 1] This figure shows an example configuration of a visual inspection system including an inspection path generation device according to the first embodiment of the present invention. [Figure 2] This figure shows an example of robot configuration information data. [Figure 3] This figure shows an example of data for inspection equipment configuration information. [Figure 4] This figure shows an example of data for the area to be inspected. [Figure 5] This figure shows an example of data for inspection condition information. [Figure 6] This figure shows an example of the connection relationship between nodes as represented by the inter-node connection information. [Figure 7] This figure shows an example of travel cost information data. [Figure 8] This figure shows an example of the output data. [Figure 9] This is an example of a flowchart for the inspection route planning process according to the first embodiment of the present invention. [Figure 10] This is an example of a flowchart for determining the inspection route. [Figure 11] This is an explanatory diagram of how travel costs are calculated. [Figure 12] This figure shows an example of the output screen in the inspection route generation device according to the first embodiment of the present invention. [Figure 13] It is a diagram showing another example of an output screen in an inspection path generation device according to a first embodiment of the present invention. [Figure 14] It is a diagram showing a configuration example of an appearance inspection system including an inspection path generation device according to a second embodiment of the present invention. [Figure 15] It is an example of a flowchart of the inspection path planning process according to a second embodiment of the present invention. [Figure 16] It is an example of a flowchart of an inspection path change process. [Figure 17] It is an explanatory diagram of a method for selecting an inspection position to be changed. [Figure 18] It is an explanatory diagram of a method for setting candidate positions and candidate postures.

Mode for Carrying Out the Invention

[0009] Hereinafter, embodiments of the present invention will be described with reference to the drawings. The following description and drawings are examples for explaining the present invention, and for the sake of clarity of explanation, omissions and simplifications are made as appropriate. The present invention can be implemented in various other forms. Unless otherwise limited, each component may be singular or plural.

[0010] In the drawings, the positions, sizes, shapes, ranges, etc. of each component shown may not represent the actual positions, sizes, shapes, ranges, etc. in order to facilitate understanding of the invention. Therefore, the present invention is not necessarily limited to the positions, sizes, shapes, ranges, etc. disclosed in the drawings.

[0011] When there are a plurality of components having the same or similar functions, they may be described with the same reference numeral and different subscripts. However, when it is not necessary to distinguish these plurality of components, the subscripts may be omitted in the description.

[0012] Furthermore, while the following explanation may describe the processes performed by executing a program, the processor (e.g., CPU, GPU) executes the program, performing defined processes using memory resources (e.g., memory) and / or interface devices (e.g., communication ports) as appropriate. Therefore, the processor may be the primary entity performing the processes. Similarly, the primary entity performing the processes by executing a program may be a controller, device, system, computer, or node having a processor. The primary entity performing the processes by executing a program may be any arithmetic unit, and may include dedicated circuits (e.g., FPGAs or ASICs) that perform specific processes.

[0013] A program may be installed from its program source into a device such as a computer. The program source may be, for example, a program distribution server or a computer-readable storage medium. If the program source is a program distribution server, the program distribution server includes a processor and storage resources for storing the program to be distributed, and the processor of the program distribution server may distribute the program to other computers. Furthermore, in the following description, two or more programs may be implemented as a single program, or one program may be implemented as two or more programs.

[0014] -First Embodiment- A first embodiment of the present invention will be described below.

[0015] [Explanation of the device configuration] Figure 1 shows an example configuration of a visual inspection system including an inspection route generation device according to the first embodiment of the present invention. The visual inspection system shown in Figure 1 is configured such that the inspection route generation device 100 and the visual inspection device 200 are connected to each other via a communication line 300. The communication line 300 is configured using, for example, Ethernet (registered trademark) or wireless LAN, and transmits various information that is sent and received between the inspection route generation device 100 and the visual inspection device 200.

[0016] The visual inspection device 200 comprises an imaging unit 210, a robot unit 220, and a processing unit 230. The imaging unit 210 captures an object to be inspected (not shown) and outputs the captured image to the processing unit 230. The robot unit 220 is composed of a combination of multiple joints that are rotatable along a predetermined rotational direction and links connecting each joint. By adjusting the angle of each joint according to the control of the processing unit 230, the imaging unit 210 attached to the tip is moved to an arbitrary inspection position and positioned according to an arbitrary imaging direction. The processing unit 230 controls the robot unit 220 and determines whether the appearance of the object to be inspected is normal or not based on the captured image input from the imaging unit 210. This enables the visual inspection of the object to be inspected.

[0017] The inspection path generation device 100 generates the order of each inspection position that the imaging unit 210 passes through when the visual inspection device 200 performs a visual inspection of the object to be inspected, and the orientation of the imaging unit 210 at each inspection position, as an inspection path for the visual inspection device 200. The generated inspection path information is then transmitted to the visual inspection device 200 to support the visual inspection of the object to be inspected by the visual inspection device 200.

[0018] The inspection path generation device 100 includes an input device 110, an output device 120, a storage unit 130, and an arithmetic unit 140.

[0019] The input device 110 is a device for inputting the various types of information stored in the memory unit 130, as described below, through user operation. The input device 110 is configured using, for example, a keyboard, mouse, or touch panel.

[0020] The output device 120 is a device for outputting information such as the generated inspection route and providing it to the user. The output device 120 is configured using, for example, a display, printer, smartphone, or tablet PC.

[0021] The storage unit 130 stores information input from the user using the input device 110, as well as information such as the calculation results of the arithmetic unit 140. The storage unit 130 is configured using a well-known information storage device capable of reading and writing data, such as an HDD (Hard Disk Drive) or SSD (Solid State Drive). The storage unit 130 stores inspection device configuration information 131, inspection target information 132, inspection condition information 133, path graph information 134, and output information 135. Other information, such as programs executed by the arithmetic unit 140, may also be stored in the storage unit 130.

[0022] The inspection device configuration information 131 is information relating to the configuration of the visual inspection device 200, and includes robot configuration information 1310 and inspection equipment configuration information 1311. The robot configuration information 1310 represents information such as the shape of the robot section 220 in the visual inspection device 200, the range of each joint angle, and the limit speed. The inspection equipment configuration information 1311 represents information such as the arrangement of the imaging section 210 and the robot section 220 that constitute the visual inspection device 200.

[0023] The inspection object information 132 is information about an object to be inspected by the visual inspection device 200, and includes shape information 1320 and inspection area information 1321. The shape information 1320 is information representing the shape of the object to be inspected. The inspection area information 1321 is information representing the area of ​​the object to be inspected that is subject to visual inspection.

[0024] The inspection condition information 133 is information regarding the constraints on the visual inspection device 200 when performing a visual inspection of an object to be inspected, and includes inspection position information 1330, inspectable area information 1331, and camera movable range information 1332. The inspection position information 1330 represents the inspection position in which the imaging unit 210 takes images of the object to be inspected, and the orientation of the imaging unit 210 at each inspection position. The inspectable area information 1331 represents the inspectable area for each inspection position. The inspectable area represented by the inspectable area information 1331 refers to the surface area in the object to be inspected that the visual inspection device 200 can perform a visual inspection on from the image acquired by the imaging unit 210 at each inspection position, out of a plurality of surface areas that are subject to visual inspection. The camera movable range information 1332 represents the movable range of the imaging unit 210 at each inspection position. The movable range represented by the camera movable range information 1332 corresponds to the constraints on the orientation of the imaging unit 210 at each inspection position.

[0025] The path graph information 134 is information about the path graph used in calculations performed by the arithmetic unit 140 when generating the inspection path of the visual inspection device 200, and includes node connection information 1340 and travel cost information 1341. The node connection information 1340 is information representing the connection relationship between nodes corresponding to each inspection position. The travel cost information 1341 is information representing the travel cost according to the length of travel time when the imaging unit 210 moves between each node represented by the node connection information 1340.

[0026] Output information 135 represents the calculation results from the arithmetic unit 140 and includes inspection path information 1350, camera angle information 1351, and robot posture information 1352. Inspection path information 1350 represents the optimal inspection path of the visual inspection device 200 generated by the calculations of the arithmetic unit 140. Camera angle information 1351 represents the rotation angle of the imaging unit 210 at each inspection position in the inspection path represented by the inspection path information 1350. Robot posture information 1352 represents the posture of the robot unit 220 at each inspection position in the inspection path represented by the inspection path information 1350.

[0027] The arithmetic unit 140 is the part that performs various calculations for the inspection path generation device 100 to generate an inspection path, and is composed of well-known elements such as a CPU, GPU, RAM, and ROM. The arithmetic unit 140 has the following functional blocks: a path determination unit 141, an inspectionable area calculation unit 142, a camera movable range calculation unit 143, a path graph generation unit 144, a movement cost calculation unit 145, a minimum cost path selection unit 146, and an output information generation unit 147. The arithmetic unit 140 can realize these functional blocks, for example, by executing a predetermined program that is stored in advance.

[0028] The route determination unit 141 determines the inspection route using the functional blocks from the inspectionable area calculation unit 142 to the output information generation unit 147. Details of the processing performed by the route determination unit 141 will be explained later with reference to Figures 9 and 10.

[0029] The inspectionable area calculation unit 142 calculates the inspectionable area for each inspection position and stores the calculation result as inspectionable area information 1331 in the storage unit 130.

[0030] The camera movable range calculation unit 143 calculates the movable range of the imaging unit 210 for each inspection position and stores the calculation result as camera movable range information 1332 in the storage unit 130.

[0031] The route graph generation unit 144 sets nodes corresponding to each inspection location and generates a route graph representing the connection relationships between each node. Then, it stores the information of the generated route graph as inter-node connection information 1340 in the storage unit 130.

[0032] The movement cost calculation unit 145 calculates the movement cost when the imaging unit 210 moves between each node of the path graph based on the inspection device configuration information 131 and the inspection condition information 133, and stores the calculation result as movement cost information 1341 in the storage unit 130.

[0033] The minimum cost route selection unit 146 selects the minimum cost route that passes through all nodes based on the route graph generated by the route graph generation unit 144 and the travel cost calculated by the travel cost calculation unit 145. This generates the optimal inspection route for the visual inspection device 200 when performing a visual inspection of the object to be inspected.

[0034] The output information generation unit 147 determines the rotation angle (imaging direction) of the imaging unit 210 at each node and the angles of each joint of the robot unit 220 at each node for the lowest cost path selected by the lowest cost path selection unit 146. Then, using this determined information and the inspection path information generated by the lowest cost path selection unit 146, it generates output information 135 and stores it in the storage unit 130.

[0035] [Data Description] Next, the data used in the inspection path generation device according to the first embodiment of the present invention will be described below with reference to Figures 2 to 8.

[0036] Figure 2 shows an example of the data of the robot configuration information 1310, which is part of the inspection device configuration information 131 stored in the memory unit 130. In the example in Figure 2, the robot configuration information 1310 has data shown in data columns 101 to 103 for each component that makes up the robot unit 220. Data column 101 stores classification names for each component that makes up the robot unit 220, such as "robot," "link," and "joint." Data column 102 stores the names of the data items. Data column 103 stores information corresponding to the names of the items in data column 102, such as data values ​​and file names.

[0037] Figure 3 shows an example of the data of the inspection equipment configuration information 1311, which is part of the inspection device configuration information 131 stored in the memory unit 130. In the example in Figure 3, the inspection equipment configuration information 1311 has the data shown in data sequences 111 to 114 for each piece of equipment that makes up the visual inspection device 200. Data sequence 111 stores the names of each piece of equipment that makes up the visual inspection device 200. Data sequence 112 stores information about the mounting target equipment corresponding to each piece of equipment, that is, information indicating which other equipment each piece of equipment is attached to. Data sequence 113 stores information about the relative position of each piece of equipment with respect to the mounting target equipment. Data sequence 114 stores information about the relative orientation of each piece of equipment with respect to the mounting target equipment.

[0038] Figure 4 shows an example of the data for the inspection target area information 1321, which is part of the inspection target information 132 stored in the memory unit 130. In the example in Figure 4, the inspection target area information 1321 has the data shown in data sequences 211 to 218 for each inspection target that the visual inspection device 200 performs a visual inspection on. Data sequence 211 stores the part name of each inspection target. Data sequences 211 to 218 each store information indicating whether or not a region F1 to Fn (where n is a natural number) is subject to visual inspection, assuming that the surface of each inspection target is divided into multiple regions. Specifically, in data sequences 211 to 218, "1" indicates that the region is subject to inspection, and "0" indicates that the region is not subject to inspection. The data representing the regions F1 to Fn of the inspection target is stored in the memory unit 130 as shape information 1320 within the inspection target information 132, and can be represented in a well-known polygon data format such as STL.

[0039] Figure 5 shows an example of the data for inspection condition information 133 stored in the memory unit 130. In the example in Figure 5, the inspection condition information 133 has data shown in data sequences 301-307, 311-316, and 321-322 for each inspection position where the imaging unit 210 takes an image of the object to be inspected when the visual inspection device 200 performs a visual inspection. Data sequence 301 stores information representing each inspection position. In the example in Figure 5, the inspection positions are designated as A-L, and information representing these inspection positions A-L is stored.

[0040] Data sequences 302 to 304 store information representing the relative position of each inspection location with respect to the object being inspected. Data sequences 305 to 307 store information representing the relative orientation of the imaging unit 210 at each inspection location with respect to the object being inspected. These data sequences 302 to 307 together constitute the inspection location information 1330.

[0041] Data sequences 311 to 316 each store information indicating whether the visual inspection device 200 can perform a visual inspection of regions F1 to Fn based on the image of the object to be inspected acquired by the imaging unit 210 at each inspection position. Specifically, in data sequences 311 to 316, "1" indicates that the region can be inspected based on the image obtained at each inspection position, and "0" indicates that the region cannot be inspected based on the image obtained at each inspection position. These data sequences 311 to 316 constitute the inspectable region information 1331.

[0042] Data sequences 321 and 322 each store information representing the range within which the robot unit 220 can move the imaging unit 210 at each inspection position. In the example in Figure 5, the minimum rotation angle stored in data sequence 321 and the maximum rotation angle stored in data sequence 322 represent the rotation angle range when the imaging unit 210 is rotated in the yaw direction. Specifically, for example, for inspection position A, "30" is stored in data sequence 321 and "200" is stored in data sequence 322. This indicates that at inspection position A, the imaging unit 210 can assume the posture necessary for imaging the object under inspection within a yaw rotation angle range of 30° to 200°. On the other hand, for inspection position L, no numerical values ​​are stored in either data sequence 321 or 322. This indicates that at inspection position L, the imaging unit 210 cannot assume the posture necessary for imaging the object under inspection at any yaw rotation angle. These data sequences 321 and 322 constitute the camera movable range information 1332. In the example shown in Figure 5, the camera range of motion information 1332 is represented by the yaw rotation angle. However, the camera range of motion information 1332 may also be represented by, for example, the roll rotation angle or the pitch rotation angle, or a combination of these may be used to represent the camera range of motion information 1332.

[0043] Figure 6 shows an example of the connection relationships between nodes represented by the inter-node connection information 1340, which is part of the path graph information 134 stored in the memory unit 130. In the inter-node connection information 1340, for example, as shown in Figure 6, nodes corresponding to each inspection position A to G are set, and the connection relationships between these nodes are represented by arrows. The information of the arrows between each node shown in Figure 6 is stored in the memory unit 130 as inter-node connection information 1340, which represents the combination of sequences in which the imaging unit 210 moves to inspection positions A to G when the visual inspection device 200 performs a visual inspection of the object to be inspected.

[0044] Figure 7 shows an example of the data for travel cost information 1341, which is part of the route graph information 134 stored in the memory unit 130. In the example in Figure 7, the travel cost information 1341 stores a numerical value indicating the magnitude of the travel cost for each combination of a column 411 representing the starting node and a row 422 representing the ending node, for each connection relationship between nodes represented by the inter-node connection information 1340. Note that no numerical value is stored for combinations of nodes where the travel cost is known to be high, such as the combination of starting node B and ending node C in Figure 7. Furthermore, the numerical values ​​of the travel cost stored for each node are asymmetrical, so that the numerical value changes if the starting and ending points are swapped, in order to impose constraints such as circulating the inspection positions in a clockwise direction. These numerical values ​​of travel cost are calculated by the travel cost calculation unit 145 by determining the travel time between each inspection position, the change in the attitude of the imaging unit 210, the direction of travel, etc., based on the inspection device configuration information 131 and the inspection condition information 133.

[0045] Figure 8 shows an example of the data of output information 135 stored in the memory unit 130. In the example in Figure 8, the output information 135 has data shown in data sequences 501-507, 511, 521-525 for each inspection position where the imaging unit 210 takes an image of the object to be inspected when the visual inspection device 200 performs a visual inspection. Data sequence 501 stores information representing each inspection position in the order in which the imaging unit 210 passes through the inspection path generated by the minimum cost path selection unit 146.

[0046] Data columns 502 to 504 store information representing the relative position of each inspection location with respect to the object being inspected. Data columns 505 to 507 store information representing the relative orientation of the imaging unit 210 at each inspection location with respect to the object being inspected. These data from data columns 502 to 507 constitute the inspection route information 1350. Note that the data in each row of the inspection route information 1350 corresponds to the data in each row of the inspection location information 1330, and the same data is stored for the same inspection location.

[0047] The data sequence 511 stores information representing the rotation angle of the imaging unit 210 at each inspection position. In the example in Figure 8, it represents the yaw rotation angle of the imaging unit 210. The data in this data sequence 511 constitutes the camera angle information 1351. In the example in Figure 8, the camera angle information 1351 is shown as being represented by the yaw rotation angle, but the camera angle information 1351 may also be represented by, for example, the roll rotation angle or the pitch rotation angle, or a combination of these may be used to form the camera angle information 1351.

[0048] Data sequences 521 to 525 store information representing the angles of each joint J1 to Jn (where n is a natural number) of the robot unit 220 at each inspection position. The data from these data sequences 521 to 525 constitute the robot posture information 1352.

[0049] [Explanation of the flowchart] Next, the inspection route planning process performed in the inspection route generation device according to the first embodiment of the present invention will be described below with reference to the flowcharts in Figures 9 and 10 and the explanatory diagram in Figure 11.

[0050] Figure 9 is an example of a flowchart of the inspection route planning process according to the first embodiment of the present invention. The process shown in the flowchart of Figure 9 is executed by the route determination unit 141 in the arithmetic unit 140.

[0051] In step S101, the inspection device configuration information 131, the object to be inspected information 132, and the inspection position information 1330 are read from the storage unit 130 and input to the arithmetic unit 140 as information to be used in the processing of step S102. This information is assumed to be pre-stored in the storage unit 130 based on information previously entered by the user using the input device 110.

[0052] In step S102, an inspection route determination process is executed to determine the optimal inspection route based on the information entered in step S101. Details of the inspection route determination process performed in step S102 will be described later with reference to the flowchart in Figure 10.

[0053] In step S103, the inspection route information determined in step S102 is stored in the storage unit 130 as output information 135 and output to the output device 120.

[0054] Figure 10 is an example of a flowchart of the inspection route determination process performed in step S102. The process shown in the flowchart of Figure 10 is executed in the arithmetic unit 140 by the route determination unit 141 appropriately calling the inspectionable area calculation unit 142, the camera movable range calculation unit 143, the route graph generation unit 144, the movement cost calculation unit 145, the minimum cost route selection unit 146, and the output information generation unit 147, using these functional blocks.

[0055] In step S200, the inspectable area of ​​the visual inspection device 200 at each inspection location is calculated based on the inspection target information 132 and inspection location information 1330 read in step S101 in Figure 9. Here, for example, the inspectable area calculation unit 142 compares the position and orientation of the imaging unit 210 with the position and orientation of the inspection target area F1 to Fn at each inspection location, based on the relative position and orientation of the imaging unit 210 at each inspection location represented by the inspection location information 1330 and the shape information 1320 included in the inspection target information 132. As a result, if the difference between these is within a predetermined range, the area is determined to be an inspectable area; if it is outside the predetermined range, the area is determined not to be an inspectable area. This allows the inspectable area to be calculated. The information on the inspectable area calculated in step S200 is stored in the storage unit 130 as inspectable area information 1331 included in the inspection condition information 133. Furthermore, based on the inspection target area information 1321 contained in the inspection target information 132, the inspection target areas are extracted from the calculated inspectionable areas, and the area of ​​each extracted inspectionable area is calculated based on the shape information 1320. The area of ​​each inspectionable area calculated here is used in calculating the total inspectionable areas Ao and Ac in steps S204 and S207 described later.

[0056] In step S201, based on the inspection device configuration information 131 and inspection position information 1330 read in step S101 in Figure 9, the movable range of the imaging unit 210 corresponding to the posture that the robot unit 220 can take at each inspection position is calculated. Here, for example, the camera movable range calculation unit 143 obtains information such as the shape of the robot unit 220, the angle range and limit speed of each joint, and the shape and arrangement of each piece of equipment in the visual inspection device 200 from the robot configuration information 1310 and inspection equipment configuration information 1311 included in the inspection device configuration information 131. Based on this information, the range of postures that the robot unit 220 can take at each inspection position is calculated. Then, from the calculated range of postures of the robot unit 220, the movable range of the imaging unit 210 at each inspection position, i.e., the constraint conditions for the posture of the imaging unit 210, is calculated. The information on the movable range of the imaging unit 210 calculated in step S201 is stored in the storage unit 130 as camera movable range information 1332 included in the inspection condition information 133.

[0057] In step S202, the movement cost between each inspection position is calculated and a path graph is generated based on the inspection position information 1330 read in step S101 in Figure 9 and the camera movable range information 1332 calculated in step S201. Here, for example, the path graph generation unit 144 sets nodes corresponding to each inspection position based on the inspection position information 1330 and sets a movement path that represents the connection relationship of the imaging unit 210 with lines connecting each node. Then, the movement cost calculation unit 145 calculates the path length (distance between nodes), the amount of change in the posture of the imaging unit 210, the direction of travel, etc. for each set movement path based on the inspection position information 1330 and the camera movable range information 1332, and uses these calculation results to calculate the movement cost between each node. This allows a path graph to be generated. The movement path information and movement cost information calculated in step S202 are stored in the storage unit 130 as node connection information 1340 and movement cost information 1341, respectively, included in the path graph information 134.

[0058] Figure 11 is an explanatory diagram of the method for calculating travel costs. Below, we will explain an example of calculating travel costs between four nodes, A to D, as shown in Figure 11.

[0059] The movement cost calculation unit 145 calculates the length (distance) of each movement path connecting the two points, the amount of attitude change of the imaging unit 210, and the direction of travel for all combinations when two points are selected from the four nodes A, B, C, and D shown in Figure 11. Figure 11 shows examples of the distance, attitude change of the imaging unit 210, and direction of travel for each movement path calculated in this way. For example, in a movement path starting at node A and ending at node B, the distance is 0.2 and the attitude change of the imaging unit 210 is 30, indicating that the imaging unit 210 moves in a counterclockwise direction.

[0060] The movement cost calculation unit 145 calculates the movement cost between each node and sets the movement cost information 1341 by evaluating the length of time required for the imaging unit 210 to move along each movement path based on the above information calculated for each movement path. At this time, for movement paths where the amount of change in the posture of the imaging unit 210 is greater than a predetermined threshold, for example, it is preferable not to set a movement cost in the movement cost information 1341, as in the combination of starting point B and ending point C in the example of Figure 7. In this way, in the processing from step S203 onwards described later, candidate inspection paths with high movement costs can be excluded in advance, and the inspection path can be determined efficiently.

[0061] Returning to the explanation of Figure 10, in step S203, the minimum cost route selection unit 146 generates an initial inspection route candidate Ro using the route graph generated in step S202. Here, the initial inspection route candidate Ro is generated by randomly selecting the order in which nodes are passed through in the route graph, and this is set as the initial value of the inspection route candidate Rc. At this time, it is not necessary to select a route that passes through all nodes. A route that passes through at least two or more nodes can be selected as the initial inspection route candidate Ro. Furthermore, for travel routes for which no cost has been set in the travel cost information 1341, for example, the travel route corresponding to the combination of starting point B and ending point C in Figure 7, it is preferable to generate the initial inspection route candidate Ro so as not to include such travel routes.

[0062] In step S204, the minimum cost route selection unit 146 calculates the total travel cost Co and the total inspectable area Ao of the initial inspection route candidate Ro generated in step S203. For example, the total travel cost Co can be calculated by summing the travel costs calculated in step S202 for each travel path included in the initial inspection route candidate Ro, according to the order in which the nodes selected in the initial inspection route candidate Ro are passed. In addition, the total inspectable area Ao can be calculated by summing the areas of the inspectable areas calculated in step S200 for each inspection position corresponding to each node in the initial inspection route candidate Ro, avoiding overlaps. The total travel cost Co and the total inspectable area Ao of the initial inspection route candidate Ro calculated in step S204 correspond to the initial values ​​of the total travel cost Cc and the total inspectable area Ac of the inspection route candidate Rc, respectively.

[0063] In steps S205 to S210, a loop process is executed. Here, the processes from steps S206 to S210 are repeatedly executed until the total travel cost Cc of the candidate inspection path Rc and the total inspectable area Ac, which were set during the loop process, each exceed a predetermined threshold, or until the number of loop iterations exceeds a predetermined threshold.

[0064] In step S206, the minimum cost route selection unit 146 generates new inspection route candidates Rc from the route graph generated in step S202. Here, similar to step S203, unselected inspection routes can be generated as inspection route candidates Rc by randomly selecting the order in which nodes are passed through the route graph. Alternatively, inspection route candidates Rc may be generated by swapping, adding, or deleting some nodes in the current inspection route R in the loop processing. In this case as well, similar to step S203, it is not necessary to select a route that passes through all nodes. A route that passes through at least two or more nodes can be selected as an inspection route candidate Rc. Furthermore, it is preferable to generate inspection route candidates Rc that do not include travel routes for which no cost has been set in the travel cost information 1341.

[0065] In step S207, the minimum cost route selection unit 146 calculates the total travel cost Cc and the total inspectable area Ac of the inspection route candidate Rc generated in step S206. Here, the total travel cost Cc and the total inspectable area Ac can be calculated in the same way as in step S204.

[0066] In step S208, the minimum cost route selection unit 146 compares the total travel cost Cc of the candidate inspection route Rc calculated in step S207 with the total travel cost C of the current inspection route R. If Cc <= C, the process proceeds to step S209; otherwise, the process proceeds to step S205.

[0067] In step S209, the minimum cost route selection unit 146 compares the total inspectable area Ac of the candidate inspection route Rc calculated in step S207 with the total inspectable area A of the current inspection route R. If Ac >= A, the process proceeds to step S210; otherwise, the process proceeds to step S205.

[0068] In step S210, the minimum cost route selection unit 146 updates the inspection route R, the total travel cost C of the inspection route R, and the total inspectable area A. Here, the candidate inspection route Rc generated in the most recent step S206 is set as the inspection route R, and the total travel cost Cc and total inspectable area Ac of the candidate inspection route Rc are set as the total travel cost C and total inspectable area A of the inspection route R, respectively, thereby updating these values. After the processing in step S210 is completed, the process proceeds to step S205.

[0069] After completing the loop processing in steps S205 to S210, the inspection route R is ultimately determined to be the one among the multiple candidate inspection routes Rc that has the smallest total travel cost Cc and the largest total inspectable area Ac.

[0070] In step S211, the output information generation unit 147 calculates the rotation angle of the imaging unit 210 for each inspection position in the inspection path R, which was finally determined in the loop processing of steps S205 to S210. Here, according to the order in which each inspection position in the inspection path R is traversed, a combination of rotation angles is selected for each inspection position in the inspection path R that minimizes the change in the yaw rotation angle of the imaging unit 210 within the camera's movable range calculated in step S201. The same calculation method can also be applied when setting roll rotation angle, pitch rotation angle, etc., for the imaging unit 210, as described above. This calculates the attitude P of the imaging unit 210 in the inspection path R.

[0071] In step S212, output information 135 is generated based on the inspection path R finally determined in the loop processing of steps S205 to S210 and the posture P calculated in step S212. Here, for example, the output information generation unit 147 determines the angle of each joint of the robot unit 220 at each node of the inspection path R based on the inspection device configuration information 131 and inspection position information 1330 read in step S101 of Figure 9. Then, the inspection position information 1330 read in step S101 is rearranged according to the order in which each inspection position is traversed in the inspection path R, and the posture P and the angle of each joint corresponding to each inspection position are set to determine the inspection path information 1350, camera angle information 1351, and robot posture information 1352. This enables the generation of output information 135. The output information 135 generated in step S212 is stored in the storage unit 130 and output to the output device 120 in step S103 of Figure 9. In addition, information such as the total travel cost C of the inspection route R and the total inspectable area A may be included in the output information 135.

[0072] In addition, in the flowchart of Figure 10 described above, the travel cost between each node may be omitted when generating the path graph in step S202. In this case, when calculating the total travel cost Co of the initial inspection path candidate Ro in step S204, and when calculating the total travel cost Cc of the inspection path candidate Rc in step S207, it is preferable to calculate the total travel costs Co and Cc by considering the total distance between nodes in the entire inspection path candidate, the amount of change in the attitude of the imaging unit 210, and the change in the direction of travel.

[0073] For example, the method for calculating the total travel cost Cc when a path passing through the four nodes A to D shown in Figure 11 in the order A→B→C→D is selected as the candidate inspection path Rc is explained below. In this case, from the distance information between each node, the total path length L of the candidate inspection path Rc can be calculated as L=0.2+0.1+0.2=0.5. Also, from the direction of travel information between each node, the direction of travel of the candidate inspection path Rc is counterclockwise→counterclockwise→counterclockwise, so the number of changes in direction of travel D is D=0. In this way, by reducing the number of changes in direction of travel D, the inspection path becomes smoother, and the path error that occurs when the imaging unit 210 operates continuously without stopping at each waypoint can be reduced. Also, from the camera attitude change information between each node, the total attitude change amount Y of the imaging unit 210 in the candidate inspection path Rc can be calculated as Y=30+20+5=55.

[0074] Based on the total path length L, the number of changes in direction D, and the total change in attitude Y obtained as described above, the total travel cost Cc of the candidate inspection path Rc can be calculated, for example, using the following equation (1). Cc = αL + βD + γY ...Equation (1) In equation (1), α, β, and γ are weighting coefficients, and each can be set to any value.

[0075] In the flowchart of Figure 10, during the loop processing from steps S205 to S210, the candidate inspection path Rc that minimizes the total movement cost Cc, expressed by equation (1), is ultimately selected as the inspection path R. This makes it possible to generate an inspection path in which the movement distance of the imaging unit 210 is short, the change in posture is small, and the change of direction is minimal.

[0076] Furthermore, in the flowchart of Figure 10, the order of steps S208 and S209 may be reversed, or only one of steps S208 or S209 may be performed.

[0077] If only step S208 is performed, the inspection path candidate Rc with the lowest travel cost is ultimately selected as the inspection path R in the loop processing of steps S205 to S210. As a result, an inspection path is obtained that allows the visual inspection of the object to be inspected by the visual inspection device 200 to be completed in a short time. In this case, instead of performing the determination in step S209, it may be determined whether the total inspectable area Ac of the inspection path candidate Rc is above a predetermined threshold, and the processing in step S210 is performed only if it is above the threshold. In this way, it is possible to prevent the generation of an inspection path in which the inspectable area of ​​the object to be inspected is small, and therefore the desired inspection accuracy cannot be obtained.

[0078] On the other hand, if only step S208 is performed, the inspection path candidate Rc with the largest inspectable area is ultimately selected as the inspection path R in the loop processing of steps S205 to S210. Therefore, an inspection path is obtained that maximizes the inspectable area in the visual inspection of the object to be inspected by the visual inspection device 200. In this case, instead of making the determination in step S208, it may be determined whether the total travel cost Cc of the inspection path candidate Rc is below a predetermined threshold, and the processing in step S210 is performed only if it is below the threshold. In this way, it is possible to prevent the generation of an inspection path with a high travel cost that would prevent the visual inspection from being performed within the desired time.

[0079] Alternatively, instead of making the determinations in steps S208 and S209, the total cost W1, which is a weighted sum of the total travel cost Cc of the candidate inspection route Rc and the total inspectable area Ac, may be calculated using the following formula (2), and the candidate inspection route Rc may be evaluated using this total cost W1. W1 = δCc + ε(A all -Ac) …Formula (2) In equation (2), δ and ε are weight coefficients, and any value can be set for each. Also, A all This represents the total area of ​​the inspection region within the object being inspected.

[0080] Furthermore, instead of performing the processing in steps S203 to S210, the total cost W1 for all travel paths included in the path graph generated in step S202 may be calculated using equation (2) above, and then the path that minimizes the total cost W1 may be found as the inspection path R. When finding such an inspection path R, the optimal solution can be obtained using well-known methods, for example, by treating it as the Traveling Salesperson Problem. Alternatively, the inspection path R that minimizes the total cost W1 may be calculated using general methods such as brute force or greedy algorithms.

[0081] Alternatively, instead of performing the processing in steps S203 to S211, a total cost W2, separate from the total cost W1, may be calculated for all paths represented by the path graph generated in step S202, and the path that minimizes the total cost W2 may be found as the inspection path R. The total cost W2 can be calculated, for example, as a weighted sum of the total movement cost Cc, total inspectable area Ac, and total attitude change amount Y for each inspection path candidate Rc obtained from the path graph, using, for example, the following equation (3). W2 = δCc + ε(A all -Ac)+γ(MAX(Y)-MIN(Y))...Formula (3) In equation (3), MAX(Y) is the maximum value of the total attitude change Y, and MIN(Y) is the minimum value of the total attitude change Y.

[0082] By using the total cost W2 calculated by equation (3) to determine the inspection path R, the inspection path R, including the change in the attitude of the imaging unit 210, can be optimized at once. In this case, the optimal solution for the inspection path R can be solved as a traveling salesman problem, or it can be calculated using general methods such as brute force or greedy methods.

[0083] [Output Description] Next, an example of the output of the processing result in the inspection path generation device according to the first embodiment of the present invention will be described below with reference to Figures 12 and 13.

[0084] Figure 12 shows an example of an output screen in an inspection route generation device according to the first embodiment of the present invention. The output screen 1100 shown in Figure 12 is displayed in the output device 120 based on the output information 135 output in step S103 of Figure 9. The output screen 1100 consists of selection frames 1101 to 1103, operation buttons 1104 to 1107, an inspection route list table 1108, and a display frame 1109.

[0085] The user can select any inspection equipment as the visual inspection device 200 using the input device 110. Information about the selected inspection equipment is displayed in the selection frame 1101. When any inspection equipment is selected, if the user selects the operation button 1104 on the output screen 1100, the inspection equipment configuration information 131 corresponding to that inspection equipment is stored in the storage unit 130.

[0086] The user can select any object to be inspected using the input device 110. Information about the selected object to be inspected is displayed in the selection frame 1102. When any object to be inspected is selected, and the user selects the operation button 1105 on the output screen 1100, the object to be inspected information 132 and inspection location information 1330 corresponding to that object to be inspected are stored in the storage unit 130.

[0087] After the inspection device configuration information 131, inspection target information 132, and inspection location information 1330 are stored in the storage unit 130 as described above, the user can perform an operation by selecting an operation button 1106 on the output screen 1100, which will cause the arithmetic unit 140 to execute route planning processing. At this time, the arithmetic unit 140 executes processing according to the flowcharts in Figures 9 and 10 by the route determination unit 141.

[0088] The inspection route list table 1108 displays a list of inspection routes generated by the route planning process. Here, information such as the execution time of the visual inspection for each inspection route, the amount of change in the posture of the imaging unit 210, and the coverage rate of the inspection target area are displayed, based on the output information 135 determined in the route planning process plus the total travel cost C and the total inspectable area A mentioned above. The user can select an execution recipe for when the visual inspection device 200 performs a visual inspection by selecting one of the inspection routes in the inspection route list table 1108. The information of the selected execution recipe is displayed in the selection frame 1103.

[0089] When an execution recipe is selected, and the user selects the operation button 1107 on the output screen 1100, the visual inspection process performed by that execution recipe is displayed in the display frame 1109. From the contents of this display frame 1109, the user can visually confirm the movement of the visual inspection device 200, the change in the posture of the imaging unit 210, the inspectable area of ​​the object to be inspected, etc., and decide which inspection path to adopt.

[0090] Figure 13 shows another example of an output screen in an inspection route generation device according to the first embodiment of the present invention. The output screen 1200 shown in Figure 13 is displayed in the output device 120 based on the output information 135 output in step S103 of Figure 9, similar to the output screen 1100 in Figure 12. The output screen 1200 is configured to include a display frame 1201 instead of the display frame 1109 in the output screen 1100 of Figure 12.

[0091] The display frame 1201 shows the changes in the orientation of the imaging unit 210 during the visual inspection based on the selected execution recipe, such as the change in the yaw rotation angle. From the information displayed in the display frame 1201, the user can check the extent of the changes in the orientation of the imaging unit 210 during the visual inspection and decide which inspection path to adopt.

[0092] According to the first embodiment of the present invention described above, the following effects are achieved.

[0093] (1) The inspection path generation device 100 generates an inspection path for the visual inspection device 200, which performs a visual inspection of an object to be inspected based on an image captured by the imaging unit 210. The inspection path generation device 100 includes a storage unit 130 that stores inspection position information 1330 representing multiple inspection positions in which the imaging unit 210 takes images of the object to be inspected, and inspection device configuration information 131 representing the configuration of the visual inspection device 200, and a path determination unit 141. Based on the inspection position information 1330 and the inspection device configuration information 131, the path determination unit 141 calculates the path length between each inspection position and the amount of change in the posture of the imaging unit 210 when the imaging unit 210 moves between multiple inspection positions (step S202), and determines the inspection path based on the calculated path length and change in posture (steps S203 to S210). In this way, the visual inspection device 200, which performs a visual inspection of an object to be inspected, can be provided with an inspection path that can shorten the inspection time while suppressing a decrease in inspection accuracy.

[0094] (2) The path determination unit 141 calculates the travel costs Co and Cc based on the path length between each inspection position when the imaging unit 210 moves between multiple inspection positions, the amount of change in the posture of the imaging unit 210, and the direction of travel (steps S202, S204, S206), and determines the inspection route based on the calculated travel costs Co and Cc (steps S208 to S210). In this way, the inspection route that has the shortest inspection time within a predetermined range of inspection accuracy can be reliably determined.

[0095] (3) The path determination unit 141 calculates the constraint conditions for the posture of the imaging unit 210 at each inspection position based on the inspection device configuration information 131 (step S201), and calculates the amount of posture change of the imaging unit 210 based on the calculated constraint conditions (step S202). In this way, the amount of posture change of the imaging unit 210 can be accurately calculated in the visual inspection device 200, taking into account the range of postures that the imaging unit 210 can take.

[0096] (4) In calculating the movement cost in step S202, the route determination unit 141 does not need to set a movement cost if the amount of change in the posture of the imaging unit 210 between any inspection positions is greater than or equal to a predetermined threshold. In this way, multiple inspection route candidates Rc are generated in the loop processing of steps S205 to S210, and when determining the inspection route R from among the multiple generated inspection route candidates Rc, the inspection route R is determined by excluding the inspection route candidate Rc that falls under this condition. This makes it possible to efficiently determine the inspection route by excluding inspection route candidates that would result in excessive inspection time in advance.

[0097] (5) The inspection path generation device 100 uses a calculation device 140 to perform the following processes: (step S101) acquiring inspection position information 1330 representing multiple inspection positions where the imaging unit 210 of the visual inspection device 200 takes images of the object to be inspected, and inspection device configuration information 131 representing the configuration of the visual inspection device 200; (step S202) calculating the path length between each inspection position and the amount of change in the posture of the imaging unit 210 when the imaging unit 210 moves between multiple inspection positions based on the acquired inspection position information 1330 and inspection device configuration information 131; and (step S203 to S210) determining the inspection path based on the calculated path length and change in posture.

[0098] -Second Embodiment- A second embodiment of the present invention will be described below.

[0099] [Explanation of the device configuration] Figure 14 shows an example configuration of a visual inspection system including an inspection route generation device according to a second embodiment of the present invention. The visual inspection system shown in Figure 14 is configured such that the inspection route generation device 100A and the visual inspection device 200 are connected to each other via a communication line 300. The difference between the inspection route generation device 100 shown in Figure 1, which was described in the first embodiment, and the inspection route generation device 100A shown in Figure 14 is that the calculation unit 140A further includes the function blocks of a route change unit 151, a change target inspection position selection unit 152, a candidate position setting unit 153, and a candidate position selection unit 154, in addition to the function blocks of the route determination unit 141, the inspectable area calculation unit 142, the camera movable range calculation unit 143, the route graph generation unit 144, the movement cost calculation unit 145, the minimum cost route selection unit 146, and the output information generation unit 147, which have already been described. The following will describe this embodiment focusing on these differences.

[0100] The route change unit 151 modifies the inspection route determined by the route determination unit 141 using the functional blocks of the inspection location selection unit 152, the candidate location setting unit 153, and the candidate location selection unit 154. Details of the processing performed by the route change unit 151 will be explained later with reference to Figures 15 and 16.

[0101] The inspection location selection unit 152 selects one of several inspection locations along the inspection route determined by the route determination unit 141 as the inspection location to be changed. The selection of the inspection location to be changed by the inspection location selection unit 152 is based, for example, on the cost of moving between inspection locations or on specifications from the user.

[0102] The candidate position setting unit 153 sets a combination of multiple candidate positions, which are candidates for the target inspection position, and the orientation of the imaging unit 210 at each candidate position, around the inspection position to be changed selected by the inspection position selection unit 152.

[0103] The candidate position selection unit 154 selects from among the multiple candidate positions set by the candidate position setting unit 153 and the orientation of the imaging unit 210 the combination that does not reduce the inspectable area and minimizes the total movement cost for the entire inspection path, as the changed inspection position and orientation of the imaging unit 210.

[0104] [Explanation of the flowchart] Next, the inspection route planning process performed in the inspection route generation device according to the second embodiment of the present invention will be described below with reference to the flowcharts in Figures 15 and 16 and the explanatory diagrams in Figures 17 and 18.

[0105] Figure 15 is an example of a flowchart of the inspection route planning process according to a second embodiment of the present invention. The process shown in the flowchart of Figure 15 is executed by the route determination unit 141 and the route modification unit 151 in the arithmetic unit 140A.

[0106] In step S301, the inspection device configuration information 131, the object to be inspected information 132, and the inspection position information 1330 are read from the storage unit 130 and input to the arithmetic unit 140A as information to be used in the processing of step S302. This information is assumed to be pre-stored in the storage unit 130 based on information previously entered by the user using the input device 110.

[0107] In step S302, an inspection route determination process is executed to determine the optimal inspection route based on the information entered in step S301. In this step S302, the inspection route determination process is executed according to the flowchart in Figure 10 described in the first embodiment.

[0108] In step S303, an inspection route change process is executed to modify the inspection route determined in step S302. Details of the inspection route change process executed in step S303 will be described later with reference to the flowchart in Figure 15.

[0109] In step S304, it is determined whether the visual inspection using the inspection route changed in step S303 is executable by the visual inspection device 200. For example, information such as the shape of the robot unit 220, the angle range of each joint, and the limit speed of the visual inspection device 200, as well as information such as the shape and arrangement of each piece of equipment constituting the visual inspection device 200, are obtained from the inspection device configuration information 131 input in step S301. Based on this information, a buffer determination is made to determine whether the operation according to the changed inspection route is executable by the robot unit 220. If it is determined that it is executable, the process proceeds to step S305; if it is determined that it is not executable, the process proceeds to step S302. If the process proceeds to step S302, the inspection route determination process is executed again in step S302, and a different inspection route from the previous one is determined.

[0110] In step S305, the information of the inspection route that was changed in step S303 is stored in the storage unit 130 as output information 135 and output to the output device 120.

[0111] Figure 16 is an example of a flowchart of the inspection route change process performed in step S303. The process shown in the flowchart of Figure 16 is executed in the arithmetic unit 140A by the route change unit 151 appropriately calling the inspectionable area calculation unit 142, the minimum cost route selection unit 146, the inspection position selection unit 152, the candidate position setting unit 153, and the candidate position selection unit 154, using these functional blocks.

[0112] In step S401, the inspection route R determined in step S302 in Figure 15, along with the orientation P of the inspection route R, the total movement cost C, and the total inspectable area A, are input to the route change unit 151.

[0113] Steps S402 to S407 execute a loop process. Here, the processes from steps S403 to S407 are repeatedly executed until the number of loop iterations exceeds a predetermined threshold.

[0114] In step S403, the inspection position selection unit 152 selects, as the inspection position X to be changed, an inspection position among multiple inspection positions included in the inspection path R input in step S401, where the change in the posture P of the imaging unit 210 is greater compared to the inspection positions before and after.

[0115] Figure 17 is an explanatory diagram of the method for selecting the inspection position to be changed. In Figure 17, for an inspection path R that passes through inspection positions A to G as exemplified in Figure 6, the changes in posture P at each inspection position and the range of yaw rotation angles of the imaging unit 210 at each inspection position are shown. Here, the range of yaw rotation angles that the imaging unit 210 can take is determined taking into consideration the range of motion of the robot unit 220.

[0116] The arrows in the graph in Figure 17 indicate the change in posture P set for each inspection position, i.e., the change in the yaw rotation angle of the imaging unit 210. The graph in Figure 17 shows that the change in posture P is larger when moving from inspection position D to inspection position E and when moving from inspection position E to inspection position F than when moving between other inspection positions. Therefore, for this inspection path R, inspection position E is selected as the inspection position X to be changed.

[0117] In step S403, the inspection position X to be changed may be selected using other methods. For example, in the output screen illustrated in Figures 12 and 13 above, the user may be able to select the inspection position X to be changed by arbitrarily selecting any of the inspection positions. Alternatively, the user may select an inspection position that matches pre-set conditions as the inspection position X to be changed. In addition to the above, it is also possible to select the inspection position X to be changed based on various information entered by the user.

[0118] Returning to the explanation of Figure 16, in step S404, the candidate position setting unit 153 randomly generates N combinations (where N is any natural number) of candidate positions X' that replace the inspection position X set in step S403, and candidate poses P' at those candidate positions X', in the vicinity of the inspection position X to be changed.

[0119] Figure 18 is an explanatory diagram of the method for setting candidate positions and candidate postures. In Figure 18, examples of inspection positions D, E, and F from the inspection positions A to G exemplified in Figure 6, along with candidate positions X' and candidate posture P', are shown. As mentioned above, when inspection position E is selected as the inspection position X to be changed in step S403, N combinations of candidate positions X' and candidate posture P' are randomly generated around this inspection position E, as shown in Figure 18. At this time, it is preferable to set the values ​​of candidate positions X' and candidate posture P' so that the difference from the original inspection position E and posture P is within a predetermined range.

[0120] In step S404, it is also possible to generate only one of the candidate position X' and candidate posture P' for the inspection position X to be changed, and use the original value for the other. In other words, the inspection path R can be changed by changing at least one of the position and posture of the imaging unit 210 at the inspection position X to be changed.

[0121] Returning to the explanation of Figure 16, in step S405, the inspectable area calculation unit 142 calculates the inspectable area of ​​the visual inspection device 200 at each candidate position X' generated in step S404. Here, based on the inspection target information 132 read in step S301 of Figure 15, the inspectable area of ​​the visual inspection device 200 at each candidate position X' can be calculated in the same manner as in step S200 of Figure 10. That is, based on the relative position of the imaging unit 210 at each candidate position X', the candidate orientation P' representing the relative orientation of the imaging unit 210 at each candidate position X', and the shape information 1320 included in the inspection target information 132, the position and orientation of the imaging unit 210 and the position and orientation of the inspection target area F1 to Fn are compared for each candidate position X'. As a result, if the difference between these is within a predetermined range, the area is determined to be an inspectable area; if it is outside the predetermined range, the area is determined not to be an inspectable area. Furthermore, based on the inspection target area information 1321 contained in the inspection target information 132, the inspection target areas are extracted from the calculated inspectionable areas, and the area of ​​each extracted inspectionable area is calculated based on the shape information 1320. The area of ​​each inspectionable area calculated here is used when calculating the total inspectionable area A' in step S406 described later.

[0122] In step S406, the candidate position selection unit 154 calculates the total travel cost C' and total inspectable area A' of the inspection path R when the inspection position X to be changed in the inspection path R is changed to each candidate position X'. Here, for the N sets of candidate position X' and candidate posture P' combinations generated in step S404, the total travel cost C' and total inspectable area A' are calculated using the area of ​​each inspectable area calculated in step S405, in the same manner as in steps S204 and S207 of Figure 10. This makes it possible to calculate the total travel cost C' based on the path length between each inspection position, the amount of posture change of the imaging unit 210 and the direction of travel when the imaging unit 210 moves between multiple inspection positions when at least one of the position and posture of the imaging unit 210 is changed at the inspection position X to be changed, and the total inspectable area A' which represents the area of ​​the object to be inspected that the visual inspection device 200 can inspect.

[0123] In step S407, the candidate position selection unit 154 updates the inspection path R and posture P based on the total movement cost C' and total inspectable area A' calculated in step S406. Here, among the total movement cost C' and total inspectable area A' calculated for each candidate position X', the candidate position X' where the total inspectable area A' is greater than or equal to the total inspectable area A of the original inspection path R, and the total movement cost C' is minimized, is selected as the modified inspection position. The inspection path R is then updated by replacing the inspection position X to be changed with this inspection position, and posture P is updated by setting the posture at this inspection position as the candidate posture P'. This allows the position and posture of the imaging unit 210 to the inspection position X to be changed to be determined, and the inspection path R and posture P are updated in a way that maintains inspection accuracy while achieving the shortest possible inspection time.

[0124] In step S408, the inspection route R and attitude P updated in step S407 are output as the inspection route R and attitude P after the inspection route change process. Based on the information output here, the contents of the output information 135 stored in the storage unit 130 are updated so that the inspection route generation device 100A reflects the changed inspection route R and attitude P. The storage unit 130 may also store both the information of the inspection route R and attitude P before the change and the information of the inspection route R and attitude P after the change.

[0125] According to the second embodiment of the present invention described above, in addition to (1) to (5) described in the first embodiment, the following further effects are achieved.

[0126] (6) The inspection route generation device 100A includes a route change unit 151 that changes the inspection route determined by the route determination unit 141. The storage unit 130 stores inspection object information 132 that represents the shape of the object to be inspected. Based on the inspection object information 132, the route change unit 151 calculates an inspectable area that represents the area of ​​the object to be inspected that the visual inspection device 200 can inspect when the inspection route is changed (steps S405, S406), and determines the changed inspection route based on the calculated inspectable area (step S407). In this way, the visual inspection device 200 that performs the visual inspection of the object to be inspected can change the inspection route once it has been determined while suppressing a decrease in inspection accuracy.

[0127] (7) The route changing unit 151 selects one of a plurality of inspection positions as the inspection position X to be changed (step S403), and changes the inspection route by changing at least one of the position and orientation of the imaging unit 210 at the selected inspection position X to be changed (step S404). In this way, the inspection route can be appropriately changed while making use of the original inspection route to some extent.

[0128] (8) When the position and orientation of the imaging unit 210 are changed at least one of the inspection position X to be changed, the path change unit 151 calculates the path length between each inspection position when the imaging unit 210 moves to multiple inspection positions on the inspection path R, the amount of change in the orientation of the imaging unit 210 and the direction of travel, as well as the inspectable area A' (step S406). Based on the calculated movement cost C' and inspectable area A', the unit determines the changed position and orientation of the imaging unit 210 with respect to the inspection position X to be changed, and determines the changed inspection path R based on the determined changed position and orientation of the imaging unit 210 (step S407). Specifically, multiple candidate positions X' are set in the vicinity of the inspection position X to be changed (step S404), and among the set multiple candidate positions X', the candidate position X' in which the inspectable area A' does not shrink and the movement cost C' is minimized is determined as the changed position of the imaging unit 210 with respect to the inspection position X (step S407). In this way, the modified inspection route R can be determined to achieve the shortest possible inspection time while maintaining inspection accuracy.

[0129] (9) In step S403, the route change unit 151 may select the inspection location X to be changed based on the information entered by the user. In this way, the user can arbitrarily select the inspection location X to be changed and change the inspection route.

[0130] The embodiments and various modifications described above are merely examples, and the present invention is not limited to these, as long as the features of the invention are not impaired. Furthermore, although various embodiments and modifications have been described above, the present invention is not limited to these. Other embodiments conceivable within the scope of the technical idea of ​​the present invention are also included within the scope of the present invention. [Explanation of symbols]

[0131] 100, 100A... Inspection path generation device 110...Input device 120... Output device 130...Storage section 131...Inspection device configuration information 132...Information on items to be inspected 133...Inspection Conditions Information 134...Route graph information 135... Output Information 140,140A...Arithmetic unit 141... Route determination unit 142...Inspectable Area Calculation Unit 143...Camera Movable Range Calculation Unit 144...Path graph generation unit 145...Movement cost calculation unit 146...Minimum cost route selection section 147...Output Information Generation Unit 151...Route change section 152...Selection section for inspection location to be changed 153... Candidate position setting section 154... Candidate position selection section 200...Visual inspection device 210...Imaging Unit 220...Robotics Department 230... Processing Unit 300...communication lines 1310... Robot configuration information 1311...Inspection equipment configuration information 1320...Shape information 1321...Inspection target area information 1330...Inspection location information 1331...Inspectable Area Information 1332...Camera range of motion information 1340... Inter-node connection information 1341... Travel cost information 1350...Inspection route information 1351...Camera angle information 1352...Robot posture information

Claims

1. An inspection path generation device that generates an inspection path for an appearance inspection device that performs an appearance inspection of an object to be inspected based on an image captured by an imaging unit, A storage unit that stores inspection position information representing multiple inspection positions where the imaging unit performs imaging of the object to be inspected, and inspection device configuration information representing the configuration of the visual inspection device, The system includes a path determination unit that calculates the path length between each of the multiple inspection positions and the amount of change in the posture of the imaging unit when the imaging unit moves between the multiple inspection positions based on the inspection position information and the inspection device configuration information, and determines the inspection route based on the calculated path length and the amount of change in posture, The aforementioned visual inspection device comprises an imaging unit and a robot unit to which the imaging unit is attached at its tip. The robot section is composed of a combination of multiple joints that can rotate freely along a predetermined rotational direction and links connecting each joint, and the position and orientation of the imaging section can be controlled by adjusting the angle of each joint. The inspection device configuration information includes robot configuration information representing the shape of each link of the robot section, the angular range and limit speed of each joint, and inspection equipment configuration information representing the arrangement of the imaging unit and the robot section in the visual inspection device. The aforementioned route determination unit, Based on the inspection position information, the robot configuration information and inspection equipment configuration information included in the inspection device configuration information, the range of postures that the robot unit can assume at each inspection position is calculated. Based on the calculated range of posture at each inspection position, the range of motion of the imaging unit at each inspection position is calculated as a constraint condition for the posture of the imaging unit. Based on the calculated constraints, the amount of change in posture is calculated, The imaging unit generates a candidate inspection route for each inspection location in the order in which it passes through them as it moves between the multiple inspection locations, thereby generating a plurality of candidate inspection routes. Based on the path length between each inspection position in each of the generated multiple candidate inspection paths, the amount of attitude change of the imaging unit, and the direction of travel, the total path length obtained by summing the path lengths is L, the number of changes in the direction of travel is D, and the total attitude change amount obtained by summing the attitude change amounts is Y, and α, β, and γ are predetermined weighting coefficients, the total movement cost Cc for each of the multiple candidate inspection paths is calculated using the following formula (1): Cc=αL+βD+γY...Formula (1) Based on the calculated total travel cost Cc, one of the multiple candidate inspection routes is selected as the inspection route. An inspection path generating device that selects a combination of rotation angles of the imaging unit at each inspection position such that the amount of change in the rotation angle of the imaging unit in a predetermined direction between each inspection position in the determined inspection path is minimized.

2. In the inspection route generation device according to claim 1, The system includes a route changing unit that changes the inspection route determined by the route determination unit, The storage unit stores information about the object to be inspected, representing the shape of the object to be inspected. The route changing unit is an inspection route generation device that calculates an inspectable area representing the area of ​​the object to be inspected that the visual inspection device can inspect when the inspection route is changed, based on the information of the object to be inspected, and determines the changed inspection route based on the calculated inspectable area.

3. In the inspection route generation device according to claim 2, The aforementioned route changing unit is Select one of the aforementioned multiple inspection locations as the inspection location to be changed. An inspection path generating device that changes the inspection path by changing at least one of the position and orientation of the imaging unit at the selected inspection location to be changed.

4. In the inspection route generation device according to claim 3, The aforementioned route changing unit is When at least one of the position and orientation of the imaging unit is changed at the inspection position to be changed, the total travel cost Cc of the inspection path based on the path length between each inspection position when the imaging unit moves between the plurality of inspection positions, the amount of change in the orientation of the imaging unit, and the direction of travel, and the inspectable area are calculated. Based on the calculated total movement cost Cc and the inspectable area, the position and orientation of the imaging unit after the change are determined relative to the inspection position to be changed. An inspection path generation device that determines the modified inspection path based on the determined modified position and orientation of the imaging unit.

5. In the inspection route generation device according to claim 4, The route changing unit sets a plurality of candidate positions near the inspection position to be changed, and determines, among the set plurality of candidate positions, the candidate position that does not reduce the inspectable area and minimizes the total movement cost Cc of the inspection route as the position of the imaging unit after the change relative to the inspection position to be changed.

6. In the inspection route generation device according to claim 3, The aforementioned route modification unit is an inspection route generation device that selects the inspection location to be modified based on information input by the user.

7. In the inspection route generation device according to claim 1, The aforementioned path determination unit is an inspection path generation device that determines the inspection path by excluding from a plurality of generated inspection path candidates any inspection path candidates in which the amount of change in the posture of the imaging unit between arbitrary inspection positions is greater than or equal to a predetermined threshold.

8. A method for generating an inspection path for an appearance inspection device that performs an appearance inspection of an object to be inspected based on an image captured by an imaging unit, The aforementioned visual inspection device comprises an imaging unit and a robot unit to which the imaging unit is attached at its tip. The robot section is composed of a combination of multiple joints that can rotate freely along a predetermined rotational direction and links connecting each joint, and the position and orientation of the imaging section can be controlled by adjusting the angle of each joint. The computing unit, A process for acquiring inspection position information representing multiple inspection positions where the imaging unit performs imaging of the object to be inspected, and inspection device configuration information representing the configuration of the visual inspection device, Based on the acquired inspection position information and inspection device configuration information, the process calculates the path length between each inspection position and the amount of change in the attitude of the imaging unit when the imaging unit moves between the plurality of inspection positions. The process of determining the inspection route based on the calculated route length and the amount of change in posture is performed, The inspection device configuration information includes robot configuration information representing the shape of each link of the robot section, the angular range and limit speed of each joint, and inspection equipment configuration information representing the arrangement of the imaging unit and the robot section in the visual inspection device. In the process for calculating the amount of change in posture, Based on the inspection position information, the robot configuration information and inspection equipment configuration information included in the inspection device configuration information, the range of postures that the robot unit can assume at each inspection position is calculated. Based on the calculated range of posture at each inspection position, the range of motion of the imaging unit at each inspection position is calculated as a constraint condition for the posture of the imaging unit. Based on the calculated constraints, the amount of change in posture is calculated, In the process of determining the inspection route, The imaging unit generates a candidate inspection route for each inspection location in the order in which it passes through them as it moves between the multiple inspection locations, thereby generating a plurality of candidate inspection routes. Based on the path length between each inspection position in each of the generated multiple candidate inspection paths, the amount of attitude change of the imaging unit, and the direction of travel, the total path length obtained by summing the path lengths is L, the number of changes in the direction of travel is D, and the total attitude change amount obtained by summing the attitude change amounts is Y, and α, β, and γ are predetermined weighting coefficients, the total movement cost Cc for each of the multiple candidate inspection paths is calculated using the following formula (1): Cc=αL+βD+γY...Formula (1) Based on the calculated total travel cost Cc, one of the multiple candidate inspection routes is selected as the inspection route. A method for generating an inspection path, which selects a combination of rotation angles of the imaging unit at each inspection position such that the amount of change in the rotation angle of the imaging unit in a predetermined direction between each inspection position in the determined inspection path is minimized.

9. In the inspection route generation method according to claim 8, The aforementioned computing device, A process for obtaining inspection object information representing the shape of the object to be inspected, Based on the acquired information of the object to be inspected, a process is performed to calculate an inspectable area representing the area of ​​the object to be inspected that the visual inspection device can inspect when the inspection route is changed, An inspection path generation method that performs a process to change the inspection path by determining the modified inspection path based on the calculated inspectable area.

10. In the inspection route generation method according to claim 9, An inspection path generation method for changing the inspection path, which involves selecting one of the plurality of inspection positions as the inspection position to be changed, and changing at least one of the position and orientation of the imaging unit at the selected inspection position to be changed, thereby changing the inspection path.

11. In the inspection route generation method according to claim 10, The calculation device performs the following processes when at least one of the position and orientation of the imaging unit is changed at the inspection position to be changed: calculate the total travel cost Cc of the inspection path based on the path length between each inspection position when the imaging unit moves between the plurality of inspection positions, the amount of change in the orientation of the imaging unit, and the direction of travel; and calculate the inspectable area. An inspection path generation method that, in the process of changing the inspection path, determines the position and orientation of the imaging unit after the change relative to the inspection position to be changed, based on the calculated total movement cost Cc and the inspectable area, and determines the changed inspection path based on the determined changed position and orientation of the imaging unit.

12. In the inspection route generation method according to claim 11, In the process of calculating the total travel cost Cc of the inspection route and the process of calculating the inspectable area, multiple candidate positions are set near the inspection position to be changed, and the total travel cost Cc and the inspectable area are calculated, respectively. In the process of changing the inspection path, the inspection path generation method determines, among the set of multiple candidate positions, the candidate position that does not reduce the inspectable area and minimizes the total movement cost Cc of the inspection path, as the position of the imaging unit after the change relative to the inspection position to be changed.

13. In the inspection route generation method according to claim 10, The process for calculating the inspectable area involves obtaining information input from the user and selecting the inspection location to be changed based on the obtained information to generate the inspection path.

14. In the inspection route generation method according to claim 8, An inspection path generation method for determining the inspection path, wherein the process of determining the inspection path involves excluding from a plurality of generated inspection path candidates any inspection path candidates in which the amount of change in the posture of the imaging unit between arbitrary inspection positions is greater than or equal to a predetermined threshold.

Images