Inspection support device and inspection support method

Abstract

To allow an operator to efficiently inspect a work-piece.SOLUTION: An inspection support device includes: a real data acquisition unit that acquires real data indicating a shape, a position, and a posture of a work-piece placed in a real space; a conversion unit that provides a conversion operation of overlapping design data with the real data for an inspection position on the design data indicating a shape on a design of the work-piece to calculate an inspection position on the work-piece in the real space; a sensor position acquisition unit that acquires a position of a sensor for inspecting the work-piece in the real space; and a display control unit that allows a display device to display a direction in which a difference between the inspection position on the work-piece in the real space and the position of the sensor is within an allowable position shift range, as a direction in which the sensor is moved.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to an inspection support device and an inspection support method. [Background technology]

[0002] When a robot equipped with an inspection sensor automatically inspects a workpiece, a method has been proposed in which the sensor is moved to a target position on the workpiece (Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2021-189003 A Summary of the Invention [Problem to be solved by the invention]

[0004] However, in order to perform more flexible inspection, a worker, instead of a robot, may manually inspect whether there are defects in a workpiece such as a part by holding a sensor. When measuring a point of interest in a workpiece with a sensor, the worker determines the position and direction of the sensor based on experience, which makes it difficult to perform appropriate measurements efficiently.

[0005] The technology described in Patent Document 1 is a technology for guiding a robot to a target position on a workpiece, and cannot be applied to cases where inspection is done manually by a person.

[0006] The present invention has been made in consideration of the above circumstances, and has an object to enable an operator to efficiently inspect a workpiece. [Means for solving the problem]

[0007] The present application includes a number of means for solving at least some of the above problems, examples of which are as follows.

[0008] In order to solve the above problems, an inspection support device according to one embodiment of the present invention has a reality data acquisition unit that acquires reality data indicating the shape, position, and posture of a workpiece placed in real space, a conversion unit that calculates an inspection position on the workpiece in real space by performing a conversion operation on an inspection position on design data indicating the designed shape of the workpiece by overlaying the design data on the reality data, a sensor position acquisition unit that acquires the position in real space of a sensor that inspects the workpiece, and a display control unit that causes a display device to display, as the direction in which the sensor should be moved, a direction in which the difference between the inspection position on the workpiece in real space and the position of the sensor falls within an allowable positional deviation range. Effect of the Invention

[0009] According to the present invention, an operator can efficiently inspect a workpiece.

[0010] Problems, configurations and effects other than those described above will become apparent from the following description of the embodiments. [Brief description of the drawings]

[0011] [Figure 1] FIG. 1 is a schematic diagram showing an example of the configuration of an examination support system according to the first embodiment. [Figure 2A] FIG. 2A is a schematic diagram showing an example of a defect detection sensor (part 1) according to the first embodiment. [Figure 2B] FIG. 2B is a schematic diagram showing an example of a defect detection sensor (part 2) according to the first embodiment. [Figure 2C] FIG. 2C is a schematic diagram showing an example of a film thickness or material measurement sensor according to the first embodiment. [Figure 2D] FIG. 2D is a schematic diagram showing an example of the internal defect detection sensor according to the first embodiment. [Diagram 3] FIG. 3 is a schematic diagram showing an example of a data structure of the inspection condition information according to the first embodiment. [Figure 4] FIG. 4 is a schematic diagram showing an example of a data structure of the test result information according to the first embodiment. [Diagram 5] FIG. 5 is a schematic diagram showing an example of a conversion operation performed by the conversion unit according to the first embodiment. [Figure 6A] FIG. 6A is a schematic diagram showing an example of a display mode (part 1) of the display device according to the first embodiment. [Figure 6B] FIG. 6B is a schematic diagram showing an example of a display mode (part 2) of the display device according to the first embodiment. [Figure 6C] FIG. 6C is a schematic diagram showing an example of a display mode (part 3) of the display device according to the first embodiment. [Figure 6D] FIG. 6D is a schematic diagram showing an example of a display mode (part 4) of the display device according to the first embodiment. [Figure 7] FIG. 7 is a flowchart illustrating an example of processing performed by the inspection support device according to the first embodiment. [Figure 8] FIG. 8 is a flowchart illustrating an example of processing performed by the inspection support device according to the first embodiment. [Figure 9] FIG. 9 is a schematic diagram showing an example of a hardware configuration of the test support device according to the first to third embodiments. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. The examples are illustrative for explaining the present invention, and are omitted and simplified as appropriate for clarity of explanation. The present invention can be implemented in various other forms. Unless otherwise specified, each component may be singular or plural.

[0013] In order to facilitate understanding of the invention, the position, size, shape, range, etc. of each component shown in the drawings may not represent the actual position, size, shape, range, etc. Therefore, the present invention is not necessarily limited to the position, size, shape, range, etc. disclosed in the drawings.

[0014] As examples of various information, the various information may be described using expressions such as "table," "list," and "queue," but the various information may be expressed using data structures other than these. For example, various information such as "XX table," "XX list," and "XX queue" may be expressed as "XX information." When describing identification information, expressions such as "identification information," "identifier," "name," "ID," and "number" are used, but these are mutually interchangeable.

[0015] When there are multiple components having the same or similar functions, they may be described by using the same reference numerals with different subscripts, or when there is no need to distinguish between these multiple components, the subscripts may be omitted.

[0016] In the embodiments, a process performed by executing a program may be described. Here, a computer executes the program using a processor (e.g., CPU, GPU), and performs the process defined by the program while using a storage resource (e.g., memory) and an interface device (e.g., communication port). Therefore, the subject of the process performed by executing the program may be the processor. Similarly, the subject of the process performed by executing the program may be a controller, device, system, computer, or node having a processor. The subject of the process performed by executing the program may be a calculation unit, and may include a dedicated circuit that performs specific processing. Here, the dedicated circuit is, for example, an FPGA (Field Programmable Gate Array), an ASIC (Application Specific Integrated Circuit), or a CPLD (Complex Programmable Logic Device).

[0017] The program may be installed in the computer from a program source. The program source may be, for example, a program distribution server or a computer-readable storage medium. When the program source is a program distribution server, the program distribution server may include a processor and a storage resource that stores the program to be distributed, and the processor of the program distribution server may distribute the program to be distributed to other computers. In addition, in the embodiment, two or more programs may be realized as one program, and one program may be realized as two or more programs.

[0018] First Embodiment 1 is a schematic diagram showing an example of the configuration of an inspection support system according to the present embodiment. The inspection support system 10 is a system that supports an operator in manually inspecting a workpiece 200 such as a part.

[0019] As an example, the inspection support system 10 includes an inspection support device 100, a 3D scanner 111, a position and orientation measurement sensor 121, a sensor 131, a display device 140, and a work table 201.

[0020] The work table 201 is a table on which a work 200 such as a component is placed. In this example, a plurality of markers 124 are provided on the surface of the work table 201. An inspection position 210 is set in advance on the work 200, and an operator performs inspection at the inspection position 210 with a sensor 131.

[0021] The 3D scanner 111 is a three-dimensional measuring device that measures the shape, position, and orientation of the workpiece 200 in real space. The 3D scanner 111 specifies the positions of each of the multiple markers 124, and thereby sets a coordinate system in which the surface of the workpiece table 201 is the coordinate plane, for example. This coordinate system is hereinafter referred to as a stage coordinate system. The 3D scanner 111 then outputs reality data 112 that indicates the shape, position, and orientation of the workpiece 200 based on the stage coordinate system. By using a three-dimensional measuring device such as the 3D scanner 111, the reality data 112 that indicates the shape, position, and orientation of the workpiece 200 can be obtained in this manner.

[0022] The sensor 131 is a device for inspecting the workpiece 200. Figures 2A to 2D are schematic diagrams showing examples of the sensor 131 used in this embodiment.

[0023] 2A is a schematic diagram showing an example of an image-based defect detection sensor 131. The defect detection sensor 131 captures an image of the surface of the workpiece 200 with a camera lens 132 while illuminating the workpiece 200 with multiple lights 133. If the workpiece has a defect 211 such as a scratch or crack, the defect 211 can be detected as a difference in brightness on the image.

[0024] Furthermore, by switching which of the multiple lights 133 is turned on, the direction of the light on the defect 211 changes, and the image of the defect 211 changes. In this example, when the right light 133 is turned on and the left light 133 is turned off, an image can be obtained in which only the left side of the defect 211 is bright. On the other hand, when the left light 133 is turned on and the right light 133 is turned off, an image can be obtained in which only the right side of the defect 211 is bright. In this way, the defect detection sensor 131 measures the position, unevenness, size, etc. of the defect 211.

[0025] 2B is a schematic diagram of an example of a three-dimensional shape sensor in which the illumination 133 of the image-based defect detection sensor 131 (FIG. 2A) is replaced with a pattern illumination 133'. The pattern illumination 133' projects a striped pattern onto the workpiece 200 using a laser beam, a liquid crystal projector, or the like. The projected pattern is deformed by the unevenness of the surface of the workpiece 200. Using this, the three-dimensional shape sensor 131 captures an image of the projected pattern with a camera lens 132, and measures the three-dimensional shape of the surface of the workpiece 200 by calculating the amount of deformation of the pattern using the principle of triangulation.

[0026] 2C is a schematic diagram showing an example of a film thickness or material measurement sensor 131 using a reflection spectrum. The sensor 131 projects light from a spectrometer lens 134 onto the film 212 on the surface of the workpiece 200 and measures the spectral characteristics of the reflected light. The spectral characteristics change depending on the variations in the thickness and material of the film 212. Thus, the sensor 131 can measure the variations in the thickness and material of the film 212 by measuring and analyzing the spectral characteristics.

[0027] 2D is a schematic diagram of an example of an ultrasonic internal defect detection sensor 131. The internal defect detection sensor 131 has an ultrasonic flaw detector 135, which is brought into contact with the workpiece 200 to oscillate ultrasonic waves, which are reflected by the internal defect 213 and return to the ultrasonic flaw detector 135. The internal defect detection sensor 131 can also detect the thickness of the workpiece 200 using a similar principle. Moreover, even if an eddy current is used instead of ultrasonic waves, the internal defect detection sensor 131 can also detect the internal defect 213 and the thickness of the workpiece 200.

[0028] Referring again to Figure 1, the sensor 131 is fitted with an attachment 122 having a plurality of markers 123 provided thereon.

[0029] The position and orientation measuring sensor 121 is a motion capture camera that measures the position and orientation of the sensor 131 in three-dimensional real space by recognizing the marker 123 on the attachment 122. By providing a plurality of position and orientation measuring sensors 121, it is possible to measure the three-dimensional position of the marker 123 by stereoscopic vision. Furthermore, in this embodiment, three or more markers 123 are provided on the attachment 122. As a result, the position and orientation measuring sensor 121 can measure the three-dimensional orientation and position of the sensor 131 attached to the attachment 122.

[0030] Moreover, the position and orientation measurement sensor 121 identifies the above-mentioned stage coordinate system, which uses the surface of the work table 201 as a coordinate plane, by recognizing the position of the marker 124 on the work table 201. Then, the position and orientation measurement sensor 121 outputs sensor position and orientation data 113 indicating the position and orientation of the sensor 131 based on the stage coordinate system. Note that the position of the sensor 131 is, for example, the position of the center of gravity of the sensor 131 in the stage coordinate system. Also, the orientation of the sensor 131 is the direction in which the sensor 131 is facing. If the sensor 131 has an optical system, the optical axis direction of the optical system is the direction in which the sensor 131 is facing. That direction may be the direction in which any of the three coordinate axes of the local coordinate system set in the sensor 131 is facing.

[0031] The position and orientation measuring sensor 121 is not limited to a motion capture camera, and other sensors may be used as the position and orientation measuring sensor 121. One such sensor is, for example, a sensor called a laser tracker. This sensor irradiates a laser light onto a retroreflective mirror (not shown) attached to the attachment 122, and measures the direction and distance of the mirror based on the deviation of the reflected light and the round-trip time of the laser light. In this case, the orientation of the sensor 131 can also be measured by recognizing the positions of multiple markers 123 with a camera. Note that a single laser tracker is sufficient instead of multiple ones.

[0032] The inspection support device 100 is a computer such as a PC (Personal Computer) or a server, and is a device that supports an operator in inspecting a workpiece 200 with a sensor 131. As an example, the inspection support device 100 includes an overall control unit 104, a storage unit 105, an input unit 107, a display unit 108, a memory unit 109, a reality data acquisition unit 110, a sensor position acquisition unit 120, and a sensor data acquisition unit 130.

[0033] The storage unit 109 stores inspection condition information 101, CAD (Computer Aided Design) data 102, and inspection result information 106.

[0034] The CAD data 102 is three-dimensional CAD data of the workpiece 200, and is an example of design data that indicates the designed shape of the workpiece 200. For example, data that collects the position coordinates of each point on the surface of the workpiece 200 is the CAD data 102. Hereinafter, a coordinate system that is the reference for the position coordinates in the CAD data 102 is referred to as a workpiece coordinate system.

[0035] 3 is a schematic diagram showing an example of the data structure of the inspection condition information 101. The inspection condition information 101 is information that associates an inspection position, an inspection method, and an allowable range with each other.

[0036] The inspection position is the position coordinate of the inspection position on the CAD data 102 that employs the above-mentioned workpiece coordinate system, and is a position on the surface of the workpiece 200 indicated by the CAD data 102 that is scheduled to be inspected in advance.

[0037] The inspection method is the ID of the sensor 131 used for inspection at the inspection position, and the inspection direction (posture) of the sensor 131 during inspection. The inspection direction is set based on the workpiece coordinate system.

[0038] The inspection direction is the direction (posture) in which the sensor 131 faces during inspection, and is, for example, the extension direction of a reference line of the sensor 131. For example, if the sensor 131 has an optical system, the reference line of the sensor 131 is the optical axis of the optical system. The inspection direction is the normal direction at the inspection position of the workpiece 200 indicated by the CAD data 102. However, depending on the type of the sensor 131, the inspection direction may be a direction tilted from the normal direction.

[0039] The allowable ranges are an allowable positional deviation range, an allowable distance range, and an allowable angle range. The allowable positional deviation range is an allowable range of deviation between the inspection position in real space and the position of the sensor 131 when these positions are projected onto a plane perpendicular to the reference line of the sensor 131. The allowable distance range is an allowable range of the distance between the inspection position in real space and the position of the sensor 131. The allowable angle range is an allowable range of the angle between the inspection direction and the direction in which the sensor 131 is actually facing in real space.

[0040] 4 is a schematic diagram showing an example of the data structure of the inspection result information 106. The inspection result information is information that associates an inspection position on the CAD data 102 with the inspection result at that inspection position. The inspection result is the result of the inspection at the inspection position by the sensor 131. For example, the sensor data 114 and information indicating the presence or absence of an abnormality, such as "scratched" or "not scratched", are stored in the inspection result.

[0041] Referring again to FIG. 1, the reality data acquisition unit 110 acquires reality data 112 indicating the shape, position, and orientation of the workpiece 200 based on the stage coordinate system from the 3D scanner 111. In this example, the 3D scanner 111 performs the conversion to the stage coordinate system, but the reality data acquisition unit 110 may perform the conversion. In this case, the 3D scanner 111 outputs shape data indicating the shape of the workpiece 200 and position data indicating the position of the marker 124 to the reality data acquisition unit 110. Then, the reality data acquisition unit 110 generates reality data 112 indicating the shape, position, and orientation of the workpiece 200 based on the stage coordinate system based on the shape data and the position data.

[0042] The sensor position acquisition unit 120 acquires sensor position and orientation data 113 indicating the position and orientation of the sensor 131 based on the stage coordinate system from the position and orientation measurement sensor 121. Note that in this example, the position and orientation measurement sensor 121 performs conversion to the stage coordinate system, but the sensor position acquisition unit 120 may perform the conversion. In this case, the position and orientation measurement sensor 121 outputs shape data indicating the shape of the sensor 131 and position data indicating the position of the marker 123 to the sensor position acquisition unit 120. Then, the sensor position acquisition unit 120 generates sensor position and orientation data 113 indicating the position and orientation of the sensor 131 based on the stage coordinate system based on the shape data and position data.

[0043] The sensor data acquisition unit 130 acquires the sensor data 114 indicating the inspection result of the sensor 131 .

[0044] The overall control unit 104 has a function of controlling the entire test support device 100, and includes, for example, a conversion unit 104a and a display control unit 104b.

[0045] The conversion unit 104a calculates an inspection position on the workpiece 200 in real space by performing a conversion operation on the inspection position in the inspection condition information 101 to superimpose the CAD data 102 on the real data 112. The conversion unit 104a also performs the above conversion operation on the inspection direction included in the inspection method of the inspection condition information 101 to convert the inspection direction into an inspection direction on the workpiece 200 in real space.

[0046] Fig. 5 is a schematic diagram showing an example of a conversion operation performed by the conversion unit 104a. In Fig. 5, a workpiece 300 as designed, which is indicated by the CAD data 102 in the workpiece coordinate system, is shown together with a workpiece 200 in the stage coordinate system in real space.

[0047] First, the conversion unit 104 a acquires the real data 112 from the real data acquisition unit 110 , and acquires the CAD data 102 from the storage unit 109 .

[0048] Then, the conversion unit 104a specifies a conversion operation that causes the workpiece 300 represented by the CAD data 102 to overlap with the workpiece 200 represented by the real data 112. Such a conversion operation is a combination of a rotation operation T for aligning the directions of the coordinate axes of the workpiece coordinate system and the stage coordinate system, and a translation operation v for translating the workpiece coordinate system so that its origin coincides with the origin of the stage coordinate system. By applying such a conversion operation to the workpiece 300, the workpiece 300 can be overlapped with the workpiece 200.

[0049] Hereinafter, the matrix representing the rotation operation T is also referred to as the rotation matrix T. Hereinafter, the vector representing the translation operation v is also referred to as the translation vector v.

[0050] Furthermore, the positions, shapes, and orientations of the workpieces 300 and 200 are respectively represented by the CAD data 102 and the real data 112. Therefore, hereinafter, performing a conversion operation on the workpiece 300 is also referred to as performing a conversion operation on the CAD data 102. Moreover, superimposing the workpiece 300 on the workpiece 200 is also referred to as superimposing the CAD data 102 on the real data 112.

[0051] The conversion unit 104a obtains, from among such conversion operations, a rotation operation T and a translation operation v that constitute a conversion operation that minimizes the deviation between the converted CAD data 102 and the real data 112. As an algorithm for superimposing the CAD data 102 on the real data 112 in this way, an iterative distance deviation minimization algorithm called ICP (Iterative Closest Point) can be used. This makes it possible to obtain a conversion operation that minimizes the difference between the real data 112 and the converted CAD data 102.

[0052] The conversion unit 104a calculates the inspection position 210 on the workpiece 200 by performing the conversion operation thus obtained on the inspection position 310 on the workpiece 300. For example, when the position vector of the inspection position 310 in the workpiece coordinate system is Ap, the conversion unit 104a calculates the position vector of the inspection position 210 in the stage coordinate system by T·Ap+v.

[0053] Furthermore, the conversion unit 104a converts the inspection direction Bp on the workpiece 300 into an inspection direction T·Bp on the workpiece 200 by a rotation operation T that constitutes the above conversion operation.

[0054] The position vector Ap is stored in advance as the inspection position in the inspection condition information 101 (see FIG. 3). The inspection direction Bp is stored in advance as the inspection method in the inspection condition information 101. If the inspection direction Bp is not defined in the inspection method, the normal direction in the position vector Ap of the workpiece 300 may be substituted for the inspection direction Bp.

[0055] By the above operations, the inspection support device 100 can identify the inspection position T·Ap+v and the inspection direction T·Bp on the workpiece 200 in the real space.

[0056] 1 again, the display control unit 104b displays on the display device 140 the movement direction of the sensor 131 that reduces the positional deviation between the inspection position and the sensor 131, the deviation between the inspection direction and the attitude of the sensor 131, and the like.

[0057] 6A to 6D are schematic diagrams showing examples of display modes of the display device 140. In FIG.

[0058] 6A shows an example in which a tablet terminal is used as the display device 140. An image showing the movement direction 141, the swing direction 142, and the rotation direction 144 of the sensor 131 is displayed on the display device 140. In this example, an arrow is displayed as the image.

[0059] The movement method 141 is a direction in which the amount of deviation between the inspection position T·Ap+v on the workpiece 200 and the actual position of the sensor 131 when these positions are projected onto a plane perpendicular to the direction in which the sensor 131 is facing falls within the allowable position deviation range included in the “allowable range” of the inspection condition information 101.

[0060] By moving the sensor 131 in the movement direction 141, the worker can keep the positional deviation of the sensor 131 within the allowable positional deviation range without relying on his / her own experience.

[0061] The swing direction 142 is a direction in which the sensor 131 is swung while a single point of the sensor 131 is fixed, and the angle formed by the inspection direction T·Bp on the workpiece 200 and the direction in which the sensor 131 is actually facing is within an allowable angle range included in the "allowable range" of the inspection condition information 101. The example of Fig. 6A shows a "pan" in which the sensor 131 is swung from below to above.

[0062] By changing the attitude of the sensor 131 in accordance with the swing direction 142, the worker can keep the deviation of the attitude of the sensor 131 within the allowable angle range without relying on his / her own experience.

[0063] The rotation direction 144 indicates a direction in which the sensor 131 is rotated around the reference line of the sensor 131 as a rotation axis. For example, the rotation direction 144 indicates a direction in which the angle between the direction perpendicular to the reference line of the sensor 131 and a reference direction set at the inspection position T·Ap+v on the workpiece 200 (for example, a direction in a tangent plane of the workpiece 200 at the inspection position) falls within an allowable range. The value of the allowable range is stored, for example, in the allowable range of the inspection condition information 101.

[0064] The display device 140 also displays a distance direction 143. The distance direction 143 indicates the direction in which the distance between the inspection position T·Ap+v on the workpiece 200 and the position of the sensor 131 falls within the allowable distance range included in the "allowable range" of the inspection condition information 101. The distance direction 143 has an inner circle and an outer circle. The inner circle indicates the allowable distance range included in the allowable range of the inspection condition information 101 (see FIG. 3), and its size is fixed. On the other hand, the size of the outer circle changes depending on the distance between the inspection position T·Ap+v and the actual position of the sensor 131. When the distance falls within the allowable distance range, the outer circle falls within the inner circle. On the other hand, when the distance exceeds the allowable distance range, the outer circle becomes larger than the inner circle.

[0065] By moving the sensor 131 closer to or farther away from the workpiece 200 so that the outer circle of the distance direction 143 fits within the inner circle, the operator can bring the distance between the inspection position T·Ap+v and the actual position of the sensor 131 within the allowable distance range without relying on his or her own experience.

[0066] The display control unit 104b identifies the deviations between the actual position and orientation of the sensor 131 indicated by the sensor position and orientation data 113 and the inspection position T·Ap+v and inspection direction T·Bp on the workpiece 200, and displays each direction 141, 142, 144 and distance direction 143 depending on whether each deviation falls within the allowable position deviation range, allowable distance range, and allowable angle range, respectively.

[0067] Furthermore, the display control unit 104b may change the display mode of the display device 140 when the deviation falls within an allowable range. For example, when the difference between the inspection position and the actual position of the sensor 131 falls within the allowable position deviation range, the arrows of the movement direction 141 and the rotation direction 144 may be erased or made lighter. This allows the operator to recognize that there is no need to move the sensor 131 any further and that inspection may begin with the sensor 131.

[0068] Furthermore, the display control unit 104b may display the degree of deviation by the color, density, size, blinking speed, or the like of the directions 141 to 144.

[0069] Moreover, a small display may be used as the display device 140 and attached to the attachment 122 .

[0070] 6B shows an example in which a glasses-type display device (e.g., smart glasses) is used as the display device 140. By using the glasses-type display device, directions 141 to 144 are displayed in the field of view of the worker, allowing the worker to work efficiently.

[0071] FIG. 6C is an example of a CG (Computer Graphics) image of the workpiece 200 displayed on the display device 140. This CG image is an image of the workpiece 200 seen from the sensor 131, and the inspection position 210 is also displayed in the CG image. For example, the display control unit 104b generates a CG image including the inspection position 210 based on the CAD data 102 that has been subjected to a conversion operation as shown in FIG. 5. The display control unit 104b may generate a CG image based on the reality data 112. The display control unit 104b also generates a CG image seen from the sensor 131 by specifying the relative positional relationship between the workpiece 200 and the sensor 131 in the real space based on the sensor position and orientation data 113 and the reality data 112. The display control unit 104b also displays a crosshair 145 indicating the position of the sensor 131 on the display device 140. The deviation between the intersection of the crosshair 145 and the inspection position 210 is the position deviation. Therefore, in this example, it is possible to eliminate the need to display the movement direction 141 and the rotation direction 144 that indicate the position deviation.

[0072] The swing direction 142 may be indicated by making each of the inner and outer circles of the distance direction 143 elliptical with the swing direction 142 as its minor axis. In this case, each of the inner and outer circles may be made perfect when the angle between the inspection direction and the direction in which the sensor 131 is facing falls within the allowable angle range. This makes it unnecessary to display the swing direction 142.

[0073] Fig. 6D shows an example in which a display such as a light-emitting diode or a liquid crystal display is provided on the attachment 122 as the display device 140. In Fig. 6D, the black parts indicate the parts that are emitting light. In this way, an arrow or circle light-emitting pattern is incorporated in advance into the display device 140, and the light-emitting pattern is illuminated when the deviation exceeds the allowable range, so that the worker can grasp the direction in which the deviation falls within the allowable range. The magnitude of the deviation may also be expressed by the color, intensity, size, blinking speed, etc. of the light-emitting pattern.

[0074] 1 again. The storage unit 105 generates inspection result information 106 by associating the inspection position on the CAD data 102 with the inspection result at the inspection position, and stores the inspection result information 106 in the memory unit 109. For example, the storage unit 105 acquires the inspection position on the CAD data 102 via the conversion unit 104a, and acquires the sensor data 114 including the inspection result at the inspection position from the sensor data acquisition unit 130. Then, the storage unit 105 associates the inspection position with the inspection result and stores it in the inspection result information 106. This allows the operator to later verify at which position on the CAD data 102 there is an abnormality.

[0075] The input unit 107 is an input device such as a keyboard or a mouse through which an operator inputs various data to the inspection support device 100 .

[0076] The display unit 108 is a display device such as a liquid crystal display that displays various data to the operator when the data is being input.

[0077] Next, the process performed by the test support device 100 according to this embodiment will be described.

[0078] FIG. 7 is a flowchart showing an example of processing performed by the test support device 100 according to this embodiment.

[0079] First, the reality data acquisition unit 110 acquires reality data 112 indicating the shape, position, and orientation of the workpiece 200 based on the stage coordinate system from the 3D scanner 111 (step S1).

[0080] For example, the 3D scanner 111 measures the shape of the workpiece 200 on the workpiece table 201, and then identifies the positions of multiple markers 124 on the workpiece table 201 to generate the reality data 112. Note that if the entire shape of the workpiece 200 cannot be measured by measurement from one direction, the worker may move the workpiece 200 so that the 3D scanner 111 measures the shape from multiple directions.

[0081] Next, the conversion unit 104a acquires the CAD data 102 from the storage unit 109 (step S2).

[0082] Next, the conversion unit 104a calculates a rotation matrix T and a translation vector v that realize a conversion operation for superimposing the CAD data 102 on the real data 112, as shown in FIG. 5 (step S3).

[0083] Next, the display control unit 104b displays a message on the display device 140 to prompt the preparation of the i-th sensor 131 (step S4). In this example, there are multiple sensors 131 identified by IDs of integers i (=1, ..., N), and when step S4 is performed for the first time, a message to prepare the sensor 131 with i=1 is displayed on the display device 140.

[0084] Based on this message, the worker performs the work of turning on the power of the sensor 131 and attaching the attachment 122 to the sensor 131. Note that if the attachment 122 is already provided on each of the multiple sensors 131, the work of attaching the attachment 122 is not necessary. Also, depending on the type of sensor 131, a calibration work for the sensor 131 may be performed.

[0085] Next, the overall control unit 104 reads out the inspection condition information 101 from the storage unit 109 (step S5).

[0086] Next, the overall control unit 104 reads the pth record from the top of the inspection condition information 101, and judges whether the "inspection method" of that record specifies the use of the ith sensor (step S6). When executing step S6 for the first time, the overall control unit 104 reads the record with p=1. Even when executing step S6 for the first time, the overall control unit 104 also reads the record with p=1 when switching to the next sensor 131 ("NO" in step S15). If the judgement in step S6 is YES, the process proceeds to step S7, and if the judgement is NO, the process proceeds to step S14.

[0087] In step S7, the display control unit 104b displays a CG image of the workpiece 200 and the inspection position 210 on the display device 140 as illustrated in Fig. 6C. For example, the display control unit 104b displays a position T·Ap+v obtained by performing transformations of a rotation matrix T and a translation vector v on the inspection position Ap stored in the p-th record of the inspection condition information 101 as the inspection position 210. This allows the operator to move the sensor 131 closer to the inspection position 210. Note that if a CG image is not displayed on the display device 140 as in Figs. 6A, 6B, and 6D, step S7 may be skipped.

[0088] Next, the sensor position acquisition unit 120 acquires the sensor position and orientation data 113 of the i-th sensor 131 from the position and orientation measurement sensor 121 (step S8).

[0089] Next, the display control unit 104b identifies the deviations between the actual position and orientation of the sensor 131 indicated by the sensor position and orientation data 113 and the inspection position T·Ap+v and inspection direction T·Bp on the workpiece 200. Then, the display control unit 104b displays each direction 141-144 on the display device 140 as shown in Figs. 6A-6D depending on whether each deviation falls within the allowable position deviation range, the allowable distance range, and the allowable angle range (step S9). Note that the inspection position Ap and the inspection direction Bp used in step S9 are the inspection position and the inspection direction in the workpiece coordinate system stored in the p-th record of the inspection condition information 101, respectively.

[0090] Next, the operator moves the sensor 131 according to the display, and the sensor position acquisition unit 120 acquires the sensor position and orientation data 113 of the sensor 131 after the movement (step S10). Then, the display control unit 104b judges whether the deviation is within the allowable range (step S11). If the judgment here is NO, the process returns to step S7.

[0091] On the other hand, if the answer is YES, the process proceeds to step S12, where the operator performs an inspection at the pth inspection position using the i-th sensor 131, and the sensor data acquisition unit 130 acquires the sensor data 114 indicating the inspection results.

[0092] Next, the storage unit 105 stores the p-th inspection position on the CAD data 102 and the inspection result at the inspection position in the inspection result information 106 in an associated manner (step S13).

[0093] Next, the overall control unit 104 determines whether the inspection at all inspection positions has been completed (step S14). For example, the overall control unit 104 determines whether the inspection at the inspection position of the last row where p is the maximum in the inspection condition information 101 has been completed. If it is determined as NO here, p is incremented by 1 and the process returns to step S5.

[0094] On the other hand, if it is determined as YES in step S14, the process proceeds to step S15. In step S15, the overall control unit 104 determines whether the inspection has been performed with all the sensors 131 from the first to the N-th (step S15). For example, if i < N, the overall control unit 104 determines that the inspection has not been performed with all the sensors 131 (NO), and if i = N, it determines that the inspection has been performed with all the sensors 131 (YES).

[0095] If it is determined as NO here, i is incremented by 1 and the process returns to step S4. On the other hand, if it is determined as YES, the process ends.

[0096] According to the present embodiment described above, in step S9, the display control unit 104b displays each of the directions 141 to 144 as shown in FIGS. 6A to 6D on the display device 140. As a result, the operator can move the sensor 131 relying on each of the directions 141 to 144, so that the deviation in the position and orientation of the sensor 131 can be kept within the allowable range. As a result, the operator can determine the position and orientation of the sensor 131 without relying on his / her own experience, and can efficiently perform the inspection of the workpiece 200.

[0097] <Second Embodiment> In the first embodiment, the 3D scanner 111 measures the shape, position, and orientation of the workpiece 200 to generate reality data 112. In contrast, in the present embodiment, an operator scans the workpiece 200 with the solid shape sensor 131 described in FIG. 2B, and the sensor position acquisition unit 120 measures the shape, position, and orientation of the workpiece 200 at the time of scanning. Then, the inspection support device 100 generates reality data 112 based on the sensor position and orientation data 113 generated by the sensor position acquisition unit 120. In this case, the marker 124 on the workpiece table 201 may be omitted.

[0098] As another example, a camera that captures an image of the workpiece 200 may be used as the sensor 131, and the reality data 112 may be generated from the obtained image sequence using an algorithm called SFM (Structure from Motion) or visual SLAM (Simultaneous Localization and Mapping). In this case, the marker 124 is used to calibrate the magnification of the measurement result.

[0099] <Third embodiment> In the first embodiment, the inspection positions of the workpiece 200 are predefined in the inspection condition information 101 (see FIG. 3). In contrast, in the present embodiment, a new inspection position is added in addition to the already defined inspection positions.

[0100] FIG. 8 is a flowchart showing an example of processing performed by the test support device 100 according to this embodiment.

[0101] First, if the result of the determination in step S15 in FIG. 7 is YES, the display control unit 104b identifies a position on the CAD data 102 where the difference between the CAD data 102 after the transformation T·Ap+v and the real data 112, i.e., the difference in shape, is equal to or greater than a predetermined value (step S21).

[0102] Next, the display control unit 104b identifies the position where the abnormality was detected in the inspection in step S12 among the positions on the CAD data 102 (step S22). For example, the display control unit 104b identifies the position where the abnormality was detected by referring to the inspection result information 106.

[0103] Next, the storage unit 105 updates the inspection condition information 101 by adding a record including the positions identified in steps S21 and S22 as the inspection position to the inspection condition information 101 (step S23). The inspection method and the tolerance in the newly added record may be set appropriately by the operator. The overall control unit 104 may also set the inspection method and the tolerance in the newly added record. For example, the overall control unit 104 may set a predetermined inspection method and tolerance according to the position and shape of the position identified in step S21. The overall control unit 104 may also set a predetermined inspection method and tolerance according to the position and shape of the position where an abnormality is present, the type of abnormality, and the like. Furthermore, the overall control unit 104 may set a sensor 131 different from that used in the previous inspection as the inspection method for the inspection position where an abnormality is detected.

[0104] Then, the process returns to step S4. As a result, the process moves to step S12, whereby an additional test is performed. The sensor 131 to be used may be different between the additional test and the initial test performed before steps S22 and S23 are performed. For example, in the initial test, "sensor A" is assigned as the sensor 131 at i=1, and "sensor B" is assigned as the sensor 131 at i=2. In addition, in the additional test, "sensor A" is assigned as the sensor 131 at i=3, and "sensor C" is assigned as the sensor 131 at i=4. In the initial test, when i=2 is reached in step S15, the result is determined as YES, and then i=3 is set before returning to step S4. In addition, after completing step S23, p may be reset to 1 to perform tests with "sensor A" and "sensor C" for all test positions, or p may be reset to the value of p corresponding to the record added in step S23 before returning to step S4.

[0105] This completes the process performed by the test support device 100 according to this embodiment. In the above example, both steps S21 and S22 are executed, but only one of these steps may be executed. Furthermore, in this example, steps S21 and S22 are executed after step S15, but steps S21 and S22 may be executed at a different timing.

[0106] According to the present embodiment described above, among the positions on the CAD data 102, a position where the difference from the actual data 112 is equal to or greater than a predetermined value or where an abnormality is found is newly set as an inspection position. Therefore, an operator can inspect the newly set position with another sensor 131 to check what kind of abnormality exists at that position.

[0107] Next, a hardware configuration of the test support device 100 according to the first to third embodiments will be described.

[0108] FIG. 9 is a schematic diagram showing an example of a hardware configuration of the test support device 100 according to the first to third embodiments.

[0109] 9, the test support device 100 includes a storage device 100a, a memory 100b, a processor 100c, an input device 100d, a display device 100e, a communication interface 100f, and a medium reading device 100g. These components are connected to each other by a bus 100i.

[0110] The storage device 100a is a non-volatile storage device such as a hard disk drive (HDD) or a solid state drive (SSD), and stores the inspection support program 50 according to the first to third embodiments.

[0111] The inspection support program 50 may be recorded in a computer-readable recording medium 100h, and the processor 100c may read the inspection support program 50 from the recording medium 100h.

[0112] The recording medium 100h may be, for example, a physically portable recording medium such as a CD-ROM (Compact Disc - Read Only Memory), a DVD (Digital Versatile Disc), or a USB (Universal Serial Bus) memory. A semiconductor memory such as a flash memory or a hard disk drive may also be used as the recording medium 100h.

[0113] The inspection support program 50 may be stored in a device connected to a public line, the Internet, a LAN (Local Area Network), etc. In that case, the processor 100c may read and execute the inspection support program 50.

[0114] The memory 100b is hardware that temporarily stores data, such as a dynamic random access memory (DRAM), and the inspection support program 50 is developed thereon.

[0115] The processor 100c is a CPU (Central Processing Unit) or a GPU (Graphical Processing Unit) that controls each part of the test support device 100. The processor 100c executes the test support program 50 in cooperation with the memory 100b, thereby realizing the overall control unit 104, the storage unit 105, the reality data acquisition unit 110, the sensor position acquisition unit 120, and the sensor data acquisition unit 130 in FIG.

[0116] The storage unit 109 in FIG. 1 is realized by the storage device 100a and the memory 100b.

[0117] The input device 100d is an input device such as a keyboard or a mouse for implementing the input unit 107 in FIG.

[0118] The display device 100e is a display device such as a liquid crystal display for implementing the display unit 108 in FIG.

[0119] The communication interface 100f is hardware such as a network interface card (NIC) for connecting the inspection support device 100 to a network such as the Internet or a LAN. The reality data acquisition unit 110, the sensor position acquisition unit 120, and the sensor data acquisition unit 130 in Fig. 1 acquire the reality data 112, the sensor position and orientation data 113, and the sensor data 114 via the communication interface 100f.

[0120] The medium reading device 100g is hardware such as a USB reader for reading data recorded on the recording medium 100h.

[0121] The effects described in this specification are merely examples and are not limiting, and other effects may also be present.

[0122] The present invention is not limited to the above-described embodiments, and various modifications are included. For example, the above-described embodiments have been described in detail to clearly explain the present invention, and the present invention is not necessarily limited to those including all of the components described. In addition, it is possible to replace a part of the configuration of one embodiment with the configuration of another embodiment, and it is also possible to add the configuration of another embodiment to the configuration of one embodiment. In addition, it is possible to add, delete, or replace a part of the configuration of each embodiment with another configuration.

[0123] In addition, each of the above configurations, functions, processing units, processing means, etc. may be realized in hardware by designing a part or all of them as an integrated circuit, for example. In addition, each of the above configurations, functions, etc. may be realized in software by a processor interpreting and executing a program that realizes each function. Information such as a program, a judgment table, and a file that realizes each function can be placed in a memory, a storage device such as an HDD or SSD, or a recording medium such as an IC (Integrated Circuit) card, an SD (Secure Digital) card, or a DVD (Digital Versatile Disc). In addition, the control lines and information lines are shown as those considered necessary for explanation, and not all control lines and information lines are necessarily shown in the product. In reality, it may be considered that almost all configurations are connected to each other. [Explanation of symbols]

[0124] 10...inspection support system, 50...inspection support program, 100...inspection support device, 100a...storage device, 100b...memory, 100c...processor, 100d...input device, 100e...display device, 100f...communication interface, 100g...medium reading device, 100h...recording medium, 100i...bus, 101...inspection condition information, 102...CAD data, 104...overall control unit, 104a...conversion unit, 104b...display control unit, 105...storage unit, 106...inspection result information, 107...input unit, 108...display unit, 109...storage unit, 110...real data acquisition unit, 111...3D scanner, 112...real data , 113...sensor position and orientation data, 114...sensor data, 120...sensor position acquisition unit, 121...position and orientation measurement sensor, 122...attachment, 123, 124...marker, 130...sensor data acquisition unit, 131...sensor, 132...camera lens, 133...lighting, 133'...pattern lighting, 134...spectrometer lens, 135...ultrasonic flaw detector, 140...display device, 141...movement direction, 142...swing direction, 143...distance direction, 144...rotation direction, 145...crosshair, 200, 300...work, 201...work table, 210, 310...inspection position, 211...defect, 212...membrane, 213...internal defect.

Claims

1. a reality data acquisition unit that acquires reality data indicating a shape, a position, and an orientation of a workpiece placed in a real space; a conversion unit that calculates an inspection position on the workpiece in real space by performing a conversion operation of superimposing design data indicating a design shape of the workpiece on the real data; and a sensor position acquisition unit that acquires a position in real space of a sensor that inspects the workpiece; a display control unit that causes a display device to display a direction in which a difference between the inspection position on the workpiece in real space and the position of the sensor falls within an allowable positional deviation range as a direction in which the sensor should be moved; An inspection support device having the above configuration.

2. The inspection support device according to claim 1 , The conversion unit converts an inspection direction on the design data into an inspection direction on the workpiece in real space by the conversion operation, The sensor position acquisition unit acquires a direction in which the sensor is facing, the display control unit causes the display device to display, as a direction in which the sensor is to be swung, a direction in which an angle formed between the inspection direction on the workpiece in real space and the direction in which the sensor is facing falls within an allowable angle range. Inspection support equipment.

3. The inspection support device according to claim 1 , a storage unit that stores the inspection position on the design data and the inspection result at the inspection position in a storage unit in association with each other; The inspection support device further comprises:

4. The inspection support device according to claim 1 , the reality data acquisition unit acquires the reality data from a three-dimensional measuring device that measures the shape, the position, and the orientation of the workpiece in a real space; Inspection support equipment.

5. The inspection support device according to claim 1 , the conversion unit performs a rotation operation and a translation operation on the design data as the conversion operation, and performs the conversion operation on the inspection position such that a deviation between the converted design data and the actual data is minimized; Inspection support equipment.

6. The inspection support device according to claim 1 , The sensor is any one of a defect detection sensor using an image, a film thickness or material measurement sensor using a reflection spectrum, and an internal defect detection sensor using ultrasonic waves or eddy currents. Inspection support equipment.

7. The inspection support device according to claim 1 , the display control unit causes the display device to display a direction in which a distance between the inspection position on the workpiece in real space and the position of the sensor falls within an allowable distance range as a direction in which the sensor is moved closer to or farther away from the inspection position. Inspection support equipment.

8. The inspection support device according to claim 1 , the display control unit changes a display mode of the display device when a difference between the inspection position on the workpiece in real space and the position of the sensor falls within the allowable positional deviation range. Inspection support equipment.

9. The inspection support device according to claim 1 , the display control unit sets, among the positions on the design data, a position where a difference between the design data on which the conversion operation has been performed and the actual data is equal to or greater than a predetermined value as the inspection position on the design data. Inspection support equipment.

10. The inspection support device according to claim 1 , the display control unit sets a position on the design data where an abnormality is found by the sensor as the inspection position on the design data. Inspection support equipment.

11. The computer acquiring reality data indicative of a shape, position, and orientation of a workpiece placed in a real space; A step of calculating an inspection position on the workpiece in real space by performing a conversion operation of superimposing design data indicating a design shape of the workpiece on the real data. acquiring a position in real space of a sensor that inspects the workpiece; displaying, on a display device, a direction in which a difference between the inspection position on the workpiece in real space and the position of the sensor falls within an allowable positional deviation range as a direction in which the sensor should be moved; An inspection support method for performing the above.

12. The inspection support method according to claim 11, The computer includes: converting an inspection direction on the design data into an inspection direction on the workpiece in real space by the conversion operation; obtaining a direction in which the sensor is pointing; a step of displaying on the display device, as a direction in which the sensor is to be swung, a direction in which an angle formed between the inspection direction on the workpiece in real space and the direction in which the sensor is facing falls within an allowable angle range; The inspection support method further comprises:

13. The inspection support method according to claim 11, The computer includes: storing the inspection position on the design data and the inspection result at the inspection position in a storage unit in association with each other; The inspection support method further comprises:

14. The inspection support method according to claim 11, The computer acquires the real data from a three-dimensional measuring device that measures the shape, the position, and the orientation of the workpiece in real space. Inspection support methods.

15. The inspection support method according to claim 11, the computer performs a rotation operation and a translation operation on the design data as the transformation operation, and performs the transformation operation on the inspection position such that a deviation between the transformed design data and the actual data is minimized; Inspection support methods.

Images