Defect inspection system and defect inspection method

Abstract

To provide a defect inspection system and a defect inspection method in which a sensor for inspecting a defect can be selected according to a defect type.SOLUTION: A defect inspection system includes: a camera 100 for imaging an inspection object 101 to obtain a captured image of the inspection object 101; and a processing device 120 comprising a database 500, a defect inspection part 104, and a sensor selection part 107. The database 500 stores sensor data that shows a relationship between a type of a defect of the inspection object 101 and a sensor 108, 109 to be used for inspecting the defect. The defect inspection part 104 subjects the captured image to image processing to detect a defect from the captured image, and obtains a defect type. The sensor selection part 107 selects a sensor 108, 109 to be used for inspecting a defect according to a defect type, on the basis of the sensor data stored in the database 500.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a system and method for inspecting an object for defects. [Background technology]

[0002] The quality and reliability of industrial products and their parts are controlled by inspecting them for defects such as scratches and rust. An example of a conventional defect inspection system is described in Patent Document 1. The surface defect inspection device described in Patent Document 1 has an imaging device that images a planar inspection object, an image processing device that processes the captured image to detect defects on the surface of the inspection object and obtain position information of the defects, and a precision inspection device that precisely inspects the defects on the surface of the inspection object based on the position information of the defects, and inspects the defects in detail using the precision inspection device. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2002-168793 A Summary of the Invention [Problem to be solved by the invention]

[0004] The object to be inspected may have various types of defects, and the sensor (device) required for detailed inspection of the defect differs depending on the type of defect. Therefore, detailed inspection of a defect requires a sensor appropriate for the type of defect.

[0005] However, in conventional defect inspection systems, the sensor (device) used for detailed inspection of defects is predetermined to be a specific sensor. For example, in the surface defect inspection device described in Patent Document 1, the precision inspection device that detects the shape and depth of a defect is predetermined to be a microscope (e.g., a differential interference microscope and a two-beam interference microscope). Therefore, there is a demand for a defect inspection system that can select a sensor according to various types of defects.

[0006] An object of the present invention is to provide a defect inspection system and a defect inspection method capable of selecting a sensor for inspecting a defect depending on the type of defect. [Means for solving the problem]

[0007] A defect inspection system according to the present invention includes a camera that captures an image of an object to be inspected to obtain the captured image of the object to be inspected, and a processing device that includes a database, a defect detection unit, and a sensor selection unit. The database stores sensor data indicating a relationship between a type of defect in the object to be inspected and a sensor to be used to inspect the defect. The defect detection unit performs image processing to detect the defect from the captured image and determine the type of the defect. The sensor selection unit selects a sensor to be used to inspect the defect according to the type of the defect based on the sensor data stored in the database.

[0008] A defect inspection method according to the present invention includes an imaging step of imaging an object to be inspected with a camera to obtain an image of the object to be inspected, a defect detection step of a processing device performing image processing to detect defects in the object to be inspected from the image and to determine the type of the defect, and a sensor selection step of the processing device selecting a sensor to be used for inspecting the defect according to the type of defect based on sensor data indicating the relationship between the type of defect and the sensor to be used for inspecting the defect. Effect of the Invention

[0009] According to the present invention, it is possible to provide a defect inspection system and a defect inspection method capable of selecting a sensor for inspecting a defect depending on the type of the defect.

[0010] Problems, configurations and effects other than those described above will become apparent from the following description of the preferred embodiment of the invention. [Brief description of the drawings]

[0011] [Figure 1] 1 is a diagram illustrating an example of the configuration of a defect inspection system according to a first embodiment of the present invention. [Diagram 2] FIG. 2 is a diagram showing a process flow of the defect inspection method according to the first embodiment of the present invention. [Figure 3A] FIG. 13 is a diagram for explaining a method in which the defect detection unit detects a defect and determines a defect type from an image captured by a camera, and is a diagram showing an example of a portion of a captured image that includes a defect that is a scratch. [Figure 3B] FIG. 13 is a diagram for explaining a method in which the defect detection unit detects defects and determines the defect type from an image captured by a camera, and is a diagram showing an example of a portion of a captured image that includes a defect that is rust. [Figure 3C] FIG. 13 is a diagram showing an example of a repair location of an inspection object obtained by simulation. [Figure 3D] 11 is a diagram showing an example of repair locations of an inspection object determined based on the history of the inspection object; FIG. [Figure 4A] FIG. 13 is a diagram showing a method using motion capture as a method for the defect calculation unit to calculate defect coordinates. [Figure 4B] FIG. 13 is a diagram showing a method in which a defect calculation unit uses two cameras to calculate defect coordinates. [Figure 4C] FIG. 13 is a diagram showing a method in which a structure-from-motion method is used as a method in which a defect calculation unit calculates defect coordinates. [Diagram 5] FIG. 4 is a diagram illustrating an example of data stored in a sensor database. [Figure 6A] 11 is a diagram for explaining an example of a method in which a sensor position and orientation calculation unit instructs an operator on a measurement position of the sensor. FIG. [Figure 6B] 11 is a diagram for explaining an example of a method in which a working robot measures an inspection object using a selected sensor. FIG. [Figure 7] 11 is a diagram for explaining a method in which a repairability determining unit determines whether a defect is repairable or not. FIG. [Figure 8] FIG. 13 is a diagram for explaining a method for a worker to repair a defect. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] In the defect inspection system and defect inspection method according to the present invention, the sensor for inspecting the defect can be selected according to the type of defect, thereby enabling detailed inspection of various defects that occur in the object to be inspected, and enabling more effective management of the quality and reliability of the object to be inspected.

[0013] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. The examples are illustrative for explaining the present invention, and are omitted and simplified as appropriate for clarity of explanation. The present invention can be implemented in various other forms. Unless otherwise specified, each component may be singular or plural.

[0014] In order to facilitate understanding of the invention, the position, size, shape, range, etc. of each component shown in the drawings may not represent the actual position, size, shape, range, etc. Therefore, the present invention is not necessarily limited to the position, size, shape, range, etc. disclosed in the drawings.

[0015] When there are multiple components having the same or similar functions, they may be described by using the same reference numerals with different subscripts, or when there is no need to distinguish between these multiple components, the subscripts may be omitted.

[0016] In the embodiments, a process performed by executing a program may be described. Here, a computer executes the program using a processor (e.g., CPU, GPU), and performs the process defined by the program while using a storage resource (e.g., memory) and an interface device (e.g., communication port). Therefore, the subject of the process performed by executing the program may be the processor. Similarly, the subject of the process performed by executing the program may be a controller, device, system, computer, or node having a processor. The subject of the process performed by executing the program may be a calculation unit, and may include a dedicated circuit that performs specific processing. Here, the dedicated circuit is, for example, an FPGA (Field Programmable Gate Array), an ASIC (Application Specific Integrated Circuit), or a CPLD (Complex Programmable Logic Device).

[0017] The program may be installed in the computer from a program source. The program source may be, for example, a program distribution server or a computer-readable storage medium. When the program source is a program distribution server, the program distribution server may include a processor and a storage resource that stores the program to be distributed, and the processor of the program distribution server may distribute the program to be distributed to other computers. In addition, in the embodiment, two or more programs may be realized as one program, and one program may be realized as two or more programs. EXAMPLES

[0018] A defect inspection system and a defect inspection method according to a first embodiment of the present invention will be described with reference to the drawings.

[0019] FIG. 1 is a diagram showing an example of the configuration of a defect inspection system according to this embodiment.

[0020] The defect inspection system according to this embodiment includes a camera 100, a motion capture camera 102, a processing device 120, and a display device 130, and inspects an inspection target 101 for defects. The defect inspection system according to this embodiment may or may not include a plurality of sensors and a work robot. Note that in FIG. 1, the processing device 120 is depicted as being divided into two for ease of understanding.

[0021] In this embodiment, defect inspection (measurement) using a sensor and defect repair are performed by an operator or a working robot. If the defect inspection system according to this embodiment does not include a sensor, an operator prepares and uses a sensor, and if the system is equipped with a sensor, the working robot uses the sensor. The working robot uses sensors 108, 109 selected by the defect inspection system from among the multiple sensors equipped in the defect inspection system. The working robot can attach the sensor selected by the defect inspection system using, for example, a tool changer or the like.

[0022] In this embodiment, as an example, a case will be described in which a worker or a working robot uses two sensors 108, 109, but the worker or the working robot may use one sensor or three or more sensors.

[0023] The inspection object 101 is, for example, an industrial product or a part thereof, which has a defect caused during the manufacturing process, or an industrial product or a part thereof that has been collected after use, which has a defect caused by use.

[0024] The camera 100 captures an image of the inspection object 101 and obtains a captured image of the inspection object 101. The image captured by the camera 100 is input to a processing device 120.

[0025] Reflective markers 103 for motion capture are attached to the camera 100 and the inspection object 101.

[0026] The motion capture camera 102 captures images of the camera 100 and a reflective marker 103 attached to an inspection object 101. The image captured by the motion capture camera 102 is input to a processing device 120.

[0027] The processing device 120 is configured with a calculator such as a computer, and includes a defect detection unit 104, a camera position and orientation calculation unit 105, a defect calculation unit 106, a sensor selection unit 107, a sensor database 500, a sensor position and orientation calculation unit 114, a measurement result input unit 112, a repair possibility determination unit 116, a threshold value database 115, and a repair amount database 113. The sensor database 500, the threshold value database 115, and the repair amount database 113 may be included in a single database.

[0028] The defect detection unit 104, the camera position and orientation calculation unit 105, the defect calculation unit 106, the sensor selection unit 107, and the sensor database 500 are components for detecting defects in the inspection object 101 and selecting sensors 108, 109 according to the type of defect (defect type).

[0029] The defect detection unit 104 inputs an image captured by the camera 100 (a captured image of the inspection object 101). The defect detection unit 104 performs image processing on the input image to detect defects in the inspection object 101 and determine the position of the defect within the image (two-dimensional coordinates in the captured image of the inspection object 101). Furthermore, the defect detection unit 104 identifies the type of the detected defect from the captured image of the inspection object 101. The type of defect (defect type) is, for example, a scratch or rust. Other examples of defect types will be described later. A method in which the defect detection unit 104 detects defects and determines the defect type will be described later.

[0030] An image captured by the motion capture camera 102 (an image of the camera 100 and a reflective marker 103 attached to the inspection object 101) is input to a camera position and orientation calculation unit 105. The camera position and orientation calculation unit 105 calculates the position and orientation of the camera 100 with respect to the inspection object 101 from the input image.

[0031] The defect calculation unit 106 inputs the processing results of the defect detection unit 104 and the camera position and orientation calculation unit 105, and determines what type of defect exists at what position on the inspection object 101 from the position of the defect in the image, the type of this defect (defect type), and the position and orientation of the camera 100 relative to the inspection object 101. Hereinafter, the position (three-dimensional coordinates) of the defect on the inspection object 101 will be referred to as defect coordinates. The defect calculation unit 106 determines the defect coordinates from the image captured by the camera 100 (image of the defect in the inspection object 101) and the position and orientation of the camera 100 relative to the inspection object 101. Then, the defect calculation unit 106 determines the defect type at these defect coordinates.

[0032] The defect calculation unit 106 can associate defect coordinates with defect types to create a defect map of the inspection object 101. As shown in Fig. 1, for example, the defect map specifies the positions of defects by three-dimensional coordinates (defect coordinates) on the inspection object 101 and indicates the defect types (for example, scratches and rust) at these positions.

[0033] The sensor selection unit 107 refers to the data stored in the sensor database 500 and selects the sensors 108 and 109 to be used for detailed inspection of the defect depending on the type of defect detected by the defect detection unit 104 .

[0034] The sensor selection unit 107 notifies the worker of the selected sensors 108, 109 by displaying the selected sensors 108, 109 on the display device 130. In addition, the sensor selection unit 107 notifies the working robot of the selected sensors 108, 109 by transmitting a signal representing the selected sensors 108, 109.

[0035] The sensor database 500 stores in advance data indicating the relationship between the defect types and the sensors 108, 109 used for detailed defect inspection. The relationship between the defect types and the sensors 108, 109 used for detailed defect inspection is determined in advance.

[0036] The sensor position and orientation calculation unit 114, the repair feasibility determination unit 116, the threshold value database 115, and the repair amount database 113 are components for performing detailed defect inspection and defect repair using the sensors 108 and 109 selected according to the defect type.

[0037] In this embodiment, as an example, an example will be described in which the inspection target 101 has two defects. It is assumed that the defect type of one defect is a scratch, and a roughness meter (surface roughness measuring instrument) is selected as the sensor 108 for this defect type. It is assumed that the defect type of the other defect is rust, and a film thickness meter is selected as the sensor 109 for this defect type.

[0038] The positions (coordinates) of defects to be inspected or repaired using the sensors 108 and 109 are indicated by the defect coordinates calculated by the defect calculation unit 106.

[0039] The sensors 108 and 109 selected by the sensor selection unit 107 are fitted with reflective markers 103 for motion capture.

[0040] The motion capture camera 102 captures images of the sensors 108 and 109 and the reflective marker 103 attached to the inspection object 101. The images captured by the motion capture camera 102 are input to a processing device 120.

[0041] The sensor position and orientation calculation unit 114 calculates the positions and orientations of the sensors 108 and 109 relative to the inspection object 101 when performing detailed inspection of defects using the sensors 108 and 109 selected by the sensor selection unit 107. The sensor position and orientation calculation unit 114 inputs images captured by the motion capture camera 102 (images of the sensors 108 and 109 and the reflective markers 103 attached to the inspection object 101). The sensor position and orientation calculation unit 114 calculates the positions and orientations of the sensors 108 and 109 relative to the inspection object 101 from the input images.

[0042] A sensor position and orientation calculation unit 114 calculates the difference between the calculated positions of the sensors 108 and 109 and the position of the defect (defect coordinates) calculated by the defect calculation unit 106, and instructs a worker or a work robot on the measurement positions of the sensors 108 and 109.

[0043] The worker or the work robot adjusts the positions of the sensors 108, 109 so that this difference becomes small, and the sensors 108, 109 inspect (measure) the defect at the position indicated by the defect coordinates. Then, the worker or the work robot inspects (measures) the defect using the sensors 108, 109.

[0044] The measurement result input unit 112 inputs the size of the defect obtained by the sensors 108 and 109 as the measurement result. For example, if the sensor 108 is a roughness meter, the sensor 108 measures the location of the scratch that is the defect, and outputs the measurement result (area and depth of the scratch) to the processing device 120 via the controller 110 of the sensor 108. For example, if the sensor 109 is a film thickness meter, the sensor 109 measures the location of the rust that is the defect, and outputs the measurement result (area and thickness of the rust) to the processing device 120 via the controller 111 of the sensor 109. The measurement result input unit 112 inputs the area and depth of the scratch or the area and thickness of the rust as the size of the defect.

[0045] The repair feasibility determination unit 116 associates the measurement results (size of the defect) of the sensors 108, 109, the positions of the sensors 108, 109 when the sensors 108, 109 obtained the measurement results, and the position of the defect (defect coordinates). The measurement results of the sensors 108, 109 are obtained from a measurement result input unit 112. The positions of the sensors 108, 109 are the positions of the sensors 108, 109 with respect to the inspection target 101, and are obtained from a sensor position and orientation calculation unit 114. The defect coordinates are obtained from the defect calculation unit 106.

[0046] Then, a repairability determining unit 116 uses the size of the defect measured by the sensors 108 and 109 and a threshold value stored in a threshold value database 115 to determine whether or not the defect is repairable.

[0047] The threshold database 115 stores in advance a threshold for determining whether or not a defect is repairable for each type of defect. This threshold can be arbitrarily determined in advance, for example, based on information about defects that have occurred in the past. For example, this threshold is a threshold for the size of the defect. For example, if the defect is a scratch, the repairable depth of the scratch can be found from information about the past defect, and the threshold can be determined based on the found depth of the scratch. This threshold may also include a threshold for the cost of repairing the defect (repair cost). The repair cost can be arbitrarily determined in advance, for example, based on the type and size of the defect. The threshold for the repair cost can be arbitrarily determined in advance, for example, based on the repair cost required when repairing a defect in the past.

[0048] The threshold database 115 may use a threshold determined by simulation as the threshold for the size of the defect. For example, the performance of the inspection object 101 when it has a defect is calculated by simulation, and the maximum size of the defect that allows the inspection object 101 to maintain the required performance is determined. Then, the maximum size of the defect that allows the inspection object 101 to maintain the required performance is set as the threshold for the size of the defect.

[0049] Repairability determination unit 116 determines that the defect is repairable if the size of the defect measured by sensors 108, 109 is smaller than the threshold value for the size of the defect stored in threshold database 115. Repairability determination unit 116 also calculates the cost of repairing the defect based on the type and size of the defect in accordance with a predetermined procedure, and determines that the defect is repairable if this cost is smaller than the threshold value for the repair cost stored in threshold database 115. Then, repairability determination unit 116 determines that the defect is not repairable if the size of the defect or the cost of repairing the defect is equal to or greater than the respective threshold values.

[0050] If the repairability determination unit 116 determines that the defect is repairable, it instructs the worker or the work robot on the location of the defect (i.e., the location to be repaired on the inspection object 101) and the amount of repair required to repair the defect. The repairability determination unit 116 refers to the repair amount stored in the repair amount database 113 and determines the amount of repair for the defect according to the size of the defect.

[0051] The repair amount database 113 stores in advance data indicating the relationship between the size of the defect (e.g., the depth of the scratch or the thickness of the rust) and the repair amount for repairing the defect for each type of defect. The relationship between the size of the defect and the repair amount is determined arbitrarily in advance. For example, if the type of defect is a scratch, the relationship between the depth of the scratch (size of the defect) and the amount of build-up (repair amount) is determined. Also, if the type of defect is rust, the relationship between the thickness of the rust (size of the defect) and the amount of rust removed (repair amount) is determined.

[0052] Fig. 1 shows a schematic example of instructions 117 given to a worker or a work robot by the repair feasibility determination unit 116. The instructions 117 shown in Fig. 1 include the positions of defects 301 and 302, which are the repair locations of the inspection object 101, a repair amount 118 (amount of build-up: 25 µm) for the defect 301, which is a scratch, and a repair amount 119 (amount of rust removal: 15 µm) for the defect 302, which is rust.

[0053] The repair feasibility determination unit 116 communicates instructions 117 to the worker by displaying the positions of the defects 301, 302 and the repair amounts 118, 119 on the display device 130. The repair feasibility determination unit 116 also communicates instructions 117 to the work robot by transmitting signals indicating the positions of the defects 301, 302 and the repair amounts 118, 119.

[0054] If the defect is repairable, the worker or the work robot repairs the defect according to the defect position (repair location on the inspection object 101) and repair amount instructed by the repair feasibility determination unit 116 of the processing device 120. To repair the defect, a tool selected based on the defect position, type, size, etc. can be used. If the defect is not repairable, the worker or the work robot discards the inspection object 101.

[0055] The display device 130 may be, for example, one or both of a display and AR (Augmented Reality) glasses worn by the worker, and is connected to the processing device 120 by wire or wirelessly. The processing device 120 can instruct the worker to inspect (measure) the defects and repair the defects by displaying the selected sensors 108 and 109, the determined positions, types and sizes of the defects, the judged possibility of repairing the defects, and the determined repair amount on the display device 130. The display device 130 can also display the difference between the positions of the sensors 108 and 109 and the positions of the defects, which are determined by the sensor position and orientation calculation unit 114, and instruct the worker on the measurement positions of the sensors 108 and 109. For example, the display device 130 can display the instruction 117 shown in FIG. 1.

[0056] Based on the information displayed on the display device 130, the worker can use the selected sensors 108, 109 to inspect (measure) the defect and repair the defect.

[0057] When the defect inspection system according to this embodiment includes sensors 108, 109 and a working robot, the working robot can inspect (measure) defects and repair the defects using the selected sensor 108, 109 in accordance with instructions from the processing device 120. The processing device 120 can inform the working robot of the selected sensor 108, 109, the determined position, type and size of the defect, whether or not the defect can be repaired, the determined amount of repair, etc.

[0058] 2 is a diagram showing a process flow of the defect inspection method according to the first embodiment of the present invention. The process flow of the defect inspection method according to this embodiment will be described with reference to FIG.

[0059] In step 201, a worker or a working robot places an inspection target 101 at a desired position.

[0060] In step 202, the camera 100 captures an image of the inspection target 101 through the operation of a worker or a working robot.

[0061] In step 203, the defect detection unit 104 of the processing device 120 performs image processing on the image captured by the camera 100 (captured image of the inspection object 101) and detects defects in the inspection object 101 from the captured image. Then, the defect detection unit 104 obtains an approximate size of the detected defect (for example, the area and depth of the defect) through image processing.

[0062] In step 204, defect detection unit 104 judges whether the defect detected in step 203 is a fatal defect. A fatal defect is a defect that is too large to be repaired (for example, a scratch with a depth of 1 mm or more). Detailed inspection using a sensor in a subsequent step is not performed on inspection object 101 that has a fatal defect. If the size of the defect detected in step 203 is larger than a predetermined threshold, defect detection unit 104 judges that the defect is a fatal defect, and if the size is equal to or smaller than the predetermined threshold, judges that the defect is not a fatal defect.

[0063] For defects that are not fatal defects, the process proceeds to step 205. For defects that are fatal defects, the process proceeds to step 213.

[0064] In step 205, the defect detection unit 104 obtains the position of the defect in the image (two-dimensional coordinates in the captured image of the inspection object 101) for the defect detected in step 203 from the captured image of the inspection object 101, and identifies the type of the defect. Next, the defect calculation unit 106 obtains the defect coordinates, i.e., the position (three-dimensional coordinates) of the defect in the inspection object 101, and associates the defect coordinates with the defect type.

[0065] In step 206 , the sensor selection unit 107 selects the sensors 108 and 109 to be used for detailed inspection of the defects, depending on the defect type identified by the defect detection unit 104 .

[0066] In step 207, the sensor position and orientation calculation unit 114 calculates the positions and orientations of the sensors 108, 109 relative to the inspection object 101, and instructs an operator or a work robot on the measurement positions of the sensors 108, 109 based on the calculated positions of the sensors 108, 109 and the defect coordinates calculated by the defect calculation unit 106.

[0067] In step 208, the worker or the working robot uses sensors 108, 109 to measure the defect (the defect detected in step 203) in the inspection object 101 at the specified measurement position. In step 208, a detailed inspection (measurement) of the defect is performed. That is, in the measurement in step 208, the size of a defect smaller than the size of the defect found in step 203 (for example, a scratch with a depth of less than 1 mm) is measured by sensors 108, 109.

[0068] In step 209, the repairability determination unit 116 determines, based on the measurement results of the sensors 108 and 109, whether or not the defect detected in step 203 is repairable.

[0069] For repairable defects, the process proceeds to step 210. For non-repairable defects, the process proceeds to step 213.

[0070] In step 210, the repair possibility determination unit 116 instructs the worker or the working robot on the location of the defect (the part to be repaired on the inspection object 101) and the amount of repair required to repair this defect.

[0071] In step 211, the worker or the work robot repairs the defect according to the instructions from the repair possibility determination unit 116.

[0072] When the worker or the working robot finishes repairing the defect in step 212, the process of the defect inspection method according to this embodiment is completed.

[0073] In step 213, the worker or the working robot discards the inspection target 101.

[0074] 3A and 3B are diagrams for explaining a method in which the defect detection unit 104 detects defects and determines the defect type from an image captured by the camera 100 (a captured image of the inspection object 101).

[0075] The defect detection unit 104 detects defects from the image captured by the camera 100 by image processing, and identifies the defect type based on the features corresponding to the defect type that are recorded in the captured image.

[0076] 3A is a diagram showing an example of a portion of a captured image including a defect 301, which is a scratch. If the defect 301 is a scratch, a change in brightness occurs at the edge of the defect 301 in the captured image by the camera 100 due to illumination light. The defect detection unit 104 can detect the defect 301, which is a scratch, by detecting the edge, for example, through image processing. Furthermore, if a change in brightness is observed at the edge of the defect in the captured image, the defect detection unit 104 determines that the type of this defect is a scratch.

[0077] 3B is a diagram showing an example of a portion of a captured image including a defect 302 that is rust. If the defect 302 is rust, there is a difference in color between the surface (part where no rust has occurred) of the inspection object 101 and the rust in the captured image of the camera 100. The defect detection unit 104 can detect the defect 302 that is rust by detecting such a difference in color, for example, through image processing. Furthermore, if there is a difference in color between the surface (part where no defect has occurred) of the inspection object 101 and the defect in the captured image, the defect detection unit 104 determines that the type of the defect is rust.

[0078] The inspection object 101 may have a defect inside. This internal defect is one type of defect. The image captured by the camera 100 may not show the characteristics of the internal defect. In such a case, a simulation may be performed in advance to find locations where internal defects are likely to occur, and these locations may be designated as repair locations for the inspection object 101. In this simulation, locations where force or heat is applied are found, and the found locations are designated as locations where stress acts and internal defects are likely to occur.

[0079] When the defect type is an internal defect, the defect is inspected, for example, by an ultrasonic sensor (see FIG. 5).

[0080] FIG. 3C is a diagram showing an example of a repair location 303 of the inspection target 101 obtained by simulation.

[0081] Furthermore, it is also possible to specify the locations and defect types in the inspection object 101 where defects are likely to occur based on the history of the inspection object 101, and to set these locations as repair locations of the inspection object 101. For example, it is also possible to specify the locations and defect types in the inspection object 101 where defects are likely to occur based on the past usage record of the inspection object 101 and the past defect repair history, and to set these locations as repair locations of the inspection object 101.

[0082] FIG. 3D is a diagram showing an example of a repair location 304 of the inspection object 101 determined based on the history of the inspection object 101.

[0083] An example of a method in which the defect calculation unit 106 calculates defect coordinates (the position of a defect in the inspection object 101) will be described with reference to FIGS. 4A, 4B, and 4C.

[0084] FIG. 4A is a diagram showing a method using motion capture as a method for the defect calculation unit 106 to calculate defect coordinates.

[0085] Reflective markers 103 for motion capture are attached to the camera 100 and the inspection object 101. A motion capture camera 102 captures images of the camera 100 and the inspection object 101. A camera position and orientation calculation unit 105 calculates the position and orientation 400 of the camera 100 with respect to the inspection object 101 from the image captured by the motion capture camera 102.

[0086] The defect calculation unit 106 calculates the three-dimensional position 402 of the defect in the object to be inspected 101 as defect coordinates from the position 401 of the defect in the image captured by the camera 100 obtained by the defect detection unit 104 and the position and orientation 400 of the camera 100 relative to the object to be inspected 101 obtained by the camera position and orientation calculation unit 105.

[0087] FIG. 4B is a diagram showing a method in which the defect calculation unit 106 calculates defect coordinates using two cameras 100a and 100b.

[0088] The two cameras 100a and 100b may be the same or different and are installed at different positions. The two cameras 100a and 100b capture the same location of the inspection object 101.

[0089] The defect calculation unit 106 obtains the three-dimensional position of the defect in the inspection object 101 from the difference in the viewing angles of the images captured by the two cameras 100a and 100b, for example, by using a stereo imaging method, and calculates the defect coordinates.

[0090] FIG. 4C is a diagram showing a method in which the defect calculation unit 106 uses a structure-from-motion method to calculate defect coordinates.

[0091] The camera 100 captures an image of the inspection object 101 multiple times while shifting the relative position of the inspection object 101 with respect to the camera 100 .

[0092] The defect calculation unit 106 obtains the three-dimensional position of the defect in the inspection object 101 from a plurality of captured images by the camera 100 by a structure from motion (SfM) method, and calculates the defect coordinates. Fig. 4C shows an example in which the shape of the inspection object 101 is rotationally symmetric. For the inspection object 101 having such a shape, the relative position of the inspection object 101 with respect to the camera 100 may be shifted by rotating the inspection object 101 about the axis of rotational symmetry.

[0093] 5 is a diagram showing an example of data stored in the sensor database 500. The relationship between the defect type and the sensor used for detailed inspection of the defect is determined in advance and stored in the sensor database 500 in advance.

[0094] The sensor selection unit 107 refers to the data stored in the sensor database 500 and selects a sensor to be used for detailed inspection of the defect according to the type of defect detected by the defect detection unit 104. In the sensor database 500, a plurality of sensors may be defined for one defect type. When a plurality of sensors are defined for one defect type in the sensor database 500, the sensor selection unit 107 may select an optimal sensor from among the plurality of sensors in consideration of conditions such as measurement accuracy, measurement range, measurement speed, and equipment cost according to the required inspection specifications. These conditions are determined in advance and stored in the sensor database 500 in advance. The worker or the work robot inputs the necessary conditions from among these conditions to the sensor selection unit 107.

[0095] For example, for defects whose defect type is surface wear, a three-dimensional shape measuring device is specified as a sensor used for detailed inspection of the defect. The three-dimensional shape measuring device measures the surface shape of the inspection object 101 to measure the degree to which the surface of the inspection object 101 has worn away. As the three-dimensional shape measuring device, for example, a sensor that projects a stripe pattern to measure the three-dimensional shape, a light-cutting type sensor that measures the three-dimensional shape, and a laser range finder that scans two dimensions can be used.

[0096] For example, for defects that are scratches and defects that are surface roughness, contact and non-contact roughness gauges are specified as sensors to be used for detailed inspection of defects. Contact roughness gauges have the advantages of high measurement accuracy and low equipment costs, but the disadvantages of a narrow measurement range and slow measurement speed. Non-contact roughness gauges are, for example, optical types, and have the advantages of a wider measurement range and faster measurement speed than contact types, but the disadvantage of high equipment costs.

[0097] In addition, for example, for defects in which the defect type is a reduction in the thickness of the surface coating of the object to be inspected 101 and defects in which the defect type is rust, a contact type film thickness gauge and a non-contact type film thickness gauge are specified as sensors to be used for detailed inspection of the defect.

[0098] Furthermore, for example, for defects whose defect type is an internal defect, ultrasonic sensors, X-ray inspection devices, and sensors for eddy current testing (ECT) are specified as sensors to be used for detailed inspection of the defect.

[0099] Fig. 6A is a diagram for explaining an example of a method in which the sensor position and orientation calculation unit 114 instructs the operator 600 on the measurement position of the sensor. An example of a method in which the sensor position and orientation calculation unit 114 calculates the position and orientation of the sensor with respect to the inspection target 101 and instructs the operator 600 on the measurement position of the sensor will be explained using Fig. 6A. In the following, as an example, an example will be explained in which the defect type is a scratch and the sensor 108 is selected.

[0100] The worker 600 attaches a reflective marker 103 for motion capture to the selected sensor 108 .

[0101] The motion capture camera 102 captures images of the reflective marker 103 attached to the sensor 108 and the reflective marker 103 attached to the inspection object 101 .

[0102] The sensor position and orientation calculation unit 114 calculates the position and orientation of the sensor 108 relative to the inspection target 101, and determines a difference 601 between the calculated position of the sensor 108 and the position of the defect (defect coordinates) determined by the defect calculation unit 106. If the difference 601 is greater than a predetermined threshold, the sensor position and orientation calculation unit 114 displays on the display device 130 a work instruction 602 to adjust the position of the sensor 108 so as to reduce the difference 601. In this way, the sensor position and orientation calculation unit 114 instructs the operator 600 on the measurement position of the sensor 108.

[0103] When the difference 601 becomes equal to or smaller than a predetermined threshold, the sensor position and orientation calculation unit 114 displays on the display device 130 an instruction to the operator 600 to measure the inspection target 101 using the sensor 108 .

[0104] When the working robot measures the inspection object 101 using the selected sensor 108, the following process may be performed.

[0105] 6B is a diagram for explaining an example of a method in which the working robot 603 measures the inspection object 101 using the selected sensor 108. The working robot 603 may measure the inspection object 101 using the selected sensor 108 according to the following procedure.

[0106] The defect calculation section 106 outputs the determined defect coordinates 604 as a measurement position to the working robot 603. The working robot 603 moves the sensor 108 to the position of the input defect coordinates 604 and measures the inspection target 101.

[0107] By carrying out the example process described with reference to Figures 6A and 6B, detailed inspection of defects can be performed using the selected sensor.

[0108] FIG. 7 is a diagram for explaining a method in which the repairability determining unit 116 determines whether or not a defect is repairable.

[0109] If the size of the defect input from measurement result input unit 112 is smaller than the threshold value for the size of the defect stored in threshold database 115, repairability determination unit 116 determines that the defect is repairable.

[0110] Furthermore, the repairability determination unit 116 can also determine whether or not a defect is repairable, taking into account the cost of repairing the defect. For example, if a defect is of a repairable size but the cost of repairing it would be high due to its position on the inspection object 101, the repairability determination unit 116 determines that the defect is not repairable. As described above, the threshold database 115 can include a threshold for the cost of repairing a defect (repair cost). If the repair cost of the defect is equal to or greater than the threshold, the repairability determination unit 116 can determine that the defect is not repairable.

[0111] FIG. 8 is a diagram for explaining a method in which a worker 600 repairs a defect.

[0112] 1, the repair feasibility determination unit 116 displays instructions 117 on the display device 130 to instruct the worker 600 to repair the defects. For example, the instructions 117 include the positions of the defects 301 and 302 which are the repair locations of the inspection target 101, a repair amount 118 (amount of build-up is 25 μm) for the defect 301 which is a scratch, and a repair amount 119 (amount of rust removal is 15 μm) for the defect 302 which is rust.

[0113] Assume that a worker 600 repairs the rust defect 302 using a tool 801. The worker 600 attaches a reflective marker 103 for motion capture to the tool 801.

[0114] The motion capture camera 102 captures images of the reflective marker 103 attached to the tool 801 and the reflective marker 103 attached to the inspection object 101 .

[0115] The sensor position and orientation calculation unit 114 calculates the position of the tool 801 in the same manner as it calculates the position and orientation of the sensor 108, and calculates a difference 802 between the calculated position of the tool 801 and the position of the defect (defect coordinates) calculated by the defect calculation unit 106. If the difference 802 is greater than a predetermined threshold, the sensor position and orientation calculation unit 114 displays on the display device 130 a work instruction 803 to adjust the position of the tool 801 so as to reduce the difference 802. In this way, the sensor position and orientation calculation unit 114 instructs the worker 600 on the work position of the tool 801 (the repair location of the inspection target 101).

[0116] When the difference 802 becomes equal to or smaller than a predetermined threshold, the sensor position and orientation calculation unit 114 displays on the display device 130 an instruction to the worker 600 to repair the defect.

[0117] As described above, the defect inspection system and defect inspection method according to this embodiment can select the sensor for inspecting the defect depending on the type of defect, and can perform detailed inspection and repair of various defects that occur in the object 101 to be inspected. [Explanation of symbols]

[0118] 100, 100a, 100b...camera, 101...object to be inspected, 102...motion capture camera, 103...reflective marker, 104...defect detection unit, 105...camera position and orientation calculation unit, 106...defect calculation unit, 107...sensor selection unit, 108, 109...sensor, 110, 111...controller, 112...measurement result input unit, 113...repair amount database, 114...sensor position and orientation calculation unit, 115...threshold value database, 116...repair possibility determination unit, 117...instruction, 118, 119...repair amount , 120... processing device, 130... display device, 301, 302... defect, 303, 304... repair area, 400... position and orientation of camera relative to inspection object, 401... position of defect in image captured by camera, 402... three-dimensional position of defect in inspection object, 500... sensor database, 600... worker, 601... difference between sensor position and defect coordinates, 602... work instructions, 603... work robot, 604... defect coordinates, 801... tool, 802... difference between tool position and defect coordinates, 803... work instructions.

Claims

1. a camera for capturing an image of an object to be inspected to obtain an image of the object to be inspected; A processing device including a database, a defect detection unit, and a sensor selection unit; Equipped with the database stores sensor data indicating a relationship between a type of defect in the inspection object and a sensor used to inspect the defect; the defect detection unit performs image processing to detect the defect from the captured image and determine a type of the defect; the sensor selection unit selects a sensor to be used for inspecting the defect according to the type of the defect based on the sensor data stored in the database. A defect inspection system comprising:

2. The processing device includes a defect calculation unit, the defect calculation unit determines a position of the defect in the inspection object from the captured image. The defect inspection system of claim 1 .

3. The processing device includes a repairability determination unit, the database stores a threshold value for the size of the defect and repair amount data indicating a relationship between the size of the defect and a repair amount for repairing the defect; the repairability determination unit determines that the defect is repairable if the size of the defect is smaller than the threshold value stored in the database; If the defect is repairable, the repair feasibility determination unit refers to the repair amount data stored in the database to determine the repair amount. The defect inspection system according to claim 2 .

4. A display device is provided, The sensor selection unit displays the selected sensor on the display device. The defect inspection system of claim 1 .

5. Equipped with a work robot, the sensor selection unit transmits a signal representing the selected sensor to the work robot; The defect inspection system of claim 1 .

6. A display device is provided, the repair possibility determination unit displays the position of the defect and the repair amount on the display device. The defect inspection system according to claim 3 .

7. Equipped with a work robot, the repair possibility determination unit transmits a signal representing the position of the defect and the amount of repair to the work robot; The defect inspection system according to claim 3 .

8. an imaging step of imaging an object to be inspected with a camera to obtain an image of the object to be inspected; a defect detection step in which a processing device performs image processing to detect defects in the inspection object from the captured image and determine the type of the defects; a sensor selection step in which the processing device selects a sensor to be used for inspecting the defect in accordance with the type of the defect based on sensor data indicating a relationship between the type of the defect and a sensor to be used for inspecting the defect; A defect inspection method comprising the steps of:

Images