Defect inspection system and defect inspection method

Abstract

Provided are a defect inspection system and a defect inspection method with which it is possible to select a sensor for inspecting a defect in accordance with the defect type. A defect inspection system according to the present invention is provided with: a camera (100) that images an inspection target object (101) to obtain a captured image of the inspection target object (101); and a processing device (120) that is provided with a database (500), a defect detecting unit (104), and a sensor selecting unit (107). The database (500) stores sensor data indicating a relationship between the types of defects of the inspection target object (101) and sensors (108, 109) used to inspect the defects. The defect detecting unit (104) performs image processing to detect a defect from the captured image, and to determine the type of the defect. The sensor selecting unit (107) selects the sensor (108, 109) to be used to inspect the defect, in accordance with the type of the defect., on the basis of the sensor data stored in the database (500).

Description

Defect inspection system and defect inspection method

[0001] The present invention relates to a system and method for inspecting an object for defects.

[0002] The quality and reliability of industrial products and their parts are controlled by inspecting them for defects such as scratches and rust. An example of a conventional defect inspection system is described in Patent Document 1. The surface defect inspection device described in Patent Document 1 includes an imaging device that captures an image of a planar inspection object, an image processing device that processes the captured image to detect defects on the surface of the inspection object and obtain position information of the defects, and a precision inspection device that precisely inspects the defects on the surface of the inspection object based on the position information of the defects, and the defects are inspected in detail using the precision inspection device.

[0003] Japanese Patent Application Laid-Open No. 2002-168793

[0004] The object to be inspected may have various types of defects, and the sensors (devices) required for detailed inspection of the defects differ depending on the type of defect. Therefore, detailed inspection of defects requires sensors appropriate for the type of defect.

[0005] However, in conventional defect inspection systems, the sensor (device) used for detailed defect inspection is predetermined to be a specific sensor. For example, in the surface defect inspection device described in Patent Document 1, the precision inspection device that detects the shape and depth of defects is predetermined to be a microscope (e.g., a differential interference microscope and a two-beam interference microscope). Therefore, there is a need for a defect inspection system that can select a sensor according to various types of defects.

[0006] An object of the present invention is to provide a defect inspection system and a defect inspection method that can select a sensor for inspecting defects depending on the type of defect.

[0007] A defect inspection system according to the present invention includes a camera that captures an image of an object to be inspected and obtains the captured image of the object to be inspected, and a processing device that includes a database, a defect detection unit, and a sensor selection unit. The database stores sensor data indicating the relationship between the type of defect in the object to be inspected and the sensor to be used to inspect the defect. The defect detection unit performs image processing to detect the defect from the captured image and determine the type of the defect. The sensor selection unit selects a sensor to be used to inspect the defect according to the type of defect based on the sensor data stored in the database.

[0008] The defect inspection method according to the present invention comprises an imaging step of imaging an object to be inspected with a camera to obtain an image of the object to be inspected; a defect detection step of using a processing device to process the image to detect defects in the object to be inspected from the image and determine the type of the defect; and a sensor selection step of using the processing device to select a sensor to be used to inspect the defect in accordance with the type of defect based on sensor data indicating the relationship between the type of defect and the sensor to be used to inspect the defect.

[0009] According to the present invention, it is possible to provide a defect inspection system and a defect inspection method that can select a sensor for inspecting a defect depending on the type of defect.

[0010] Problems, configurations, and effects other than those described above will become apparent from the following description of the preferred embodiments of the invention.

[0011] FIG. 1 is a diagram illustrating an example of the configuration of a defect inspection system according to a first embodiment of the present invention. FIG. 2 is a diagram illustrating a processing flow of a defect inspection method according to the first embodiment of the present invention. FIG. 3 is a diagram illustrating a method in which a defect detection unit detects defects from an image captured by a camera to determine a defect type, and is a diagram illustrating an example of a portion of a captured image including a defect that is a scratch. FIG. 4 is a diagram illustrating a method in which a defect detection unit detects defects from an image captured by a camera to determine a defect type, and is a diagram illustrating an example of a portion of a captured image including a defect that is rust. FIG. 5 is a diagram illustrating an example of a repair location on an inspection object determined by simulation. FIG. 6 is a diagram illustrating an example of a repair location on an inspection object determined based on the history of the inspection object. FIG. 7 is a diagram illustrating a method in which motion capture is used as a method in which a defect calculation unit determines defect coordinates. FIG. 8 is a diagram illustrating a method in which two cameras are used as a method in which a defect calculation unit determines defect coordinates. FIG. 9 is a diagram illustrating a method in which a structure from motion method is used as a method in which a defect calculation unit determines defect coordinates. FIG. 10 is a diagram illustrating an example of data stored in a sensor database. FIG. 11 is a diagram illustrating an example of a method in which a sensor position and orientation calculation unit instructs a worker on the measurement position of a sensor. FIG. 12 is a diagram illustrating an example of a method in which a work robot measures an inspection object using a selected sensor. 10A and 10B are diagrams for explaining a method by which a repairability determining unit determines whether a defect is repairable, and a method by which a worker repairs a defect.

[0012] The defect inspection system and defect inspection method according to the present invention allow the sensor for inspecting defects to be selected according to the type of defect, thereby enabling detailed inspection of various defects occurring in the object to be inspected, and enabling more effective management of the quality and reliability of the object to be inspected.

[0013] Hereinafter, embodiments of the present invention will be described with reference to the drawings. The examples are illustrative for explaining the present invention, and some omissions and simplifications have been made as appropriate for clarity of explanation. The present invention can be implemented in various other forms. Unless otherwise specified, each component may be singular or plural.

[0014] In order to facilitate understanding of the invention, the position, size, shape, range, etc. of each component shown in the drawings may not represent the actual position, size, shape, range, etc. Therefore, the present invention is not necessarily limited to the position, size, shape, range, etc. disclosed in the drawings.

[0015] When there are multiple components having the same or similar functions, they may be described using the same reference numeral with different subscripts. When there is no need to distinguish between these multiple components, the subscripts may be omitted.

[0016] In the embodiments, processing performed by executing a program may be described. Here, a computer executes the program using a processor (e.g., a CPU or a GPU) and performs processing defined by the program using storage resources (e.g., memory) and interface devices (e.g., communication ports). Therefore, the entity performing the processing by executing the program may be the processor. Similarly, the entity performing the processing by executing the program may be a controller, device, system, computer, or node having a processor. The entity performing the processing by executing the program may be any computing unit, and may include a dedicated circuit that performs specific processing. Here, the dedicated circuit may be, for example, an FPGA (Field Programmable Gate Array), an ASIC (Application Specific Integrated Circuit), or a CPLD (Complex Programmable Logic Device).

[0017] A program may be installed on a computer from a program source. The program source may be, for example, a program distribution server or a computer-readable storage medium. When the program source is a program distribution server, the program distribution server may include a processor and storage resources for storing the program to be distributed, and the processor of the program distribution server may distribute the program to be distributed to other computers. In addition, in the embodiments, two or more programs may be realized as one program, or one program may be realized as two or more programs.

[0018] A defect inspection system and a defect inspection method according to a first embodiment of the present invention will be described with reference to the drawings.

[0019] FIG. 1 is a diagram showing an example of the configuration of a defect inspection system according to this embodiment.

[0020] The defect inspection system according to this embodiment includes a camera 100, a motion capture camera 102, a processing device 120, and a display device 130, and inspects an inspection target 101 for defects. The defect inspection system according to this embodiment may or may not include multiple sensors and work robots. Note that in FIG. 1, the processing device 120 is depicted as being divided into two parts for ease of understanding.

[0021] In this embodiment, defect inspection (measurement) using sensors and defect repair are performed by an operator or a work robot. If the defect inspection system according to this embodiment does not include sensors, the operator prepares and uses the sensors, and if the system includes sensors, the work robot uses the sensors. The work robot uses sensors 108 and 109 selected by the defect inspection system from among the multiple sensors included in the defect inspection system. The work robot can attach the sensors selected by the defect inspection system using, for example, a tool changer.

[0022] In this embodiment, an example will be described in which the worker or the work robot uses two sensors 108 and 109, but the worker or the work robot may use one or three or more sensors.

[0023] The inspection object 101 may be, for example, an industrial product or a part thereof that has a defect that occurred during the manufacturing process, or may be an industrial product or a part thereof that has been collected after use and has a defect that occurred during use.

[0024] The camera 100 captures an image of the inspection object 101 and obtains a captured image of the inspection object 101. The image captured by the camera 100 is input to the processing device 120.

[0025] Reflective markers 103 for motion capture are attached to the camera 100 and the inspection object 101 .

[0026] The motion capture camera 102 captures images of the camera 100 and the reflective markers 103 attached to the inspection object 101. The images captured by the motion capture camera 102 are input to the processing device 120.

[0027] The processing device 120 is configured with a computer or other such computing device, and includes a defect detection unit 104, a camera position and orientation calculation unit 105, a defect calculation unit 106, a sensor selection unit 107, a sensor database 500, a sensor position and orientation calculation unit 114, a measurement result input unit 112, a repair feasibility determination unit 116, a threshold database 115, and a repair amount database 113. The sensor database 500, the threshold database 115, and the repair amount database 113 may be included in a single database.

[0028] The defect detection unit 104, the camera position and orientation calculation unit 105, the defect calculation unit 106, the sensor selection unit 107, and the sensor database 500 are components for detecting defects in the inspection object 101 and selecting sensors 108 and 109 according to the type of defect (defect type).

[0029] The defect detection unit 104 inputs an image captured by the camera 100 (a captured image of the inspection object 101). The defect detection unit 104 performs image processing on the input image to detect defects in the inspection object 101 and determine the position of the defect within the image (two-dimensional coordinates in the captured image of the inspection object 101). Furthermore, the defect detection unit 104 identifies the type of the detected defect from the captured image of the inspection object 101. The type of defect (defect type) is, for example, a scratch or rust. Other examples of defect types will be described later. A method by which the defect detection unit 104 detects defects and determines the defect type will be described later.

[0030] The camera position and orientation calculation unit 105 receives an input of an image captured by the motion capture camera 102 (an image of the camera 100 and the reflective markers 103 attached to the inspection object 101). The camera position and orientation calculation unit 105 calculates the position and orientation of the camera 100 relative to the inspection object 101 from the input image.

[0031] The defect calculation unit 106 inputs the processing results of the defect detection unit 104 and the camera position and orientation calculation unit 105, and determines what type of defect is located at what position on the inspection object 101 from the position of the defect in the image, the type of this defect (defect type), and the position and orientation of the camera 100 relative to the inspection object 101. Hereinafter, the position (three-dimensional coordinates) of the defect on the inspection object 101 will be referred to as defect coordinates. The defect calculation unit 106 determines the defect coordinates from the image captured by the camera 100 (image of the defect in the inspection object 101) and the position and orientation of the camera 100 relative to the inspection object 101. Then, the defect calculation unit 106 determines the defect type at these defect coordinates.

[0032] The defect calculation unit 106 can associate defect coordinates with defect types to create a defect map of the inspection object 101. As shown in FIG. 1 , for example, the defect map identifies the positions of defects by three-dimensional coordinates (defect coordinates) on the inspection object 101 and indicates the defect types (for example, scratches or rust) at these positions.

[0033] The sensor selection unit 107 refers to the data stored in the sensor database 500 and selects the sensors 108 and 109 to be used for detailed inspection of the defect depending on the type of defect detected by the defect detection unit 104 .

[0034] The sensor selection unit 107 notifies the worker of the selected sensors 108, 109 by displaying the selected sensors 108, 109 on the display device 130. The sensor selection unit 107 also notifies the work robot of the selected sensors 108, 109 by transmitting a signal representing the selected sensors 108, 109 to the work robot.

[0035] The sensor database 500 stores in advance data indicating the relationship between the defect types and the sensors 108 and 109 used for detailed defect inspection. The relationship between the defect types and the sensors 108 and 109 used for detailed defect inspection is determined in advance.

[0036] The sensor position and orientation calculation unit 114, the repair feasibility determination unit 116, the threshold database 115, and the repair amount database 113 are components for performing detailed defect inspection and defect repair using sensors 108 and 109 selected according to the defect type.

[0037] In this embodiment, an example will be described in which the inspection target 101 has two defects. The defect type of one defect is a scratch, and a roughness meter (surface roughness measuring instrument) is selected as the sensor 108 for this defect type. The defect type of the other defect is rust, and a film thickness meter is selected as the sensor 109 for this defect type.

[0038] The positions (coordinates) of defects to be inspected or repaired using the sensors 108 and 109 are indicated by the defect coordinates calculated by the defect calculation unit 106 .

[0039] The sensors 108 and 109 selected by the sensor selection unit 107 are fitted with reflective markers 103 for motion capture.

[0040] The motion capture camera 102 captures images of the sensors 108 and 109 and the reflective marker 103 attached to the inspection object 101. The images captured by the motion capture camera 102 are input to a processing device 120.

[0041] The sensor position and orientation calculation unit 114 calculates the positions and orientations of the sensors 108 and 109 relative to the inspection object 101 when performing detailed defect inspection using the sensors 108 and 109 selected by the sensor selection unit 107. The sensor position and orientation calculation unit 114 inputs images captured by the motion capture camera 102 (images of the sensors 108 and 109 and the reflective markers 103 attached to the inspection object 101). The sensor position and orientation calculation unit 114 calculates the positions and orientations of the sensors 108 and 109 relative to the inspection object 101 from the input images.

[0042] The sensor position and orientation calculation unit 114 calculates the difference between the calculated positions of the sensors 108 and 109 and the position of the defect (defect coordinates) calculated by the defect calculation unit 106, and instructs the worker or work robot on the measurement positions of the sensors 108 and 109.

[0043] The worker or the work robot adjusts the positions of the sensors 108 and 109 to reduce this difference, and the sensors 108 and 109 inspect (measure) the defect at the position indicated by the defect coordinates. The worker or the work robot then inspects (measures) the defect using the sensors 108 and 109.

[0044] The measurement result input unit 112 inputs the size of the defect obtained by the sensors 108 and 109 as the measurement result. For example, if the sensor 108 is a roughness meter, the sensor 108 measures the location of the scratch that is the defect and outputs the measurement result (area and depth of the scratch) to the processing device 120 via the controller 110 of the sensor 108. For example, if the sensor 109 is a film thickness meter, the sensor 109 measures the location of the rust that is the defect and outputs the measurement result (area and thickness of the rust) to the processing device 120 via the controller 111 of the sensor 109. The measurement result input unit 112 inputs the area and depth of the scratch or the area and thickness of the rust as the size of the defect.

[0045] The repairability determination unit 116 associates the measurement results (defect size) of the sensors 108, 109 with the positions of the sensors 108, 109 when they obtained the measurement results, and the position of the defect (defect coordinates). The measurement results of the sensors 108, 109 are obtained from a measurement result input unit 112. The positions of the sensors 108, 109 are the positions of the sensors 108, 109 relative to the inspection target 101, and are obtained from a sensor position and orientation calculation unit 114. The defect coordinates are obtained from the defect calculation unit 106.

[0046] Then, the repairability determining unit 116 determines whether or not the defect is repairable, using the size of the defect measured by the sensors 108 and 109 and the threshold value stored in the threshold value database 115 .

[0047] The threshold database 115 pre-stores a threshold for determining whether a defect is repairable for each type of defect. This threshold can be arbitrarily determined in advance, for example, based on information about defects that have occurred in the past. For example, this threshold is a threshold for the size of the defect. For example, if the defect is a scratch, the repairable depth of the scratch can be determined from information about the past defects, and the threshold can be determined based on the determined depth of the scratch. The threshold may also include a threshold for the cost of repairing the defect (repair cost). The repair cost can be arbitrarily determined in advance, for example, based on the type and size of the defect. The threshold for the repair cost can be arbitrarily determined in advance, for example, based on the repair cost required to repair the defect in the past.

[0048] The threshold database 115 may use a threshold determined by simulation as the threshold for the defect size. For example, the performance of the inspection object 101 when it has a defect is calculated by simulation, and the maximum defect size at which the inspection object 101 can maintain the required performance is determined. The maximum defect size at which the inspection object 101 can maintain the required performance is then set as the threshold for the defect size.

[0049] The repairability determination unit 116 determines that the defect is repairable if the size of the defect measured by the sensors 108 and 109 is smaller than the threshold value for the defect size stored in the threshold database 115. The repairability determination unit 116 also calculates the cost of repairing the defect based on the type and size of the defect according to a predetermined procedure, and determines that the defect is repairable if this cost is smaller than the threshold value for the repair cost stored in the threshold database 115. The repairability determination unit 116 then determines that the defect is not repairable if the size of the defect or the cost of repairing the defect is equal to or greater than the respective threshold value.

[0050] If the repair feasibility determination unit 116 determines that the defect is repairable, it instructs the worker or the work robot on the location of the defect (i.e., the location to be repaired on the inspection object 101) and the amount of repair required to repair the defect. The repair feasibility determination unit 116 references the repair amount stored in the repair amount database 113 and determines the amount of repair for the defect according to the size of the defect.

[0051] The repair amount database 113 stores in advance data indicating the relationship between the size of the defect (for example, the depth of the scratch or the thickness of the rust) and the repair amount for repairing the defect for each type of defect. The relationship between the size of the defect and the repair amount is predetermined as desired. For example, if the type of defect is a scratch, the relationship between the depth of the scratch (the size of the defect) and the amount of buildup (the repair amount) is predetermined. Also, if the type of defect is rust, the relationship between the thickness of the rust (the size of the defect) and the amount of rust removal (the repair amount) is predetermined.

[0052] 1 schematically shows an example of instructions 117 given to a worker or a work robot by the repair feasibility determination unit 116. The instructions 117 shown in Fig. 1 include the positions of defects 301 and 302, which are repair locations in the inspection object 101, a repair amount 118 (amount of build-up: 25 µm) for the defect 301, which is a scratch, and a repair amount 119 (amount of rust removal: 15 µm) for the defect 302, which is rust.

[0053] The repair possibility determination unit 116 communicates instructions 117 to the worker by displaying the positions of the defects 301, 302 and the repair amounts 118, 119 on the display device 130. The repair possibility determination unit 116 also communicates instructions 117 to the work robot by transmitting signals indicating the positions of the defects 301, 302 and the repair amounts 118, 119.

[0054] If the defect is repairable, the worker or the work robot repairs the defect in accordance with the defect location (repair location on the inspection object 101) and repair amount instructed by the repair feasibility determination unit 116 of the processing device 120. To repair the defect, a tool selected based on the defect location, type, size, etc. can be used. If the defect is not repairable, the worker or the work robot discards the inspection object 101.

[0055] The display device 130 may be, for example, one or both of a display and augmented reality (AR) glasses worn by the worker, and is connected to the processing device 120 via a wired or wireless connection. The processing device 120 can instruct the worker to inspect (measure) the defects and repair them by displaying the selected sensors 108 and 109, the determined position, type, and size of the defects, whether or not the defects can be repaired, and the determined amount of repair on the display device 130. The display device 130 can also display the difference between the positions of the sensors 108 and 109 and the positions of the defects, as determined by the sensor position and orientation calculation unit 114, and instruct the worker on the measurement positions of the sensors 108 and 109. For example, the display device 130 can display the instruction 117 shown in FIG. 1 .

[0056] Based on the information displayed on the display device 130, the worker can inspect (measure) the defect using the selected sensors 108, 109 and repair the defect.

[0057] When the defect inspection system according to this embodiment includes sensors 108, 109 and a work robot, the work robot can inspect (measure) defects and repair the defects using the selected sensor 108, 109 in accordance with instructions from the processing device 120. The processing device 120 can inform the work robot of the selected sensor 108, 109, the determined position, type, and size of the defect, whether or not the defect can be repaired, the determined amount of repair, etc.

[0058] 2 is a diagram showing a process flow of the defect inspection method according to the first embodiment of the present invention. The process flow of the defect inspection method according to this embodiment will be described with reference to FIG.

[0059] In step 201, a worker or a work robot places an inspection target 101 at a desired position.

[0060] In step 202, the camera 100 captures an image of the inspection object 101 by operation of a worker or a work robot.

[0061] In step 203, the defect detection unit 104 of the processing device 120 performs image processing on the image captured by the camera 100 (captured image of the inspection object 101) and detects defects in the inspection object 101 from the captured image. Then, the defect detection unit 104 obtains the approximate size of the detected defect (for example, the area and depth of the defect) through image processing.

[0062] In step 204, the defect detection unit 104 determines whether the defect detected in step 203 is a fatal defect. A fatal defect is a defect that is too large to be repaired (for example, a scratch with a depth of 1 mm or more). The inspection object 101 that has a fatal defect is not subjected to detailed inspection using a sensor in a subsequent step. If the size of the defect detected in step 203 is larger than a predetermined threshold, the defect detection unit 104 determines that the defect is a fatal defect, and if the size is equal to or smaller than the predetermined threshold, the defect detection unit 104 determines that the defect is not a fatal defect.

[0063] For defects that are not fatal defects, the process of step 205 is executed, whereas for defects that are fatal defects, the process of step 213 is executed.

[0064] In step 205, the defect detection unit 104 determines the position of the defect in the image (two-dimensional coordinates in the captured image of the inspection object 101) and identifies the type of the defect for the defect detected in step 203 from the captured image of the inspection object 101. Next, the defect calculation unit 106 determines the defect coordinates, i.e., the position of the defect in the inspection object 101 (three-dimensional coordinates), and associates the defect coordinates with the defect type.

[0065] In step 206, the sensor selection unit 107 selects the sensors 108 and 109 to be used for detailed inspection of the defects according to the defect type identified by the defect detection unit 104.

[0066] In step 207, the sensor position and orientation calculation unit 114 calculates the positions and orientations of the sensors 108 and 109 relative to the inspection object 101, and instructs the worker or work robot on the measurement positions of the sensors 108 and 109 based on the calculated positions of the sensors 108 and 109 and the defect coordinates calculated by the defect calculation unit 106.

[0067] In step 208, the worker or the work robot uses sensors 108 and 109 to measure the defect (the defect detected in step 203) in inspection object 101 at the specified measurement position. In step 208, a detailed inspection (measurement) of the defect is performed. That is, in the measurement in step 208, sensors 108 and 109 measure the size of a defect smaller than the size of the defect determined in step 203 (for example, a scratch with a depth of less than 1 mm).

[0068] In step 209, the repairability determining unit 116 determines whether the defect detected in step 203 is repairable based on the measurement results of the sensors 108 and 109.

[0069] For repairable defects, the process proceeds to step 210. For non-repairable defects, the process proceeds to step 213.

[0070] In step 210, the repair possibility determination unit 116 instructs the worker or the work robot on the location of the defect (the location to be repaired on the inspection object 101) and the amount of repair required to repair this defect.

[0071] In step 211, the worker or the work robot repairs the defect in accordance with the instructions of the repair possibility determination unit 116.

[0072] When the worker or the work robot finishes repairing the defect in step 212, the processing of the defect inspection method according to this embodiment is completed.

[0073] In step 213, the worker or the work robot discards the inspection target 101.

[0074] 3A and 3B are diagrams for explaining a method in which the defect detection unit 104 detects defects and determines the defect type from the image captured by the camera 100 (captured image of the inspection object 101).

[0075] The defect detection unit 104 detects defects from the image captured by the camera 100 by image processing, and identifies the defect type based on the characteristics corresponding to the defect type recorded in the captured image.

[0076] 3A is a diagram showing an example of a portion of a captured image including a defect 301, which is a scratch. If the defect 301 is a scratch, a change in brightness occurs at the edge of the defect 301 in the image captured by the camera 100 due to illumination light. The defect detection unit 104 can detect the defect 301, which is a scratch, by detecting the edge, for example, through image processing. Furthermore, if a change in brightness is observed at the edge of the defect in the captured image, the defect detection unit 104 determines that the type of this defect is a scratch.

[0077] 3B is a diagram showing an example of a portion of a captured image including a defect 302 that is rust. If the defect 302 is rust, there will be a difference in color between the surface (a portion where no rust has occurred) of the inspection object 101 and the rust in the captured image by the camera 100. The defect detection unit 104 can detect the rust defect 302 by detecting such a color difference, for example, through image processing. Furthermore, if there is a difference in color between the surface (a portion where no defect has occurred) of the inspection object 101 and the defect in the captured image, the defect detection unit 104 determines that the type of defect is rust.

[0078] The inspection object 101 may have a defect inside it. This internal defect is one type of defect. The image captured by the camera 100 may not show the characteristics of the internal defect. In such a case, it is possible to determine in advance by simulation where the internal defect is likely to occur, and to designate this location as the repair location of the inspection object 101. In this simulation, the locations where force or heat is applied are determined, and the determined locations are designated as locations where stress acts and internal defects are likely to occur.

[0079] If the defect type is an internal defect, the defect is inspected using, for example, an ultrasonic sensor (see FIG. 5).

[0080] FIG. 3C is a diagram showing an example of a repair location 303 of the inspection object 101 obtained by simulation.

[0081] Furthermore, it is also possible to identify locations and defect types in the inspection object 101 where defects are likely to occur based on the history of the inspection object 101, and to designate these locations as repair locations for the inspection object 101. For example, it is also possible to identify locations and defect types in the inspection object 101 where defects are likely to occur based on the past usage record of the inspection object 101 and the history of past defect repairs, and to designate these locations as repair locations for the inspection object 101.

[0082] FIG. 3D is a diagram showing an example of a repair location 304 of the inspection object 101 determined based on the history of the inspection object 101.

[0083] 4A, 4B, and 4C, an example of a method in which the defect calculation unit 106 calculates defect coordinates (the position of a defect in the inspection object 101) will be described.

[0084] FIG. 4A is a diagram showing a method using motion capture as a method for the defect calculation unit 106 to calculate defect coordinates.

[0085] Motion capture reflective markers 103 are attached to the camera 100 and the inspection object 101. A motion capture camera 102 captures images of the camera 100 and the inspection object 101. A camera position and orientation calculation unit 105 calculates the position and orientation 400 of the camera 100 relative to the inspection object 101 from the images captured by the motion capture camera 102.

[0086] The defect calculation unit 106 calculates the three-dimensional position 402 of the defect in the object to be inspected 101 as defect coordinates from the position 401 of the defect in the image captured by the camera 100 obtained by the defect detection unit 104 and the position and orientation 400 of the camera 100 relative to the object to be inspected 101 obtained by the camera position and orientation calculation unit 105.

[0087] FIG. 4B is a diagram showing a method in which the defect calculation unit 106 calculates defect coordinates using two cameras 100a and 100b.

[0088] The two cameras 100a and 100b may be the same or different and are installed at different positions. The two cameras 100a and 100b capture images of the same location on the inspection object 101.

[0089] The defect calculation unit 106 determines the three-dimensional position of the defect in the inspection object 101 from the difference in the viewing angles of the images captured by the two cameras 100a and 100b, for example, using a stereo imaging method, and calculates the defect coordinates.

[0090] FIG. 4C is a diagram showing a method in which the defect calculation unit 106 uses the structure-from-motion method to calculate defect coordinates.

[0091] The camera 100 captures an image of the inspection object 101 multiple times while shifting the relative position of the inspection object 101 with respect to the camera 100 .

[0092] The defect calculation unit 106 obtains the three-dimensional position of the defect in the inspection object 101 from the multiple images captured by the camera 100 by a structure from motion (SfM) method, and calculates the defect coordinates. Fig. 4C shows an example in which the shape of the inspection object 101 is rotationally symmetric. For an inspection object 101 having such a shape, the relative position of the inspection object 101 with respect to the camera 100 may be shifted by rotating the inspection object 101 about the axis of rotational symmetry of the inspection object 101.

[0093] 5 is a diagram showing an example of data stored in the sensor database 500. The relationship between the defect type and the sensor used for detailed defect inspection is predetermined and stored in the sensor database 500 in advance.

[0094] The sensor selection unit 107 references the data stored in the sensor database 500 and selects a sensor to be used for detailed defect inspection depending on the type of defect detected by the defect detection unit 104. The sensor database 500 may define multiple sensors for one defect type. When the sensor database 500 defines multiple sensors for one defect type, the sensor selection unit 107 may select the optimal sensor from the multiple sensors in accordance with the required inspection specifications, taking into account conditions such as measurement accuracy, measurement range, measurement speed, and equipment cost. These conditions are defined in advance and stored in the sensor database 500. The worker or the work robot inputs the necessary conditions from among these conditions into the sensor selection unit 107.

[0095] For example, for defects whose defect type is surface wear, a three-dimensional shape measuring device is specified as a sensor to be used for detailed inspection of the defect. The three-dimensional shape measuring device measures the surface shape of the inspection object 101 to determine the degree of wear of the surface of the inspection object 101. Examples of three-dimensional shape measuring devices that can be used include a sensor that projects a stripe pattern to measure the three-dimensional shape, a light-cutting sensor that measures the three-dimensional shape, and a laser rangefinder that scans two-dimensionally.

[0096] Furthermore, for example, for defects whose defect type is scratches and defects whose defect type is surface roughness, contact-type roughness gauges and non-contact-type roughness gauges are specified as sensors to be used for detailed defect inspection. Contact-type roughness gauges have the advantages of high measurement accuracy and low equipment costs, but have the disadvantages of a narrow measurement range and slow measurement speed. Non-contact-type roughness gauges are, for example, optical types, and have the advantages of a wider measurement range and faster measurement speed than contact-type types, but have the disadvantage of high equipment costs.

[0097] Furthermore, for example, for defects in which the defect type is a decrease in the film thickness of the surface coating of the object to be inspected 101 and defects in which the defect type is rust, a contact film thickness gauge and a non-contact film thickness gauge are specified as sensors to be used for detailed inspection of the defect.

[0098] Furthermore, for example, for defects whose defect type is an internal defect, ultrasonic sensors, X-ray inspection devices, and sensors for eddy current testing (ECT) are specified as sensors to be used for detailed inspection of the defect.

[0099] 6A is a diagram for explaining an example of a method in which the sensor position and orientation calculation unit 114 instructs the operator 600 on the measurement position of the sensor. Using FIG. 6A , an example of a method in which the sensor position and orientation calculation unit 114 calculates the position and orientation of the sensor with respect to the inspection target 101 and instructs the operator 600 on the measurement position of the sensor will be explained. Below, as an example, an example will be explained in which the defect type is a scratch and the sensor 108 is selected.

[0100] The worker 600 attaches a reflective marker 103 for motion capture to the selected sensor 108 .

[0101] The motion capture camera 102 captures images of the reflective markers 103 attached to the sensor 108 and the reflective markers 103 attached to the inspection object 101 .

[0102] The sensor position and orientation calculation unit 114 calculates the position and orientation of the sensor 108 relative to the inspection target 101, and determines a difference 601 between the calculated position of the sensor 108 and the position of the defect (defect coordinates) determined by the defect calculation unit 106. If the difference 601 is greater than a predetermined threshold, the sensor position and orientation calculation unit 114 displays on the display device 130 a work instruction 602 to adjust the position of the sensor 108 so as to reduce the difference 601. In this way, the sensor position and orientation calculation unit 114 instructs the operator 600 on the measurement position of the sensor 108.

[0103] When the difference 601 is equal to or smaller than a predetermined threshold, the sensor position and orientation calculation unit 114 displays on the display device 130 an instruction to the operator 600 to measure the inspection target 101 using the sensor 108 .

[0104] When the work robot measures the inspection target 101 using the selected sensor 108, the following process may be performed.

[0105] 6B is a diagram illustrating an example of a method in which the work robot 603 measures the inspection object 101 using the selected sensor 108. The work robot 603 may measure the inspection object 101 using the selected sensor 108 in accordance with the following procedure.

[0106] Defect calculation unit 106 outputs the determined defect coordinates 604 as a measurement position to work robot 603. Work robot 603 moves sensor 108 to the position of input defect coordinates 604 and measures inspection object 101.

[0107] By performing the example process described with reference to Figures 6A and 6B, detailed inspection of defects can be performed using the selected sensor.

[0108] FIG. 7 is a diagram for explaining a method by which the repairability determining unit 116 determines whether a defect is repairable or not.

[0109] If the size of the defect input from the measurement result input unit 112 is smaller than the threshold value for the size of the defect stored in the threshold database 115, the repairability determination unit 116 determines that the defect is repairable.

[0110] The repairability determination unit 116 can also determine whether a defect is repairable by taking into account the cost of repairing the defect. For example, if a defect is of a repairable size but the cost of repairing it would be high due to its location on the inspection object 101, the repairability determination unit 116 determines that the defect is unrepairable. As described above, the threshold database 115 can include a threshold for the cost of repairing a defect (repair cost). If the repair cost of the defect is equal to or greater than the threshold, the repairability determination unit 116 can determine that the defect is unrepairable.

[0111] FIG. 8 is a diagram for explaining a method in which the worker 600 repairs the defect.

[0112] 1, the repair possibility determination unit 116 displays instructions 117 on the display device 130 to instruct the worker 600 to repair the defects. For example, the instructions 117 include the positions of the defects 301 and 302, which are the repair locations of the inspection object 101, a repair amount 118 (amount of build-up: 25 μm) for the defect 301, which is a scratch, and a repair amount 119 (amount of rust removal: 15 μm) for the defect 302, which is rust.

[0113] It is assumed that a worker 600 repairs a rust defect 302 using a tool 801. The worker 600 attaches a reflective marker 103 for motion capture to the tool 801.

[0114] The motion capture camera 102 captures images of the reflective marker 103 attached to the tool 801 and the reflective marker 103 attached to the inspection object 101 .

[0115] The sensor position and orientation calculation unit 114 calculates the position of the tool 801 in the same way as it calculates the position and orientation of the sensor 108, and calculates a difference 802 between the calculated position of the tool 801 and the position of the defect (defect coordinates) calculated by the defect calculation unit 106. If the difference 802 is greater than a predetermined threshold, the sensor position and orientation calculation unit 114 displays on the display device 130 a work instruction 803 to adjust the position of the tool 801 so as to reduce the difference 802. In this way, the sensor position and orientation calculation unit 114 instructs the worker 600 on the work position of the tool 801 (the location to repair on the inspection target 101).

[0116] If the difference 802 is equal to or smaller than a predetermined threshold, the sensor position and orientation calculation unit 114 displays on the display device 130 an instruction to the worker 600 to repair the defect.

[0117] As described above, the defect inspection system and defect inspection method according to this embodiment can select the sensor to inspect the defect depending on the type of defect, and can perform detailed inspection and repair of various defects that occur in the object 101 to be inspected.

[0118] 100, 100a, 100b...camera, 101...inspection object, 102...motion capture camera, 103...reflective marker, 104...defect detection unit, 105...camera position and orientation calculation unit, 106...defect calculation unit, 107...sensor selection unit, 108, 109...sensor, 110, 111...controller, 112...measurement result input unit, 113...repair amount database, 114...sensor position and orientation calculation unit, 115...threshold value database, 116...repair possibility determination unit, 117...instruction, 118, 119...repair amount , 120...processing device, 130...display device, 301, 302...defect, 303, 304...repair location, 400...position and attitude of camera relative to inspection object, 401...position of defect in image captured by camera, 402...three-dimensional position of defect in inspection object, 500...sensor database, 600...worker, 601...difference between sensor position and defect coordinates, 602...work instructions, 603...work robot, 604...defect coordinates, 801...tool, 802...difference between tool position and defect coordinates, 803...work instructions.

Claims

1. A defect inspection system comprising: a camera that captures an image of an object to be inspected to obtain an image of the object to be inspected; and a processing device comprising a database, a defect detection unit, and a sensor selection unit, wherein the database stores sensor data indicating a relationship between a type of defect in the object to be inspected and a sensor to be used to inspect the defect, the defect detection unit performs image processing to detect the defect from the captured image and determines the type of the defect, and the sensor selection unit selects a sensor to be used to inspect the defect according to the type of defect based on the sensor data stored in the database.

2. The defect inspection system according to claim 1, wherein the processing device includes a defect calculation unit, and the defect calculation unit determines the position of the defect in the object to be inspected from the captured image.

3. The defect inspection system described in claim 2, wherein the processing device is equipped with a repair feasibility determination unit, the database stores a threshold value for the size of the defect and repair amount data indicating the relationship between the size of the defect and the repair amount for repairing the defect, the repair feasibility determination unit determines that the defect is repairable if the size of the defect is smaller than the threshold value stored in the database, and if the defect is repairable, the repair feasibility determination unit refers to the repair amount data stored in the database to determine the repair amount.

4. The defect inspection system according to claim 1, further comprising a display device, wherein the sensor selection unit displays the selected sensor on the display device.

5. A defect inspection system as described in claim 1, further comprising a work robot, wherein the sensor selection unit transmits a signal representative of the selected sensor to the work robot.

6. A defect inspection system according to claim 3, further comprising a display device, wherein the repairability determination unit displays the position of the defect and the amount of repair on the display device.

7. A defect inspection system as described in claim 3, further comprising a work robot, wherein the repair feasibility determination unit transmits a signal representing the position of the defect and the amount of repair to the work robot.

8. A defect inspection method comprising: an imaging step of imaging an object to be inspected with a camera to obtain an image of the object to be inspected; a defect detection step of a processing device performing image processing to detect defects in the object to be inspected from the image to determine the type of the defect; and a sensor selection step of the processing device selecting a sensor to be used for inspecting the defect according to the type of defect based on sensor data indicating the relationship between the type of defect and the sensor to be used for inspecting the defect.

Images