Defect inspection system and defect inspection method

Abstract

To provide a defect inspection system capable of precisely inspecting a defect of an inspection object.SOLUTION: A defect inspection system comprises: a camera 100 for capturing an image of an inspection object 101; a database 500; and a computer 120 having a defect cleaning section 104, a sensor selection section 107, and a measurement position calculation section 108. In the database 500, a sensor used for inspection of a defect of the inspection object 101 is stored along with characteristic data of the sensor according to the type of the defect. The defect screening section 104 obtains the type of the defect and an occurrence range of the defect in the inspection object 101 from the image captured by the camera 100. The sensor selection section 107 refers to the database 500 to select a sensor used for inspection according to the type of the defect. The measurement position calculation section 108 obtains a measurement start position and a measurement ending position in the inspection on the basis of characteristic data and the occurrence range of the defect for the selected sensor.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a system and method for inspecting an object for defects. [Background technology]

[0002] The quality and reliability of industrial products and their parts are controlled by inspecting them for defects such as scratches and rust. An example of a conventional defect inspection system is described in Patent Document 1. The surface defect inspection device described in Patent Document 1 includes an imaging device that captures an image of a planar inspection object, an image processing device that processes the captured image to detect defects on the surface of the inspection object and obtain position information of the defects, and a precision inspection device that precisely inspects the defects on the surface of the inspection object based on the position information of the defects, and uses the precision inspection device to inspect the defects in detail and measure the shape and size of the defects. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2002-168793 Summary of the Invention [Problem to be solved by the invention]

[0004] An object to be inspected may have various types of defects, and the sensor (device) required for detailed inspection of the defects differs depending on the type of defect. Therefore, detailed inspection of an object to be inspected requires a sensor that is appropriate for the type of defect.

[0005] Furthermore, when inspecting for defects in an object, it is necessary to scan the sensor in an appropriate direction according to the distribution of defects. Therefore, in order to inspect defects accurately, it is necessary to appropriately determine the scanning position and scanning direction of the sensor (i.e., the measurement start position and measurement end position).

[0006] However, for example, when the object to be inspected is a recovered product after use, the type and location of defects are unspecified and diverse, and when scanning a recovered product with a sensor to inspect the defect, it is often difficult to determine the scanning position and scanning direction of the sensor in advance using conventional technology. In such cases, appropriately determining the sensor's measurement start and end positions requires skilled work and a considerable amount of time, placing a heavy burden on the worker. Furthermore, for example, recovered products often require 100% inspection, and for these reasons, reducing the burden on the worker is also an issue.

[0007] Conventional techniques have not necessarily been able to provide a defect inspection system that can solve these problems and inspect an object for defects with high accuracy.

[0008] An object of the present invention is to provide a defect inspection system and a defect inspection method that can inspect an object for defects with high accuracy. [Means for solving the problem]

[0009] A defect inspection system according to the present invention comprises a computer including a camera that captures an image of an inspection object, a database, a defect screening unit, a sensor selection unit, and a measurement position calculation unit. The database stores sensors to be used in inspecting defects in the inspection object along with characteristic data of the sensors according to the type of defect. The defect screening unit determines the type of defect and the extent to which the defect occurs in the inspection object from the image captured by the camera. The sensor selection unit refers to the database and selects the sensor to be used in inspecting the defect according to the type of defect determined by the defect screening unit. The measurement position calculation unit determines a measurement start position and a measurement end position for the sensor selected by the sensor selection unit based on the characteristic data acquired from the database and the extent to which the defect occurs determined by the defect screening unit.

[0010] A defect inspection method according to the present invention includes a defect screening step in which a database stores sensors to be used in inspecting defects in an object to be inspected, along with characteristic data of the sensors, according to the type of defect; a computer determines the type of defect and the range of occurrence of the defect in the object to be inspected from an image of the object to be inspected taken by a camera; a sensor selection step in which the computer refers to the database and selects the sensor to be used in inspecting the defect according to the type of defect determined in the defect screening step; and a measurement position calculation step in which the computer determines a measurement start position and a measurement end position for the sensor selected in the sensor selection step, based on the characteristic data acquired from the database and the range of occurrence of the defect determined in the defect screening step. [Effects of the Invention]

[0011] According to the present invention, it is possible to provide a defect inspection system and a defect inspection method that can inspect an object for defects with high accuracy. Problems, configurations, and effects other than those described above will become apparent from the following description of the preferred embodiment of the invention. [Brief explanation of the drawings]

[0012] [Figure 1] 1 is a block diagram showing an outline of the configuration of a defect inspection system according to a first embodiment of the present invention. [Figure 2A] FIG. 1 illustrates a method for determining the position and orientation of a camera relative to an object under inspection using motion capture technology. [Figure 2B] FIG. 10 is a diagram showing a method for determining the position and orientation of a camera relative to an object to be inspected using a rotary stage and a linear stage. [Figure 2C] FIG. 10 is a diagram illustrating a method for determining the position and orientation of a camera relative to an object to be inspected using a work robot. [Figure 3A] FIG. 10 is a diagram showing an example of a portion of a captured image including a defect that is a scratch. [Figure 3B] FIG. 10 is a diagram showing an example of a portion of a captured image including a rust defect. [Figure 3C] FIG. 10 is a diagram showing an example of an internal defect of an inspection object obtained by simulation. [Figure 3D] 10A and 10B are diagrams showing examples of defects in an inspection object determined based on the past usage record of the inspection object; [Figure 4] FIG. 10 is a diagram showing an example of a defect map created by a defect screening unit. [Figure 5] 10 is a flowchart showing a process executed by a sensor selection unit. [Figure 6] FIG. 2 is a diagram illustrating an example of data stored in a database. [Figure 7] FIG. 2 is a top view showing a scratch, which is a defect that has occurred on the surface of an object to be inspected. [Figure 8A] FIG. 1 is a diagram showing a scratch, which is a defect that has occurred on the surface of an object to be inspected, and a sensor that measures the scratch. [Figure 8B] FIG. 1 is a diagram showing a scratch, which is a defect that has occurred on the surface of an object to be inspected, and a sensor that measures the scratch. [Figure 9] FIG. 10 is a diagram showing the field of view and offset height of a sensor when the sensor is an optical two-dimensional area sensor. [Figure 10A] FIG. 10 is a diagram showing an example of a method in which an operator inspects defects occurring in an inspection object using a sensor selected by a sensor selection unit. [Figure 10B] FIG. 10 is a diagram showing an example of a method in which a working robot uses a sensor selected by a sensor selection unit to inspect defects occurring in an inspection object. DETAILED DESCRIPTION OF THE INVENTION

[0013] Hereinafter, examples of embodiments of the present invention will be described with reference to the drawings. The examples are illustrative for explaining the present invention, and for clarity of explanation, appropriate omissions and simplifications have been made. The present invention can be implemented in various other forms. Unless otherwise specified, each component may be singular or plural.

[0014] In order to facilitate understanding of the invention, the position, size, shape, range, etc. of each component shown in the drawings may not represent the actual position, size, shape, range, etc. Therefore, the present invention is not necessarily limited to the position, size, shape, range, etc. disclosed in the drawings.

[0015] In the drawings referred to in this specification, identical or corresponding components are designated by the same reference numerals, and repeated explanations of these components may be omitted. When there are multiple components having the same or similar functions, they may be described by using the same reference numerals with different subscripts. Furthermore, when it is not necessary to distinguish between these multiple components, the subscripts may be omitted.

[0016] In the embodiments, processing performed by executing a program may be described. Here, a computer executes the program using a processor (e.g., a CPU or a GPU) and performs processing defined by the program using storage resources (e.g., a memory) and interface devices (e.g., a communication port). Therefore, the entity performing the processing by executing the program may be the processor. Similarly, the entity performing the processing by executing the program may be a controller, device, system, computer, or node having a processor. The entity performing the processing by executing the program may be any computing unit, and may include a dedicated circuit that performs specific processing. Here, the dedicated circuit may be, for example, an FPGA (Field Programmable Gate Array), an ASIC (Application Specific Integrated Circuit), or a CPLD (Complex Programmable Logic Device).

[0017] A program may be installed on a computer from a program source. The program source may be, for example, a program distribution server or a computer-readable storage medium. When the program source is a program distribution server, the program distribution server may include a processor and a storage resource for storing the program to be distributed, and the processor of the program distribution server may distribute the program to be distributed to other computers. In addition, in an embodiment, two or more programs may be realized as one program, or one program may be realized as two or more programs.

[0018] Furthermore, in the following embodiment examples, the components (including element steps, etc.) are not necessarily essential unless otherwise specified or considered to be clearly essential in principle. Furthermore, when the terms "consist of A," "comprises A," "has A," or "includes A" are used, it goes without saying that they do not exclude other elements unless otherwise specified to include only that element. Similarly, in the following embodiment examples, when referring to the shape, positional relationship, etc. of components, etc., it is intended to include those that are substantially similar or similar to that shape, etc., unless otherwise specified or considered to be clearly otherwise in principle.

[0019] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS A defect inspection system and a defect inspection method according to an embodiment of the present invention will be described below with reference to the accompanying drawings. [Example]

[0020] First Embodiment A defect inspection system and a defect inspection method according to a first embodiment of the present invention will be outlined with reference to FIG.

[0021] 1 is a block diagram showing an outline of the configuration of a defect inspection system according to a first embodiment of the present invention. The defect inspection system according to this embodiment includes a camera 100, a computer 120, and a database 500, and inspects an object for defects. The computer 120 includes a defect screening unit 104, a sensor selection unit 107, and a measurement position calculation unit 108. The defect inspection system according to this embodiment can also include a display device 130 connected to the computer 120.

[0022] The object to be inspected may be, for example, an industrial product or a part thereof that has a defect that occurred during the manufacturing process. Alternatively, the object to be inspected may be an industrial product or a part thereof that has been recovered after use (i.e., a recovered product) that has a defect that occurred during use. Recovered products may have various defects depending on the conditions of use.

[0023] The camera 100 inspects the inspection object in the first inspection step. For example, the camera 100 captures an image of the appearance of the inspection object. The defect inspection system inputs the captured image obtained in the first inspection step into the computer 120.

[0024] The defect screening unit 104 calculates and determines the type of defect and the range of occurrence of the defect on the surface of the object to be inspected.

[0025] The sensor selection unit 107 refers to the database 500 and selects the optimum sensor for inspecting the defect in detail according to the type of defect determined by the defect screening unit 104 .

[0026] The measurement position calculation unit 108 acquires characteristic data from the database 500 for the sensor selected by the sensor selection unit 107, and based on the acquired sensor characteristic data and the defect occurrence range obtained by the defect screening unit 104, determines the measurement start position and measurement end position of the selected sensor so that the selected sensor can scan at an appropriate position and attitude relative to the defect occurrence range.

[0027] In the second inspection step, defects occurring in the object to be inspected are inspected in detail using the sensor selected by the sensor selection unit 107, according to the measurement start position and measurement end position calculated by the measurement position calculation unit .

[0028] If the defect screening unit 104 finds multiple types of defects or if the sensor selection unit 107 selects multiple types of sensors, the second inspection step is performed multiple times.

[0029] The defect inspection system and defect inspection method according to this embodiment will be described in detail below.

[0030] <First inspection process> 2A to 2C are diagrams showing the details of the first inspection step. In the first inspection step, a camera 100 is operated by a worker or a work robot to inspect an inspection object 101. The camera 100 is a sensor that inspects the appearance of the inspection object 101, and acquires, for example, an image of the appearance of the inspection object 101.

[0031] The camera 100 acquires an image of the appearance of the inspection object 101 as a captured image while changing the relative position and attitude of the camera 100 and the inspection object 101. The camera 100 may acquire an image of the entire appearance of the inspection object 101, or may acquire an image of the appearance of all areas in which defects are expected to occur depending on the intended use of the inspection object 101.

[0032] The computer 120 (FIG. 1) inputs an image of the appearance of the inspection object 101 acquired by the camera 100 (i.e., an image captured by the camera 100) in the first inspection step. The computer 120 can also determine the position and orientation 105 of the camera 100 with respect to the inspection object 101 in the first inspection step. For example, the computer 120 can determine the relative position and orientation 105 of the camera 100 with respect to the inspection object 101 by determining the respective positions and orientations of the camera 100 and the inspection object 101.

[0033] An example of a method for determining the position and orientation 105 of the camera 100 relative to the inspection object 101 will be described with reference to FIGS. 2A to 2C.

[0034] FIG. 2A is a diagram illustrating how the position and orientation 105 of a camera 100 relative to an object under inspection 101 can be determined using motion capture techniques.

[0035] The motion capture technology uses reflective markers 103 for motion capture and multiple motion capture cameras 102. The reflective markers 103 for motion capture are attached to a camera 100 and an object under inspection 101. The multiple motion capture cameras 102 capture images of the reflective markers 103. A computer 120 processes data using the positions of the reflective markers 103 captured by the motion capture camera 102 to determine the position and orientation 105 of the camera 100 relative to the object under inspection 101.

[0036] The method of using motion capture technology to determine the position and orientation 105 of the camera 100 relative to the object under inspection 101 is an effective method from the viewpoint of work efficiency when capturing images of multiple objects under inspection 101 with a wide range of shapes or when an operator holds the camera 100 to capture images.

[0037] FIG. 2B is a diagram showing a method for determining the position and orientation 105 of the camera 100 relative to the inspection object 101 using the rotation stage 110 and the linear stage 109.

[0038] A rotation stage 110 or a linear stage 109 is installed on at least one of the camera 100 and the object to be inspected 101. A computer 120 calculates the position and orientation 105 of the camera 100 relative to the object to be inspected 101 based on coordinate command values ​​for the rotation stage 110 and the linear stage 109 or values ​​indicated by encoders attached to these stages.

[0039] FIG. 2B shows an example in which a linear stage 109 is installed on the camera 100, and a rotation stage 110 is installed on the inspection object 101 via a jig 112.

[0040] This method is effective when inspecting multiple inspection objects 101 of the same shape, or when a part of the inspection object 101 that is manufactured with high manufacturing precision (for example, the rotation axis of a rotating part) is fixed and supported on a jig 112, and it allows the position and attitude 105 of the camera 100 relative to the inspection object 101 to be determined with high accuracy and high reproducibility.

[0041] FIG. 2C is a diagram showing a method for determining the position and orientation 105 of the camera 100 relative to the inspection object 101 using the working robot 603.

[0042] The working robot 603 holds the camera 100 and captures an image of the inspection object 101 with the camera 100. Alternatively, the working robot 603 holds the inspection object 101 and moves the inspection object 101 so that the camera 100 can capture an image of the inspection object 101. The computer 120 calculates the position and orientation 105 of the camera 100 relative to the inspection object 101 based on a command value to the working robot 603 or a value indicated by an encoder provided on the working robot 603.

[0043] FIG. 2C shows an example in which working robot 603 holds camera 100 and captures an image of inspection object 101 with camera 100.

[0044] As will be described later, the defect screening unit 104 (FIG. 1) can determine the defect occurrence range 401 on the image from the image of the appearance of the inspection object 101 obtained in the first inspection step. Also, as will be described later, the defect screening unit 104 can determine the three-dimensional position 402 of the defect on the inspection object 101 from the defect occurrence range 401 on the image and the position and orientation 105 of the camera 100 with respect to the inspection object 101 obtained in the first inspection step.

[0045] <Defect Screening Department> The defect screening unit 104 receives the image captured by the camera 100 in the first inspection step (i.e., an image of the appearance of the inspection target 101) and calculates and determines the type of defect and the range 401 in which the defect occurs on the image from the image captured by the camera 100. The type of defect and the range 401 in which the defect occurs on the image are included in the defect detection results of the defect screening unit 104. Typical types of defects (defect types) include scratches, rust, surface wear, film thickness reduction, surface roughness, and internal defects.

[0046] Furthermore, the defect screening unit 104 can determine the three-dimensional positions 402 of defects in the inspection object 101 based on the position and orientation 105 of the camera 100 relative to the inspection object 101 obtained in the first inspection step and the defect detection results. Then, the defect screening unit 104 creates a defect map indicating the types of defects and the three-dimensional positions 402 of the defects in the three-dimensional model data of the inspection object 101.

[0047] The defect screening unit 104 can determine the range in which the defect occurs in the inspection object 101 from the three-dimensional position 402 of the defect in the inspection object 101 .

[0048] First, a process will be described in which the defect screening unit 104 determines the type of defect and the range 401 in which the defect occurs on the image from the image captured by the camera 100 (an image of the appearance of the inspection object 101). In this embodiment, the defect screening unit 104 determines the type of defect and the range 401 in which the defect occurs on the image based on color information and brightness changes.

[0049] 3A and 3B are diagrams for explaining a method in which the defect screening unit 104 detects defects and determines the types of defects (defect types) from the images captured by the camera 100. FIG.

[0050] The defect screening unit 104 detects defects from the image captured by the camera 100 by image processing, and identifies the defect type based on the characteristics corresponding to the defect type recorded in the image.

[0051] 3A is a diagram showing an example of a portion of a captured image including a defect 301, which is a scratch. If the defect 301 is a scratch, a change in brightness occurs at the edge of the defect 301 in the image captured by the camera 100 due to illumination light. The defect screening unit 104 can detect the defect 301, which is a scratch, by detecting the edge, for example, through image processing, and can determine the area 401 in which the defect occurs on the image. Furthermore, if a change in brightness is observed at the edge of the defect in the captured image, the defect screening unit 104 determines that the type of defect is a scratch.

[0052] FIG. 3B is a diagram showing an example of a portion of a captured image containing a rust defect 302. If the defect 302 is rust, there will be a difference in color between the rust and the surface (non-rust portion) of the inspection object 101 in the captured image by the camera 100. The defect screening unit 104 can detect the rust defect 302 by detecting such a color difference, for example, through image processing, and can determine the defect occurrence range 401 in the image. In image processing, the color difference is detected by classifying each pixel in an RGB color space, for example. Furthermore, if there is a difference in color between the surface (non-defect portion) of the inspection object 101 and the defect in the captured image, the defect screening unit 104 determines that the type of defect is rust.

[0053] The defect screening unit 104 is not limited to the examples described above, but can detect defects using general image processing methods including classification by machine learning, and can determine the type of defect and the range 401 where the defect occurs on the image.

[0054] The inspection object 101 may have an internal defect. This internal defect is one type of defect, but the characteristics of the internal defect do not appear in the image captured by the camera 100, making it difficult to detect from the image of the exterior of the inspection object 101.

[0055] For such internal defects, locations where they are likely to occur can be determined in advance through simulation and set as the defect occurrence range 401. In this simulation, locations where force or heat is applied are determined, and the determined locations are set as locations where stress acts and internal defects are likely to occur.

[0056] If the defect type is an internal defect, the defect is inspected using, for example, an eddy current testing (ECT) sensor, an ultrasonic sensor, an X-ray inspection device, or the like.

[0057] FIG. 3C is a diagram showing an example of a defect 303 inside the inspection object 101 obtained by simulation.

[0058] Furthermore, the locations where defects are likely to occur in the inspection object 101 and the types of defects can be identified based on the past usage history of the inspection object 101, and these locations can be set as the defect occurrence range 401. Furthermore, in the case of an inspection object 101 that has been repeatedly collected after use, the defect occurrence range 401 can be determined based on the history of past defect repairs.

[0059] FIG. 3D is a diagram showing an example of a defect 304 in the inspection object 101 determined based on the past usage record of the inspection object 101.

[0060] Next, the defect map created by the defect screening unit 104 will be described.

[0061] Fig. 4 is a diagram showing an example of the defect map 106 created by the defect screening unit 104. The defect map 106 displays three-dimensional model data 101a of the inspection object 101, and the types of defects (defect types) and three-dimensional positions 402 of the defects in this three-dimensional model data 101a. The XYZ coordinate system shown in Fig. 4 is a global coordinate system of the three-dimensional model data 101a.

[0062] The defect screening unit 104 uses the relative position and orientation 105 of the camera 100 with respect to the inspection object 101 to determine the three-dimensional position 402 of the defect, that is, the range in which the defect occurs in the inspection object 101, from the image captured by the camera 100.

[0063] The defect map 106 includes data indicating the relationship between the type of defect (defect kind) and the three-dimensional position 402 of the defect (the range in which the defect occurs in the inspection object 101).

[0064] The defect screening unit 104 uses known techniques to determine the three-dimensional positions 402 of defects in the inspection object 101 based on the position and orientation 105 of the camera 100 relative to the inspection object 101 and the defect detection results (the type of defect and the range 401 in which the defect occurs on the image).The defect screening unit 104 then inputs or creates three-dimensional model data 101a of the inspection object 101, and displays the type of defect and the three-dimensional positions 402 of the defect in this three-dimensional model data 101a to create a defect map 106.

[0065] The defect map 106 can be displayed on a display device 130 (FIG. 1). In the defect map 106 displayed on the monitor of the display device 130, the display of defects in the inspection object 101 can be changed by changing the position and orientation of the three-dimensional model data 101a by operating a mouse or a touch panel.

[0066] 4, defect map 106 may display, for example, a front view, a plan view, a side view, and a back view of inspection object 101, and display the defect type and three-dimensional position 402 of the defect on these views. Furthermore, if inspection object 101 has a rotationally symmetric shape, a cylindrical coordinate system with the center of rotation as its axis may be defined, and a plane defined by the axial direction and circumferential direction in this cylindrical coordinate system may be displayed on defect map 106. The side of inspection object 101 may be represented on this plane, and the defect type and three-dimensional position 402 of the defect may be displayed on defect map 106.

[0067] <Sensor selection section> The sensor selection unit 107 will be described with reference to Figures 5 and 6. The sensor selection unit 107 refers to a database 500 and selects a sensor to be used in the second inspection step according to the type of defect determined by the defect screening unit 104.

[0068] FIG. 5 is a flowchart showing the processing executed by the sensor selection unit 107.

[0069] In S501, the sensor selection unit 107 inputs the inspection items for the second inspection process through operation by the operator. For example, the operator inputs which types of defects will be inspected in detail in the second inspection process. In addition, the sensor selection unit 107 inputs, through operation by the operator, the inspection specifications required for defect inspection, such as inspection accuracy and inspection throughput, for each inspection object 101.

[0070] In S502, the sensor selection unit 107 refers to the database 500 and extracts candidate sensors to be used in the second inspection process based on the type of defect and the occurrence range determined by the defect screening unit 104. For example, the sensor selection unit 107 extracts sensors corresponding to the type of defect (defect type) as candidate sensors.

[0071] In S503, the sensor selection unit 107 refers to the database 500 and selects a sensor to be used in the second inspection process from the extracted sensor candidates according to the inspection specifications. For example, the database 500 stores data on the measurement accuracy and measurement speed of the sensor as sensor characteristic data. The sensor selection unit 107 uses the sensor characteristic data acquired from the database 500 to select a sensor to be used in the second inspection process according to the inspection specifications (e.g., inspection accuracy and inspection throughput) input in S501.

[0072] 6 is a diagram showing an example of data stored in the database 500. The database 500 stores candidate sensors to be used in the second inspection process according to the type of defect (defect kind) together with characteristic data of the sensors.

[0073] Sensors used in the second inspection process include, for example, a fringe pattern projection type measuring device, a light section type measuring device, a contact type roughness gauge, a non-contact type roughness gauge, a contact type film thickness gauge, a non-contact type film thickness gauge, a sensor for eddy current testing (ECT), an ultrasonic sensor, and an X-ray inspection device.

[0074] The sensor characteristic data includes, for example, the defect types that can be handled, the type of measurement range, the size of the measurement range, the measurement accuracy, the sensor angle of view, and the measurement speed for each sensor.

[0075] The type of measurement range indicates the shape of the measurement range of the sensor, including, for example, a surface, a line, and a point.

[0076] The size of the measurement range indicates the range (width) that the sensor can measure, and is stored for each of the x, y, and z directions in the coordinates (local coordinates) defined for the sensor. The size of this measurement range indicates the size of the range that the sensor can measure.

[0077] The sensor field of view is stored only for optical sensors. In the example shown in Figure 6, the optical sensors are a fringe pattern projection type measuring device, a light section type measuring device, a non-contact roughness meter, and a non-contact film thickness meter.

[0078] In the database 500, appropriate sensors are registered, among those owned by the worker, taking into consideration conditions such as measurement accuracy, type and size of measurement range, measurement speed, and equipment cost.

[0079] For example, a case will be described in which the type of defect determined by the defect screening unit 104 is surface wear, and the surface wear is inspected in detail in the second inspection step.

[0080] In S501 shown in FIG. 5, the sensor selection unit 107 inputs surface wear as an inspection item in the second inspection step in response to an operation by an operator.

[0081] In S502, the sensor selection unit 107 refers to the database 500 and extracts sensors corresponding to the surface wear as candidate sensors to be used in the second inspection process. For example, the sensor selection unit 107 extracts a stripe pattern projection type measurement device and a light-section type measurement device as candidate sensors.

[0082] In S503, the sensor selection unit 107 selects a sensor to be used in the second inspection process from the extracted sensor candidates according to the inspection specifications. For example, the sensor selection unit 107 refers to the sensor characteristic data stored in the database 500 and selects a stripe pattern projection measurement device as the sensor to be used in the second inspection process according to the required inspection specifications (e.g., inspection throughput and inspection accuracy).

[0083] In the second inspection step, a fringe pattern projection measuring device is used to measure the surface shape of the inspection object 101, and the measured shape is compared with the design shape to measure the amount of wear on the surface of the inspection object 101. The fringe pattern projection measuring device measures the surface shape of the inspection object 101 by projecting illumination with a fringe pattern.

[0084] Also, for example, if the type of defect determined by the defect screening unit 104 is a scratch, in S502, the sensor selection unit 107 refers to the database 500 and extracts a contact-type roughness meter and a non-contact-type roughness meter as candidate sensors.

[0085] Also, for example, if the type of defect determined by the defect screening unit 104 is a decrease in the film thickness of the surface coating of the inspection object 101, in S502 the sensor selection unit 107 extracts a contact film thickness meter and a non-contact film thickness meter as candidate sensors.

[0086] Also, for example, if the type of defect determined by the defect screening unit 104 is the depth of rust on the surface of the inspection object 101, in S502 the sensor selection unit 107 extracts a contact-type film thickness meter as a candidate sensor.

[0087] As described above, the defect type includes an internal defect. When the defect type is an internal defect, the sensor selection unit 107 extracts a sensor for eddy current testing (ECT), an ultrasonic sensor, and an X-ray inspection device as sensor candidates.

[0088] In S503, the sensor selection unit 107 selects a sensor to be used in the second inspection process from among the candidate sensors according to the required inspection specifications (for example, inspection throughput and inspection accuracy). Note that the operator may select a sensor to be used in the second inspection process from among the candidate sensors extracted by the sensor selection unit 107. Also, the operator may use a sensor different from the sensor selected by the sensor selection unit 107 in the second inspection process.

[0089] <Measurement position calculation section> The measurement position calculation unit 108 will be described with reference to Figures 7 to 9. The measurement position calculation unit 108 determines the measurement start position and measurement end position of the sensor when the sensor selected by the sensor selection unit 107 is used in the second inspection step.

[0090] In the following, as an example, a case will be described in which the type of defect determined by the defect screening unit 104 is a scratch, and the sensor selected by the sensor selection unit 107 is a contact or non-contact roughness meter.

[0091] When inspecting the defect scratches in detail in the second inspection step, it is preferable to scan the sensor in a direction approximately perpendicular to the length of the scratches. For this reason, it is preferable to determine the measurement start position and measurement end position when scanning the sensor and determine the scanning direction of the sensor.

[0092] Fig. 7 is a top view showing a scratch 701, which is a defect that has occurred on the surface of the inspection target 101. The coordinate system shown in Fig. 7 is a coordinate system (local coordinate system) defined for the sensor.

[0093] For ease of explanation, it is assumed that the orientation of the sensor is determined so that the length direction of the flaw 701 is the x direction, the width direction of the flaw 701 is the y direction, and the direction perpendicular to the surface of the inspection object 101 is the z direction. In the second inspection step, the sensor is oriented in this way relative to the flaw 701 and inspection is performed.

[0094] As described above, when inspecting the flaw 701 in the second inspection step, it is preferable to scan the sensor in a direction approximately perpendicular to the length direction of the flaw 701. Therefore, the measurement position calculation unit 108 determines the length direction of the flaw 701 from the occurrence range of the flaw 701 (the occurrence range of the defect in the inspection object 101), and sets the direction approximately perpendicular to the length direction of the flaw 701 as the measurement direction (scanning direction) of the sensor. In other words, the measurement direction of the sensor is the width direction of the flaw 701.

[0095] First, the measurement position calculation unit 108 calculates the length direction, width direction, and width of the flaw 701 (thickness of the flaw 701) from the occurrence range of the flaw 701. The length direction of the flaw 701 is, for example, the major axis direction of an ellipse when an ellipse is fitted to the pixel area extracted as the flaw 701 in the image captured by the camera 100 acquired in the first inspection step (i.e., the pixel area corresponding to the occurrence range of the defect calculated by the defect screening unit 104). This ellipse includes the entire flaw 701. The width direction of the flaw 701 is a direction approximately perpendicular to the length direction of the flaw 701, and is the minor axis direction of this ellipse. The width length of the flaw 701 is the maximum length in the minor axis direction of the ellipse of the area extracted as the flaw 701 (the occurrence range of the defect).

[0096] When calculating the length direction, width direction, and width of the flaw 701, the measurement position calculation unit 108 may fit a rectangle to the flaw 701 instead of an ellipse. Alternatively, the measurement position calculation unit 108 may fit a straight line to the flaw 701, and determine the extension direction of the line as the length direction of the flaw 701, and the direction perpendicular to this line as the width direction of the flaw 701.

[0097] Next, the measurement position calculation unit 108 calculates a measurement start coordinate 710, which is the measurement start position of the sensor, and a measurement end coordinate 720, which is the measurement end position of the sensor, so that the width of the flaw 701 is sufficiently included. For example, the measurement position calculation unit 108 determines an arbitrary position that is an arbitrary distance away from the flaw 701 along the width direction of the flaw 701 as the measurement start coordinate 710. Next, the measurement position calculation unit 108 moves the measurement start coordinate 710 along the width direction of the flaw 701 and determines a position that is an arbitrary distance beyond the flaw 701 as the measurement end coordinate 720. In other words, the measurement end coordinate 720 is located at a position beyond the flaw 701 along the width direction of the flaw 701 (i.e., the measurement direction of the sensor) from the measurement start coordinate 710.

[0098] The measurement start coordinate 710 and the measurement end coordinate 720 are located outside the defect, i.e., the scratch 701. The line segment connecting the measurement start coordinate 710 and the measurement end coordinate 720 intersects with the scratch 701 and is parallel to the width direction of the scratch 701. The distance between the measurement start coordinate 710 and the measurement end coordinate 720 is longer than the width of the scratch 701. In other words, the line segment connecting the measurement start coordinate 710 and the measurement end coordinate 720 encompasses the width of the scratch 701.

[0099] Some sensors are equipped with a scanning mechanism for themselves and perform scanning. When using such a sensor, it is preferable that the measurement position calculation unit 108 calculates the measurement start coordinate 710 and the measurement end coordinate 720 so that the measurement start coordinate 710 and the measurement end coordinate 720 are included in the scanning range in the measurement direction of the sensor. In this way, the sensor can measure the flaw 701 by scanning with the scanning mechanism. When the sensor itself is equipped with a scanning mechanism, the database 500 (FIG. 6) may store the scanning range of the sensor as the size of the measurement range.

[0100] Some sensors have a sensor scanning mechanism and multiple sensor scanning ranges, from which a scanning range to be used for measurement can be selected. For such sensors, the database 500 (FIG. 6) may store the multiple sensor scanning ranges as the sizes of multiple measurement ranges. The measurement position calculation unit 108 may refer to the database 500 to select an appropriate measurement range from the multiple measurement range sizes, and calculate the measurement start coordinate 710 and the measurement end coordinate 720 so that the line segment connecting the measurement start coordinate 710 and the measurement end coordinate 720 includes the width of the flaw 701.

[0101] Furthermore, the measurement position calculation unit 108 may obtain multiple pairs of the measurement start coordinate 710 and the measurement end coordinate 720 depending on the condition of the flaw 701. For example, if the defects (flaws) detected by the defect screening unit 104 include multiple regions in which the brightness in the image captured by the camera 100 exceeds a predetermined threshold, the measurement position calculation unit 108 may obtain multiple pairs of the measurement start coordinate 710 and the measurement end coordinate 720 so as to measure each of these multiple regions. In other words, the measurement position calculation unit 108 may obtain multiple pairs of the measurement start coordinate 710 and the measurement end coordinate 720 so that the line segment connecting the measurement start coordinate 710 and the measurement end coordinate 720 intersects with each of these multiple regions.

[0102] Areas where the brightness exceeds a predetermined threshold are considered to be areas with large defects (flaws). Therefore, by measuring not just one defect but multiple areas including such an area, the inspection target 101 can be inspected more accurately.

[0103] Furthermore, if the sensor is a film thickness meter, there is no need to move the sensor, as it measures only at the position where the sensor is in contact with the inspection object 101. In such a case, the measurement position calculation unit 108 determines only the measurement start coordinates 710.

[0104] The measurement position calculation unit 108 determines whether to obtain both the measurement start coordinate 710 and the measurement end coordinate 720, or to obtain only the measurement start coordinate 710, by referring to the information on the type of measurement range stored in the database 500 (FIG. 6).

[0105] The measurement position calculation unit 108 can not only determine the x-coordinate and y-coordinate of the measurement start position and measurement end position (i.e., the measurement start coordinate 710 and the measurement end coordinate 720) on a plane along the surface of the object to be inspected 101, but also the coordinate (z-coordinate) in a direction perpendicular to the surface of the object to be inspected 101 depending on the shape of the surface of the object to be inspected 101.

[0106] 8A and 8B, a method for determining the distance between the sensor and the inspection object 101 in a direction perpendicular to the surface of the inspection object 101 will be described below. The distance between the sensor and the inspection object 101 in a direction perpendicular to the surface of the inspection object 101 is called the "offset height." The offset height is expressed by the z-coordinate. From the offset height, the coordinate (z-coordinate) in the direction perpendicular to the surface of the inspection object 101 can be determined.

[0107] 8A and 8B are diagrams showing a scratch 701, which is a defect that has occurred on the surface of the inspection object 101, and a sensor 800 that measures the scratch 701. 8A and 8B show an example in which the sensor 800 is a contact-type roughness meter. The coordinate systems shown in 8A and 8B are coordinate systems (local coordinate systems) defined for the sensor.

[0108] The contact-type roughness meter, which is the sensor 800, measures the flaw 701, which is a defect on the surface of the inspection object 101. The measurement position calculation unit 108 calculates the offset height 810 at this time, for example, as follows.

[0109] The contact-type roughness meter includes a probe 800a, and obtains a profile of the surface of the test object 101 by varying the position of the tip of the probe 800a approximately parallel to the normal to the surface of the test object 101. The range of variation in the position of the tip of the probe 800a is the measurement range 800z of the contact-type roughness meter, and is stored in the database 500 (FIG. 6) as the size of the measurement range z.

[0110] The measurement position calculation unit 108 refers to the database 500 to obtain the value of the measurement range 800z, and calculates the position of the tip of the probe 800a so that the surface of the inspection target 101 falls within this measurement range 800z. The measurement position calculation unit 108 then determines the offset height 810 of the sensor 800, taking into account the length 800b of the probe 800a. The length 800b of the probe 800a is a value specific to the sensor 800, and may therefore be stored in the database 500.

[0111] Typically, the depth (size in the z direction) of the scratch 701 is unknown in the process of calculating the offset height 810. For this reason, it is preferable to calculate the offset height 810 by positioning the sensor 800 so that the position on the surface of the inspection object 101 closest to the sensor 800 is the upper limit of the measurement range 800z of the sensor 800. By calculating the offset height 810 in this manner, it is possible to measure the deepest possible scratch 701.

[0112] Furthermore, if the sensor 800 is an optical sensor and the type of measurement range is a surface, i.e., if the sensor 800 is an optical two-dimensional area sensor, the field of view range (measurable range) of the sensor 800 is determined by the angle of view of the sensor 800 and depends on the offset height 810.

[0113] Fig. 9 is a diagram showing the field of view and offset height 810 of the sensor 800 when the sensor 800 is an optical two-dimensional area sensor. Fig. 9 shows an example in which the inspection range 900 of the inspection object 101 is rectangular.

[0114] The sensor 800 scans along the longitudinal direction of the inspection area 900. That is, the measurement direction of the sensor 800 is the longitudinal direction of the inspection area 900. The direction perpendicular to the longitudinal direction of the inspection area 900 is the width direction of the inspection area 900.

[0115] The measurement position calculation unit 108 acquires the field of view information of the sensor 800 from the database 500 (FIG. 6), and uses this field of view information to determine the offset height 810 so that the field of view range of the sensor 800 sufficiently includes the width of the inspection range 900.

[0116] As described above, the measurement position calculation unit 108 can determine the measurement start position and measurement end position (measurement start coordinate 710 and measurement end coordinate 720) of the sensor 800 according to the type of measurement range of the sensor 800 (surface, line, or point). For example, if the type of measurement range is a surface, the calculation takes into consideration the angle of view (field of view) of the sensor 800. If the type of measurement range is a line, the calculation takes into consideration the width of the defect (flaw 701). If the type of measurement range is a point, only the measurement start coordinate 710 is calculated.

[0117] As described above, the measurement position calculation unit 108 refers to the database 500 and finds the measurement start position and measurement end position (measurement start coordinate 710 and measurement end coordinate 720) of the sensor 800 according to the sensor 800, thereby automatically and appropriately determining the scanning position and scanning direction of the sensor 800. Therefore, the defect inspection system according to this embodiment can reduce the burden on the operator and can inspect the inspection object 101 for defects with high accuracy in the second inspection step.

[0118] <Second inspection process> The second inspection process will be described with reference to Figures 10A and 10B. The second inspection process is performed by a worker or a work robot. The worker or work robot inspects the inspection object 101 for defects according to the measurement start position and measurement end position (measurement start coordinate 710 and measurement end coordinate 720) calculated by the measurement position calculation unit 108.

[0119] Fig. 10A is a diagram showing an example of a method in which an operator 600 uses a sensor 800 selected by a sensor selection unit 107 to inspect a defect occurring in an inspection object 101. Fig. 10A shows an example in which an operator 600 uses motion capture technology to inspect a defect in an inspection object 101.

[0120] The worker 600 attaches reflective markers 103 for motion capture to the selected sensor 800 and the inspection target 101. The worker 600 holds the sensor 800 to which the reflective markers 103 are attached.

[0121] A plurality of motion capture cameras 102 capture images of the reflective markers 103 attached to the sensor 800 and the reflective markers 103 attached to the inspection object 101 .

[0122] The computer 120 (FIG. 1) calculates and obtains the position and orientation 114 of the sensor 800 relative to the inspection target 101, and obtains a difference 601 between the obtained position of the sensor 800 and the measurement start coordinate 710 obtained by the measurement position calculation unit 108. If the difference 601 is greater than a predetermined threshold, the computer 120 displays, as work instructions 602 on the display device 130 (FIG. 1), an instruction to adjust the position of the sensor 800 so as to reduce the difference 601, and the orientation of the sensor 800 for determining the measurement direction of the sensor 800. The measurement direction of the sensor 800 is obtained from the measurement start coordinate 710 and the measurement end coordinate 720. In this way, the computer 120 instructs the operator 600 on the measurement start position and orientation of the sensor 800.

[0123] When the difference 601 becomes equal to or smaller than a predetermined threshold, the computer 120 displays on the display device 130 an instruction to the operator 600 to measure the inspection object 101 using the sensor 800 .

[0124] The display device 130 can be, for example, one or both of a display and AR (Augmented Reality) glasses worn by the worker 600, and is connected to the computer 120 by wire or wirelessly. The computer 120 can display the measurement start coordinates 710, the measurement end coordinates 720, the difference 601, the attitude of the sensor 800, and the like as work instructions 602 on the display device 130.

[0125] If the sensor 800 has its own scanning mechanism or if the sensor 800 has an input channel for inputting a measurement start trigger, the computer 120 instructs the sensor 800 to start measurement.

[0126] FIG. 10B is a diagram showing an example of a method in which working robot 603 uses sensor 800 selected by sensor selection unit 107 to inspect for defects occurring in inspection object 101.

[0127] The work robot 603 is connected to the computer 120 by wire or wirelessly. The computer 120 can output measurement start coordinates 710 and measurement end coordinates 720 to the work robot 603 or the control device of the work robot 603. The work robot 603 holds the selected sensor 800 and inspects the inspection target 101 for defects, for example, according to the following procedure.

[0128] Computer 120 outputs the determined measurement start coordinates 710 and measurement end coordinates 720 to working robot 603 or the control device of working robot 603. Working robot 603 receives measurement start coordinates 710 and measurement end coordinates 720 from computer 120, moves sensor 800 to the position of measurement start coordinates 710, and begins inspecting object 101. Working robot 603 moves sensor 800 from the position of measurement start coordinates 710 to the position of measurement end coordinates 720, and inspects object 101.

[0129] As described above, the defect inspection system and defect inspection method according to this embodiment can select the sensor 800 that will inspect the defect according to the type of defect, and can determine the measurement start position and measurement end position (measurement start coordinate 710 and measurement end coordinate 720) of the sensor 800 according to the selected sensor 800. In the defect inspection system according to this embodiment, the scanning position and scanning direction of the sensor 800 can be automatically and appropriately determined, thereby reducing the burden on the operator and enabling the inspection object 101 to be inspected for defects with high accuracy.

[0130] It should be noted that the present invention is not limited to the above-described embodiments, and various modifications are possible. For example, the above-described embodiments have been described in detail to clearly explain the present invention, and the present invention is not necessarily limited to embodiments that include all of the configurations described. It is also possible to replace part of the configuration of one embodiment with the configuration of another embodiment. It is also possible to add the configuration of another embodiment to the configuration of one embodiment. It is also possible to delete part of the configuration of each embodiment, or to add or replace other configurations. Furthermore, each component of the device of the present invention may be realized in hardware, for example, by designing it as an integrated circuit. Alternatively, each component may be realized in software, with a processor interpreting and executing a program that realizes each function. Information such as programs, tables, files, measurement information, and calculation information that realize each function can be recorded on a recording device such as a memory, a hard disk drive, or an SSD (Solid State Drive), or on a recording medium such as an IC card, an SD card, or a DVD. Therefore, each component of the device of the present invention can realize each function as a processing unit, a processing unit, a program module, or the like.

[0131] In addition, in each drawing, the control lines and information lines that are considered necessary for explanation are shown, and not all control lines and information lines necessary for the product are shown. In an actual product, it can be assumed that almost all components are interconnected.

[0132] Furthermore, the configuration of the device of the present invention can be further divided into more components depending on the processing content, and can also be divided so that one component performs even more processing. [Explanation of symbols]

[0133] 100...camera, 101...inspection object, 101a...three-dimensional model data, 102...motion capture camera, 103...reflective marker, 104...defect screening unit, 105...camera position and orientation, 106...defect map, 107...sensor selection unit, 108...measurement position calculation unit, 109...linear stage, 110...rotation stage, 112...jig, 114...sensor position and orientation, 120...computer, 130...display device, 3 01, 302, 303, 304...defect, 401...area of ​​defect occurrence, 402...3D position of defect in object to be inspected, 500...database, 600...worker, 601...difference, 602...work instructions, 603...work robot, 701...flaw, 710...measurement start coordinate, 720...measurement end coordinate, 800...sensor, 800a...measuring probe, 800b...measuring probe length, 800z...measurement range, 810...offset height, 900...inspection range.

Claims

1. a camera for capturing an image of an object to be inspected; A database, a computer including a defect screening unit, a sensor selection unit, and a measurement position calculation unit; Equipped with the database stores sensors used in inspecting the defects of the inspection object together with characteristic data of the sensors according to the types of the defects; the defect screening unit determines the type of the defect and the range of occurrence of the defect in the inspection object from the image captured by the camera; the sensor selection unit refers to the database and selects the sensor to be used for inspecting the defect according to the type of the defect determined by the defect screening unit; the measurement position calculation unit determines a measurement start position and a measurement end position for the defect inspection for the sensor selected by the sensor selection unit, based on the characteristic data acquired from the database and the defect occurrence range determined by the defect screening unit; A defect inspection system comprising:

2. the measurement position calculation unit determines the measurement start position and the measurement end position so that the measurement start position and the measurement end position are located outside the defect and a line segment connecting the measurement start position and the measurement end position intersects with the defect. The defect inspection system according to claim 1 .

3. The types of defects include scratches, rust, surface wear, film thickness reduction, surface roughness, and internal defects. The defect inspection system according to claim 1 .

4. the defect screening unit determines the defect occurrence range from the image captured by the camera using the relative position and orientation of the camera with respect to the inspection object; The defect inspection system according to claim 1 .

5. The sensors include a fringe pattern projection type measuring device, a light section type measuring device, a roughness gauge, a film thickness gauge, an eddy current flaw detection sensor, an ultrasonic sensor, and an X-ray inspection device. The defect inspection system according to claim 1 .

6. a display device connected to the computer; the computer displays the measurement start position and the measurement end position on the display device; The defect inspection system according to claim 1 .

7. the computer outputs the measurement start position and the measurement end position to a work robot connected to the computer or a control device for the work robot; The defect inspection system according to claim 1 .

8. the sensor selection unit inputs inspection specifications required for the defect inspection, and selects the sensor to be used for the defect inspection in accordance with the inspection specifications using the characteristic data acquired from the database. The defect inspection system according to claim 1 .

9. The database stores sensors used in inspecting defects in the inspection object along with characteristic data of the sensors according to the types of the defects; a defect screening step in which a computer determines the type of defect and the range of occurrence of the defect in the inspection object from an image of the inspection object captured by a camera; a sensor selection step in which the computer refers to the database and selects the sensor to be used for inspecting the defect according to the type of the defect determined in the defect screening step; a measurement position calculation step in which the computer calculates a measurement start position and a measurement end position for the defect inspection based on the characteristic data acquired from the database and the defect occurrence range calculated in the defect screening step for the sensor selected in the sensor selection step; A defect inspection method comprising:

Images