Inspection apparatus, inspection method, and recording medium

Abstract

The position detection unit (312) applies the inspection target image to a position detection model generated by machine learning using image information of the learning target image, and calculates a score with respect to a portion of the inspection target image. Further, the position detection unit (312) detects the position of the metal member in the inspection target image by determining a portion of the image whose calculated score becomes a score of a predetermined relationship. Then, the position output unit (314) outputs the position of the metal member detected by the position detection unit (312).

Description

TECHNICAL FIELD

[0001] The present disclosure relates to an inspection apparatus, an inspection method, and a program. BACKGROUND

[0002] An inspection apparatus that inspects a shape, a disposed position, and the like of an inspection target by image processing is known.

[0003] As one example of such an inspection apparatus, an inspection apparatus that illuminates a metal terminal housed in a connector with an illumination device and takes an image, and inspects whether or not the shape and position of the metal terminal are good or not based on the degree of coincidence of the taken image with a reference pattern is disclosed in Patent Literature 1. The inspection apparatus excludes, from the taken image, pixels around a shiny portion of the metal terminal based on the illumination, and calculates the degree of coincidence with the reference pattern, thereby reducing errors in the inspection due to differences in the position, size, and the like of the shiny portion.

[0004] PRIOR ART DOCUMENTS

[0005] PATENT LITERATURE

[0006] Patent Literature 1: Japanese Patent Application Laid-Open No. 10-91785 SUMMARY

[0007] Depending on the kind of the inspection, there are cases where even if an abnormality such as rust, warping, a scratch, an adhering matter, or the like exists on the surface of the inspection target, it is desired to detect only the disposed position while tolerating the existence of the abnormality. However, the inspection apparatus described in Patent Literature 1 has a problem that when such an abnormality exists on the surface of the metal terminal, the shape and position of the shiny portion change, so the degree of coincidence of the taken image with the reference pattern decreases and it is not possible to detect the position of each metal terminal.

[0008] The present disclosure was achieved in view of the above-described circumstances, and aims to be able to inspect the position at which the inspection target is disposed even if an abnormality exists on the surface of the inspection target.

[0009] In order to achieve the above-described object, the inspection apparatus of the present disclosure includes a position detection section and a position output section. The position detection section applies an image for inspection to a position detection model generated by machine learning using image information of an image for learning, calculates a score with respect to a portion of the image for inspection, and determines a portion of the image for inspection for which the score becomes a predetermined relationship, thereby detecting the position of the inspection target in the image for inspection. The position output section outputs the position of the inspection target detected by the position detection section.

[0010] According to the present disclosure, a portion of an image in which a relationship is set between the image and a score calculated by a position detection model generated by machine learning using image information of a learning image is determined from an inspection image, and a position of an inspection object in the inspection image is detected and output. Thus, the inspection device can inspect a position of an inspection object even if there is an abnormality on a surface of the inspection object, by generating a position detection model by machine learning using image information of a learning image including an inspection object having an abnormality on a surface. BRIEF DESCRIPTION OF DRAWINGS

[0011] Figure 1 is a diagram of an inspection device of Embodiment 1 of the present disclosure.

[0012] Figure 2A is a side view of a camera device of Embodiment 1.

[0013] Figure 2B is a cross-sectional view of the camera device of Embodiment 1 as viewed from the 2B-2B line toward the lower side. Figure 2A

[0014] Figure 3 is a diagram showing a functional configuration of the inspection device of Embodiment 1.

[0015] Figure 4 is a block diagram showing a hardware configuration of a computer of Embodiment 1.

[0016] Figure 5A is a diagram showing one example of device setting information of Embodiment 1.

[0017] Figure 5B is a diagram showing one example of product setting list information of Embodiment 1.

[0018] Figure 5C is a diagram showing one example of position inspection setting information of Embodiment 1.

[0019] Figure 5D is a diagram showing one example of abnormality inspection setting information of Embodiment 1.

[0020] Figure 6 is a diagram showing one example of position learning information of Embodiment 1.

[0021] Figure 7 is a diagram showing one example of abnormality learning information of Embodiment 1.

[0022] Figure 8 is a diagram showing a display example of an initial screen of Embodiment 1.

[0023] Figure 9 ​FIG. 1 is a diagram showing a display example of a learning screen of Embodiment 1.

[0024] Figure 10 FIG. 2 is a diagram showing a display example of an inspection screen of Embodiment 1.

[0025] Figure 11 FIG. 3 is a flowchart showing a position detection model generation process of Embodiment 1.

[0026] Figure 12 FIG. 4 is a flowchart showing an abnormality detection model generation process of Embodiment 1.

[0027] Figure 13 FIG. 5 is a flowchart showing an abnormality inspection process of Embodiment 1.

[0028] Figure 14 FIG. 6 is a flowchart showing a position inspection process of Embodiment 1.

[0029] Figure 15 FIG. 7 is a flowchart following Figure 14 FIG. 8.

[0030] Figure 16 FIG. 9 is a diagram showing one example of position inspection setting information of Embodiment 2.

[0031] Figure 17 FIG. 10 is a flowchart showing a position inspection process of Embodiment 2.

[0032] Figure 18 FIG. 11 is a flowchart following Figure 17 FIG. 12.

[0033] Figure 19 FIG. 13 is a diagram showing a functional structure of an inspection device of Embodiment 3.

[0034] Figure 20 FIG. 14 is a diagram showing one example of position inspection setting information of Embodiment 3.

[0035] Figure 21 FIG. 15 is a flowchart showing a position detection model generation process of Embodiment 3.

[0036] Figure 22 FIG. 16 is a flowchart showing a position inspection process of Embodiment 3.

[0037] Figure 23 FIG. 17 is a diagram showing one example of position inspection setting information of Embodiment 4.

[0038] Figure 24 FIG. 18 is a diagram showing one example of a fitting process of Embodiment 4.

[0039] Figure 25 FIG. 19 is a flowchart showing a position inspection process of Embodiment 4.

[0040] (Symbol explanation)

[0041] 1: inspection device; 2: camera device; 3: computer; 4: substrate; 5: initial screen; 6: learning screen; 7: inspection screen; 50: internal bus; 51: control section; 52: main storage section; 53: external storage section; 54: operation section; 55: display section; 56: transceiver section; 59: control program; 201: base plate; 202 to 205: support; 206: top plate; 207: lateral positioning block; 208: longitudinal positioning block; 209: imaging section; 210, 211: illumination light output section; 301: input section; 302: imaging control section; 303: illumination light control section; 304: setting information storage section; 305: image information storage section; 306: position learning information storage section; 307: abnormality learning information storage section; 308: reference position calculation section; 309: shielded image generation section; 310: position detection model generation section; 311: position detection model storage section; 312: position detection section; 313: position standard judgment section; 314: position output section; 315: abnormality detection model generation section; 316: abnormality detection model storage section; 317: abnormality detection section; 318: abnormality standard judgment section; 319: abnormality output section; 320: initial screen display section; 321: learning screen display section; 322: inspection screen display section; 323: comprehensive judgment section; 324: comprehensive judgment result output section; 325: comprehensive judgment result storage section; 326: coarse detection model generation section; 327: coarse detection model storage section; 401: metal part; 402: glossy portion; 501: learning screen display button; 502: inspection screen display button; 503: end button; 601: learning image display field; 602, 702: product selection field; 603: learning image selection field; 604: learning image imaging button; 605: learning image read-in button; 606: learning image deletion button; 607: position learning information display field; 608: abnormality learning information display field; 609: position learning information reflection button; 610: 1st abnormality learning information reflection button; 611: enlarged display field; 612: 2nd abnormality learning information reflection button; 613, 708: save button; 614: learning button; 615, 709: return button; 616: abnormality category selection field; 617: abnormality category addition button; 618: abnormality category deletion button; 619: cursor; 620: position corresponding to glossy portion; 621: position corresponding to abnormality; 622: area display frame; 623: image of abnormality; 701: inspection image display field; 703: position inspection information display field; 704: abnormality inspection information display field; 705: mark display switching field; 706: comprehensive judgment field; 707: inspection execution button; 710: position corresponding to measured value; 711: mark for measured value. DETAILED DESCRIPTION

[0042] An inspection device, an inspection method, and a program for implementing the mode of the present disclosure will be described in detail below with reference to the drawings. Furthermore, the same symbols are attached to the same or equivalent portions in the drawings.

[0043] Embodiment 1

[0044] The inspection device, the inspection method, and the program of Embodiment 1 of the present disclosure inspect the shape of an inspection target and a position at which the inspection target is arranged.

[0045] [About the inspection device 1]

[0046] Figure 1 is an explanatory diagram of the inspection device of Embodiment 1. Figure 2A is a side view of the imaging device of Embodiment 1. Figure 2B is a cross-sectional view of the inspection device of Embodiment 1 as viewed from the 2B-2B line toward the lower side. Figure 2A

[0047] As shown in Figure 1 , the inspection device 1 of Embodiment 1 is provided with an imaging device 2 that images an inspection target and a computer 3 that inspects an abnormality on the surface of the inspection target and a position at which the inspection target is arranged from the imaged image including the inspection target. In Embodiment 1, as shown in Figure 2A , Figure 2B , a substrate 4 is set as the inspection target of the abnormality inspection and a metal member 401 provided to the substrate 4 is set as the inspection target of the position inspection.

[0048] In the inspection device 1, image information of a learning-use image of the substrate 4 imaged by the imaging device 2 is subjected to machine learning, a position detection model and an abnormality detection model are generated, and the generated position detection model and abnormality detection model are stored. Hereinafter, the learning-use image is referred to as a learning target image.

[0049] The inspection device 1 images the image of the substrate 4 as the inspection target by the imaging device 2, and acquires an inspection-use image. Hereinafter, the inspection-use image is referred to as an inspection target image. In addition, the inspection device 1 applies the inspection target image to the position detection model, and determines whether the position of the metal member 401 is in conformity with the standard. In addition, the inspection device 1 applies the inspection target image to the abnormality detection model, and determines whether the substrate 4 is in conformity with the standard. In addition, the inspection device 1 performs comprehensive determination of the position inspection result and the abnormality inspection result.

[0050] [About the imaging device 2]

[0051] As shown in Figure 2A , Figure 2B ​As shown, the imaging device 2 includes: a base plate 201, which is a mounting platform for a substrate 4, the object to be inspected for anomaly detection; cylindrical support pillars 202-205, which are supported at the four corners of the base plate 201 and extend upwards; and a top plate 206, which is supported on the support pillars 202-205. The imaging device 2 has a transverse positioning block 207 on the base plate 201 that abuts against one end of the substrate 4 in the X direction to determine its mounting position, and a longitudinal positioning block 208 that abuts against one end of the substrate 4 in the Y direction to determine its mounting position. In addition, the imaging device 2 includes: an imaging unit 209 disposed at the center of the bottom surface of the top plate 206 to capture images of the substrate 4 below; and illumination light output units 210 and 211 disposed at both ends of the top plate 206 in the X direction and extending in the Y direction to illuminate the substrate 4 below. In addition, the camera unit 209 is, for example, a camera, and the lighting output units 210 and 211 are, for example, lighting fixtures with dimming function.

[0052] The substrate 4 includes a metal component 401, which serves as an object for position checking. The metal component 401 is a semi-cylindrical metal plate disposed at the center of the upper surface of the substrate 4 and extending longitudinally. Regarding the metal component 401, when illumination light is output from the illumination light output units 210 and 211, the illumination light is reflected, resulting in a glossy portion 402 extending longitudinally at the transverse center. Furthermore, while the metal component is semi-cylindrical, it is not limited to this shape; for example, it can be any shape such as a cuboid, hemisphere, or cone. In this case, the shape of the glossy portion 402 also changes.

[0053] [About Computers 3]

[0054] like Figure 1 As shown, computer 3 is a personal computer. However, computer 3 is not limited to a personal computer. For example, computer 3 could also be a supercomputer, a workbench, or a GPU server equipped with a GPU (Graphics Processing Unit). Additionally, computer 3 could also be other computers capable of performing machine learning and object inspection calculations, such as those equipped with a microcomputer board or an FPGA board equipped with a Field Programmable Gate Array (FPGA). Furthermore, in inspection device 1, the camera device 2 is separate from computer 3, but this is not a limitation; for example, computer 3 and camera device 2 could be integrated into one device.

[0055] Figure 3 This is a diagram illustrating the functional structure of the inspection device according to Embodiment 1. Figure 3As shown, the computer 3 includes an input unit 301 that receives an input from a user, an imaging control unit 302 that controls the imaging unit 209, an illumination light control unit 303 that controls the illumination light output units 210 and 211, and a setting information storage unit 304 that stores setting information used in the inspection device 1. In addition, the computer 3 includes an image information storage unit 305 that stores image information including an image of the substrate 4. In addition, the computer 3 includes a position learning information storage unit 306 that stores position learning information used for machine learning for detecting a position of the metal member 401, and an abnormality learning information storage unit 307 that stores abnormality learning information used for machine learning for detecting an abnormality of a surface of the substrate 4.

[0056] In addition, the computer 3 includes a reference position calculation unit 308 that calculates a reference position for position detection, and a mask image generation unit 309 that generates a mask image that masks an image of a portion of the metal member 401. In addition, the computer 3 includes a position detection model generation unit 310 that generates a position detection model that represents a relationship between an image and a position of the metal member 401 by machine learning, and a position detection model storage unit 311 that stores the generated position detection model. In addition, the computer 3 includes a position detection unit 312 that detects a position of the metal member 401 from an image including the substrate 4 imaged by the imaging unit 209. In addition, the computer 3 includes a position standard determination unit 313 that determines whether or not the detected position of the metal member 401 conforms to a standard, and a position output unit 314 that outputs the detected position of the metal member 401.

[0057] In addition, the computer 3 includes an abnormality detection model generation unit 315 that generates an abnormality detection model that represents a relationship between an image and an abnormality of a surface of the substrate 4 by machine learning, and an abnormality detection model storage unit 316 that stores the generated abnormality detection model. In addition, the computer 3 includes an abnormality detection unit 317 that detects an abnormality of a surface of the substrate 4 from an image including the substrate 4 imaged by the imaging unit 209. In addition, the computer 3 includes an abnormality standard determination unit 318 that determines whether or not the detected abnormality of a surface of the substrate 4 conforms to a standard, and an abnormality output unit 319 that outputs the detected abnormality of a surface of the substrate 4.

[0058] Further, the computer 3 has an initial screen display section 320 that displays an initial screen that the user inputs in order to use the inspection device 1. Further, the computer 3 has a learning screen display section 321 that displays a learning screen that the user inputs in order to generate the position detection model and the abnormality detection model by machine learning. Further, the computer 3 has an inspection screen display section 322 that displays an inspection screen that the user inputs in order to inspect the abnormality of the surface of the substrate 4 or inspect the position of the metal member 401. Further, the computer 3 has a comprehensive determination section 323 that comprehensively determines the inspection result of the position of the metal member 401 and the inspection result of the abnormality of the surface of the substrate 4, a comprehensive determination result output section 324 that outputs the comprehensive determination result, and a comprehensive determination result storage section 325 that stores the comprehensive determination result.

[0059] Figure 4 is a block diagram showing a hardware structure of the computer of Embodiment 1. As shown in Figure 4 The computer 3 has a control section 51 that performs processing in accordance with a control program 59. The control section 51 has a CPU (Central Processing Unit). The control section 51 functions as the imaging control section 302, the illumination light control section 303, the reference position calculation section 308, the shield image generation section 309, the position detection model generation section 310, the position detection section 312, the position standard determination section 313, and the position output section 314 of the computer 3 in accordance with the control program 59. Further, the control section 51 functions as the abnormality detection model generation section 315, the abnormality detection section 317, the abnormality standard determination section 318, the abnormality output section 319, the comprehensive determination section 323, and the comprehensive determination result output section 324 of the computer 3 in accordance with the control program 59.

[0060] Further, the computer 3 has a main storage section 52 that loads the control program 59 and is used as a work area of the control section 51. The main storage section 52 has a RAM (Random-Access Memory).

[0061] Further, the computer 3 is provided with an external storage section 53 that stores a control program 59 in advance. The external storage section 53 provides data stored by the program to the control section 51 and stores data provided from the control section 51 in accordance with an instruction from the control section 51. The external storage section 53 is provided with a nonvolatile memory such as a flash memory, an HDD (Hard Disk Drive), an SSD (Solid State Drive), a DVD (Digital Versatile Disc)-RAM, a DVD-RW (ReWritable), and the like. The external storage section 53 functions as a setting information storage section 304, an image information storage section 305, a position learning information storage section 306, an abnormality learning information storage section 307, a position detection model storage section 311, an abnormality detection model storage section 316, and a comprehensive determination result storage section 325.

[0062] Further, the computer 3 is provided with an operation section 54 that is operated by a user. Information input is provided to the control section 51 via the operation section 54. The operation section 54 is provided with an information input section such as a keyboard and a mouse. The operation section 54 functions as an input section 301.

[0063] Further, the computer 3 is provided with a display section 55 that displays information input via the operation section 54 and information output from the control section 51. The display section 55 is provided with a display device such as an LCD (Liquid Crystal Display), a PDP (Plasma Display Panel), and an organic EL (Electro-Luminescence) display. The display section 55 functions as an initial screen display section 320, a learning screen display section 321, and an inspection screen display section 322.

[0064] Further, the computer 3 is provided with a transceiving section 56 that transmits and receives information. The transceiving section 56 is provided with an information communication section such as a network terminal device and a wireless communication device.

[0065] Further, in the computer 3, the main storage section 52, the external storage section 53, the operation section 54, the display section 55, and the transceiving section 56 are connected to the control section 51 via the internal bus 50.

[0066] Figure 1 、 Figure 3The illustrated computer 3 realizes the functions of the input section 301, the imaging control section 302, and the illumination light control section 303 by the control section 51 using the main storage section 52, the external storage section 53, the operation section 54, the display section 55, and the transceiver section 56 as resources. For example, the computer 3 executes the input step performed by the input section 301, the imaging control step performed by the imaging control section 302, and the illumination light control step performed by the illumination light control section 303. In addition, the control section 51 realizes the functions of the setting information storage section 304, the image information storage section 305, the position learning information storage section 306, and the abnormality learning information storage section 307 by using the main storage section 52, the external storage section 53, the operation section 54, the display section 55, and the transceiver section 56 as resources. For example, the computer 3 executes the setting information storage step performed by the setting information storage section 304 and the image information storage step performed by the image information storage section 305. In addition, for example, the computer 3 executes the position learning information storage step performed by the position learning information storage section 306 and the abnormality learning information storage step performed by the abnormality learning information storage section 307.

[0067] In addition, the computer 3 realizes the functions of the reference position calculation section 308 and the shield image generation section 309 by the control section 51 using the main storage section 52, the external storage section 53, the operation section 54, the display section 55, and the transceiver section 56 as resources. For example, the computer 3 executes the reference position calculation step performed by the reference position calculation section 308 and the shield image generation step performed by the shield image generation section 309. In addition, the computer 3 realizes the functions of the position detection model generation section 310, the position detection model storage section 311, the position detection section 312, the position standard determination section 313, and the position output section 314 by the control section 51 using the main storage section 52, the external storage section 53, the operation section 54, the display section 55, and the transceiver section 56 as resources. In addition, for example, the computer 3 executes the position detection model generation step performed by the position detection model generation section 310, the position detection model storage step performed by the position detection model storage section 311, and the position detection step performed by the position detection section 312. In addition, for example, the computer 3 executes the position standard determination step performed by the position standard determination section 313 and the position output step performed by the position output section 314.

[0068] Furthermore, the computer 3 utilizes the main storage unit 52, external storage unit 53, operation unit 54, display unit 55, and transceiver unit 56 as resources by the control unit 51 to realize the functions of the anomaly detection model generation unit 315, the anomaly detection model storage unit 316, and the anomaly detection unit 317. For example, the computer 3 executes the anomaly detection model generation step performed by the anomaly detection model generation unit 315, the anomaly detection model storage step performed by the anomaly detection model storage unit 316, and the anomaly detection step performed by the anomaly detection unit 317. Additionally, the computer 3 utilizes the main storage unit 52, external storage unit 53, operation unit 54, display unit 55, and transceiver unit 56 as resources by the control unit 51 to realize the functions of the anomaly standard determination unit 318, the anomaly output unit 319, the comprehensive determination unit 323, the comprehensive determination result output unit 324, and the comprehensive determination result storage unit 325. For example, the computer 3 executes the anomaly standard determination step performed by the anomaly standard determination unit 318 and the anomaly output step performed by the anomaly output unit 319. In addition, for example, the computer 3 executes the comprehensive determination step performed by the comprehensive determination unit 323, executes the comprehensive determination result output step performed by the comprehensive determination result output unit 324, and executes the comprehensive determination result storage step performed by the comprehensive determination result storage unit 325.

[0069] [Detailed information about the functional structure of Computer 3]

[0070] like Figure 3 As shown, the camera control unit 302 controls the camera unit 209 to capture images of the substrate 4. The camera control unit 302 controls the camera unit 209 to capture images of the substrate 4 not only when acquiring an image of the object to be inspected during inspection, but also when acquiring an image of the object to be learned before performing machine learning.

[0071] The illumination control unit 303 controls the dimming of the illumination light output by the illumination light output units 210 and 211 when the camera control unit 302 controls the camera unit 209 to capture images of the substrate 4.

[0072] The setting information storage unit 304 stores the setting information used in the inspection device 1. The setting information includes those described later. Figure 5A to Figure 5D The diagram shows the storage area for information used in the inspection device 1, device setting information for the image size of the image captured by the camera device 2, product setting overview information for each product of the substrate 4, position inspection setting information for the position inspection of the metal part 401, and anomaly inspection setting information for the anomaly inspection of the substrate 4.

[0073] Figure 5A This is a diagram illustrating an example of device setting information according to Embodiment 1. The device setting information is as follows: Figure 5AThe information shown includes items such as "Product Name" indicating the product name of substrate 4, "Temporary Storage Area for Inspection Image Information" indicating the location of a temporary storage area for image information of the image of the object to be inspected, "Storage Area for Learning Image Information" indicating the location of a storage area for image information of the image of the object to be learned, "Storage Area for Position Learning Information" indicating the location of a storage area for position learning information used for position detection, "Storage Area for Abnormal Learning Information" indicating the location of a storage area for abnormal learning information used for abnormal detection, and "Image Size" indicating the image size of the image of the object to be inspected. This information is categorized for each product name. Furthermore, as described above, the temporary storage area for image information of the image of the object to be inspected and the storage area for image information of the image of the object to be learned are located in the image information storage unit 305, the position learning information storage area is located in the position learning information storage unit 306, and the abnormal learning information storage area is located in the abnormal learning information storage unit 307.

[0074] For example, in the device setting information, in Figure 5A In the first row shown, for the product with "Product Name" "Substrate A", an area for temporarily storing inspection image information is shown at the position defined by "FA1". Additionally, in the device setting information, for the product with "Product Name" "Substrate A", an area for storing learning image information is shown at the position defined by "FA2", and an area for storing position learning information is shown at the position defined by "FA3". Furthermore, in the device setting information, for the product with "Product Name" "Substrate A", an area for storing anomaly learning information is shown at the position defined by "FA4", and the size of the image used for learning and inspection is set to "SZA1".

[0075] Figure 5B This is a diagram illustrating an example of product setting overview information in Embodiment 1. Furthermore, the product setting overview information is as follows: Figure 5BThe illustrated information includes items of "product name" indicating the product name of the substrate 4, "position check reference" indicating the coordinate position on the design of the substrate 4 as a reference of the position of the metal member 401, "position check criterion" indicating a threshold value of the determination value described later as a criterion used in the position check of the metal member 401, and "abnormality check criterion" indicating a threshold value of the dimension described later as a criterion used in the abnormality check of the substrate 4, and the information of these items is classified for each product name. Further, in Embodiment 1, the image of the substrate 4 is imaged by the imaging device 2 in a state where the substrate 4 is butted against the lateral positioning block 207 and the longitudinal positioning block 208, and thus the coordinate position of the substrate 4 is replaced with the coordinate position of the image imaged by the imaging device 2. Hereinafter, the coordinate position on the design of the substrate 4 is sometimes described as a reference value of the X coordinate and the Y coordinate. Further, the determination value is an absolute value of a difference value between a measured value of the X coordinate and the Y coordinate of the coordinate position of the substrate 4 detected at the time of the position detection and a reference value. That is, the threshold value of the determination value as a criterion used in the position check refers to a value allowed as an absolute value of a difference value between a measured value of the X coordinate and the Y coordinate of the substrate 4 detected at the time of the position detection and a reference value of the X coordinate and the Y coordinate on the design of the substrate 4.

[0076] For example, in the product setting list information, in the 1st row of Figure 5B , with respect to the product of which the "product name" is "substrate A", it is shown that the coordinate position on the design of the metal member 401 is (XA1, YA1), and with respect to the position of the metal member 401, the threshold value of the difference value between the measured value and the reference value of the X coordinate is "XA2". Further, in the product setting list information, with respect to the product of which the "product name" is "substrate A", it is shown that the threshold value of the absolute value of the difference value between the measured value and the reference value of the Y coordinate is "YA2", and the threshold value of the dimension allowed with respect to the surface abnormality of the substrate 4 is "SZA3". Hereinafter, the threshold value of the difference value is sometimes described as a criterion value. Further, in Embodiment 1, the absolute value of the difference value is set as the determination value, and the threshold value of the absolute value of the difference value is set as the criterion value, but it is not limited thereto, and for example, the difference value can be set as the determination value, and a range from a lower limit value to an upper limit value of the difference value can be set as the criterion. In this case, with respect to the information of the 1st row of Figure 5B , it can be expressed as "(−YA2) or more and (+YA2) or less" or "±YA2" instead of "YA2", or it can be set as "YA21 or more and YA22 or less" using "YA21" and "YA22" as different numerical values.

[0077] Figure 5C is a view illustrating one example of the position check setting information of Embodiment 1. The position check setting information is as shown in Figure 5CThe information shown includes items such as "Product Name" (representing the product name of substrate 4), "Position Detection Range" (representing the range of X-coordinate values ​​from the start point to the end point and the range of Y-coordinate values ​​from the start point to the end point as position detection range information in the image of the object to be inspected), "Pattern Size" (representing the width and height of the image extracted from the image of the learning object or the image of the object to be inspected), "Offset" (representing the offset of the measured values ​​of the X-coordinate and Y-coordinate used to correct the position of the metal part 401), "Model Form" (representing the form of the learning model of machine learning), "Shielding Size" (representing the length of the width and height of the image extracted from the image of the learning object or the image of the object to be inspected), and "Score Threshold" (representing the threshold for the score calculated about the image extracted during inspection). This information is categorized for each product name.

[0078] For example, in the location check settings information, in Figure 5C In the first line, regarding the product with "Product Name" "Substrate A", the image is shown as the inspection object, extracted within a rectangular detection range with its starting point (XSA1, YSA1) and ending point (XEA1, YEA1) as diagonal vertices. The image's width WA1 (X-direction dimension) and width LA1 (Y-direction dimension) are used as the inspection dimensions. Furthermore, in the position inspection settings information, for the product with "Product Name" "Substrate A", the offset value of the measured X-coordinate for correcting the position of the metal part 401 is shown as "XOA1", and the offset value of the measured X-coordinate for correcting the position is shown as "YOA1". Additionally, in the position inspection settings information, for the product with "Product Name" "Substrate A", the learning model used for inspection is shown as "MA1", the vertical dimension for masking the image is "MWA1", the horizontal dimension is "MLA1", and the scoring threshold is "SA1".

[0079] Figure 5D This is a diagram illustrating an example of the anomaly check setting information in Embodiment 1. The anomaly check setting information is as follows: Figure 5DAs shown, the information includes items such as "Product Name" (representing the product name of substrate 4), "Anomaly Detection Range" (representing the range of X-coordinate values ​​from the start point to the end point and the range of Y-coordinate values ​​from the start point to the end point as anomaly detection range information in the image of the object being inspected), "Pattern Size" (representing the width and height of the image extracted from the learning object image or the image of the object being inspected), "Model Form" (representing the form of the learning model of machine learning), and "Score Threshold" (representing a threshold indicating whether the score obtained from the image extracted during inspection is sufficiently high). This information is categorized for each product name. Furthermore, the "Score Threshold" item is categorized as "Rust," "Warping," "Scratches," and "Attachments" as examples of anomaly categories. While "Rust," "Warping," "Scratches," and "Attachments" are examples of anomaly categories, the categories are not limited to these; for example, "Wounds" and "Gap" could also be added as anomaly categories.

[0080] For example, in the anomaly check settings, in Figure 5D In the first row, for the product with "Product Name" "Substrate A", a rectangular shape with its starting point (XSA2, YSA2) and ending point (XEA2, YEA2) as diagonal vertices is shown as the inspection range. Additionally, in the anomaly inspection settings information, for the product with "Product Name" "Substrate A", an image with a width of "WA2" in the X direction and a height of "LA2" in the Y direction is extracted from the inspection range and used as the inspection object. Furthermore, in the anomaly inspection settings information, for the product with "Product Name" "Substrate A", the learning model used for inspection is shown to be "MA2", the scoring threshold for "rust" is "SA21", the scoring threshold for "warping" is "SA22", the scoring threshold for "scratches" is "SA23", and the scoring threshold for "attachments" is "SA24".

[0081] The image information storage unit 305 stores image information of multiple types of learning object images acquired in advance. Additionally, the image information storage unit 305 temporarily stores image information of inspection object images acquired during inspection.

[0082] Figure 6 This is a diagram illustrating an example of the position learning information in Embodiment 1. The position learning information storage unit 306 stores, as described above, machine learning information used for detecting the position of the metal part 401, i.e., position learning information for position learning. Here, the position learning information is information representing the measured X and Y coordinates of the position of the glossy portion 402 of the metal part 401 in the learning object image. Figure 6As shown, the position learning information includes information of an item of "measurement value" indicating the measurement values of the X coordinate and the Y coordinate of the glossy portion 402, and information of items of "X coordinate" and "Y coordinate" indicating the measurement values of the X coordinate and the Y coordinate, which are classified for each product name. Further, the position of the position learning information is determined from the information of the "position learning information storage area" set in the storage area 306. Figure 5A

[0083] For example, in the position learning information, as shown in Figure 6 , regarding the learning target image of "substrate A" acquired at the time of learning, it is shown that the glossy portion 402 exists at a position of "X coordinate" of "261" and a position of "Y coordinate" of "115". Further, this storage position is determined from "FA3" from the information of the "position learning information storage area" of Figure 5A

[0084] Figure 7 is a diagram showing one example of the abnormality learning information of Embodiment 1. The abnormality learning information storage section 307 stores, as described above, machine learning for detecting an abnormality of the surface of the substrate 4, that is, abnormality learning information for abnormality learning. Here, the abnormality learning information is information indicating measurement values of the X coordinate and the Y coordinate of the position of the abnormality of the surface of the substrate 4 in the learning target image, a category of the abnormality, and a size of the abnormality. As shown in Figure 7 , the abnormality learning information includes information of items of "X coordinate" and "Y coordinate" indicating measurement values of the X coordinate and the Y coordinate of the actual or determined abnormality through simulation, "category" indicating the category of the abnormality, and "size" indicating the size of the abnormality, which are classified for each product name. Further, if a plurality of abnormalities need to be learned with respect to one product, the information is classified for each abnormality to be learned.

[0085] For example, in the abnormality learning information, as shown in Figure 7 , regarding the learning target image of "substrate A" acquired at the time of learning, it is shown that the abnormality of the foreign matter "attached" having a size of "16" as the first abnormality is at a position of "X coordinate" of "110" and "Y coordinate" of "170".

[0086] Figure 3 ​​The reference position calculating section 308 calculates values of the X coordinate and the Y coordinate of the reference position that is the detected position at the time of position learning of the metal member 401. In detail, the reference position calculating section 308 acquires the measured values of the "X coordinate" and the "Y coordinate" of the position learning information corresponding to the product name from the position determined by the information of the item of the "position learning information storage area" of the device setting information stored in the setting information storage section 304. In addition, the reference position calculating section 308 acquires the value of the offset of the measured values of the corrected X coordinate and Y coordinate from the information of the item of the "offset" of the position inspection setting information corresponding to the product name of the metal member 401 stored in the setting information storage section 304. Then, the reference position calculating section 308 calculates the reference values of the X coordinate and the Y coordinate by adding the measured value of the X coordinate to the value of the offset and adding the measured value of the Y coordinate to the value of the offset. Here, the offset is provided, for example, in the case where the position of the object within the imaging range is different due to the deviation of the setting of the camera as in the case where a plurality of inspection devices exist, in order to suppress the deviation of the measured values of each inspection device. In detail, as the value of the offset, a negative value obtained by inverting each of the positive and negative of the values of the X coordinate and the Y coordinate of the upper left end of the substrate 4 in the image of the substrate 4 is set for each inspection device. Then, the reference position calculating section 308 adds the values of the offset of the X coordinate and the Y coordinate to the values of the X coordinate and the Y coordinate of the position of the metal member 401 in the image of the substrate 4. As a result, the inspection device 1 can calculate the measured values of the position of the metal member 401 as the relative position from the upper left end of the substrate 4 in a state where the deviation of each inspection device is eliminated.

[0087] For example, the reference position calculating section 308, when acquiring the learning target image of "substrate A", determines the address of storing the position learning information from "FA3" of the "position learning information storage area", and acquires "261" and "115" as the measured values of the X coordinate and the Y coordinate of the position of the metal member 401 from the position learning information. In addition, the reference position calculating section 308 acquires "XOA1" and "YOA1" as the values of the offset of the X coordinate and the Y coordinate from the position inspection setting information. Then, the reference position calculating section 308 calculates (261+XOA1, 115+YOA1) as the reference values of the X coordinate and the Y coordinate.

[0088] The shield image generation section 309 generates a shield image when an abnormality is detected in the pattern size image extracted from the learning target image or the image for detection, and shields. The shield image generation section 309 acquires the information of the item of "shield size" of the position inspection setting information corresponding to the product name of the metal member 401 stored in the setting information storage section 304 when an abnormality is detected, that is, the length of the horizontal width and the vertical width of the shield. Then, the shield image generation section 309 generates a shield image by shielding the length of the horizontal width and the vertical width acquired with the coordinate position of the value of the X coordinate and the Y coordinate at which the abnormality is detected as the center. Here, the shield is an image filled with white. In addition, regarding the shield, it is not limited to an image filled with white, and for example, it can be an image filled with a color other than white, or an image filled with an average color of the range of the acquired horizontal width and vertical width. In addition, for example, it can be an image including a plurality of kinds of colors, or an image in which the range of the acquired horizontal width and vertical width is blurred. In addition, for example, it can be an image of a specific pattern such as a check pattern or a stripe pattern.

[0089] For example, the shield image generation section 309 acquires the horizontal width MWA1 and the vertical width MLA1 of the item of "shield size" of the position inspection setting information when the learning target image of "substrate A" is acquired. In addition, the shield image generation section 309 acquires "110" and "170" as the measured values of the X coordinate and the Y coordinate of the coordinate position of the abnormality from the items of "X coordinate" and "Y coordinate" of the abnormality learning information. In addition, the shield image generation section 309 determines whether the position of (110, 170) is included in the extracted pattern size image. In addition, the shield image generation section 309, for example, in the case where it is determined that the position of (110, 170) is included in the extracted pattern size image, replaces the image of the range of MWA1 in the horizontal width and MLA1 in the vertical width with the coordinate position of (110, 170) as the center in the extracted pattern size image with an image filled with white.

[0090] The position detection model generation section 310 generates a position detection model indicating the relationship of the image and the position of the metal member 401 by machine learning as described above. The position detection model generation section 310 acquires the information of the item of "model form" of the position check setting information corresponding to the product name of the metal member 401 stored in the setting information storage section 304, that is, the form of the learning model of the machine learning, and generates a rudiment of the position detection model. Here, the method of the machine learning performed by the position detection model generation section 310 is CNN (Convolutional Neural Network) which is commonly used in deep learning of image recognition. In addition, the form of the learning model of the machine learning which the position detection model generation section 310 can generate can be selected from any of the deep learning models of "AlexNet", "VGG", and "ResNet" which are representative deep learning models in CNN. However, the form of the learning model is not limited to these, and a learning model other than CNN can be selected. Furthermore, the method of the machine learning is not limited to CNN, and can be any method of machine learning which can perform image recognition. In this case, the form of the learning model of the set method of machine learning needs to be able to be selected.

[0091] In addition, the position detection model generation section 310 acquires the image information of the learning target image stored in the image information storage section 305 with reference to the information of the item of "learning image information storage area" of the device setting information stored in the setting information storage section 304. In addition, the position detection model generation section 310 acquires the information of the item of "pattern size" of the position check setting information corresponding to the product name of the metal member 401, that is, the length of the horizontal width and the vertical width of the image extracted from the learning target image. In addition, the position detection model generation section 310 extracts the image of the pattern size of the horizontal width and the vertical width acquired with the center of the reference values of the X coordinate and the Y coordinate calculated by the reference position calculation section 308 from the learning target image. Here, the extracted image is an image extracted from the reference values based on the measured values of the X coordinate and the Y coordinate of the position of the glossy portion 402 of the metal member 401 in the learning target image, and is an image in which the position of the metal member 401 can be determined. Thus, the extracted image is an image which is an image in which the portion in which the position is detected can be determined in the position detection, and is an image which is an image of the so-called OK classification. Therefore, the position detection model generation section 310 performs machine learning by using the extracted image as the image of the OK classification, and thereby updates the position detection model.

[0092] Further, in the position detection model generation section 310, when the masking image generation section 309 generates a masking image with respect to the extracted image, machine learning is performed with respect to the masking image as an OK classification image, and the position detection model is updated. In addition, the position detection model generation section 310 extracts an image of a pattern size of a horizontal width and a vertical width obtained with a center at a coordinate position expressed by an arbitrary value different from a reference value of an X coordinate and a Y coordinate from the learning target image, performs machine learning on the extracted image as an NG classification image, and updates the position detection model. Further, the NG classification image is extracted with a center at a coordinate position expressed by an arbitrary value different from a reference value of an X coordinate and a Y coordinate, but is not limited thereto. For example, the NG classification image can be centered at a coordinate position expressed by a value obtained by adding or subtracting a predetermined value to a reference value of an X coordinate, or can be centered at a coordinate position expressed by a value obtained by adding or subtracting a predetermined value to a reference value of a Y coordinate. That is, the position detection model generation section 310 can extract the NG classification image with a center at a coordinate position separated by a predetermined interval from a reference position.

[0093] For example, when the position detection model generation section 310 acquires the learning target image of "substrate A", the position detection model generation section 310 acquires "MA1" of the item of "model form" of the position check setting information. Here, for example, if "MA1" is "VGG16" of "VGG" indicating 16 layers, the position detection model generation section 310 generates a prototype of the position detection model in which the form of the learning model is "VGG16". In addition, the position detection model generation section 310 refers to "FA2" of the "learning image information storage area" of the device setting information, determines an address at which the image information of the learning target image is stored, and acquires the image information of the learning target image. In addition, the position detection model generation section 310 acquires the horizontal width WA1 and the vertical width LA1 of the pattern size of the position check setting information, and extracts an image of a range of a horizontal width of WA1 and a vertical width of LA1 with a center at a coordinate position of (261+XOA1, 115+YOA1) from the learning target image. In addition, the position detection model generation section 310 performs machine learning on the extracted image as an OK classification image, and updates the position detection model. Further, the position detection model generation section 310 performs machine learning on a masking image of the extracted image if the masking image is generated, and updates the position detection model. In addition, the position detection model generation section 310 extracts an image of a range of a horizontal width of WA1 and a vertical width of LA1 with a center at a coordinate position other than (261+XOA1, 115+YOA1) from the learning target image. In addition, the position detection model generation section 310 performs machine learning on the extracted image as an NG classification image, and updates the position detection model.

[0094] The position detection section 312 detects the position of the metal member 401 from the inspection target image including the substrate 4 imaged by the imaging section 209 as described above. The position detection section 312 acquires the image information of the inspection target image temporarily stored in the image information storage section 305 with reference to the information of the item of "inspection image information temporary storage area" of the device setting information corresponding to the product name of the metal member 401 stored in the setting information storage section 304. In addition, the position detection section 312 acquires the value of the item of "image size" of the device setting information, and acquires the information of the items of "position detection range" and "pattern size" of the position inspection setting information corresponding to the product name of the metal member 401 stored in the setting information storage section 304. That is, the position detection section 312 acquires the values of the X coordinate and the Y coordinate as the start point of the position detection range in the inspection target image, and the values of the X coordinate and the Y coordinate as the end point, and the length of the horizontal width and the vertical width of the extracted image.

[0095] In addition, the position detection section 312 extracts the image of the pattern size represented by the horizontal width and the vertical width acquired centering on the coordinate position represented by the values of the X coordinate and the Y coordinate selected from the position detection range from the inspection target image. In addition, the position detection section 312 acquires the position detection model stored in the position detection model storage section 311, applies the position detection model to the extracted image, and calculates the score as the evaluation value of the image. Here, the calculated score is the score representing the possibility that the image is the OK classification, and the score becomes higher as the possibility that the extracted image is the OK classification becomes higher, and the score becomes lower as the possibility that the extracted image is the OK classification becomes lower. In addition, the position detection section 312 extracts the images of all the pattern sizes that can be extracted from the position detection range, and calculates the scores of the extracted images, respectively. In addition, the position detection section 312 acquires the information of the item of "score threshold value" of the position inspection setting information corresponding to the product name of the metal member 401, that is, the score threshold value, and determines whether there is an image of which the calculated score becomes equal to or higher than the acquired score threshold value. In addition, if there is a score that becomes equal to or higher than the score threshold value, the position detection section 312 calculates the measured values of the X coordinate and the Y coordinate at the time of acquisition of the image of which the score becomes the maximum score, and thereby detects the position of the metal member 401.

[0096] For example, the position detection section 312, when acquiring the inspection target image of "substrate A", refers to "FA1" of the item of "inspection image information temporary storage area" of the device setting information, determines the address where the image information of the inspection target image is stored, and acquires the image information of the inspection target image. In addition, the position detection section 312 acquires "SZA1" of "image size" of the device setting information, and acquires the range XSA1 to XEA1 of the value of the X coordinate, the range YSA1 to YEA1 of the value of the Y coordinate of "position detection range" of the position inspection setting information, and the horizontal width WA1 and the vertical width LA1 of "pattern size". In addition, the position detection section 312 extracts, from the inspection target image, an image of a range having a horizontal width of WA1 and a vertical width of LA1 with a coordinate position selected within the rectangular shape position detection range having the start point (XSA1, YSA1) and the end point (XEA1, YEA1) as the diagonal vertices as the center.

[0097] In addition, the position detection section 312, when N selectable coordinate positions exist within the position detection range, applies the acquired position detection model to the extracted N images, and calculates the scores SC11 to SC1N of the N images. In addition, the position detection section 312 acquires "SA1" of "score threshold" of the position inspection setting information, and determines whether there is an image of SA1 or more based on the calculated scores SC11 to SC1N. In addition, the position detection section 312, when the maximum score among the scores of SA1 or more is SC1M, detects the measured values of the X coordinate and the Y coordinate, for example, from the image of which the maximum score SC1M is obtained. Specifically, the position detection section 312, if the values of the X coordinate and the Y coordinate acquired from the image of which the maximum score SC1M is obtained are (X0, Y0), calculates (X0+XOA1, Y0+YOA1) obtained by adding the values of the X coordinate and the Y coordinate with the offset (XOA1, YOA1) to the measured values of the X coordinate and the Y coordinate, and thereby detects the position of the metal member 401. Here, let (X1, Y1) = (X0+XOA1, Y0+YOA1), and hereafter, the measured values of the X coordinate and the Y coordinate are described as (X1, Y1).

[0098] The position criterion determination section 313 determines whether the position of the detected metal member 401 complies with the criterion as described above. The position criterion determination section 313 acquires the information of the items of "position check reference" and "position check criterion" of the product setting list information corresponding to the product name of the metal member 401 stored in the setting information storage section 304, that is, the threshold values of the determination values as the reference values and the criterion values of the X coordinate and the Y coordinate in the position check of the metal member 401. In addition, the position criterion determination section 313 subtracts the reference values from the measured values of the X coordinate determined by the position detection section 312 and from the measured values of the Y coordinate determined by the position detection section 312, thereby calculating the difference values of the X coordinate and the Y coordinate. Then, the position criterion determination section 313 determines that the position of the detected metal member 401 complies with the criterion if the absolute values of the difference values as the determination values of the calculated X coordinate and the calculated Y coordinate are equal to or less than the threshold values. In addition, the position criterion determination section 313 determines that the position of the detected metal member 401 does not comply with the criterion if the absolute value of the difference value of the calculated X coordinate exceeds the threshold value or the absolute value of the difference value of the calculated Y coordinate exceeds the threshold value.

[0099] For example, the position criterion determination section 313 acquires the reference values of the X coordinate and the Y coordinate (XA1, YA1) of "position check reference" and the threshold values of the difference values of the X coordinate XA2 and the difference values of the Y coordinate YA2 of "position check criterion" of the product setting list information when acquiring the inspection target image of "substrate A". In addition, the position criterion determination section 313 calculates the difference values (X1-XA1, Y1-YA1) of the measured values (X1, Y1) of the X coordinate and the Y coordinate from the reference values (XA1, YA1). Then, if |X1-XA1|≤XA2 and |Y1-YA1|≤YA2, the position criterion determination section 313 determines that it complies with the criterion, whereas if |X1-XA1|>XA2 or |Y1-YA1|>YA2, the position criterion determination section 313 determines that it does not comply with the criterion.

[0100] The position output section 314 outputs the measured values of the X coordinate and the Y coordinate calculated by the position detection section 312 as the position of the detected metal member 401. In addition, the position output section 314 outputs the reference value and the standard value of the X coordinate and the Y coordinate acquired by the position standard determination section 313, and the absolute value of the difference value of the determination values of the X coordinate and the Y coordinate calculated by the position standard determination section 313. In addition, the position output section 314 outputs determination OK information indicating that the standard is met as determination result information when the position standard determination section 313 determines that the standard is met, and outputs determination NG information indicating that the standard is not met as determination result information when the position standard determination section 313 determines that the standard is not met. Further, the determination OK information and the determination NG information are, for example, information of "OK" and "NG". In addition, the position output section 314 outputs check failure information in a case where the position detection section 312 fails to detect the position of the metal member 401. Further, the check failure information is, for example, information of "N / A".

[0101] For example, the position output section 314 outputs (X1, Y1), (XA1, YA1), (XA2, YA2), (X1-XA1, Y1-YA1) when acquiring the inspection target image of "substrate A". In addition, the position output section 314 outputs determination OK information if |X1-XA1|≤XA2 and |Y1-YA1|≤YA2, and outputs determination NG information if |X1-XA1|>XA2 or |Y1-YA1|>YA2.

[0102] The abnormality detection model generation section 315 generates an abnormality detection model representing the relationship of the image and the surface of the substrate 4 with the abnormality by machine learning as described above. The abnormality detection model generation section 315 acquires the form of the learning model of the machine learning, that is, the form of the item of "model form" of the abnormality inspection setting information corresponding to the product name of the metal member 401 stored in the setting information storage section 304, and generates a prototype of the abnormality detection model. In addition, the abnormality detection model generation section 315 acquires the image information of the learning target image stored in the image information storage section 305 with reference to the item of information of the "learning image information storage area" of the device setting information stored in the setting information storage section 304. In addition, the abnormality detection model generation section 315 acquires the items of information of "X coordinate" and "Y coordinate" of the abnormality learning information corresponding to the product name of the metal member 401 stored in the abnormality learning information storage section 307 with reference to the item of information of the "abnormality learning information storage area" of the device setting information corresponding to the product name of the metal member 401 stored in the setting information storage section 304. That is, the abnormality detection model generation section 315 acquires the measured values of the X coordinate and the Y coordinate.

[0103] Further, the abnormality detection model generation unit 315 acquires information of an item of "pattern size" of the abnormality inspection setting information, that is, the length of the horizontal width and the vertical width of the image extracted from the learning target image. Further, the abnormality detection model generation unit 315 extracts an image of the pattern size of the horizontal width and the vertical width acquired with the acquired measurement values of the X coordinate and the Y coordinate as the center from the learning target image. Further, the abnormality detection model generation unit 315 acquires information of an item of "category" of the abnormality learning information corresponding to the product name of the metal member 401, that is, the category of the abnormality to be learned, and performs machine learning with the extracted image as an image of the abnormality classified in the acquired category, thereby updating the abnormality detection model.

[0104] For example, the abnormality detection model generation unit 315 acquires "MA2" of "model form" of the abnormality inspection setting information when acquiring the learning target image of "substrate A", and generates a prototype of the abnormality detection model. Here, for example, if "MA2" is "VGG16", the abnormality detection model generation unit 315 generates a prototype of the abnormality detection model in which the form of the learning model is "VGG16". Further, the abnormality detection model generation unit 315 refers to "FA2" of the "learning image information storage area", and acquires image information of the learning target image.

[0105] Further, the abnormality detection model generation unit 315 refers to "FA4" of the "abnormality learning information storage area", and determines an address in which the abnormality learning information is stored, and acquires "110" and "170" as measurement values of the X coordinate and the Y coordinate of the position of the abnormality on the surface of the substrate 4 from the abnormality learning information. Further, the abnormality detection model generation unit 315 acquires the horizontal width WA2 and the vertical width LA2 of "pattern size" from the abnormality inspection setting information. Further, the abnormality detection model generation unit 315 extracts an image of a range with the horizontal width of WA2 and the vertical width of LA2 with the coordinate position of (110, 170) as the center from the learning target image. Further, the abnormality detection model generation unit 315 acquires "adhering matter" of an item of "category" of the abnormality learning information, and performs machine learning with the extracted image as an image of the abnormality classified in the adhering matter, thereby updating the abnormality detection model.

[0106] The abnormality detection section 317 detects an abnormality of the surface of the substrate 4 from the image including the substrate 4 imaged by the imaging section 209 as described above. The abnormality detection section 317 refers to the information of the item of the "temporary storage area of image information for inspection" of the device setting information corresponding to the product name of the metal member 401 stored in the setting information storage section 304, acquires the image information of the image of the inspection target temporarily stored in the image information storage section 305. In addition, the abnormality detection section 317 acquires the value of the item of the "image size" of the device setting information, and acquires the information of the items of the "abnormality detection range" and the "pattern size" of the abnormality inspection setting information corresponding to the product name of the metal member 401 stored in the setting information storage section 304. That is, the abnormality detection section 317 acquires the values of the X coordinate and the Y coordinate which are the start point of the abnormality detection range in the image of the inspection target, and the values of the X coordinate and the Y coordinate which are the end point, and the length of the horizontal width and the vertical width of the extracted image.

[0107] In addition, the abnormality detection section 317 extracts the image of the pattern size of the horizontal width and the vertical width acquired with the coordinate position indicated by the values of the X coordinate and the Y coordinate selected from the abnormality detection range as the center from the image of the inspection target. In addition, the abnormality detection section 317 acquires the abnormality detection model stored in the abnormality detection model storage section 316, applies the abnormality detection model to the extracted image, and calculates the score which is the evaluation value of the image for each of the categories of the abnormality. Here, the calculated score is the score indicating the possibility that the image is the image including the abnormality of the specific category. For example, regarding the score of "rust", the score becomes higher as the extracted image is more likely to be the image including the abnormality of "rust", and the score becomes lower as the extracted image is less likely to be the image including the abnormality of "rust".

[0108] In addition, the abnormality detection section 317 acquires the information of the item of the "score threshold value" of the position inspection setting information corresponding to the product name of the metal member 401, that is, the score threshold value of each of the categories of the abnormality, determines whether or not the calculated score becomes the category of the score threshold value or more. In addition, if there is a score that becomes the score threshold value or more, the abnormality detection section 317 determines the abnormality of the category that becomes the maximum score, calculates the measured values of the X coordinate and the Y coordinate of the image determined to be the abnormality, and calculates the size of the portion determined to be the abnormality by image processing. Furthermore, if there is no score that becomes the score threshold value or more, the abnormality detection section 317 does not detect the abnormality of the surface of the substrate 4. Then, the abnormality detection section 317 extracts all the images of the pattern size within the abnormality detection range, calculates the score of each of the categories of the abnormality in the extracted image, and if there is a category that becomes the score threshold value or more, determines the category of the abnormality that becomes the maximum score, and calculates the measured values of the X coordinate and the Y coordinate and the size.

[0109] For example, when the abnormality detection section 317 acquires the inspection target image of "substrate A", it acquires the image information of the inspection target image with reference to "FA1" of "inspection image information temporary storage area". In addition, the abnormality detection section 317 acquires "SZA1" of the item of "image size", and acquires the range XSA2 to XEA2 of the value of the X coordinate, the range YSA2 to YEA2 of the value of the Y coordinate of "abnormality detection range" of the abnormality inspection setting information, and the horizontal width WA2 and the vertical width LA2 of "pattern size". In addition, the position detection section 312 extracts an image of a range having a horizontal width of WA2 and a vertical width of LA2 from the learning target image, with the coordinate position selected within the rectangular shape abnormality detection range having the start point (XSA2, YSA2) and the end point (XEA2, YEA2) as the diagonal vertices as the center.

[0110] In addition, the abnormality detection section 317 applies the acquired abnormality detection model to the extracted image, and calculates the score of the image for each of the categories of abnormalities. In addition, the abnormality detection section 317 acquires the score threshold value SA21 of "rust", the score threshold value SA22 of "warpage", the score threshold value SA23 of "scratch", and the score threshold value SA24 of "adhering matter" of "score threshold value" of the position inspection setting information, and determines whether there is a category in which the calculated score is equal to or higher than the acquired score threshold value. In addition, if there is a score equal to or higher than the score threshold value, the abnormality detection section 317 determines the abnormality of the category that becomes the maximum score. Specifically, if the score of "rust" is SC21, the score of "warpage" is SC22, the score of "scratch" is SC23, the score of "adhering matter" is SC24, SC21 < SC22 < SC23 < SC24, SC21 < SA21, SC22 < SA22, SC23 < SA23, and SC24 > SA24, the abnormality detection section 317 determines the abnormality of adhering matter.

[0111] Further, in Embodiment 1, if there is a score that becomes a score threshold or more, the abnormality detection section 317 determines an abnormality of the category that becomes the maximum score, but even if it is not the maximum score, as long as it is a score that becomes a score threshold or more, it can be determined as an abnormality of the category that becomes the score. For example, it can be made so that if SC22<SC23<SC21<SC24, SC21>SA21, SC22<SA22, SC23<SA23, SC24>SA24, the abnormality detection section 317 determines an abnormality of rust and adhering matter. In addition, the abnormality detection section 317, for example, if the values of the X coordinate and the Y coordinate of the image determined as an abnormality are (X2, Y2) and the size is SZ2, calculates (X2, Y2) as the measured values of the X coordinate and the Y coordinate, and calculates SZ2 as the size of the portion determined as an abnormality by image processing. Then, the abnormality detection section 317 extracts all of the images of the pattern size within the abnormality detection range, calculates the score of each abnormality of the category in the extracted images, if there is a category that becomes a score threshold or more, determines the category of the abnormality of the category that becomes the maximum score, and calculates the measured values of the X coordinate and the Y coordinate and the size.

[0112] The abnormality standard determination section 318 determines whether or not the detected abnormality of the surface of the substrate 4 conforms to the standard as described above. The abnormality standard determination section 318 acquires the information of the item of "abnormality inspection standard" of the product setting list information corresponding to the product name of the metal member 401 stored in the setting information storage section 304, that is, the standard value of the size of the abnormality in the abnormality inspection of the metal member 401. In addition, if the value of the size of the abnormality calculated by the abnormality detection section 317 is the standard value or less, even if there is a detected abnormality of the surface of the substrate 4, the abnormality standard determination section 318 determines that it conforms to the standard. In addition, if the value of the size of the abnormality calculated by the abnormality detection section 317 exceeds the standard value, the abnormality standard determination section 318 determines that it does not conform to the standard due to the detected abnormality of the surface of the substrate 4.

[0113] For example, the abnormality standard determination section 318 acquires the threshold value SZA3 of the size of "abnormality inspection standard" of the product setting list information when acquiring the inspection target image of "substrate A". In addition, if SZ2≤SZA3, the abnormality standard determination section 318 determines that it conforms to the standard, on the other hand, if SZ2>SZA3, the abnormality standard determination section 318 determines that it does not conform to the standard.

[0114] The abnormality output section 319 outputs the category of the abnormality determined by the abnormality detection section 317, the values of the X coordinate and the Y coordinate calculated by the abnormality detection section 317, and the size of the abnormality as the detected abnormality of the surface of the substrate 4. In addition, the abnormality output section 319 outputs the standard value of the size of the abnormality acquired by the abnormality standard determination section 318. In addition, the abnormality output section 319 outputs determination OK information indicating that the standard is met as determination result information when the abnormality standard determination section 318 determines that the standard is met, and outputs determination NG information indicating that the standard is not met as determination result information when the abnormality standard determination section 318 determines that the standard is not met.

[0115] For example, the abnormality output section 319 outputs "adhering matter", (X2, Y2), SZ2, and SZA3 when acquiring the inspection target image of "substrate A". In addition, the abnormality output section 319 outputs determination OK information if SZ2≤ SZA3, and outputs determination NG information if SZ2> SZA3.

[0116] Figure 8 is a diagram showing a display example of an initial screen of Embodiment 1. The initial screen display section 320 displays the initial screen 5 shown in Figure 8 when executing a program for inspection. The initial screen 5 has a learning screen display button 501 for displaying a learning screen, an inspection screen display button 502 for displaying an inspection screen, and an end button 503 for ending the program. As shown in Figure 8 , in the initial screen 5, the learning screen display button 501 is provided at the left portion of the upper side of the initial screen 5, the inspection screen display button 502 is provided at the right portion of the upper side of the initial screen 5, and the end button 503 is provided at the central portion in the left-right direction of the lower side of the initial screen 5.

[0117] Figure 9 is a diagram showing a display example of a learning screen of Embodiment 1. The learning screen display section 321 displays the learning screen 6 shown in Figure 9 when the learning screen display button 501 is operated in the initial screen 5. The learning screen 6 has a learning image display field 601 for displaying a learning target image, a product selection field 602 for selecting the product name of the substrate 4, and a learning image selection field 603 for selecting the learning target image. In addition, the learning screen 6 has a learning image imaging button 604 for imaging the learning target image, a learning image read-in button 605 for reading in the image information of the selected learning target image, and a learning image deletion button 606 for deleting the image information of the selected learning target image.

[0118] Additionally, the learning screen 6 includes a position learning information display bar 607 for displaying position learning information, an abnormality learning information display bar 608 for displaying abnormality learning information, and a position learning information reflection button 609 for reflecting the position of the metal component 401 onto the position learning information display bar 607. Furthermore, the learning screen 6 includes a first abnormality learning information reflection button 610 for reflecting the position of anomalies on the surface of the substrate 4 onto the abnormality learning information display bar 608, and a magnification display bar 611 for magnifying the image of the surface of the substrate 4. Additionally, the learning screen 6 includes a second abnormality learning information reflection button 612 for reflecting the size of anomalies on the surface of the substrate 4 onto the abnormality learning information display bar 608. Finally, the learning screen 6 includes a save button 613 for saving the contents of the position learning information display bar 607 and the abnormality learning information display bar 608 as position learning information and abnormality learning information, and a learning button 614 for performing machine learning.

[0119] Additionally, the learning screen 6 includes a return button 615 for returning to the initial screen 5, an anomaly category selection bar 616 for selecting the type of anomaly on the surface of the substrate 4, and an anomaly category addition button 617 for adding items of anomaly category on the surface of the substrate 4 to the anomaly learning information display bar 608. Furthermore, the learning screen 6 includes an anomaly category deletion button 618 for deleting items of anomaly category on the surface of the substrate 4 selected in the anomaly learning information display bar 608.

[0120] like Figure 9 As shown, in learning screen 6, the learning image display bar 601 is located on the upper left side of learning screen 6, the product selection bar 602 is located on the upper right side of learning screen 6, and the learning image selection bar 603 is located below the product selection bar 602. Additionally, the learning image camera button 604 is located on the lower left side of the learning image selection bar 603, the learning image read button 605 is located below the learning image selection bar 603, and the learning image delete button 606 is located on the lower right side of the learning image selection bar 603. Furthermore, the location learning information display bar 607 is located on the lower left side of the learning image display bar 601, the abnormal learning information display bar 608 is located below the learning image display bar 601, and the location learning information feedback button 609 is located below buttons 604 to 606.

[0121] Further, the 1st abnormality learning information reflecting button 610 is provided below the position learning information reflecting button 609, the enlarged display column 611 is provided below the 1st abnormality learning information reflecting button 610, and the 2nd abnormality learning information reflecting button 612 is provided below the enlarged display column 611. Further, the save button 613 is provided below and to the left of the 2nd abnormality learning information reflecting button 612, the learning button 614 is provided below the 2nd abnormality learning information reflecting button 612, and the return button 615 is provided below and to the right of the 2nd abnormality learning information reflecting button 612. Further, the abnormality category selection column 616 is provided below and to the right of the learning image display column 601, the abnormality category addition button 617 is provided below and to the left of the abnormality category selection column 616, and the abnormality category deletion button 618 is provided below and to the right of the abnormality category selection column 616.

[0122] The learning screen display section 321 acquires all of the information of the item of "product name" of the product name stored in the setting information storage section 304 so as to be selectable in the product selection column 602. Further, the learning screen display section 321, when the user selects the product name in the product selection column 602 via the input section 301, refers to the information of the item of the "learning image information storage area" of the device setting information corresponding to the selected product name stored in the setting information storage section 304, and displays the file names of the learning target images with respect to the product name selected in the learning image selection column 603. Further, the learning screen display section 321, when the user operates the learning image reading button 605 after selecting the file name in the learning image selection column 603 via the input section 301, acquires the image information of the learning target image of the file name selected from the image information storage section 305, and displays the learning target image in the learning image display column 601.

[0123] Further, when the user operates the learning image photographing button 604 via the input section 301 in a state where the product name is selected in the product selection column 602, the illumination light control section 303 controls the illumination light output section 210, 211 to output the illumination light, and the photographing control section 202 controls the photographing section 209 to photograph the learning target image, and stores the image information in the image information storage section 305. Then, the learning screen display section 321 displays the file name of the photographed learning target image in the learning image selection column 603, and displays the photographed learning target image in the learning image display column 601.

[0124] Further, the learning screen display section 321 displays the cursor 619 in the learning image display field 601, and the user can change the display position of the cursor 619 via the input section 301. Thus, the user can change the display position of the cursor 619 from the learning target image displayed in the learning image display field 601 to the position 620 corresponding to the glossy portion 402 of the metal member 401 or the position 621 corresponding to the abnormality of the surface of the substrate 4 via the input section 301. Further, the learning screen display section 321 displays the enlarged image of the position 620 corresponding to the glossy portion 402 in the enlarged display field 611 when the user changes the display position of the cursor 619 to the position 620 corresponding to the glossy portion 402. Further, the learning screen display section 321 displays the measured values of the X coordinate and the Y coordinate corresponding to the display position of the cursor 619 in the position learning information display field 607 when the user operates the position learning information reflection button 609 via the input section 301 in a state where the enlarged image is displayed in the enlarged display field 611.

[0125] Further, the learning screen display section 321 displays the enlarged image of the position 621 corresponding to the abnormality of the surface of the substrate 4 in the enlarged display field 611 when the user changes the display position of the cursor 619 to the position 621 corresponding to the abnormality of the surface of the substrate 4. At this time, the learning screen display section 321 displays the area display frame 622 in a state where the image 623 of the abnormality centered on the position 621 corresponding to the abnormality of the surface of the substrate 4 is surrounded with the area display frame 622. Further, the learning screen display section 321 can select the category of the abnormality of the four kinds of "rust", "warpage", "scratch", and "adhering matter" in the abnormality category selection field 616.

[0126] When the learning screen display section 321 receives an operation of the abnormality category addition button 617 from the user via the input section 301 in a state where the area display frame 622 is displayed in the enlarged display column 611, the learning screen display section 321 adds a project of abnormality learning, and displays the category of the selected abnormality in the project of "category" of the abnormality learning information display column 608. Further, when the learning screen display section 321 receives an operation of the 1st abnormality learning information reflection button 610 from the user in a state where the user selects the kind of the abnormality learning information display column 608 via the input section 301, the learning screen display section 321 displays the measured values of the X coordinate and the Y coordinate corresponding to the display position of the cursor 619 in the projects of "X coordinate" and "Y coordinate" of the selected abnormality learning of the abnormality learning information display column 608. Further, when the learning screen display section 321 receives an operation of the 2nd abnormality learning information reflection button 612 from the user in a state where the user selects the kind of the abnormality learning information display column 608 via the input section 301, the learning screen display section 321 displays the size of the area display frame 622 in the project of "size" of the selected abnormality learning of the abnormality learning information display column 608.

[0127] Further, when the learning screen display section 321 receives an operation of the abnormality category deletion button 618 from the user in a state where the user selects the kind of the abnormality learning information display column 608 via the input section 301, the learning screen display section 321 sets the project of the selected abnormality learning to non-display.

[0128] Further, when the learning screen display section 321 receives an operation of the save button 613 from the user via the input section 301 in a state where the information is displayed in the position learning information display column 607, the learning screen display section 321 refers to the project of "position learning information storage area" of the image display information of the selected product name stored in the setting information storage section 304, and stores the information displayed in the position learning information display column 607 as the position learning information of the selected product name in the position learning information storage section 306. Further, when the learning screen display section 321 receives an operation of the save button 613 from the user via the input section 301 in a state where the information is displayed in the abnormality learning information display column 608, the learning screen display section 321 refers to the project of "abnormality learning information storage area" of the image display information of the selected product name stored in the setting information storage section 304, and stores the information displayed in the abnormality learning information display column 608 as the abnormality learning information of the selected product name in the abnormality learning information storage section 307.

[0129] Further, the learning screen display section 321 causes the position detection model generation section 310 to generate the position detection model when the user operates the learning button 614 via the input section 301 in a state where the position learning information display field 607 has display. Further, the learning screen display section 321 causes the abnormality detection model generation section 315 to generate the abnormality detection model when the user operates the learning button 614 via the input section 301 in a state where the abnormality learning information display field 608 has display. Further, the learning screen display section 321 causes the position detection model generation section 310 and the abnormality detection model generation section 315 to suspend the generation when the user operates the learning button 614 via the input section 301 during the generation of the position detection model and the abnormality detection model.

[0130] Further, the learning screen display section 321 sets the selected file name of the learning image selection field 603 to non-display when the user operates the learning image deletion button 606 via the input section 301 in a state where the selected file name is selected. Further, at this time, the learning screen display section 321 sets the information associated with the learning target image of the selected file name in the learning image display field 601, the enlarged display field 611, the position learning information display field 607, and the abnormality learning information display field 608 to non-display. Further, at this time, the learning screen display section 321 causes the image information storage section 305 to delete the image information of the learning target image of the selected file name. Further, the learning screen display section 321 sets the learning screen 6 to non-display and causes the initial screen display section 320 to display the initial screen 5 when the user operates the return button 615 via the input section 301.

[0131] Figure 10 is a drawing illustrating a display example of the inspection screen of Embodiment 1. The inspection screen display section 322 displays the Figure 10The inspection screen 7 shown is equipped with an inspection image display bar 701 for displaying an image of the object being inspected, a product selection bar 702 for selecting the product name of the substrate 4, and a position inspection information display bar 703 for displaying information related to the inspection of the position of the metal component 401. Additionally, the inspection screen 7 includes an anomaly inspection information display bar 704 for displaying information related to the inspection of surface anomalies of the substrate 4, and a marker display switch bar 705 for switching the presence or absence of markers in the inspection image display bar 701 based on the measured values ​​and reference values ​​in the position inspection information display bar 703. Furthermore, the inspection screen 7 includes a comprehensive judgment bar 706 for displaying the combined judgment results of the position inspection of the metal component 401 and the surface anomaly inspection of the substrate 4, and an inspection execution button 707 for performing the inspection. Finally, the inspection screen 7 includes a save button 708 for saving information displayed in the position inspection information display bar 703 and the anomaly inspection information display bar 704, and a return button 709 for returning to the initial screen display.

[0132] like Figure 10 As shown, in inspection screen 7, the inspection image display bar 701 is located on the upper left side of inspection screen 7, and the product selection bar 702 is located on the upper right side of inspection screen 7. Additionally, the position inspection information display bar 703 is located below the inspection image display bar 701, and the anomaly inspection information display bar 704 is located to the right of the position inspection information display bar 703. Furthermore, the marker display toggle bar 705 is located below the product selection bar 702, the comprehensive judgment bar 706 is located below the marker display toggle bar 705, and the inspection execution button 707 is located below the comprehensive judgment bar 706. Additionally, the save button 708 is located to the lower left of the inspection execution button 707, and the return button 709 is located to the lower right of the inspection execution button 707.

[0133] When the inspection screen display section 322 displays the inspection screen 7, the illumination light control section 303 controls the illumination light output section 210, 211 to output the illumination light, and the imaging control section 302 controls the imaging section 209 to display the imaged inspection target image in the inspection image display field 701. The inspection screen display section 322 acquires all the information of the item of "product name" stored in the setting information storage section 304, i.e., the product name, so that it can be selected in the product selection field 702. In addition, when the user operates the inspection execution button 707 in a state where the product name is selected in the product selection field 702 via the input section 301, the inspection screen display section 322 causes the position detection section 312 to detect the position of the metal member 401 as described above. In addition, the inspection screen display section 322 causes the position standard determination section 313 to determine whether or not it conforms to the standard, and causes the position output section 314 to output the measured value, the reference value, the standard value, the determination value, and the determination result information of the X coordinate and the Y coordinate. In addition, the inspection screen display section 322 displays the measured value, the reference value, the standard value, the determination value, and the determination result information of the X coordinate and the Y coordinate output in the position inspection information display field 703.

[0134] In addition, the inspection screen display section 322 can select the item of "measured value" and the item of "reference value" in the mark display switching field 705. When the user selects the item of "measured value" in the mark display switching field 705 via the input section 301 in a state where the display of the measured value of the X coordinate and the Y coordinate is present in the position inspection information display field 703, the inspection screen display section 322 displays the mark 711 for the measured value at the position 710 corresponding to the measured value of the X coordinate and the Y coordinate in the inspection image display field 701. In addition, when the user selects the item of "reference value" in the mark display switching field 705 via the input section 301 in a state where the display of the reference value of the X coordinate and the Y coordinate is present in the position inspection information display field 703, the inspection screen display section 322 displays the mark (not illustrated) for the reference value at the position corresponding to the reference value of the X coordinate and the Y coordinate in the inspection image display field 701. In addition, the inspection screen display section 322 can move the position of the mark for the reference value via the input section 301 in a state where the mark for the reference value is displayed in the inspection image display field 701. The display of the reference value of the X coordinate and the Y coordinate is changed in the position inspection information display field 703.

[0135] Further, the inspection screen display section 322 causes the abnormality detection section 317 to detect an abnormality of the surface of the substrate 4 when the user operates the inspection execution button 707 in a state where the product name is selected in the product selection column 702 via the input section 301, as described above. Further, the inspection screen display section 322 causes the abnormality criterion determination section 318 to determine whether or not the criterion is met, and causes the abnormality output section 319 to output the category of the abnormality, the values of the X coordinate and the Y coordinate of the abnormality, the measured value of the size of the abnormality, the criterion value, and the determination result information. Further, the inspection screen display section 322 displays the output category of the abnormality, the values of the X coordinate and the Y coordinate of the abnormality, the measured value of the size of the abnormality, the criterion value, and the determination result information in the abnormality inspection information display column 704.

[0136] Further, the inspection screen display section 322 causes the comprehensive determination section 323 to perform comprehensive determination when the user operates the inspection execution button 707 in a state where the product name is selected in the product selection column 702 via the input section 301, and causes the comprehensive determination result output section 324 to output the comprehensive determination result information. Further, the inspection screen display section 322 displays the output comprehensive determination result information in the comprehensive determination column 706.

[0137] Further, the inspection screen display section 322 stores the measured value, the reference value, the criterion value, the determination value, and the determination result information of the X coordinate and the Y coordinate displayed in the position inspection information display column 703 in the comprehensive determination result storage section 325 as the position inspection information when the user operates the save button 708 via the input section 301 in a state where the position inspection information display column 703 is displayed. At this time, if the display of the reference value of the X coordinate and the Y coordinate in the position inspection information display column 703 is changed, the information of the item of "position inspection reference" of the product setting list information can be updated to the changed value in the setting information storage section 304. Further, the inspection screen display section 322 stores the category of the abnormality, the values of the X coordinate and the Y coordinate of the abnormality, the measured value of the size of the abnormality, the criterion value, and the determination result information displayed in the abnormality inspection information display column 704 in the comprehensive determination result storage section 325 as the abnormality inspection information when the user operates the save button 708 via the input section 301 in a state where the abnormality inspection information display column 704 is displayed. Further, the inspection screen display section 322 stores the comprehensive determination information displayed in the comprehensive determination column 706 in the comprehensive determination result storage section 325 when the user operates the save button 708 via the input section 301 in a state where the comprehensive determination column 706 is displayed. Further, the inspection screen display section 322 causes the initial screen display section 320 to display the initial screen 5 when the user operates the return button 709 via the input section 301.

[0138] The comprehensive determination section 323 comprehensively determines the inspection result of the position of the metal member 401 and the inspection result of the abnormality of the surface of the substrate 4 as described above. If the determination result information outputted from the position output section 314 is determination OK information and the determination result information outputted from the abnormality output section 319 is all determination OK information, the comprehensive determination section 323 determines that the comprehensive determination is OK as the comprehensive determination result. On the other hand, if the determination result information outputted from the position output section 314 includes determination NG information or one of the determination result information outputted from the abnormality output section 319 includes determination NG information, the comprehensive determination section 323 determines that the comprehensive determination is NG as the comprehensive determination result.

[0139] If the comprehensive determination section 323 determines that the comprehensive determination is OK, the comprehensive determination result output section 324 outputs determination OK information as the comprehensive determination result information, and on the other hand, if the comprehensive determination section 323 determines that the comprehensive determination is NG, determination NG information is outputted as the comprehensive determination result information. Further, the determination OK information and the determination NG information are, for example, information of "determination OK" and "determination NG".

[0140] The comprehensive determination result storage section 325 stores the position inspection information displayed in the position inspection information display field 703 of the inspection screen 7, the abnormality inspection information displayed in the abnormality inspection information display field 704 of the inspection screen 7, and the comprehensive determination information displayed in the comprehensive determination field 706 of the inspection screen 7 as described above.

[0141] Figure 11 is a flowchart showing the position detection model generation processing of Embodiment 1. Using the flowchart shown in Figure 11 The operation of the inspection device 1 to generate the position detection model will be described. The position detection model generation section 310 starts the execution of the position detection model generation processing when the user operates the learning button 614 via the input section 301 on the learning screen 6. First, as shown in Figure 11

[0142] ​Further, the position detection model generating section 310 acquires a learning target image from the image information corresponding to the product name selected in the learning screen 6 (step S102). Further, the position detection model generating section 310 acquires the measured values of the X coordinate and the Y coordinate of the position learning information corresponding to the product name selected in the learning screen 6 and the values of the offset of the X coordinate and the Y coordinate of the position check setting information, and causes the reference position calculating section 308 to calculate the reference values of the X coordinate and the Y coordinate, that is, the reference position (step S103). For example, if "board A" is selected in the learning screen 6, the reference position calculating section 308 acquires "261" and "115" from the position learning information and "(XOA1, YOA1)" from the position check setting information, and calculates (261+XOA1, 115+YOA1).

[0143] Further, the position detection model generating section 310 acquires the pattern size of the position check setting information corresponding to the product name selected in the learning screen 6, and extracts an image of the pattern size centered on the reference values of the X coordinate and the Y coordinate from the learning target image (step S104). For example, if "board A" is selected in the learning screen 6, the position detection model generating section 310 acquires "(WA1, LA1)" of the position check setting information, and extracts an image of a range having a horizontal width of WA1 and a vertical width of LA1 centered on the coordinate position of (261+XOA1, 115+YOA1) from the learning target image. Further, the position detection model generating section 310 causes the extracted image to be subjected to machine learning as an OK classified image, and updates the position detection model (step S105).

[0144] Further, the position detection model generating section 310 acquires the values of the X coordinate and the Y coordinate of the abnormality learning information stored in the abnormality learning information storage section 307, and determines whether or not an abnormality of the surface of the board 4 is detected in the extracted image (step S106). For example, if "board A" is selected in the learning screen 6, the position detection model generating section 310 acquires "110" and "170" of the abnormality learning information, and determines whether or not a position of the coordinates of (110, 170) is included in the extracted image of the pattern size.

[0145] The position detection model generation section 310, in a case where an abnormality is detected (step S106; YES), acquires the masking size from the position check setting information corresponding to the product name selected in the learning screen 6, and causes the masking image generation section 309 to generate a masking image that masks the extracted image by the masking size (step S107). For example, if "substrate A" is selected in the learning screen 6, the masking image generation section 309 acquires "(MWA1, MLA1)" from the position check setting information when the coordinate position of (110, 170) is included in the image of the pattern size that is extracted. In addition, the masking image generation section 309 replaces the range of which the horizontal width is MWA1 and the vertical width is MLA1 centered on the coordinate position of (110, 170) with an image filled in white.

[0146] In addition, the position detection model generation section 310 causes the generated masking image to be subjected to machine learning as an OK-classified image, and updates the position detection model (step S108). In addition, the position detection model generation section 310, after the position detection model based on the masking image is updated or in a case where an abnormality is not detected (step S106; NO), extracts an image of a pattern size centered on an arbitrary value other than the reference values of the X coordinate and the Y coordinate from the learning target image (step S109). For example, if "substrate A" is selected in the learning screen 6, the position detection model generation section 310 acquires "(WA1, LA1)" of the position check setting information, and extracts an image of a range of which the horizontal width is WA1 and the vertical width is LA1 centered on an arbitrary coordinate position other than (261+XOA1, 115+YOA1) from the learning target image. In addition, the position detection model generation section 310 causes the extracted image to be subjected to machine learning as an NG-classified image, and updates the position detection model (step S110).

[0147] In addition, the position detection model generation section 310 determines whether machine learning is performed using all of the learning target images (step S111). The position detection model generation section 310, in a case where machine learning is not performed using all of the learning target images (step S111; NO), repeats the processes of steps S102 to S110 until machine learning is performed using all of the learning target images. On the other hand, the position detection model generation section 310, in a case where machine learning is performed using all of the learning target images (step S111; YES), ends the processing.

[0148] Figure 12 is a flowchart showing the abnormality detection model generation processing of Embodiment 1. Using the flowchart shown in Figure 12 The operation of the inspection device 1 to generate the abnormality detection model will be described. The abnormality detection model generation section 315 starts execution of the abnormality detection model generation processing when the user operates the learning button 614 via the input section 301 in the learning screen 6. First, as inFigure 12 As shown, the abnormality detection model generation section 315 generates an abnormality detection model before machine learning from the model form of the abnormality check setting information corresponding to the product name selected in the learning screen 6 (step S201). For example, if "substrate A" is selected in the learning screen 6, the abnormality detection model generation section 315 acquires "MA2" from the abnormality check setting information, and generates an abnormality detection model of which the model form is "MA2".

[0149] In addition, the abnormality detection model generation section 315 acquires a learning target image from the image information corresponding to the product name selected in the learning screen 6 (step S202). In addition, the abnormality detection model generation section 315 acquires the values of the X coordinate and the Y coordinate of the abnormality learning information corresponding to the product name selected in the learning screen 6 and the pattern size of the abnormality check setting information (step S203). For example, if "substrate A" is selected in the learning screen 6, the abnormality detection model generation section 315 acquires "110" and "170" of the abnormality learning information and "(WA2, LA2)" of the abnormality check setting information. In addition, the abnormality detection model generation section 315 extracts an image of a pattern size centered on the values of the X coordinate and the Y coordinate from the learning target image (step S204). For example, if "substrate A" is selected in the learning screen 6, the abnormality detection model generation section 315 extracts an image of a range of which the horizontal width is WA2 and the vertical width is LA2 centered on the coordinate position of (110, 170) from the learning target image.

[0150] In addition, the abnormality detection model generation section 315 acquires the category of the item including the X coordinate and the Y coordinate acquired from the abnormality learning information, and performs machine learning of the extracted image as a classified image of the acquired category, and updates the abnormality detection model (step S205). For example, if "substrate A" is selected in the learning screen 6, the abnormality detection model generation section 315 acquires "adhering matter" of the abnormality learning information, and performs machine learning of the extracted image as an abnormal image of the category of adhering matter, thereby updating the abnormality detection model.

[0151] Further, the abnormality detection model generation unit 315 determines whether or not the values of the X coordinates and the Y coordinates of all the items of the abnormality learning information are acquired (step S206). The abnormality detection model generation unit 315 repeats the processes of steps S203 to S205 until the values of the X coordinates and the Y coordinates of all the items are acquired, in a case where the values of the X coordinates and the Y coordinates of all the items are not acquired (step S206; No). On the other hand, the abnormality detection model generation unit 315 determines whether or not the machine learning is performed using all the learning target images, in a case where the values of the X coordinates and the Y coordinates of all the items are acquired (step S206; Yes). The abnormality detection model generation unit 315 repeats the processes of steps S202 to S206 until the machine learning is performed using all the learning target images, in a case where the machine learning is not performed using all the learning target images (step S207; No). On the other hand, the abnormality detection model generation unit 315 ends the process, in a case where the machine learning is performed using all the learning target images (step S207; Yes).

[0152] Figure 13 is a flowchart showing the abnormality inspection process of Embodiment 1. Using the flowchart shown in Figure 13 , the operation of the inspection device 1 to inspect the surface of the substrate 4 for abnormalities is described. The abnormality detection unit 317 starts the execution of the abnormality inspection process when the user operates the inspection execution button 707 via the input unit 301 on the inspection screen 7. First, as shown in Figure 13 , the abnormality detection unit 317 acquires the inspection target image including the substrate 4 imaged by the imaging unit 209 (step S301). Further, the abnormality detection unit 317 acquires the image size from the device setting information corresponding to the product name selected on the inspection screen 7, and acquires the abnormality detection range and the pattern size of the abnormality inspection setting information (step S302). For example, if "substrate A" is selected on the inspection screen 7, the abnormality detection unit 317 acquires "SZA1" of the device setting information, acquires "XSA2 to XEA2, YSA2 to YEA2" and "(WA2, LA2)" of the abnormality inspection setting information.

[0153] Further, the abnormality detection section 317 extracts an image of a pattern size from the inspection target image within the abnormality detection range (step S303). For example, if "substrate A" is selected on the inspection screen 7, the abnormality detection section 317 extracts an image of a range of a horizontal width of WA2 and a vertical width of LA2 centered on coordinate positions selected within the abnormality detection range of a rectangular shape with the starting point (XSA2, YSA2) and the ending point (XEA2, YEA2) as opposite vertices. Further, the abnormality detection section 317 applies the abnormality detection model corresponding to the product name selected on the inspection screen 7 to the extracted image, and calculates the score of each of the categories of abnormalities (step S304). For example, if "substrate A" is selected on the inspection screen 7, the abnormality detection section 317 calculates the score SC21 of "rust", the score SC22 of "warpage", the score SC23 of "scratch", and the score SC24 of "adhering matter".

[0154] Further, the abnormality detection section 317 acquires the score threshold value of the category of the abnormality inspection setting information corresponding to the product name selected on the inspection screen 7, and determines whether there is a category of which the calculated score is equal to or higher than the score threshold value (step S305). For example, if "substrate A" is selected on the inspection screen 7, if SC21 < SC22 < SC23 < SC24, SC21 < SA21, SC22 < SA22, SC23 < SA23, and SC24 > SA24, the abnormality detection section 317 determines that the score SC24 of the adhering matter is equal to or higher than the score threshold value SA24. The abnormality detection section 317 determines the abnormality of the category of which the maximum score among the scores equal to or higher than the score threshold value (step S305; Yes) (step S306). For example, if "substrate A" is selected on the inspection screen 7, the abnormality detection section 317 determines the abnormality of the adhering matter of the maximum score SC24.

[0155] Further, the abnormality detection section 317 calculates the X coordinate and the Y coordinate and the size of the abnormality determined (step S307). For example, if "substrate A" is selected on the inspection screen 7, the abnormality detection section 317 calculates (X2, Y2) and SZ2. Further, the abnormality criterion determination section 318 acquires the value of the abnormality inspection criterion from the product setting list information corresponding to the product name selected on the inspection screen 7 (step S308). For example, if "substrate A" is selected on the inspection screen 7, the abnormality criterion determination section 318 acquires "SZA3" of the product setting list information.

[0156] Further, the abnormality criterion determination section 318 determines whether the calculated size value is below the abnormality check criterion value (step S309). The abnormality criterion determination section 318, in the case where it is below the abnormality check criterion value (step S309; YES), causes the abnormality output section 319 to output determination OK information as determination result information (step S310). On the other hand, the abnormality criterion determination section 318, in the case where it exceeds the abnormality check criterion value (step S309; NO), causes the abnormality output section 319 to output determination NG information as determination result information (step S311). For example, if "substrate A" is selected on the check screen 7, if SZ2≤ SZA3, the abnormality criterion determination section 318 determines that it is in conformity with the criterion, the abnormality output section 319 outputs determination OK information, on the other hand, if SZ2> SZA3, it is determined that it is not in conformity with the criterion, the abnormality output section 319 outputs determination NG information.

[0157] Further, the abnormality detection section 317, after the determination result information is output or in the case where the calculated score is not above the score threshold value (step S305; NO), determines whether the images of the pattern sizes are all extracted from the check target image within the abnormality detection range (step S312). For example, if "substrate A" is selected on the check screen 7, the abnormality detection section 317 determines whether the images of the pattern sizes are extracted from all of the selectable coordinates within the abnormality detection range of the rectangular shape having the starting point (XSA2, YSA2) and the end point (XEA2, YEA2) as diagonal vertices. The abnormality detection section 317, in the case where the images of the pattern sizes are not all extracted (step S312; NO), repeats the processing of steps S303 to S311 until the images of the pattern sizes are all extracted. On the other hand, the abnormality detection section 317, in the case where the images of the pattern sizes are all extracted (step S312; YES), ends the processing.

[0158] Figure 14 and Figure 15 is a flowchart showing the position check processing of Embodiment 1. Using the flowchart shown in Figure 14 and Figure 15 the operation of the check device 1 to check the position of the metal member 401 is explained. The position detection section 312 starts the execution of the position check processing when the user operates the check execution button 707 via the input section 301 on the check screen 7. First, as shown in Figure 14

[0159] ​Further, the position detection section 312 acquires the image size from the device setting information corresponding to the product name selected in the inspection screen 7, and acquires the position detection range and the pattern size of the position inspection setting information (step S402). For example, if "substrate A" is selected in the inspection screen 7, the position detection section 312 acquires "SZA1" of the device setting information, acquires "XSA1 to XEA1, YSA1 to YEA1" and "(WA1, LA1)" of the position inspection setting information. Further, the position detection section 312 extracts an image of the pattern size from the inspection target image within the position detection range (step S403). For example, if "substrate A" is selected in the inspection screen 7, the position detection section 312 extracts an image of a range having a horizontal width of WA1 and a vertical width of LA1 centered on the coordinate position selected within the position detection range of a rectangular shape having the starting point (XSA1, YSA1) and the end point (XEA1, YEA1) as the diagonal vertices from the inspection target image.

[0160] Further, the position detection section 312 acquires the X coordinate and the Y coordinate determined to be abnormal calculated by the abnormality detection section 317, and determines whether or not an abnormality of the surface of the substrate 4 is detected in the extracted image (step S404). For example, if "substrate A" is selected in the inspection screen 7, the position detection section 312 determines whether or not the position of the coordinate (X2, Y2) calculated by the abnormality detection section 317 is included in the extracted image. The position detection section 312 acquires the shield size from the position inspection setting information in a case where an abnormality is detected (step S404; Yes), and causes the shield image generation section 309 to generate a shield image in which the extracted image is shielded by the shield size (step S405). For example, if "substrate A" is selected in the inspection screen 7, the shield image generation section 309 acquires "(MWA1, MLA1)" from the position inspection setting information when the position of the coordinate (X2, Y2) is included in the extracted image of the pattern size. Further, the shield image generation section 309 replaces a range having a horizontal width of MWA1 and a vertical width of MLA1 centered on the coordinate position of (X2, Y2) with an image filled with white.

[0161] Furthermore, after the masking image is generated or no abnormality is detected (step S404; no), the position detection unit 312 applies the position detection model corresponding to the product name selected on the inspection screen 7 to the image and calculates a score (step S406). Additionally, the position detection unit 312 determines whether the pattern-sized image has been completely extracted from the inspection object image within the position detection range (step S407). If the pattern-sized image has not been completely extracted (step S407; no), the position detection unit 312 repeats steps S403 to S406 until the pattern-sized image has been completely extracted. For example, if "Substrate A" is selected on the inspection screen 7, the position detection unit 312 continues to extract pattern-sized images from the inspection object image until the scores SC11 to SC1N of N images are calculated.

[0162] On the other hand, such as Figure 15 As shown, when the position detection unit 312 extracts all images of the pattern size (step S407; Yes), it acquires the score threshold of the position check setting information and determines whether there is an image whose calculated score is higher than the score threshold (step S408). If there is no image whose calculated score is higher than the score threshold (step S408; No), the position detection unit 312 outputs a detection failure message (step S409) and ends the process. On the other hand, if there is an image whose calculated score is higher than the score threshold (step S408; Yes), the position detection unit 312 calculates the measured values ​​of the X and Y coordinates based on the image with the highest score among the scores that are higher than the score threshold (step S410). For example, if "Substrate A" is selected on the inspection screen 7, the position detection unit 312 calculates (X1, Y1) = (X0 + XOA1, Y0 + YOA1) based on the measured values ​​of the X and Y coordinates when acquiring the image with the highest score SC1M among the scores that are SA1 or higher.

[0163] Additionally, the position standard determination unit 313 obtains the reference values ​​of the X and Y coordinates of the position inspection reference and the standard values ​​of the X and Y coordinates of the position inspection standard from the product setting overview information corresponding to the product name selected on the inspection screen 7, i.e., the threshold values ​​of the determination values ​​(step S411). For example, if "substrate A" is selected on the inspection screen 7, the position standard determination unit 313 calculates "(XA1, YA1)" and "(XA2, YA2)" based on the product setting overview information. Furthermore, the position standard determination unit 313 calculates the difference between the measured values ​​of the X and Y coordinates and the reference values ​​of the X and Y coordinates of the position inspection reference (step S412). For example, if "substrate A" is selected on the inspection screen 7, the position standard determination unit 313 calculates (X1 - XA1, Y1 - YA1).

[0164] Further, the position criterion determination section 313 determines whether the absolute values of the difference values of the X coordinates and the Y coordinates are equal to or lower than the threshold value (step S413). The position criterion determination section 313, in a case where the absolute values of the difference values of the X coordinates and the Y coordinates are equal to or lower than the threshold value (step S413; Yes), causes the position output section 314 to output determination OK information as determination result information (step S414), and ends the processing. On the other hand, the position criterion determination section 313, in a case where any of the absolute values of the difference values of the X coordinates and the Y coordinates exceeds the threshold value (step S413; No), causes the position output section 314 to output determination NG information as determination result information (step S415), and ends the processing. For example, if |X1-XA1|≤XA2 and |Y1-YA1|≤YA2, the position criterion determination section 313 determines that the criterion is satisfied, and the position output section 314 outputs determination OK information. On the other hand, if |X1-XA1|>XA2 or |Y1-YA1|>YA2, the position criterion determination section 313 determines that the criterion is not satisfied, and the position output section 314 outputs determination NG information.

[0165] As explained above, according to the inspection device 1 of Embodiment 1, the position detection section 312 applies the inspection target image imaged by the imaging section 209 to the position detection model, and calculates a score for each image with respect to the pattern size of the image extracted from the position detection range of the inspection target image. Then, the position detection section 312 detects the position of the metal member 401 in the inspection target image by determining the image of which the score becomes the maximum score equal to or higher than the score threshold value, and the position output section 314 outputs the detected position of the metal member 401.

[0166] As a result, the inspection device 1, by generating the position detection model by machine learning using the image information including the image of the metal member 401 in which the abnormality is present on the surface, can detect the position of the metal member 401 tolerantly to the abnormality on the surface of the metal member 401, and can check whether the position of the metal member 401 satisfies the position inspection criterion.

[0167] Further, as in Embodiment 1, the inspection device 1 preferably performs machine learning using the position inspection setting information set by the user and the position learning information, but can perform machine learning using only the image information without using the position inspection setting information and the position learning information. That is, the setting information storage section 304 and the position learning information storage section 306 can be omitted.

[0168] Further, according to the inspection device 1 of Embodiment 1, the position detection model generation section 310 generates the position detection model indicating the relationship between the learning target image and the position of the metal member 401 by machine learning using the image information of the learning target image acquired in advance, the position inspection setting information set by the user, and the position learning information.

[0169] As a result, the inspection device 1 is able to learn the positional relationship of the learning target image and the position of the metal member 401 using the image information of the learning target image, the position inspection setting information, and the position learning information.

[0170] Further, in Embodiment 1, the inspection device 1 is provided with the position detection model generation section 310 and the position detection model storage section 311, but can not be provided with the position detection model generation section 310 and the position detection model storage section 311. For example, an external learning device can be provided with the position detection model generation section 310 and the position detection model storage section 311, the position detection model generation section 310 generates the position detection model by machine learning using the above information, and stores the generated position detection model in the position detection model storage section 311. In this case, the inspection device 1 needs to acquire the position detection model stored in the learning device.

[0171] Further, according to the inspection device 1 of Embodiment 1, the position standard determination section 313 determines whether the position of the metal member 401 detected by the position detection section 312 conforms to the position inspection standard decided in advance. Specifically, the position standard determination section 313 determines whether the position of the metal member 401 conforms to the position inspection standard by determining whether the difference value between the measured value of the X coordinate and the Y coordinate of the substrate 4 detected at the time of position detection and the reference value of the X coordinate and the Y coordinate of the substrate 4 on the design is a value allowed.

[0172] As a result, the inspection device 1 is able to determine whether the position of the detected metal member 401 conforms to the position inspection standard.

[0173] Further, according to the inspection device 1 of Embodiment 1, the abnormality detection section 317 applies the inspection target image imaged by the imaging section 209 to the abnormality detection model, and calculates the score of each abnormality category with respect to the image of the pattern size extracted from the abnormality detection range of the inspection target image. Then, the abnormality detection section 317 detects the abnormality of the surface of the substrate 4 in the inspection target image by determining the category of which the score becomes the maximum score of the score threshold or more, and the abnormality output section 319 outputs the category, position, and size of the detected abnormality of the surface of the substrate 4.

[0174] As a result, the inspection device 1 is able to detect the category, position, and size of the abnormality of the surface of the substrate 4, and is able to inspect whether the abnormality inspection conforms to the standard even if there is an abnormality of the surface of the substrate 4.

[0175] Further, as in Embodiment 1, the inspection device 1 preferably detects the category of the abnormality in the abnormality inspection, but can not detect the category of the abnormality.

[0176] Further, as in Embodiment 1, the inspection apparatus 1 preferably performs machine learning using the abnormality inspection setting information set by the user and the abnormality learning information, but can perform machine learning using only the image information without using the abnormality inspection setting information and the abnormality learning information. That is, the setting information storage section 304 and the abnormality learning information storage section 307 can be omitted.

[0177] Further, as in Embodiment 1, in the inspection apparatus 1, in order to perform abnormality inspection, the abnormality detection model generation section 315 preferably generates an abnormality detection model, the abnormality detection section 317 detects an abnormality of the surface of the substrate 4, and the abnormality output section 319 outputs a category, a position, and a size of the detected abnormality of the surface of the substrate 4. However, the abnormality detection model generation section 315, the abnormality detection section 317, and the abnormality output section 319 can be omitted.

[0178] Further, according to the inspection apparatus 1 of Embodiment 1, the abnormality detection model generation section 315 generates an abnormality detection model representing a relationship between the learning target image and the abnormality of the surface of the substrate 4 by machine learning using the image information of the learning target image acquired in advance, the abnormality inspection setting information set by the user, and the abnormality learning information.

[0179] As a result, the inspection apparatus 1 can learn the relationship between the learning target image and the abnormality of the surface of the substrate 4 using the image information of the learning target image, the abnormality inspection setting information, and the abnormality learning information.

[0180] Further, in Embodiment 1, the inspection apparatus 1 includes the abnormality detection model generation section 315 and the abnormality detection model storage section 316, but can not include the abnormality detection model generation section 315 and the abnormality detection model storage section 316. For example, in the inspection apparatus 1, an external learning apparatus can include the abnormality detection model generation section 315 and the abnormality detection model storage section 316, the abnormality detection model generation section 315 generates an abnormality detection model by machine learning using the above information, and stores the generated abnormality detection model in the abnormality detection model storage section 316. In this case, the inspection apparatus 1 needs to acquire the abnormality detection model stored in the learning apparatus.

[0181] Further, according to the inspection apparatus 1 of Embodiment 1, even if there is an abnormality of the surface of the substrate 4 detected by the abnormality detection section 317, the abnormality standard determination section 318 can determine whether or not the abnormality inspection standard decided in advance is satisfied. Specifically, the abnormality standard determination section 318 determines whether or not the size of the abnormality detected at the time of abnormality detection is a permissible value, and thus determines whether or not the abnormality inspection standard is satisfied even if there is an abnormality of the surface of the substrate 4.

[0182] As a result, even if an abnormality of the surface of the substrate 4 is detected, the inspection device 1 can determine whether or not the abnormality inspection criterion is satisfied.

[0183] In addition, according to the inspection device 1 of Embodiment 1, the masked image generation section 309 generates a masked image in which a portion in which the abnormality of the surface of the substrate 4 is detected is covered by masking when the abnormality is detected in the image extracted from the learning target image. Then, the position detection model generation section 310 updates the position detection model by machine learning using not only the extracted image but also image information of the masked image.

[0184] As a result, compared with the inspection device in which the position detection model is not updated by machine learning using also the image information of the masked image, the inspection device 1 allows the abnormality of the surface of the metal member 401 and is easy to detect the position of the metal member 401.

[0185] Further, as in Embodiment 1, the inspection device 1 preferably performs masking with the coordinate position of the abnormality detected in accordance with the masking size set in the position inspection setting information as the center, but is not limited thereto. The inspection device 1 can mask an arbitrary position different from the position of the detected abnormality or can mask a plurality of positions separated by a predetermined interval.

[0186] Further, as in Embodiment 1, in the inspection device 1, the masked image generation section 309 preferably generates the masked image in order to perform machine learning using the image information of the masked image, but can be omitted. That is, in the inspection device 1, the masked image can not be generated even when the abnormality of the surface of the substrate 4 is detected in the image extracted from the learning target image. In this case, the position detection model generation section 310 can update the position detection model by machine learning using image information including only the extracted image without the masked image.

[0187] In addition, according to the inspection device 1 of Embodiment 1, the masked image generation section 309 also generates a masked image in which a portion in which the abnormality of the surface of the substrate 4 is detected is covered by masking when the abnormality is detected in the image extracted from the inspection target image. Then, the position detection section 312 calculates the score by applying the masked image to the position detection model not only with respect to the extracted image.

[0188] As a result, compared with the inspection device in which the masked image is not applied to the position detection model to calculate the score, the inspection device 1 allows the abnormality of the surface of the metal member 401 and is easy to detect the position of the metal member 401.

[0189] Further, as in Embodiment 1, in the inspection device 1, the score is preferably calculated in order to apply the shield image to the position detection model, and the shield image generation section 309 generates the shield image, but the shield image generation section 309 can be omitted. That is, in the inspection device 1, the shield image can also be generated even when the abnormality of the surface of the substrate 4 is detected in the image extracted from the inspection target image. In this case, the position detection section 312 can also calculate the score without applying the shield image to the position detection model, and can apply only to the position detection model with respect to the extracted image, and calculate the score.

[0190] Further, according to the inspection device 1 of Embodiment 1, the imaging section 209 images the substrate 4 in a state where the illumination light output section 210, 211 outputs the illumination light. Further, the position detection model generation section 310 generates the position detection model by machine learning using the image information of the learning target image imaged in a state where the illumination light output section 210, 211 outputs the illumination light. Furthermore, the measured values of the X coordinate and the Y coordinate of the position learning information are set according to the position of the glossy portion 402.

[0191] As a result, the inspection device 1 can detect the position of the metal member 401 according to the position of the glossy portion 402 based on the reflected light of the illumination light output from the illumination light output section 210, 211. That is, the inspection device 1 performs machine learning with respect to the image including the reflected light when the illumination light is output to the metal member 401, and thus can detect the position of the metal member 401 according to the position, size, and range of the glossy portion 402 as a feature quantity related to the shape of the metal member 401 from the reflected light of the illumination light.

[0192] Further, as in Embodiment 1, in the inspection device 1, the imaging section 209 preferably images the substrate 4 in a state where the illumination light is output in order to detect the position of the metal member 401 according to the position of the glossy portion 402, but is not limited thereto. In the inspection device 1, the imaging section 209 can not image the substrate 4 in a state where the illumination light is output, and the illumination light output section 210, 211 and the illumination light control section 303 can be omitted.

[0193] Further, according to the inspection device 1 of Embodiment 1, the learning screen display section 321 displays the learning screen 6. Further, the user can acquire the learning target image by operating the learning screen 6 via the input section 301, or make the position learning information and the abnormality learning information, or perform machine learning to generate the position detection model and the abnormality detection model.

[0194] As a result, the inspection device 1 can visually perform machine learning for the inspection of the position of the metal member 401 and the inspection of abnormalities of the surface of the substrate 4, and the user can easily add information required for machine learning compared to an inspection device that does not display a learning screen.

[0195] Further, in Embodiment 1, the inspection device 1 acquires a learning target image via the learning screen 6, or creates position learning information and abnormality learning information, but is not limited thereto. The inspection device 1 can also acquire a learning target image via the learning screen 6, or create position learning information and abnormality learning information, without using the learning screen 6, for example, the user can acquire a learning target image via the input section 301, or create position learning information and abnormality learning information.

[0196] Further, in the inspection device 1 as in Embodiment 1, it is preferable to display the learning screen 6 in order to visually perform machine learning, but it is also possible to not display the learning screen 6, or to omit the learning screen display section 321. In this case, it is necessary to enable the user to perform machine learning via the input section 301.

[0197] In addition, according to the inspection device 1 of Embodiment 1, the inspection screen display section 322 displays the inspection screen 7. In addition, the user can perform the inspection of the position of the metal member 401 and the inspection of abnormalities of the surface of the substrate 4, or perform comprehensive determination by operating the inspection screen 7 via the input section 301 to acquire an inspection target image.

[0198] As a result, the inspection device 1 can visually perform these inspections and comprehensive determination.

[0199] Further, in Embodiment 1, the inspection device 1 performs comprehensive determination via the inspection screen 7, but it is also possible to not perform comprehensive determination via the inspection screen 7. That is, the inspection device 1 can not perform comprehensive determination, or can omit the comprehensive determination section 323, the comprehensive determination result output section 324, and the comprehensive determination result storage section 325.

[0200] Further, in the inspection device 1 as in Embodiment 1, it is preferable to display the inspection screen 7 in order to visually perform the inspection and comprehensive determination, but it is also possible to not display the inspection screen 7, or to omit the inspection screen display section 322. In this case, the inspection device 1 needs the user to perform the inspection via the input section 301.

[0201] Further, in Embodiment 1, in the inspection device 1, the initial screen display section 320 displays the initial screen 5, and the user operates the initial screen 5 via the input section 301 to display the learning screen 6, or to display the inspection screen 7, but it is also possible to not display the initial screen 5. That is, it is also possible to omit the initial screen display section 320.

[0202] Further, in Embodiment 1, in the inspection device 1, after the reference position calculating section 308 calculates the reference values of the X coordinate and the Y coordinate for position learning when the position detection model generating section 310 generates the position detection model, the image of the pattern size centered on the reference values of the X coordinate and the Y coordinate is extracted. However, the inspection device 1 can also extract the image of the pattern size centered on the measured values of the X coordinate and the Y coordinate without calculating the reference values of the X coordinate and the Y coordinate. That is, the reference position calculating section 308 and the item of "offset" of the position inspection setting information can also be omitted.

[0203] Embodiment 2

[0204] In the inspection device 1 of Embodiment 1, the position detection section 312 extracts the images of all the pattern sizes that can be extracted from the position detection range. However, the present disclosure is not limited thereto, and the position detection section 312 can also not extract the images of all the pattern sizes that can be extracted from the position detection range. In the inspection device 1 of Embodiment 2, the position detection section 312 roughly extracts the images of a plurality of kinds of pattern sizes from the position detection range. Hereinafter, the structure of the inspection device 1 of Embodiment 2 is described with reference to Figure 16 to Figure 18 , while the structure different from Embodiment 1 is described in Embodiment 2, the description of the same structure as Embodiment 1 is omitted to avoid redundancy.

[0205] The inspection device, the inspection method, and the program of Embodiment 2 of the present disclosure achieve a reduction in the processing time taken for position detection compared to the inspection device, the inspection method, and the program of Embodiment 1.

[0206] [Details of the functional structure of the computer 3]

[0207] Figure 16 is a diagram showing one example of the position inspection setting information of Embodiment 2. In Embodiment 2, the position inspection setting information includes, as the information of the item classified for each product name as shown in Figure 16 , the information of the item of "coarse detection interval" indicating the interval of the coordinate positions of the centers of the images of the pattern sizes from each other. Further, in Embodiment 2, the interval of the coordinate positions from each other is set as the length of the X direction or the Y direction of the coordinate positions from each other, but is not limited thereto as long as the interval of the coordinate positions from each other can be determined. For example, the interval of the coordinate positions from each other can also be the number of pixel pitches of the X direction or the Y direction of the pixels corresponding to the coordinate positions from each other, the amount of pixel size of the number of pixels, or the like.

[0208] For example, in the position inspection setting information, in Figure 16The first row of the product name "substrate A" shows that the image of the extraction center coordinate position in the detection range of the rectangular shape with the starting point (XSA1, YSA1) and the end point (XEA1, YEA1) of the image as the diagonal vertices is the image of the pattern size of the horizontal width WA1 and the vertical width LA1 of the interval of the other. Specifically, in the position inspection setting information, the image of the pattern size of the coordinate of the extraction center is (XSA1, YSA1), (XSA1+PA1, YSA1), (XSA1+2*PA1, YSA1),..., (XSA1, YSA1+PA1), (XSA1, YSA1+2*PA1)...

[0209] In addition, in Embodiment 2, the position detection section 312 acquires the value of the item of "image size" of the device setting information, and acquires the information of the items of "position detection range", "pattern size", and "coarse detection interval" of the position inspection setting information corresponding to the product name of the metal member 401 stored in the setting information storage section 304. In addition, the position detection section 312 extracts the pattern size image centered on the coordinate position selected at intervals of the coarse detection interval in order from the coordinate position of the starting point of the position detection range, from all the pattern size images that can be extracted in the position detection range, and calculates the score of the extracted image. In addition, if there is a score that becomes a score threshold or more, the position detection section 312 roughly detects the position of the metal member 401 by calculating the measured values of the X coordinate and the Y coordinate at the time of acquiring the image that becomes the maximum score. Hereinafter, the X coordinate and the Y coordinate of the calculated measured values will be described as the coarse detection result coordinates.

[0210] In addition, the position detection section 312 updates the information of the item of "position detection range" of the position inspection setting information corresponding to the product name of the metal member 401 according to the coarse detection result coordinates. Specifically, first, the position detection section 312 redefines the starting point and the end point with the coordinate represented by the measured values of the calculated X coordinate and the Y coordinate as the central point, thereby redefining the position detection range. Then, the position detection section 312 replaces the information of the item of "position detection range" with the values of the X coordinate and the Y coordinate of the calculated starting point and the values of the X coordinate and the Y coordinate of the end point.

[0211] For example, the position detection section 312, when acquiring the inspection target image of "substrate A", refers to "FA1" of the item of "inspection image information temporary storage area" of the device setting information, determines the address where the image information of the inspection target image is stored, and acquires the image information of the inspection target image. In addition, the position detection section 312 acquires "SZA1" of "image size" of the device setting information, and acquires the range XSA1 to XEA1 of the value of the X coordinate, the range YSA1 to YEA1 of the value of the Y coordinate, and the horizontal width WA1 and the vertical width LA1 of "pattern size" of the position inspection setting information. In addition, the position detection section 312 acquires "PA1" of "coarse detection interval" of the position inspection setting information.

[0212] In addition, the position detection section 312 extracts, from the inspection target image, an image of a range having a horizontal width of WA1 and a vertical width of LA1 centered on a coordinate position selected within the position detection range in a rectangular shape having the start point (XSA1, YSA1) and the end point (XEA1, YEA1) as diagonal vertices. At this time, the position detection section 312 extracts, from the inspection target image, an image of a range having a horizontal width of WA1 and a vertical width of LA1 centered on coordinates (XSA1+PA1, YSA1), (XSA1+2×PA1, YSA1),..., (XSA1, YSA1+PA1), (XSA1, YSA1+2×PA1),..., which are spaced apart by PA1 in the order of the X direction or the Y direction from the start point (XSA1, YSA1).

[0213] In addition, the position detection section 312, when there are M coordinate positions that can be selected within the position detection range, applies the acquired position detection model to the extracted M images, and calculates scores SC11 to SC1M of the M images. In addition, the position detection section 312, for example, when the maximum score among the scores that are equal to or higher than SA1 as the score threshold is SC1M, detects the measured values (X1, Y1) of the X coordinate and the Y coordinate from the image that becomes the maximum score SC1M. Then, the position detection section 312 calculates a new start point (XSA21, YSA21) and a new end point (XEA21, YEA21) of which the calculated measured values (X1, Y1) of the X coordinate and the Y coordinate become the center points, and updates the information of the item of "position detection range" of the position inspection setting information.

[0214] Specifically, first, the central value of the range XSA1 to XEA1 of the values of the X coordinate of the position detection range is set as XCA1, and the central value of the range YSA1 to YEA1 of the values of the Y coordinate is set as YCA1. Further, the error of the measured value X1 of the X coordinate from the central value XCA1 of the X coordinate of the position detection range is set as αx, and the error of the measured value Y1 of the Y coordinate from the central value YCA1 of the Y coordinate of the position detection range is set as αy. In this case, the position detection section 312 calculates the errors αx, αy of the X coordinate and the Y coordinate by calculating αx = X1 - XCA1, αy = Y1 - YCA1.

[0215] Further, the position detection section 312 calculates the new range XSA21 to XEA21 of the values of the X coordinate of the position detection range and the new range YSA21 to YEA21 of the values of the Y coordinate by calculating XSA21 = XSA1 - αx, XEA21 = XEA1 - αx, YSA21 = YSA1 - αy, YEA21 = YEA1 - αy. Then, the position detection section 312 updates the information of the item of "position detection range" of the position check setting information in accordance with the calculated new range XSA21 to XEA21 of the values of the X coordinate and the new range YSA21 to YEA21 of the values of the Y coordinate.

[0216] Figure 17 and Figure 18 is a flowchart showing the position check processing of Embodiment 2. Using the flowchart shown in Figure 17 and Figure 18 , the operation of the inspection device 1 of Embodiment 2 to check the position of the metal member 401 is described. First, as shown in Figure 17 , after the processing of Step S401 is executed, the position detection section 312 acquires the image size from the device setting information corresponding to the product name selected in the inspection screen 7, and acquires the position detection range, the pattern size, and the rough detection interval of the position check setting information (Step S451). For example, if "substrate A" is selected in the inspection screen 7, the position detection section 312 acquires "SZA1" of the device setting information, and acquires "XSA1 to XEA1, YSA1 to YEA1" and "(WA1, LA1)", "PA1" of the position check setting information.

[0217] Further, the position detection section 312 extracts images of the pattern size from the inspection target image at intervals of the coarse detection interval within the position detection range (step S452). For example, if "substrate A" is selected on the inspection screen 7, the position detection section 312 extracts images of a range of a horizontal width of WA1 and a vertical width of LA1 centered on coordinates (XSA1+PA1, YSA1), (XSA1+2xPA1, YSA1),..., (XSA1, YSA1+PA1), (XSA1, YSA1+2xPA1),... at intervals of PA1 in the order of the X direction or the Y direction from the starting point (XSA1, YSA1). Further, the position detection section 312 performs the processes of steps S404 to S407. The position detection section 312 repeats the processes of steps S452, S404 to S406 until all the images of the pattern size are extracted, in a case where all the images of the pattern size are not extracted (step S407; No).

[0218] On the other hand, as shown in Figure 18 step S407; Yes), the position detection section 312 performs the processes of steps S408 to S410, and updates the position detection range of the position inspection setting information based on the measured values of the calculated X coordinates and Y coordinates (step S453). For example, if "substrate A" is selected on the inspection screen 7, the position detection section 312 calculates a new starting point (XSA21, YSA21) and a new end point (XEA21, YEA21) in which a coordinate (X1, Y1) represented by the measured values of the calculated X coordinates and Y coordinates becomes a central point, and updates the information of the item of "position detection range" of the position inspection setting information. Further, the position criterion determination section 313 performs the processes of steps S411 to S415, and ends the processes.

[0219] As described above, according to the inspection device 1 of Embodiment 2, the position detection section 312 extracts images of the pattern size from within the position detection range at intervals of the coarse detection interval, thereby roughly extracting images of a plurality of kinds of pattern sizes. Then, the position detection section 312 applies the position detection model to the extracted images of the pattern size, respectively calculates scores, determines an image in which the score becomes a maximum score that is equal to or higher than a score threshold value, thereby detecting the position of the metal member 401 in the inspection target image, and the position output section 314 outputs the detected position of the metal member 401.

[0220] As a result, even if all the images of the pattern size that can be acquired are not extracted from within the position detection range, the inspection device 1 can detect the position of the metal member 401, can check whether or not the position of the metal member 401 conforms to the position inspection criterion, and can reduce the load and the time of the work for the inspection.

[0221] In particular, according to the inspection device 1 of Embodiment 2, the position detection section 312 updates the position detection range of the position inspection setting information in accordance with the calculated coarse detection result coordinates. Here, it is expected that the calculated coarse detection result coordinates become near the center of the metal member 401.

[0222] As a result, the inspection device 1 can detect the position of the metal member 401 in the inspection target image with good precision, as compared with an inspection device that does not update the position detection range in accordance with the coarse detection result coordinates. Therefore, the inspection device 1 can narrow the position detection range in accordance with the degree of error that is assumed between the coarse detection result coordinates and the actual position of the metal member 401, and can shorten the time taken for the operation of the position detection section 312. For example, if the coarse detection interval PA1 is a length of 5 pixels, even if the position detection range is set to about ±15 pixels in the X direction and the Y direction, respectively, the position detection section 312 can sufficiently detect the position of the metal member 401. As a result, the inspection device 1 can further reduce the load and the time of the work for inspection.

[0223] In addition to this, the inspection device 1 of Embodiment 2 functions to have the same effects as the inspection device 1 of Embodiment 1.

[0224] Further, as in Embodiment 2, with respect to the position detection section 312, it is preferable to update the position detection range of the position inspection setting information in accordance with the calculated coarse detection result coordinates in order to be able to detect the position of the metal member 401 even with a narrow position detection range, but is not limited thereto. For example, the position detection section 312 can update the position detection range when the error of the calculated coarse detection result coordinates and the coordinates of the center point of the set position detection range exceeds a threshold value that is decided in advance. In addition, for example, the position detection section 312 can not update the position detection range even if the coarse detection result coordinates are calculated.

[0225] Embodiment 3

[0226] In the inspection device 1 of Embodiments 1 and 2, the position detection section 312 detects the position of the metal member 401 in the inspection target image using the position detection model. However, the present disclosure is not limited thereto, and the position detection section 312 can detect the position of the metal member 401 in the inspection target image using not only the position detection model but also another learning model. In the inspection device 1 of Embodiment 3, the position detection section 312 detects the position of the metal member 401 in the inspection target image using not only the position detection model but also a coarse detection model that will be described later. Hereinafter, the position detection section 312 will be described with reference to Embodiment 3. Figure 19 to Figure 22, the configuration of the inspection apparatus 1 of Embodiment 3 is described. Furthermore, in Embodiment 3, the configuration different from Embodiments 1 and 2 is described, and regarding the configuration common to Embodiments 1 and 2, the description is omitted to avoid redundancy.

[0227] The inspection apparatus, the inspection method, and the program of Embodiment 3 of the present disclosure achieve a reduction in processing time taken for position detection, as compared with the inspection apparatus, the inspection method, and the program of Embodiment 1.

[0228] [About the computer 3]

[0229] Figure 19 is a diagram showing the functional configuration of the inspection apparatus of Embodiment 3. As shown in Figure 19 , the computer 3 further has a rough detection model generation section 326 as one example of a position rough detection model generation section that generates a rough detection model as one example of a position rough detection model representing the relationship between an image and the position of the metal member 401 by machine learning. In addition, the computer 3 further has a rough detection model storage section 327 that stores the generated rough detection model. Returning to Figure 4 , in Embodiment 3, the control section 51 functions as the rough detection model generation section 326 of the computer 3 in accordance with the control program 59. In addition, in Embodiment 3, the external storage section 53 functions as the rough detection model storage section 327.

[0230] In addition, the computer 3 realizes the functions of the rough detection model generation section 326 and the rough detection model storage section 327 by using the main storage section 52, the external storage section 53, the operation section 54, the display section 55, and the transceiver section 56 as resources by the control section 51. For example, the computer 3 executes the rough detection model generation step performed by the rough detection model generation section 326 and executes the rough detection model storage step performed by the rough detection model storage section 327.

[0231] [Detailed contents about the functional configuration of the computer 3]

[0232] Figure 20 is a diagram showing one example of the position inspection setting information of Embodiment 3. In Embodiment 3, the position inspection setting information, as shown in Figure 20 , includes information of an item of “rough detection range” as a range roughly detecting the position of the metal member 401, as information of an item classified for each product name, in addition to the information of the items of the product name, the model type, and the position detection model. In addition, the information of the item of “model form” shows not only the form of the position detection model but also the form of the rough detection model.

[0233] For example, in the position inspection setting information, inFigure 20 The first line of the 1st row shows that, for the product with the product name "substrate A", an image with a horizontal width of WA1 and a vertical width of LA1 is extracted within a rough detection range in the shape of a rectangle with the starting point (XSA3, YSA3) and the end point (XEA3, YEA3) of the image as diagonal vertices. In addition, in the position check setting information, for the product with the product name "substrate A", the position detection model of the learning model for the check is "MA1", and the rough detection model is "MA3".

[0234] The rough detection model generation section 326 generates a rough detection model representing the relationship between the image and the position of the metal member 401 by machine learning as described above. Here, the rough detection model refers to a learning model used when roughly detecting the position of the metal member 401 before the position of the metal member 401 is detected using the position detection model. The rough detection model generation section 326 acquires the information of the item of "model form" of the position check setting information corresponding to the product name of the metal member 401 stored in the setting information storage section 304, that is, the form of the learning model of machine learning, and generates a prototype of the rough detection model. Further, the form of the learning model of machine learning that the rough detection model generation section 326 can generate can be selected from any of "SSD", "YOLOv3", and "M2Det" which are representative object recognition models for the purpose of detecting the class and position of an object from an image. However, the form of the learning model is not limited to these, and for example, it can be made possible to select an object recognition model other than for the purpose of object recognition. In this case, it is necessary to make it possible to select the form of the learning model of the machine learning method set.

[0235] Further, the rough detection model generation section 326 acquires image information of the learning target image stored in the image information storage section 305 with reference to the information of the item of "learning image information storage area" of the device setting information stored in the setting information storage section 304. Further, the rough detection model generation section 326 acquires the information of the item of "pattern size" of the position inspection setting information corresponding to the product name of the metal member 401, that is, the length of the horizontal width and the vertical width of the image extracted from the learning target image. Further, the rough detection model generation section 326 extracts the image of the pattern size of the horizontal width and the vertical width acquired with the reference values of the X coordinate and the Y coordinate calculated by the reference position calculation section 308 as the center from the learning target image. Further, the rough detection model generation section 326 performs machine learning using the extracted image, the values of the X coordinate and the Y coordinate as the start point of the extraction range of the image, and the values of the X coordinate and the Y coordinate as the end point, and thereby updates the rough detection model. Furthermore, the rough detection model generation section 326 calculates the values of the X coordinate and the Y coordinate as the start point of the extraction range of the image and the values of the X coordinate and the Y coordinate as the end point from the extracted image and the reference values of the X coordinate and the Y coordinate. Further, the rough detection model generation section 326 also performs machine learning with respect to the mask image when the mask image generation section 309 generates the mask image with respect to the extracted image, and thereby updates the rough detection model.

[0236] For example, the rough detection model generation section 326 acquires "MA3" of the item of "model form" of the position inspection setting information when acquiring the learning target image of "substrate A". Here, for example, if "MA3" is "M2Det", the rough detection model generation section 326 generates a prototype of the rough detection model of which the form of the learning model is "M2Det". Further, the rough detection model generation section 326 acquires the image information of the learning target image with reference to the "FA2" of the item of "learning image information storage area" of the device setting information, and determines the address where the image information of the learning target image is stored, and acquires the image information of the learning target image. Further, the rough detection model generation section 326 acquires the horizontal width WA1 and the vertical width LA1 of the item of "pattern size" of the position inspection setting information, and extracts the image of the range of the horizontal width of WA1 and the vertical width of LA1 with the coordinate position of (261+XOA1, 115+YOA1) as the center from the learning target image. Further, the rough detection model generation section 326 performs machine learning using the extracted image, the values of the X coordinate and the Y coordinate as the start point of the extraction range of the image, and the values of the X coordinate and the Y coordinate as the end point, and thereby updates the rough detection model. Furthermore, the rough detection model generation section 326 performs machine learning using the mask image and the values of the X coordinate and the Y coordinate as the start point of the extraction range if the mask image of the extracted image is generated, and thereby updates the rough detection model.

[0237] Returning to Figure 9In the embodiment 3, the learning screen display section 321 causes the position detection model generation section 310 to generate the position detection model and causes the rough detection model generation section 326 to generate the rough detection model when the user operates the learning button 614 via the input section 301 in a state where the position learning information display field 607 is displayed. Further, the learning screen display section 321 causes the position detection model generation section 310, the rough detection model generation section 326, and the abnormality detection model generation section 315 to suspend the generation when the user operates the learning button 614 via the input section 301 during the generation of the position detection model, the rough detection model, and the abnormality detection model.

[0238] Returning to Figure 20 In the embodiment 3, the position detection section 312 acquires the value of the item of "image size" of the device setting information, and acquires the information of the items of "position detection range", "pattern size", and "rough detection range" of the position inspection setting information corresponding to the product name of the metal member 401 stored in the setting information storage section 304. That is, the position detection section 312 acquires not only the values of the X coordinate and the Y coordinate of the start point of the position detection range in the inspection target image and the values of the X coordinate and the Y coordinate of the end point and the lengths of the horizontal width and the vertical width of the extracted image, but also the values of the X coordinate and the Y coordinate of the start point of the rough detection range and the values of the X coordinate and the Y coordinate of the end point.

[0239] Further, the position detection section 312 acquires the rough detection model stored in the rough detection model storage section 327, applies the rough detection model to the image extracted from the detection target image according to the rough detection range, and calculates the values of the coordinates of the position of the metal member 401 in the extracted image as a result of the rough detection. Hereinafter, the X coordinate and the Y coordinate of the calculated values are referred to as the rough detection result coordinates.

[0240] Further, the position detection section 312 updates the information of the item of "position detection range" of the position inspection setting information corresponding to the product name of the metal member 401 on the basis of the rough detection result coordinates. Specifically, first, the position detection section 312 calculates the start point and the end point of the value of the X coordinate with the calculated value of the X coordinate as the center point, and calculates the minimum value and the maximum value of the value of the Y coordinate with the calculated value of the Y coordinate as the center value, thereby redefining the position detection range. Then, the position detection section 312 replaces the information of the item of "position detection range" with the values of the X coordinate and the Y coordinate as the calculated minimum value and the values of the X coordinate and the Y coordinate as the maximum value.

[0241] Further, the position detection section 312 extracts an image of the rough detection range from the updated position detection range, and calculates a score of the extracted image. Further, if there is a score that is equal to or higher than the score threshold value, the position detection section 312 detects the position of the metal member 401 by calculating the measured values of the X coordinate and the Y coordinate at the time of acquisition of the image having the highest score.

[0242] For example, when the position detection section 312 acquires the inspection target image of "substrate A", the position detection section 312 refers to "FA1" of the item of "image information for inspection temporary storage area" of the device setting information, determines the address of the image information in which the inspection target image is stored, and acquires the image information of the inspection target image. Further, the position detection section 312 acquires "SZA1" of "image size" of the device setting information, and acquires the range XSA1 to XEA1 of the value of the X coordinate, the range YSA1 to YEA1 of the value of the Y coordinate, and the horizontal width WA1 and the vertical width LA1 of the pattern size of the position inspection setting information. Further, the position detection section 312 acquires the range XSA3 to XEA3 of the value of the X coordinate and the range YSA3 to YEA3 of the value of the Y coordinate of the rough detection range of the position inspection setting information.

[0243] Further, the position detection section 312 extracts an image of the rough detection range from the updated position detection range, and calculates a score of the extracted image. Further, if there is a score that is equal to or higher than the score threshold value, the position detection section 312 detects the position of the metal member 401 by calculating the measured values of the X coordinate and the Y coordinate at the time of acquisition of the image having the highest score.

[0244] Specifically, first, the error of the value X3 of the X coordinate of the coarse detection result coordinate from the center value XCA1 of the X coordinate of the position detection range is set to βx, and the error of the value Y3 of the Y coordinate of the coarse detection result coordinate from the center value YCA1 of the Y coordinate of the position detection range is set to βy. In this case, the position detection section 312 calculates the errors βx, βy of the X coordinate and the Y coordinate by calculating βx = X3 - XCA1, βy = y3 - YCA1. Then, the position detection section 312 updates the acquired position detection range by calculating XSA31 = XSA1 - βx, YSA31 = YSA1 - βy, XEA31 = XEA1 - βx, YEA31 = YEA1 - βy.

[0245] In addition, the position detection section 312 extracts an image of a range having a horizontal width of WA1 and a vertical width of LA1, with a coordinate position selected within a rectangular shape position detection range having the updated new start point (XSA31, YSA31) and the new end point (XEA31, YEA31) as diagonal vertices as a center. In addition, the position detection section 312, when there are N selectable coordinate positions within the position detection range, applies the acquired position detection model to the N extracted images, and calculates scores SC11 to SC1N of the N images. Then, the position detection section 312, for example, when the maximum score among the scores that are equal to or higher than a score threshold value SA1 is SC1M, detects the measured values (X1, Y1) of the X coordinate and the Y coordinate from the image that becomes the maximum score SC1M.

[0246] Figure 21 is a flowchart showing the coarse detection model generation processing of Embodiment 3. Using the flowchart shown in Figure 21 the operation of the inspection device 1 of Embodiment 3 to generate a coarse detection model. The coarse detection model generation section 326 starts execution of the coarse detection model generation processing when the user operates the learning button 614 via the input section 301 on the learning screen 6. First, as shown in Figure 21

[0247] ​Further, the rough detection model generation section 326 acquires the learning target image from the image information corresponding to the product name selected in the learning screen 6 (step S502). Further, the rough detection model generation section 326 acquires the measured values of the X coordinate and the Y coordinate of the position learning information corresponding to the product name selected in the learning screen 6 and the values of the offset of the X coordinate and the Y coordinate of the position check setting information, and causes the reference position calculation section 308 to calculate the reference values of the X coordinate and the Y coordinate (step S503). For example, if "substrate A" is selected in the learning screen 6, the reference position calculation section 308 acquires "261" and "115" from the position learning information and "(XOA1, YOA1)" from the position check setting information, and calculates (261+XOA1, 115+YOA1).

[0248] Further, the rough detection model generation section 326 acquires the pattern size of the position check setting information corresponding to the product name selected in the learning screen 6, and extracts an image of the pattern size centered on the reference values of the X coordinate and the Y coordinate from the learning target image (step S504). For example, if "substrate A" is selected in the learning screen 6, the rough detection model generation section 326 acquires "(WA1, LA1)" of the position check setting information, and extracts an image of a range having a horizontal width of WA1 and a vertical width of LA1 centered on the coordinate position of (261+XOA1, 115+YOA1) from the learning target image. Further, the rough detection model generation section 326 performs machine learning using the extracted image, the values of the X coordinate and the Y coordinate as the start point of the extraction range of the image, and the values of the X coordinate and the Y coordinate as the end point, and updates the rough detection model (step S505).

[0249] Further, the rough detection model generation section 326 acquires the values of the X coordinate and the Y coordinate of the abnormality learning information stored in the abnormality learning information storage section 307, and determines whether or not an abnormality of the surface of the substrate 4 is detected in the extracted image (step S506). For example, if "substrate A" is selected in the learning screen 6, the rough detection model generation section 326 acquires "110" and "170" of the abnormality learning information, and determines whether or not a position of the coordinates of (110, 170) is included in the extracted image of the pattern size.

[0250] The rough detection model generation section 326, in a case where an abnormality is detected (step S506; Yes), acquires the masking size from the position check setting information corresponding to the product name selected in the learning screen 6, and causes the masking image generation section 309 to generate a masking image in which the extracted image is masked by the masking size (step S507). For example, if "substrate A" is selected in the learning screen 6, the masking image generation section 309 acquires "(MWA1, MLA1)" from the position check setting information when the coordinate position of (110, 170) is included in the extracted pattern size image. In addition, the masking image generation section 309 replaces the range having a horizontal width of MWA1 and a vertical width of MLA1 centered on the coordinate position of (110, 170) with an image filled in white.

[0251] In addition, the rough detection model generation section 326 performs machine learning using the generated masking image, the values of the X coordinate and the Y coordinate as the start point of the extraction range of the image, and the values of the X coordinate and the Y coordinate as the end point, and updates the rough detection model (step S508). In addition, the rough detection model generation section 326, after the rough detection model is updated or in a case where an abnormality is not detected (step S506; No), determines whether machine learning is performed using all of the learning target images (step S509). The rough detection model generation section 326, in a case where machine learning is not performed using all of the learning target images (step S509; No), repeats the processes of steps S502 to S508 until machine learning is performed using all of the learning target images. On the other hand, the rough detection model generation section 326, in a case where machine learning is performed using all of the learning target images (step S509; Yes), ends the processing.

[0252] Figure 22 is a flowchart showing the position check processing of Embodiment 3. Using the flowchart shown in Figure 22 , the operation of the inspection device 1 of Embodiment 3 to inspect the position of the metal member 401 will be described. First, as shown in Figure 22 , the position detection section 312, after executing the processing of step S401, acquires the image size from the device setting information corresponding to the product name selected in the inspection screen 7, and acquires the position detection range, the pattern size, and the rough detection range of the position check setting information (step S461). For example, if "substrate A" is selected in the inspection screen 7, the position detection section 312 acquires "SZA1" of the device setting information, and acquires "XSA1 to XEA1, YSA1 to YEA1" and "(WA1, LA1)", "XSA3 to XEA3, YSA3 to YEA3" of the position check setting information.

[0253] Further, the position detection section 312 applies the rough detection model to the image extracted from the inspection target image according to the rough detection range, calculates a rough detection result coordinate (step S462), and updates the position detection range of the position inspection setting information according to the calculated rough detection result coordinate (step S463). For example, if "board A" is selected in the inspection screen 7, the position detection section 312 extracts an image of a rough detection range of a rectangular shape having the start point (XSA3, YSA3) and the end point (XEA3, YEA3) as diagonal vertices from the inspection target image. Further, the position detection section 312 applies the rough detection model corresponding to the product name selected in the inspection screen 7 to the extracted image, and calculates a rough detection result coordinate (X3, Y3). Further, the position detection section 312 calculates a new start point (XSA31, YSA31) and a new end point (XEA31, YEA31) of which the rough detection result coordinate (X3, Y3) becomes a central point.

[0254] Further, the position detection section 312 extracts an image of the pattern size from the inspection target image within the updated position detection range (step S464). For example, if "board A" is selected in the inspection screen 7, the position detection section 312 extracts an image of a range of a horizontal width of WA1 and a vertical width of LA1 centered at a coordinate position selected within the position detection range of a rectangular shape having the new start point (XSA31, YSA31) and the new end point (XEA31, YEA31) as diagonal vertices from the inspection target image.

[0255] Further, the position detection section 312 performs the processes of steps S404 to S407. The position detection section 312 repeats the processes of steps S464, S404 to S406 until the image of the pattern size is extracted completely, in a case where the image of the pattern size is not extracted completely (step S407; No). For example, if "board A" is selected in the inspection screen 7, the position detection section 312 continues to extract the image of the pattern size from the inspection target image until the scores SC11 to SC1N of N images are calculated. On the other hand, as shown in the flowchart of FIG. 17, the position detection section 312 performs the processes of steps S408 to S415 and ends the processes, in a case where the image of the pattern size is extracted completely (step S407; Yes). Figure 15

[0256] ​As explained above, according to the inspection device 1 of Embodiment 3, the position detection section 312 applies the coarse detection model to the image of the pattern size extracted from the inspection target image according to the coarse detection range, and calculates the coarse detection result coordinates. In addition, the position detection section 312 extracts a plurality of kinds of images of the pattern size by extracting images of the pattern size from the new position detection range based on the calculated coarse detection result coordinates. Then, the position detection section 312 applies the position detection model to the extracted images of the pattern size, respectively calculates the scores, and determines the image of which the score becomes the maximum score that is equal to or higher than the score threshold value, thereby detecting the position of the metal member 401 in the inspection target image, and the position output section 314 outputs the detected position of the metal member 401.

[0257] As a result, the inspection device 1 can detect the position of the metal member 401 in the inspection target image using not only the position detection model but also the coarse detection model.

[0258] In particular, according to the inspection device 1 of Embodiment 3, the position detection section 312 updates the position detection range acquired from the position inspection setting information according to the calculated coarse detection result coordinates. In addition, it is expected that the calculated coarse detection result coordinates become near the center of the metal member 401.

[0259] As a result, the inspection device 1 can detect the position of the metal member 401 in the inspection target image with good precision compared to the inspection device that does not update the position detection range according to the coarse detection result coordinates. Therefore, the inspection device 1 can narrow the position detection range according to the degree of error assumed between the coarse detection result coordinates and the actual position of the metal member 401, and can shorten the time taken for the operation of the position detection section 312. For example, if the error of the coarse detection result coordinates is a length of about 20 pixels, even if the position detection range is set to about ±40 pixels in the X direction and the Y direction, respectively, the position detection section 312 can sufficiently detect the position of the metal member 401. As a result, the inspection device 1 can further reduce the load and the time of the work for inspection.

[0260] In addition to this, the inspection device 1 of Embodiment 3 has the same effects as the inspection device 1 of Embodiment 1.

[0261] Further, in Embodiment 3, the inspection device 1 includes the coarse detection model generation unit 326 and the coarse detection model storage unit 327, but can not include the coarse detection model generation unit 326 and the coarse detection model storage unit 327. For example, an external learning device can include the coarse detection model generation unit 326 and the coarse detection model storage unit 327, the coarse detection model generation unit 326 generates a coarse detection model by machine learning using the above information, and stores the generated coarse detection model in the coarse detection model storage unit 327. In this case, the inspection device 1 needs to acquire the coarse detection model stored in the learning device.

[0262] Further, in Embodiment 3, the position detection unit 312 does not extract the images of the pattern sizes at the coarse detection intervals from the position detection range as in Embodiment 2, and thus roughly extracts the images of the plurality of kinds of pattern sizes, but is not limited thereto. For example, the structures of Embodiments 2 and 3 can be combined, the position detection unit 312 updates the position detection range to a new position detection range according to the coarse detection result coordinates calculated using the coarse detection model, and extracts the images of the pattern sizes at the coarse detection intervals from the updated new position detection range, and thus roughly extracts the images of the plurality of kinds of pattern sizes.

[0263] Embodiment 4

[0264] In the inspection device 1 of Embodiments 1 to 3, the position detection unit 312 detects the position corresponding to the coordinates of the center of the image of the pattern size that becomes the maximum score as the position of the metal member 401 from the measured values of the X coordinates and the Y coordinates calculated in the inspection target image. However, the present disclosure is not limited thereto, and the position detection unit 312 can not detect the position corresponding to the coordinates of the center of the image of the pattern size that becomes the maximum score as the position of the metal member 401. In the inspection device 1 of Embodiment 4, the position detection unit 312 detects the determined position as the position of the metal member 401 from the measured values of the X coordinates and the Y coordinates calculated in the inspection target image. Hereinafter, the structure of the inspection device 1 of Embodiment 4 will be described with reference to Figure 23 to Figure 25 , the structure of the inspection device 1 of Embodiment 4 will be described with reference to

[0265] The inspection device, the inspection method, and the program of Embodiment 4 of the present disclosure achieve an improvement in the accuracy of position detection compared to the inspection device, the inspection method, and the program of Embodiment 1.

[0266] [Details of the functional structure of the computer 3]

[0267] Figure 23is a diagram showing one example of the position check setting information of Embodiment 4. In Embodiment 4, the position check setting information includes, as information of items classified for each product name, information of a maximum value of luminance values of pixels, and information of a "peak detection range" of lengths of a vertical width and a horizontal width indicating a detection range of a so-called peak, as items classified for each product name, as shown in Figure 23

[0268] For example, in the position check setting information, in the 1st row of Figure 23 , with respect to a product of which the "product name" is "substrate A", an image of which the horizontal width WA4 and the vertical width LA4 are extracted as the peak detection range is shown, and the position of the metal member 401 is detected.

[0269] Further, in Embodiment 4, the position detection section 312 acquires the horizontal width and the vertical width of the "peak detection range" of the position check setting information corresponding to the product name of the metal member 401 after calculating the measured values of the X coordinate and the Y coordinate from the image of which the pattern size that becomes the maximum score is acquired. Further, the position detection section 312 extracts an image of which the horizontal width and the vertical width acquired with the measured values of the calculated X coordinate and Y coordinate as the center from the inspection target image.

[0270] Figure 24 is a diagram showing one example of the fitting process of Embodiment 4. Here, Figure 24 the horizontal axis of the graph shown in Figure 24 indicates the value of the X coordinate, and the vertical axis indicates the average value of the luminance of the pixels arranged in the Y direction, and is a graph of a peak shape function indicating the relationship between the value of the X coordinate and the average value of the luminance of the pixels arranged in the Y direction. As described above, the glossy portion 402 of the metal member 401 is present extending in the vertical direction, that is, the Y direction. Therefore, the position detection section 312 calculates the average value of the luminance of the pixels arranged in the Y direction for each X coordinate in the extracted image. Further, the position detection section 312 performs a so-called fitting process of fitting the average value of the luminance of the pixels arranged in the Y direction of each X coordinate calculated as described above to the peak shape function so that the error becomes the minimum with respect to the peak shape function decided in advance, and calculates the value of the peak shape function of which the value of the X coordinate of the peak. Further, the peak shape function can use, for example, a Gaussian function, a Lorentz function, or the like. Then, the position detection section 312 updates the measured value of the X coordinate to the calculated value, and detects the position based on the measured values of the updated X coordinate and the Y coordinate as the position of the metal member 401.

[0271] ​For example, the position detection section 312 acquires the horizontal width WA4 and the vertical width LA4 of the "peak detection range" of the position check setting information after calculating the measured values (Xl, Yl) of the X coordinate and the Y coordinate. In addition, the position detection section 312 extracts an image that becomes the horizontal width WA4 and the vertical width LA4 with the measured values (Xl, Yl) of the X coordinate and the Y coordinate as the center, and calculates the average value of the brightness of the pixels arranged in the Y direction that can be selected from the extracted image.

[0272] Here, when there are K pieces of X coordinates that can be selected in the extracted image, the X coordinates that can be selected are set as XP1, XP2,..., XPK, and the average values of the brightness of the pixels arranged in the Y direction of the K pieces of X coordinates XP1, XP2,..., XPK are set as YP1, YP2,..., YPK. In this case, the position detection section 312 calculates the average values YP1, YP2,..., YPK of the brightness of the pixels arranged in the Y direction of the X coordinates XP1 to XPK, performs the fitting process of fitting into the peak shape function, and calculates the value of the peak shape function as the value of the X coordinate of the peak. Then, the position detection section 312 sets Xl = 381, for example, in the case where the peak shape function is represented by the graph shown in Figure 24

[0273] Figure 25 is a flowchart showing the position check processing of Embodiment 2. Using the flowchart shown in Figure 25 Figure 25 is shown, the position detection section 312 acquires the peak detection range of the position check setting information corresponding to the product name selected in the check screen 7 after performing the processing of Steps S401 to S410 (Step S471). For example, if "substrate A" is selected in the check screen 7, the position detection section 312 acquires "(WA4, LA4)" of the position check setting information.

[0274] ​​Further, the position detection section 312 extracts an image of the length of the horizontal width and the vertical width of the peak detection range acquired with the center of the coordinate of the calculated measurement value of the X coordinate and the Y coordinate from the inspection target image (step S472). Further, the position detection section 312 calculates the average value of the brightness of the pixels arranged in the Y direction of all the X coordinates selectable from the extracted image (step S473). For example, the position detection section 312 extracts an image of the horizontal width WA4 and the vertical width LA4 with the center of the coordinate of the calculated measurement value of the X coordinate and the Y coordinate (X1, Y1), and calculates the average values YP1 to YPK of the brightness of the pixels arranged in the Y direction of the X coordinates XP1 to XPK.

[0275] Further, the position detection section 312 performs a fitting process of fitting the average values of the brightness of the pixels arranged in the Y direction of each of the calculated X coordinates into a peak shape function, calculates the value of the peak shape function as the value of the X coordinate of the peak (step S474), and updates the measurement value of the X coordinate to the calculated value (step S475). For example, the position detection section 312 performs the fitting process illustrated, calculates "381" as the value of the X coordinate of the peak of the peak shape function, sets X1 = 381, and thereby updates the measurement value X1 of the X coordinate to "381". Then, the position criterion determination section 313 performs the processes of steps S411 to S415, and ends the process. Figure 24

[0276] As explained above, according to the inspection device 1 of Embodiment 2, the position detection section 312 extracts an image of the peak detection range with the center of the coordinate of the measurement value of the X coordinate and the Y coordinate, that is, the center of the coordinate of the image of the pattern size of which the maximum score is calculated. Further, the position detection section 312 calculates the average value of the brightness of the pixels arranged in the Y direction for each of the X coordinates from the extracted image, and calculates the value of the peak shape function as the value of the X coordinate of the peak of the approximation function of the average value. Then, the position detection section 312 updates the measurement value of the X coordinate to the calculated value, and detects the position of the metal member 401 based on the updated measurement values of the X coordinate and the Y coordinate.

[0277] As a result, the inspection device 1 can make the detected position of the metal member 401 close to the position of the peak of the brightness of the glossy portion 402, and can detect the position of the metal member 401 with good accuracy, compared with the inspection device that does not update the measurement value of the X coordinate to the value of the X coordinate of the peak of the peak shape function.

[0278] In addition to this, the inspection device 1 of Embodiment 4 functions to have the same effects as the inspection device 1 of Embodiment 1.

[0279] ​Moreover, in Embodiment 4, the glossy portion 402 of the metal member 401 is present extending in the Y direction, so the position detection section 312 calculates the average value of the luminance of the pixels arranged in the Y direction for each X coordinate, and calculates only the peak in the X direction, but is not limited thereto. For example, in a case where the glossy portion 402 of the metal member 401 is present extending in the X direction, the position detection section 312 can also calculate the average value of the luminance of the pixels arranged in the X direction for each Y coordinate, and calculate only the peak in the Y direction. Further, for example, in a case where the glossy portion 402 of the metal member 401 is present in a circular shape, the position detection section 312 can also calculate the average value of the luminance of the pixels arranged in the Y direction for each X coordinate, and calculate the peak in the X direction, and calculate the average value of the luminance of the pixels arranged in the X direction for each Y coordinate, and calculate the peak in the Y direction. Furthermore, the position detection section 312 can also calculate the peaks in the X direction and the Y direction in accordance with the shape of the glossy portion 402 of the metal member 401.

[0280] Further, for example, in a case where the glossy portion 402 of the metal member 401 is present extending in a slant direction inclined with respect to the X direction and the Y direction, the position detection section 312 can also calculate the average value of the luminance of the pixels arranged in the slant direction for each coordinate of a direction orthogonal to the slant direction, and calculate the peak in the slant direction. Furthermore, the position detection section 312 can also calculate the peak in the slant direction in accordance with the shape of the glossy portion 402 of the metal member 401.

[0281] Moreover, as in Embodiment 4, in order to update the measured value of the X coordinate, the position detection section 312 preferably performs fitting processing of the average value of the luminance of the pixels arranged in the Y direction for each X coordinate calculated from the extracted image and the peak shape function, and calculates the value of the peak shape function as the value of the X coordinate of the position of the peak. However, the method of updating the measured value of the X coordinate is not limited thereto. For example, the position detection section 312 can also update the measured value of the X coordinate with the value of the X coordinate where the average value of the luminance of the pixels arranged in the Y direction for each X coordinate is the greatest. In this case, the position of the detected metal member 401 can be made close to the position where the luminance of the glossy portion 402 is the peak by a simpler process than fitting processing.

[0282] Further, in Embodiment 4, the position detection section 312 does not extract the image of the pattern size from the position detection range every coarse detection interval as in Embodiment 2, but is not limited thereto. For example, the structures of Embodiments 2 and 4 can be combined. Specifically, the position detection section 312 can extract the image of the pattern size from the position detection range every coarse detection interval, and update the coordinates of the image that becomes the maximum score based on the luminance values of the pixels of the image extracted with the coordinates as the center.

[0283] Further, in Embodiment 4, the position detection section 312 does not extract the image of the pattern size after updating the position detection range based on the coarse detection result coordinates calculated using the coarse detection model as in Embodiment 3, but is not limited thereto. For example, the structures of Embodiments 3 and 4 can be combined, and the position detection section 312 can update the coordinates of the image that becomes the maximum score based on the luminance values of the pixels of the image extracted with the coordinates as the center after updating the position detection range based on the coarse detection result coordinates calculated using the coarse detection model.

[0284] Further, as a part that performs the processing of the computer 3 provided with the control section 51, the main storage section 52, the external storage section 53, the operation section 54, the display section 55, the transceiver section 56, the internal bus 50, and the like, regardless of the dedicated system, a general computer system can be used. For example, a computer program for executing the described actions can be distributed by being stored in a recording medium, such as a flexible disk, a DVD-ROM (Read-Only Memory), and the like, which can be read by the computer 3, and the computer program can be installed in the computer, thereby constituting the computer 3 that executes the above-described processing. In addition, the computer program can be stored in a storage device of a server device on a communication network, such as the Internet, and the like, and a general computer system can download and the like, thereby constituting the computer 3.

[0285] In addition, in a case where the functions of the computer 3 are realized by the sharing of an OS (Operating System) and an application program or the cooperation of the OS and the application program, only the application program part can be stored in the recording medium or the storage device.

[0286] In addition, the computer program can be superimposed on a carrier and provided via a communication network. For example, the computer program can be announced in a bulletin board (BBS, Bulletin Board System) on a communication network, and the computer program can be provided via the network. Furthermore, the computer program can be started, and the same as other application programs, the processing described above can be executed under the control of the OS.

[0287] In addition to this, the configuration of the inspection device 1, the inspection method, and the program of Embodiments 1 to 4 is one example, and can be arbitrarily changed and modified as long as the problems to be solved by the present disclosure can be solved.

[0288] The present disclosure can be variously embodied and modified without departing from the broad spirit and scope of the present disclosure. In addition, the above-described embodiments are for describing the present disclosure, and do not limit the scope of the present disclosure. That is, the scope of the present disclosure is shown not by the embodiments, but by the claims. Furthermore, various modifications made within the scope of claims and their equivalent meanings are to be regarded as being within the scope of the present disclosure.

[0289] This application is based on Japanese Patent Application No. 2019-132473 filed on July 18, 2019. In this specification, the description, claims, drawings of Japanese Patent Application No. 2019-132473 are incorporated as a whole by reference.

Claims

1. An inspection device comprising: The position detection unit extracts a portion of an image for inspection, including the object being inspected, based on a coarse detection result of the object's position. This extracted portion of the image is then applied to a position detection model generated using image information from a learning image. A score is calculated for each portion of the image, and portions whose scores satisfy a pre-defined relationship are identified. Within a pre-determined range centered on the identified portion of the image, the position of the pixel with the highest brightness value is determined, thereby detecting the position of the object being inspected in the image. The coarse detection result of the location of the object being inspected is calculated by applying the image used for inspection to a coarse location detection model generated by machine learning using image information from the image being learned; and The position output unit outputs the position of the object being inspected as detected by the position detection unit.

2. The inspection device according to claim 1, wherein, The inspection device also includes a position standard determination unit, which determines whether the position of the object to be inspected detected by the position detection unit conforms to a predetermined standard.

3. The inspection device according to claim 1, wherein, The location detection model is generated by machine learning, which also uses location detection range information representing the detection range in the image being learned.

4. The inspection device according to claim 2, wherein, The location detection model is generated by machine learning, which also uses location detection range information representing the detection range in the image being learned.

5. The inspection device according to any one of claims 1 to 4, wherein, It also has: The anomaly detection unit applies the image for inspection to an anomaly detection model generated by machine learning using image information from the image for learning, calculates a score for a portion of the image for inspection, determines the portion of the image for inspection whose score corresponds to a pre-defined relationship, and thereby detects anomalies on the surface of the object to be inspected in the image for inspection. as well as The anomaly output unit outputs the anomalies detected by the anomaly detection unit in the object under inspection.

6. The inspection device according to claim 5, wherein, The inspection device also includes an anomaly standard determination unit, which determines whether the surface of the object to be inspected, even if the anomaly detection unit detects an anomaly, meets a predetermined standard.

7. The inspection device according to claim 5, wherein, The anomaly detection model is generated by machine learning, which also uses anomaly detection range information representing the detection range in the image being studied.

8. The inspection device according to claim 6, wherein, The anomaly detection model is generated by machine learning, which also uses anomaly detection range information representing the detection range in the image being studied.

9. The inspection device according to any one of claims 1 to 4, wherein, The inspection device also includes a position detection model generation unit, which generates the position detection model by using machine learning of image information from the learning image.

10. The inspection device according to claim 9, wherein, The inspection device further includes a masking image generation unit that generates a masking image. This masking image is obtained by masking the portion of the pre-acquired learning image, which includes the inspected object where an anomaly was detected, by covering the portion where the anomaly was detected. The location detection model generation unit is able to generate a location detection model by using machine learning of the image information of the masked image.

11. The inspection apparatus according to claim 10, wherein, When an anomaly is detected in an image for inspection, the shielding image generation unit generates a shielding image that covers the portion of the image for inspection where the anomaly was detected. The position detection unit applies the shielding image to the position detection model, calculates a score for a portion of the shielding image, and determines that the score is a portion of the shielding image with a pre-defined relationship, thereby enabling the detection of the position of the object to be inspected in the shielding image.

12. The inspection apparatus according to claim 9, wherein, It also has: The video recording department is responsible for recording images of the objects being inspected; and The illumination output unit outputs illumination light to the object being inspected. The position detection model generation unit performs machine learning on an image including the reflected light when illumination light is output to the object being inspected, thereby generating the position detection model.

13. The inspection apparatus according to claim 10, wherein, It also has: The video recording department is responsible for recording images of the objects being inspected; and The illumination output unit outputs illumination light to the object being inspected. The position detection model generation unit performs machine learning on an image including the reflected light when illumination light is output to the object being inspected, thereby generating the position detection model.

14. The inspection apparatus according to claim 11, wherein, It also has: The video recording department is responsible for recording images of the objects being inspected; and The illumination output unit outputs illumination light to the object being inspected. The position detection model generation unit performs machine learning on an image including the reflected light when illumination light is output to the object being inspected, thereby generating the position detection model.

15. The inspection device according to claim 1, wherein, The inspection device also includes a coarse location detection model generation unit, which generates the coarse location detection model by using machine learning of image information from the learning image.

16. The inspection apparatus according to claim 5, wherein, The inspection device also includes an anomaly detection model generation unit, which generates the anomaly detection model by using machine learning of the image information of the learning image.

17. The inspection apparatus according to claim 6, wherein, The inspection device also includes an anomaly detection model generation unit, which generates the anomaly detection model by using machine learning of the image information of the learning image.

18. The inspection apparatus according to claim 7, wherein, The inspection device also includes an anomaly detection model generation unit, which generates the anomaly detection model by using machine learning of the image information of the learning image.

19. The inspection apparatus according to claim 8, wherein, The inspection device also includes an anomaly detection model generation unit, which generates the anomaly detection model by using machine learning of the image information of the learning image.

20. The inspection device according to claim 1, wherein, The position detection unit applies portions of the image to be inspected, extracted from the detection range of the image to be inspected at predetermined coarse detection intervals, determined based on the coarse detection results, to the position detection model to calculate the score.

21. An inspection method, comprising: The location detection step involves extracting a portion of the image containing the object to be inspected based on a coarse detection result of the object's location. This extracted portion of the image is then applied to a location detection model generated using image information from a learned image. A score is calculated for this portion of the image, and portions of the image whose scores satisfy a pre-defined relationship are identified. Within a pre-determined range centered on the identified portion of the image, the location of the pixel with the highest brightness value is determined, thereby detecting the location of the object to be inspected in the image. The coarse detection result of the object's location is calculated by applying the image to the coarse location detection model generated using image information from the learned image. The position output step outputs the position of the object under inspection detected in the position detection step.

22. A recording medium storing a program that can be read by a computer, the program enabling the computer to function as a component: The position detection unit extracts a portion of an image for inspection, including the object being inspected, based on a coarse detection result of the object's position. This extracted portion of the image is then applied to a position detection model generated using image information from a learning image. A score is calculated for each portion of the image, and portions whose scores satisfy a pre-defined relationship are identified. Within a pre-determined range centered on the identified portion of the image, the position of the pixel with the highest brightness value is determined, thereby detecting the position of the object being inspected in the image. The coarse detection result of the location of the object being inspected is calculated by applying the image used for inspection to a coarse location detection model generated by machine learning using image information from the image being learned; and The position output unit outputs the position of the object being inspected as detected by the position detection unit.

Images