Inspection device, inspection method, and inspection program

Abstract

This invention achieves test technology with which an erroneous determination is unlikely to occur. A test device (1) comprises at least one processor (12). The test device is characterized in that the processor (12) executes: a first determination process (S11) for using a model (M1) generated by machine learning to determine the class of a test target from a first image (I1) obtained by imaging the test target; and a second determination process (S12) for, if the result of the first determination process (S11) is not a predetermined specific class, determining the class of the test target included as the subject in the first image (I1) by referencing, in addition to the first image (I1), a second image (I2) representing a normal test target.

Description

[Technical Field]

[0001] The present invention relates to an inspection device, an inspection method, and an inspection program for determining the state of an object to be inspected. [Background technology]

[0002] Machine learning is used in the image inspection of industrial products. By using a model that takes an image obtained by capturing an industrial product as input and outputs a class that indicates the state of the industrial product, it is possible to obtain judgment results with accuracy comparable to that of a visual inspection by an inspector. For example, Patent Document 1 discloses such a technology. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2020-42755 [Patent Document 2] Japanese Patent Publication No. 2020-52044 Summary of the Invention [Problem to be solved by the invention]

[0004] However, in image inspection using a model generated by machine learning, erroneous determinations may occur, such as an incorrect determination result indicating that an object is normal even though it is normal, or an incorrect determination result indicating that an object is abnormal even though it is abnormal. For example, the former type of erroneous determination is likely to occur when a normal object has structural features that frequently appear in abnormal objects. Furthermore, the latter type of erroneous determination is likely to occur when an abnormal object has structural features that frequently appear in normal objects.

[0005] One aspect of the present invention has been made in view of the above problems, and its purpose is to realize an inspection technique that is less likely to result in erroneous determinations. [Means for solving the problem]

[0006] An inspection device according to one embodiment of the present invention includes at least one processor, which executes a first determination process using a model generated by machine learning to determine the class of an object to be inspected from a first image obtained by photographing the object to be inspected, and a second determination process, if the result of the first determination process is not a predetermined specific class, to determine the class of the object to be inspected included as a subject in the first image by referring to a second image representing a normal object to be inspected in addition to the first image.

[0007] An inspection method according to another aspect of the present invention includes a first determination process in which at least one processor determines the class of an object to be inspected from a first image obtained by photographing the object to be inspected using a model generated by machine learning, and a second determination process in which the processor determines the class of the object to be inspected included as a subject in the first image by referring to a second image representing a normal object to be inspected in addition to the first image if the result of the first determination process is not a predetermined specific class. [Effects of the Invention]

[0008] According to one aspect of the present invention, it is possible to reduce the occurrence of erroneous determinations in which the determination result does not indicate a specific class even though the object to be inspected belongs to that specific class. [Brief explanation of the drawings]

[0009] [Figure 1] 1 is a block diagram showing a configuration of an inspection device according to an embodiment of the present invention; [Figure 2] 2 is a flowchart showing the flow of an inspection method using the inspection device shown in FIG. 1. [Figure 3] FIG. 3 is a flowchart showing a first modified example of the inspection method shown in FIG. [Figure 4] FIG. 3 is a flowchart showing a second modified example of the inspection method shown in FIG. [Figure 5]FIG. 10 is a flowchart showing a third modified example of the inspection method shown in FIG. [Figure 6] 3 is a schematic diagram showing a modified example of the second determination process shown in FIG. 2. FIG. [Figure 7] FIG. 1 is a diagram showing an image of an object to be inspected that has a normal structure but is difficult to distinguish from the characteristic structure of an abnormal area. [Figure 8] FIG. 10 is a flowchart showing a specific example of matching processing. DETAILED DESCRIPTION OF THE INVENTION

[0010] (Configuration of inspection equipment) The configuration of an inspection device 1 according to one embodiment of the present invention will be described with reference to FIG. 1. The inspection device 1 is a device for determining the state of an object to be inspected. In this embodiment, the object to be inspected is assumed to be a circuit board. However, the object to be inspected is arbitrary and is not limited to a circuit board. Hereinafter, when referring to an object to be inspected without specifying an individual, it will be referred to as an inspection object T, and when referring to an object to be inspected by specifying an individual, it will be referred to as an inspection object T1, an inspection object T2, etc.

[0011] 1, the inspection device 1 includes a memory 11, a processor 12, and a storage 13. The memory 11, the processor 12, and the storage 13 are connected to one another via a bus (not shown). An input / output interface (not shown) may be further connected to this bus. This input / output interface is used, for example, to input an image from an external device (e.g., a camera) to the inspection device 1, or to output inspection results from the inspection device 1 to an external device (e.g., a display, etc.).

[0012] The memory 11 is configured to expand and store an inspection program P1 for performing an inspection method S1 (described later), a model M1 used in the inspection method S1 (described later), a first image I1 referenced in the inspection method S1 (described later), and a second image I2 referenced in the inspection method S1 (described later) in a state that can be referenced by the processor 12. In this specification, the term "program" refers to an algorithm generated by programming, and the term "model" refers to an algorithm generated by machine learning. For example, a semiconductor RAM (Random Access Memory) or the like can be used as the memory 11. For example, a CNN (Convolutional Neural Network) can be used as the model M1.

[0013] The processor 12 is configured to execute an inspection method S1, which will be described later, in accordance with an inspection program P1 stored in the memory 11. As the processor 12, for example, a CPU (Central Processing Unit), a GPU (Graphic Processing Unit), or a combination thereof can be used.

[0014] The storage 13 is configured to store (non-volatilely store) the inspection program P1 and the model M1. When executing an inspection method S1 described below, the processor 12 loads the inspection program P1 and the model M1 stored in the storage 13 onto the memory 11 and references them. The storage 13 may be, for example, a flash memory, an HDD (Hard Disk Drive), an SSD (Solid State Drive), or a combination thereof.

[0015] Although the configuration in which the inspection method S1 described below is executed by a single processor 12 provided in a single computer has been described, the present invention is not limited to this. That is, it is also possible to adopt a configuration in which the inspection method S1 described below is executed jointly by multiple processors provided in a single computer or distributed across multiple computers.

[0016] Also, although the configuration in which the model M1 is stored in a single storage 13 provided in a single computer has been described here, the present invention is not limited to this. That is, it is also possible to adopt a configuration in which the model M1 is stored in a plurality of storages, which are centrally provided in a single computer, or which are distributed across a plurality of computers.

[0017] The inspection program P1 for causing the processor 12 to execute the inspection method S1 described below may be recorded on a computer-readable, non-transitory, tangible recording medium. This recording medium may be the memory 11, the storage 13, or another recording medium. For example, a tape, a disk, a card, a semiconductor memory, a programmable logic circuit, etc. may be used as the other recording medium.

[0018] (Testing method flow) The flow of the inspection method S1 using the inspection device 1 will be described with reference to Fig. 2. Fig. 2 is a flow chart showing the flow of the inspection method S1.

[0019] As shown in FIG. 2, the inspection method S1 includes a first determination process S11 and a second determination process S12.

[0020] The first determination process S11 is a process for determining the class of the inspection object T1 by referring to a first image I1 obtained by photographing the inspection object T1. In this embodiment, the processor 12 executes the first determination process S11 by using a model M1 generated by machine learning. Here, the model M1 is a model that receives an image obtained by photographing the inspection object T as input and outputs the class of the inspection object T. In this embodiment, the output of the model M1 is any one of class C1 indicating that the inspection object T is normal, class C2 indicating that the inspection object T is abnormal, and class C3 indicating that the state of the inspection object T is unknown.

[0021] Note that class C1, which indicates that the inspection object T is normal, may be divided into a plurality of subclasses C11 to C1m (m is a natural number equal to or greater than 2). In this case, the model M1 may be configured to output one of the subclasses C11 to C1m, instead of outputting class C1. Similarly, class C2, which indicates that the inspection object T is abnormal, may be divided into a plurality of subclasses C21 to C2n (n is a natural number equal to or greater than 2). In this case, the model M1 may be configured to output one of the subclasses C21 to C2n, instead of outputting class C2. Furthermore, the model M1 may be configured not to output class C3. In this case, the model M1 outputs one of class C1 (or subclasses C11 to C1m) and class C2 (or subclasses C21 to C2n).

[0022] If the result of the first determination process S11 is a predetermined specific class, the processor 12 takes the result of the first determination process S11 as the final result and terminates the series of processes. In this embodiment, if the result of the first determination process S11 is class C1 (normal class), the processor 12 takes class C1 (normal class) as the final result and terminates the series of processes. On the other hand, if the result of the first determination process S11 is not a predetermined specific class, the processor 12 further executes the second determination process S12. In this embodiment, if the result of the first determination process S11 is not class C1 (normal class), that is, if the result of the first determination process S11 is class C2 (abnormal class) or class C3 (unknown class), the processor 12 further executes the second determination process S12.

[0023] If the output of the model M1 is the likelihood of each class, then "the result of the first determination process S11 is a specific class" means that "there is no other class with a higher likelihood than the specific class" or "the likelihood of the specific class exceeds a predetermined threshold." In this case, "the result of the first determination process S11 is not a specific class" means that "there is another class with a higher likelihood than the specific class" or "the likelihood of the specific class is below a threshold."

[0024] The second determination process S12 is a process for determining the class of the inspection object T1 included as a subject in the first image I1 by referring to a second image I2 representing a normal inspection object T in addition to the first image I1. In this embodiment, the processor 12 executes the second determination process S12 by performing pattern matching between the first image I1 and the second image I2. If the first image I1 matches the second image I2, the result of the second determination process S12 is class C1, which indicates that the inspection object T is normal. On the other hand, if the first image I1 does not match the second image I2, the result of the second determination process S12 is class C2, which indicates that the inspection object T is abnormal.

[0025] Instead of performing pattern matching between the first image I1 and the second image I2 to perform the second determination process S12, the second determination process S12 may be performed by comparing features between the first image I1 and the second image I2. Specific examples of features include, but are not limited to, local features such as HoG (Histogram of Gradient) features and AKAZE (Accelerated KAZE) features. Specific examples of matching methods include, but are not limited to, distance-based methods such as Manhattan distance, Euclidean distance, and dot product distance.

[0026] The second image I2 referenced in the second judgment process S12 may be an image representing a normal inspection object T generated by a camera, or may be an image representing a normal inspection object T generated by a computer.

[0027] An example of the second image I2 generated by the camera is an image obtained by capturing an image of a normal inspection object T. An example of a normal inspection object T is an inspection object T2 that has been confirmed to be normal by an inspector. When inspecting inspection objects T consecutively, an inspection object T2 that was inspected before inspection object T1 and that has been determined to be normal by the inspection device 1 can also be used as a normal inspection object image T.

[0028] Examples of the second image I2 generated by a computer include a drawing image, a CG (Computer Graphics) image, a CAD (Computer Aided Design) image, etc., which represent a normal inspection object T. The second image I2 may be created for designing or manufacturing the inspection object T, or may be created for visually representing the inspection object T. Mask data used for etching or a simulated image generated by a GAN (Generative Adversarial Network) can also be used as the second image I2.

[0029] The second determination process S12 may be performed using a model generated by machine learning. In this case, a model is used that receives the first image I1 and the second image I2 as input and outputs either a class C1 indicating that the inspection object T is normal or a class C2 indicating that the inspection object T is abnormal.

[0030] Furthermore, the second image I2 used in the second determination process S2 may be one or multiple. In the latter case, the processor 12 performs the second determination process S2 for each of the multiple second images I2. In this case, if there is a second image whose pattern (or feature values) match the first image I1, the result of the second determination process S12 is class C1, indicating that the inspection object T is normal. On the other hand, if there is no second image I2 whose pattern (or feature values) match the first image I1, the result of the second determination process S12 is class C2, indicating that the inspection object T is abnormal. In this case, pattern matching or feature matching may be performed on all second images I2, or a known neighborhood search technique may be used to narrow down the second images I2 on which pattern matching or feature matching is performed. Known neighborhood search techniques include, for example, (1) a method using Kd Tree, (2) a method of dividing a storage area according to feature values, and (3) approximate nearest neighbor search such as ANN (Approximate Nearest Neighbor) and LSH (Locality Sensitive Hashing). Furthermore, when the second determination process S2 is implemented by feature matching, it is necessary to match multiple feature values ​​of the first image I1 with multiple feature values ​​of the second image I2. In this case, too, known neighborhood search techniques may be used to narrow down the feature values ​​to be matched.

[0031] (Effects of inspection methods) In the learning phase, the model M learns structural features that frequently appear in abnormal inspection objects T, and in the inference phase, it determines that an inspection object T having such structural features is abnormal. If the inspection object T is a circuit board, for example, areas where the width of the wiring changes irregularly or areas where the wiring is discontinuous are learned as structural features that frequently appear in abnormal inspection objects T. However, even normal inspection objects T may have such structural features. In particular, if the design of the inspection object T is automated using a computer, even normal inspection objects T tend to have such structural features.

[0032] For this reason, even if the inspection object T1 is normal, the result of the first determination process S11 may not be class C1 (normal class). In response to this, in the inspection device 1, if the result of the first determination process S11 is not class C1 (normal class), the second determination process S12 is executed. In the second determination process S12, in addition to the first image I1 referenced in the first determination process S11, a second image I2 representing a normal inspection object T is referenced. Therefore, even if the inspection object T1 contains structural features that frequently appear in abnormal inspection objects T, if the inspection object T1 is normal, the result of the second determination process S12 is likely to be class C1 (normal class).

[0033] Therefore, the inspection method S1 has the effect of reducing the possibility of an erroneous determination that the inspection result is not in class C1 (normal class) even though the inspection object T1 is normal. In particular, by using, as the second image I2, an image representing the inspection object T having a structure that is difficult to distinguish from an abnormal structure by the model M1, even though it is a normal structure, the effect of reducing the possibility of the above-mentioned erroneous determination becomes more pronounced.

[0034] As an example, FIG. 7 shows a circuit board designed using a computer and a first image I1 obtained by capturing an image of a portion of the circuit board. The first image I1 shown in FIG. 7 includes a portion having a shape that is difficult to distinguish from the characteristic shape of a disconnection, even though the first image I1 has the shape as designed. Therefore, when the first determination process S11 is performed on the first image I1 shown in FIG. 7, the model M1 that has learned the characteristic shape of the disconnection may determine that the wiring is abnormal (or not normal). Even in such a case, the second determination process S12, which is performed with reference to the second image I2, can determine that the wiring is normal.

[0035] In this embodiment, the second determination process S12 is always executed when the result of the first determination process S11 is not class C1 (normal class). However, the present invention is not limited to this. For example, if the abnormality class C2 is divided into multiple subclasses C21 to C2n, the second determination process S12 may be executed only when the result of the first determination process S11 is a specific subclass C2i. For example, when a model M1 that has learned the characteristic shape of a disconnection location is used, if the result of the first determination process S11 is subclass C2i indicating a "disconnection abnormality," there is a high possibility that an erroneous determination has occurred. However, if the result of the first determination process S11 is subclass C2j indicating a "short-circuit abnormality," there is a low possibility that an erroneous determination has occurred. Therefore, by executing the second determination process S12 only in the former case, the possibility of an erroneous determination can be effectively reduced without needlessly executing the second determination process S12.

[0036] Patent Document 2 discloses an image assessment device equipped with two image assessment units, namely, a non-machine learning image assessment unit and a machine learning image assessment unit. This image assessment device utilizes the assessment result of the non-machine learning image assessment unit only when the reliability of the machine learning image assessment unit is low. Here, the reliability of the machine learning image assessment unit is determined based on the degree of agreement between the assessment results by the two image assessment units for past inspection target images, rather than the assessment result of the machine learning image assessment unit for the current inspection target image. Therefore, once the reliability of the machine learning image assessment unit is determined to be high, the assessment result of the non-machine learning image assessment unit is no longer utilized, regardless of the assessment result of the machine learning image assessment unit for the current inspection target image. In other words, the image assessment device described in Patent Document 2 has a fundamentally different configuration from the inspection device 1 according to the present embodiment, which determines whether to execute the second assessment process S12 with reference to the second image I2 depending on the assessment result (class) of the first assessment process S11 for the first image I1. For this reason, the image assessment device described in Patent Document 2 cannot achieve the effect that can be obtained for each image with the inspection method S1 of this embodiment, which is to "reduce the possibility of an erroneous judgment that the inspection result is not class C1 (normal class) even though the object to be inspected T1 is normal."

[0037] (Modification of the inspection method) In the inspection method S1 shown in Figure 2, a configuration is adopted in which a second judgment process S12 is performed when the result of the first judgment process S11 is not class C1 (normal class) in order to reduce the possibility of an erroneous judgment in which the test result is not class C1 (normal class) even though the test object T1 is normal, but the present invention is not limited to this.

[0038] For example, in order to reduce the possibility of an erroneous determination in which the inspection result is not classified as class C2 (abnormal class) even though the inspection object T1 is abnormal, a configuration may be adopted in which a second determination process S12 is performed when the result of the first determination process S11 is not classified as class C2 (abnormal class). FIG. 3 is a flow diagram showing the flow of an inspection method S1A employing such a configuration. The contents of the first determination process S11 and the second determination process S12 shown in FIG. 3 are similar to the contents of the first determination process S11 and the second determination process S12 shown in FIG. 2, and therefore a description thereof will be omitted. In this case, by using, as the second image I2, an image representing the inspection object T having a structure that is difficult to distinguish from a normal structure using the model M1, even though the structure is abnormal, the effect of reducing the possibility of the above-mentioned erroneous determination becomes more pronounced.

[0039] Furthermore, in order to reduce the possibility that the test result will be class C3 (unknown class), a configuration may be adopted in which a second determination process S12 is performed when the result of the first determination process S11 is neither class C1 (normal class) nor class C2 (abnormal class). Figure 4 is a flow diagram showing the flow of a test method S1B employing such a configuration. The contents of the first determination process S11 and the second determination process S12 shown in Figure 4 are the same as the contents of the first determination process S11 and the second determination process S12 shown in Figure 2, and therefore a description thereof will be omitted.

[0040] It should be noted that a configuration in which the first determination process S11 and the second determination process S12 are executed in parallel may be employed. Fig. 5 is a flow diagram showing the flow of an inspection method S1C employing such a configuration. In addition to the first determination process S11 and the second determination process S12, the inspection method S1C includes a third determination process S13. The contents of the first determination process S11 and the second determination process S12 shown in Fig. 5 are similar to the contents of the first determination process S11 and the second determination process S12 shown in Fig. 2, and therefore a description thereof will be omitted. The third determination process S13 is a process for deriving a final determination result from the results of the first determination process S11 and the second determination process S12.

[0041] For example, if the result of the first determination process S11 is class C1 (normal class), class C1 (normal class) is set as the result of the third determination process S13 regardless of the result of the second determination process S12. Also, if the result of the first determination process S11 is not class C1 (normal class), the result of the second determination process S12 is set as the result of the third determination process S13. This makes it possible to more efficiently execute a test equivalent to the test method S1 shown in FIG.

[0042] Alternatively, if the result of the first determination process S11 is class C2 (abnormal class), class C2 (abnormal class) is set as the result of the third determination process S13 regardless of the result of the second determination process S12. Also, if the result of the first determination process S11 is not class C2 (abnormal class), the result of the second determination process S12 is set as the result of the third determination process S13. This makes it possible to more efficiently execute a test equivalent to the test method S1A shown in FIG.

[0043] (Modification of the second determination process) In the second judgment process S12 shown in Figure 2, a configuration is adopted in which pattern matching is performed between the first image I1 and the second image I2 under the assumption that the angle of view of the first image I1 matches the angle of view of the second image I2, but the present invention is not limited to this.

[0044] For example, if the angle of view of the first image I1 is smaller than the angle of view of the second image I2, the second determination process S12A shown in FIG. 6 may be performed instead of the second determination process S12 shown in FIG.

[0045] 6 includes an extraction process S121 (an example of an "extraction step" and "identification step" in the claims) and a matching process S122 (an example of a "determination step" in the claims). Here, the extraction process S121 is a process of identifying an area in the second image I2 that corresponds to the angle of view of the first image I1 and extracting the identified area as a third image I3. Furthermore, the matching process S122 is a process of determining the class of the inspection object T1 included as a subject in the first image I1 by pattern matching between the first image I1 and the third image I3.

[0046] The extraction process S121 may be performed based on angle-of-view information about the first image I1. Here, the angle-of-view information is information specifying an area in the second image that corresponds to the angle of view of the first image I1, such as the coordinates of a representative point of the area (e.g., the upper left corner) and the width and height of the area. The angle-of-view information may be included as an image in the first image I1 or the second image I2, or may be included as metadata. A method for including the angle-of-view information as an image in the first image I1 or the second image I2 includes, for example, attaching a physical marker (e.g., a three-point marker) representing the angle-of-view information to the object under inspection and capturing an image of the object to obtain the first image I1 or the second image I2. Alternatively, a method may be employed in which an electronic marker is superimposed on the first image I1 or the second image I2 obtained by capturing an image of the object under inspection.

[0047] As shown in FIG. 8(a), the matching process S122 may be always performed regardless of the position in the second image I2 of the region extracted in the extraction process S121. Alternatively, as shown in FIG. 8(b), the matching process S122 may be performed only when the position in the second image I2 of the region extracted in the extraction process S121 coincides or substantially coincides with a predetermined position as a region prone to erroneous determination. Here, an area prone to erroneous determination is, for example, a region that contains, as a subject, a structure that is difficult to distinguish from the characteristic structure of an abnormal area, despite being a normal structure. In the second determination process S12A shown in FIG. 8(b), if the position of the region extracted in the extraction process S121 does not coincide or substantially coincide with the predetermined position, the matching process S122 is omitted, and the result of the first determination process S11 is used as the result of the second determination process S12. Therefore, the second determination process S12A shown in FIG. 8(b) can shorten the average execution time of the second determination process S12 compared to the second determination process S12A shown in FIG. 8(a).

[0048] Furthermore, in the second determination process 12A shown in FIG. 8(b), the matching process S122 may be omitted. If the position of the region extracted in the extraction process S121 in the second image I2 coincides or approximately coincides with a predetermined position, the result of the first determination process S11 is likely to be erroneous, and therefore a result different from the result of the first determination process S11 is determined as the result of the second determination process S12. For example, if the result of the first determination process S11 is class C2 (abnormal class), the result of the second determination process S12 is class C1 (normal class). That is, in this case, the class of the inspection object T included as a subject in the first image I1 is determined based on whether the position of the region identified in the extraction process S121 coincides or approximately coincides with a predetermined position as the position of a region prone to erroneous determination, and the result of the first determination process S11. This allows the matching process S122 to be always omitted, thereby further shortening the average execution time of the second determination process S12.

[0049] Furthermore, instead of executing the above-described matching process S122, a matching process (another example of the "determination step" in the claims) may be executed to determine the class of the inspection object T1 included as a subject in the first image I1 by matching feature amounts between the first image I1 and the third image I3. Specific examples of feature amounts include, but are not limited to, local feature amounts such as HoG (Histogram of Gradient) feature amounts and AKAZE (Accelerated KAZE) feature amounts. Specific examples of matching methods include, but are not limited to, distance-based methods such as Manhattan distance, Euclidean distance, and dot product distance.

[0050] Furthermore, in each of the above-described embodiments, the result of the second determination process S12 is either class C1 (normal class) or class C2 (abnormal class), but the present invention is not limited to this. For example, even in the second determination process S12, there may be cases where it is not possible to determine whether the test object T belongs to class C1 (normal class) or class C2 (abnormal class). In such cases, class C3 (unknown class) may be added to the result of the second determination process S12. In this case, the final determination result of the test method S1 will be either class C1 (normal class), class C2 (abnormal class), or class C3 (unknown class).

[0051] (summary) The inspection device according to aspect 1 of the present invention includes at least one processor, which executes a first determination process using a model generated by machine learning to determine the class of an object to be inspected from a first image obtained by photographing the object to be inspected, and a second determination process, if the result of the first determination process is not a predetermined specific class, to determine the class of the object to be inspected included as a subject in the first image by referring to a second image representing a normal object to be inspected in addition to the first image.

[0052] According to the above configuration, it is possible to reduce the occurrence of erroneous determinations in which the determination result does not indicate a specific class even though the object to be inspected is a specific class. Note that the configuration of "executing the second determination process if the result of the first determination process is not a specific class" is not limited to the configuration of "always executing the second determination process if the result of the first determination process is not a specific class without determining whether or not other conditions are satisfied." For example, it also includes the configuration of "if the result of the first determination process is not a specific class, determining whether or not other conditions are satisfied, and if other conditions are satisfied, executing the second determination process."

[0053] In addition to the configuration of aspect 1, the inspection device of aspect 2 of the present invention adopts a configuration in which the specific class is one or both of a class indicating that the object being inspected is normal and a class indicating that the object being inspected is abnormal.

[0054] According to the above configuration, it is possible to reduce either or both of the possibility of an erroneous judgment being made in which the judgment result is not normal even though the object being inspected is normal, and the possibility of an erroneous judgment being made in which the judgment result is not abnormal even though the object being inspected is abnormal.

[0055] In addition to the configuration of aspect 1, the inspection device of aspect 3 of the present invention adopts the following configuration: the specific class is a class indicating that the object being inspected is normal, and the second image is an image representing the object being inspected having a structure that is difficult to distinguish from an abnormal structure in the model, despite being a normal structure.

[0056] According to the above configuration, it is possible to further reduce the possibility of an erroneous determination in which the determination result is not abnormal even though the inspection object is normal.

[0057] In addition to the configuration of aspect 1, the inspection device of aspect 4 of the present invention adopts the following configuration: the specific class is a class indicating that the object being inspected is abnormal, and the second image is an image representing the object being inspected having a structure that is difficult to distinguish from a non-abnormal structure in the model, despite being an abnormal structure.

[0058] According to the above configuration, it is possible to further reduce the possibility of an erroneous determination being made that the determination result is not abnormal even though the object to be inspected is abnormal.

[0059] The inspection device according to aspect 5 of the present invention has the same configuration as any one of aspects 1 to 4, but further has the following configuration: the second image is an image created for the purpose of designing or manufacturing the object to be inspected.

[0060] According to the above configuration, it is possible to effectively reduce the possibility of erroneous determination by utilizing an image created for designing or manufacturing an object to be inspected.

[0061] An inspection device according to aspect 6 of the present invention has the same configuration as any of aspects 1 to 5, and further has a configuration in which the second judgment process includes an extraction step of extracting an area in the second image that corresponds to the angle of view of the first image as a third image, and a judgment step of determining the class of the object to be inspected that is included as a subject in the first image based on the first image and the third image.

[0062] According to the above configuration, when the angle of view of the first image is smaller than the angle of view of the second image, the possibility of an erroneous determination can be effectively reduced.

[0063] In addition to the configuration of aspect 6, the inspection device of aspect 7 of the present invention adopts a configuration in which the judgment step is executed only when the position of an area in the second image corresponding to the angle of view of the first image coincides or approximately coincides with a position predetermined as the position of an area where erroneous judgment is likely to occur.

[0064] According to the above configuration, the average execution time of the second determination process can be shortened.

[0065] The inspection device according to aspect 8 of the present invention has the same configuration as aspect 6 or 7, and further has the following configuration: the determination step is a step of determining the class of the object to be inspected contained as a subject in the first image by pattern matching between the first image and the third image.

[0066] According to the above configuration, the possibility of erroneous determination can be further reduced.

[0067] In addition to the configuration of aspect 6 or 7, the inspection device of aspect 9 of the present invention adopts a configuration in which the determination step is a step of determining the class of the object to be inspected contained as a subject in the first image by comparing features between the first image and the third image.

[0068] According to the above configuration, the possibility of erroneous determination can be further reduced.

[0069] An inspection device according to aspect 10 of the present invention has the same configuration as any of aspects 1 to 5, and is configured such that the second judgment process includes a step of identifying an area in the second image that corresponds to the angle of view of the first image, and the class of the object to be inspected contained as a subject in the first image is determined based on whether the position of the area identified in the identification step coincides or approximately coincides with a position predetermined as the position of an area prone to erroneous judgment, and the result of the first judgment process.

[0070] According to the above configuration, the average execution time of the second determination process can be further shortened.

[0071] An inspection method according to aspect 11 of the present invention has the same configuration as any one of aspects 1 to 10, and further comprises the step of: the processor executes the second judgment process when the result of the first judgment process is a specific subclass of a class other than the specific class.

[0072] According to the above configuration, by using a subclass that represents a determination result that is highly likely to result in an erroneous determination as the specific class, it is possible to further effectively reduce the possibility of an erroneous determination occurring.

[0073] An inspection method according to aspect 12 of the present invention includes a first determination process in which at least one processor determines the class of an object to be inspected from a first image obtained by photographing the object to be inspected using a model generated by machine learning, and a second determination process in which the processor determines the class of an object to be inspected included as a subject in the first image by referring to a second image representing a normal object to be inspected in addition to the first image if the result of the first determination process is not a predetermined specific class.

[0074] According to the above configuration, it is possible to reduce the occurrence of erroneous determinations in which the determination result does not indicate a specific class even though the object to be inspected is a specific class. Note that the configuration of "executing the second determination process if the result of the first determination process is not a specific class" is not limited to the configuration of "always executing the second determination process if the result of the first determination process is not a specific class without determining whether or not other conditions are satisfied." For example, it also includes the configuration of "if the result of the first determination process is not a specific class, determining whether or not other conditions are satisfied, and if other conditions are satisfied, executing the second determination process."

[0075] The inspection program of aspect 13 of the present invention is an inspection program for causing a computer equipped with the processor to operate as an inspection device of any of aspects 1 to 12, and causes the processor to execute the first judgment process and the second judgment process.

[0076] According to the above configuration, it is possible to reduce the occurrence of erroneous determinations in which the determination result does not indicate a specific class even though the object to be inspected belongs to that specific class.

[0077] In addition to the configuration of any one of aspects 1 to 5, this specification discloses an inspection device that employs a configuration in which the second determination process is a process of determining the class of an object to be inspected that is included as a subject in the first image by pattern matching between the first image and the second image.

[0078] According to the above configuration, when the angle of view of the first image matches the angle of view of the second image, the possibility of an erroneous determination can be effectively reduced.

[0079] This specification also discloses an inspection device that employs, in addition to the configuration of any one of aspects 1 to 5, a configuration in which the second determination process is a process of determining the class of an object to be inspected that is included as a subject in the first image by comparing features between the first image and the second image.

[0080] According to the above configuration, when the angle of view of the first image matches the angle of view of the second image, the possibility of an erroneous determination can be effectively reduced.

[0081] (Additional notes) The present invention is not limited to the above-described embodiments, and various modifications are possible within the scope of the claims. Embodiments obtained by appropriately combining the technical means included in the above-described embodiments are also included in the technical scope of the present invention. [Explanation of symbols]

[0082] 1. Inspection equipment 11. Memory 12 processors 13. Storage S1 Inspection Method S11 First determination process S12 Second determination process

Claims

1. A system comprising at least one processor, The processor: a first determination process for determining a class of an object to be inspected from a first image obtained by photographing the object to be inspected using a model generated by machine learning; a second determination process for determining a class of the inspection object included as a subject in the first image by referring to a second image representing a normal inspection object in addition to the first image, when the result of the first determination process is not a predetermined specific class; Run the specific class is a class indicating that the test object is normal, The second image is an image representing an object to be inspected that has a structure that is difficult to distinguish from an abnormal structure in the model, even though it is a normal structure. An inspection device characterized by:

2. A system comprising at least one processor, The processor: a first determination process for determining a class of an object to be inspected from a first image obtained by photographing the object to be inspected using a model generated by machine learning; a second determination process for determining a class of an object to be inspected included as a subject in the first image by referring to a second image representing an abnormal object to be inspected in addition to the first image, if the result of the first determination process is not a predetermined specific class; Run the specific class is a class indicating that the inspection object is abnormal, The second image is an image representing an object to be inspected that has an abnormal structure that is difficult to distinguish from a normal structure in the model. An inspection device characterized by:

3. A system comprising at least one processor, The processor: a first determination process for determining a class of an object to be inspected from a first image obtained by photographing the object to be inspected using a model generated by machine learning; a second determination process for determining a class of an object to be inspected included as a subject in the first image by referring to a second image representing a normal or abnormal object to be inspected in addition to the first image, if the result of the first determination process is not a predetermined specific class; Run The second determination process includes an extraction step of extracting, as a third image, an area in the second image that corresponds to the angle of view of the first image, and a determination step of determining a class of an inspection object included as a subject in the first image based on the first image and the third image. An inspection device characterized by:

4. the determining step is executed only when a position of an area in the second image corresponding to the angle of view of the first image coincides or substantially coincides with a position predetermined as a position of an area where erroneous determination is likely to occur.

4. The inspection device according to claim 3.

5. the determining step is a step of determining a class of an inspection object included as a subject in the first image by pattern matching between the first image and the third image.

4. The inspection device according to claim 3.

6. the determining step is a step of determining a class of an inspection object included as a subject in the first image by comparing feature amounts between the first image and the third image.

4. The inspection device according to claim 3.

7. A system comprising at least one processor, The processor: a first determination process for determining a class of an object to be inspected from a first image obtained by photographing the object to be inspected using a model generated by machine learning; a second determination process for determining a class of an object to be inspected included as a subject in the first image by referring to a second image representing a normal or abnormal object to be inspected in addition to the first image, if the result of the first determination process is not a predetermined specific class; Run the second determination process includes a specifying step of specifying an area in the second image that corresponds to the angle of view of the first image, and determining a class of the inspection object included as a subject in the first image based on whether or not a position of the area specified in the specifying step coincides or substantially coincides with a position predetermined as a position of an area where erroneous determination is likely to occur, and based on a result of the first determination process. An inspection device characterized by:

8. The second image is an image created for designing or manufacturing an object to be inspected.

8. The inspection device according to claim 1, wherein:

9. the processor executes the second determination process when a result of the first determination process is a specific subclass of a class other than the specific class.

8. The inspection device according to claim 1, wherein:

10. a first determination process in which at least one processor determines a class of the inspection object from a first image obtained by photographing the inspection object using a model generated by machine learning; a second determination process in which, when a result of the first determination process is not a predetermined specific class, the processor determines a class of the inspection object included as a subject in the first image by referring to a second image representing a normal inspection object in addition to the first image; the specific class is a class indicating that the test object is normal, The second image is an image representing an object to be inspected that has a structure that is difficult to distinguish from an abnormal structure in the model, even though it is a normal structure. An inspection method characterized by:

11. a first determination process in which at least one processor determines a class of the inspection object from a first image obtained by photographing the inspection object using a model generated by machine learning; a second determination process in which, when a result of the first determination process is not a predetermined specific class, the processor determines a class of the inspection object included as a subject in the first image by referring to a second image representing an abnormal inspection object in addition to the first image; the specific class is a class indicating that the inspection object is abnormal, The second image is an image representing an object to be inspected that has an abnormal structure that is difficult to distinguish from a normal structure in the model. An inspection method characterized by:

12. a first determination process in which at least one processor determines a class of the inspection object from a first image obtained by photographing the inspection object using a model generated by machine learning; a second determination process in which, when a result of the first determination process is not a predetermined specific class, the processor determines a class of the test object included as a subject in the first image by referring to a second image representing a normal or abnormal test object in addition to the first image; The second determination process includes an extraction step of extracting, as a third image, an area in the second image that corresponds to the angle of view of the first image, and a determination step of determining a class of an inspection object included as a subject in the first image based on the first image and the third image. An inspection method characterized by:

13. a first determination process in which at least one processor determines a class of the inspection object from a first image obtained by photographing the inspection object using a model generated by machine learning; a second determination process in which, when a result of the first determination process is not a predetermined specific class, the processor determines a class of the test object included as a subject in the first image by referring to a second image representing a normal or abnormal test object in addition to the first image; the second determination process includes a specifying step of specifying an area in the second image that corresponds to the angle of view of the first image, and determining a class of the inspection object included as a subject in the first image based on whether or not a position of the area specified in the specifying step coincides or substantially coincides with a position predetermined as a position of an area where erroneous determination is likely to occur, and based on a result of the first determination process. An inspection method characterized by:

14. An inspection program for causing a computer equipped with the processor to operate as the inspection device described in any one of claims 1 to 7, the inspection program causing the processor to execute the first judgment process and the second judgment process.

Images