Inspection Equipment

Abstract

To provide an inspection device that can accurately detect a defect even when a defect occurs at a specific location and the defect is difficult to detect on a rule basis. [Solution] In an inspection device comprising one or more lights for illuminating an inspection object, an imaging unit for capturing an image of the inspection object illuminated by the lights, and an inspection device main body connected to the imaging unit, the inspection device main body, which determines whether or not there is a defect in the inspection object based on the inspection image obtained by capturing the image of the inspection object, comprises an inspection area setting unit and a trained model obtained by machine learning. The inspection area setting unit sets a specific portion of the inspection image as the inspection area. The trained model reads the inspection area and outputs a determination result for determining whether or not there is a defect in the inspection area.

Description

[Technical Field]

[0001] The present invention relates to an inspection device, and more particularly to an inspection device suitable for detecting defects occurring in minute components such as electronic components exemplified by semiconductor elements. [Background technology]

[0002] BACKGROUND ART Inspection devices for inspecting electronic components such as semiconductor elements for defects are known (see, for example, Patent Documents 1 to 3).

[0003] The device described in Patent Document 1 detects abnormalities in an object under inspection using a threshold for detecting abnormalities. Specifically, the device holds a threshold indicating the degree of change in pixel values ​​that should be used to determine an abnormality, and determines an abnormality when the degree of change in pixel values ​​in an image output from an imaging device exceeds the threshold. This threshold is set by a user or the like according to inspection standards that correspond to the type, use, material, etc. of the object under inspection. In this way, the device described in Patent Document 1 detects abnormalities in an object under inspection based on a rule base that follows artificial laws. The nature of the abnormality in the abnormality is determined by AI (artificial intelligence).

[0004] The device described in Patent Document 2 uses a deep neural network to extract first defects that do not affect the evaluation of an evaluation object such as an electronic component, and sets an evaluation area by excluding the area where the first defects exist. In this way, the device described in Patent Document 2 uses AI to inspect the entire surface of the evaluation object. Then, the evaluation of the defects in the evaluation object is performed based on a rule-based system.

[0005] The device described in Patent Document 3 uses rule-based image processing to remove information that becomes noise in the quality inspection of solder joints. After that, machine learning is used to extract image features, which are then converted into index values ​​that indicate whether the product is good or bad. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] Japanese Patent Application Publication No. 2022-139796 [Patent Document 2] Japanese Patent Publication No. 2022-090513 [Patent Document 3] Japanese Patent Publication No. 2022-086149 Summary of the Invention [Problem to be solved by the invention]

[0007] However, in electronic components, defects may occur only in limited locations. For example, burrs that occur on electrodes of electronic components occur only around the electrodes, and do not occur in locations distant from the electrodes. On the other hand, it may be difficult to accurately detect the presence of defects such as burrs using a detection method based on an artificially set rule base. As such, Patent Documents 1 to 3 do not disclose configurations suitable for cases where a defect occurs in a specific location and is difficult to detect using a rule base.

[0008] In view of the above problems, one object of the present invention is to enable accurate detection of defects even when a defect occurs at a specific location and the defect is difficult to detect using a rule-based method. [Means for solving the problem]

[0009] The present invention relates to the following inspection device.

[0010] (1) An inspection device that determines whether or not an object to be inspected has a defect based on an inspection image obtained by photographing the object to be inspected, An inspection area setting unit and a trained model obtained by machine learning, the inspection area setting unit sets a specific portion of the inspection image as an inspection area; The trained model reads the inspection area and outputs a discrimination result for discriminating the presence or absence of the defect in the inspection area.

[0011] (2) The inspection device described in (1) above, wherein the inspection area setting unit determines a reference area in the inspection image that serves as a basis for setting the inspection area, and sets the inspection area based on the reference area.

[0012] (3) The inspection device according to (1) or (2), wherein the inspection area setting unit configures the inspection area by combining a plurality of small areas.

[0013] (4) there are a plurality of types of defects in the inspection object, The inspection device according to any one of (1) to (3), wherein the trained model is provided for each type of defect.

[0014] (5) The inspection area setting unit sets the inspection area at a plurality of different locations; The inspection device according to any one of (1) to (4), wherein the inspection regions at a plurality of locations are separated from each other or partially overlap each other.

[0015] (6) The inspection device according to (5), wherein the trained model is provided for each of the plurality of inspection areas.

[0016] (7) The inspection device described in (5) or (6), wherein the inspection area setting unit configures the inspection area by combining multiple small areas for each of the multiple inspection areas, and makes the configuration conditions of the small areas different between one inspection area and another inspection area.

[0017] (8) The inspection device according to any one of (1) to (7), further comprising a notification control unit that notifies the presence or absence of the defect based on the discrimination result.

[0018] (9) The inspection area setting unit configures the inspection area by combining a plurality of small areas, The inspection device according to (8), wherein the notification control unit notifies the defect when detecting the presence of the defect in at least one of the small regions in the trained model.

[0019] (10) An inspection device described in any one of (1) to (9), wherein the trained model is a model trained using images of the inspection object without the defect and images of the inspection object with the defect as training data.

[0020] (11) The inspection object includes an electronic component, The inspection device according to any one of (1) to (10), wherein the inspection area includes at least one of an area in the inspection image where a specific component of the electronic component is present and an area surrounding the component.

[0021] (12) A display control unit is further provided to control the display performed by the display device, the display control unit causes the display device to display setting content for setting the specific part of the inspection image as the inspection area; The inspection device described in any one of (1) to (11), wherein the display control unit causes the display device to display settings for setting whether or not to use the trained model to determine whether or not the object to be inspected in the inspection area is defective.

[0022] (13) A display control unit is further provided to control the display performed by the display device, the display control unit causes the display device to display a setting result indicating that the specific portion of the inspection image has been set as the inspection area; The inspection device described in any one of (1) to (11), wherein the display control unit causes the display device to display a discrimination result obtained by using the trained model to determine whether or not the object to be inspected in the inspection area is defective. [Effects of the Invention]

[0023] According to the present invention, even when a defect occurs at a specific location and this defect is difficult to detect on a rule-based basis, the defect can be detected with high accuracy. [Brief explanation of the drawings]

[0024] [Figure 1] FIG. 1 is a schematic diagram showing an example of a configuration for inspecting the appearance of an object to be inspected based on a conventional rule. [Figure 2] FIG. 2 is a schematic diagram for explaining elements of defect determination. [Figure 3] Fig. 3(A) is a schematic perspective view of an electronic component as an inspection object to show an example of a defect that is difficult to improve rule-based detection accuracy, and Fig. 3(B) is a schematic plan view of the inspection object. [Figure 4] FIG. 4 is a side view showing the appearance of the inspection device. [Figure 5] FIG. 5 is a block diagram showing the configuration of the inspection device main body. [Figure 6] 6(A) to 6(D) are diagrams for explaining the processing performed by the inspection area setting unit on the inspection object. [Figure 7] FIG. 7 is a diagram for explaining the processing performed by the inspection area setting unit on the inspection object. [Figure 8] FIG. 8 is a diagram showing an example of an image of a small region having a defect. [Figure 9] FIG. 9 is a diagram showing an example of a heat map or the like showing the results of determining whether or not there is a burr. [Figure 10] FIG. 10 is a diagram showing an example of the display content on the display unit. [Figure 11] FIG. 11 is a flowchart illustrating an example of an operation for setting conditions for constructing a trained model. [Figure 12] FIG. 12 is a flowchart illustrating an example of an operation for setting conditions for performing a defect discrimination operation using a trained model. [Figure 13]FIG. 13 is a flowchart illustrating an example of the flow of the inspection execution operation in the inspection device. [Figure 14] 14(A) and 14(B) are diagrams showing examples of the test results. [Figure 15] Fig. 15(A) is a schematic perspective view of an electronic component as an object to be inspected in embodiment 2. Fig. 15(B) is a schematic plan view of the object to be inspected. [Figure 16] FIG. 16 is a diagram for explaining the processing performed by the inspection area setting unit on the inspection object according to the second embodiment. [Figure 17] FIG. 17 is a diagram showing an example of an image of a small region having a defect F in the second embodiment. [Figure 18] FIG. 18 is a diagram showing an example of a heat map or the like showing the results of determining whether or not there is a melted damaged portion. [Figure 19] FIG. 19 is a schematic plan view of a main part for explaining the first modification. [Figure 20] FIG. 20 is a schematic plan view of a main part for explaining the second modification. [Figure 21] FIG. 21 is a block diagram showing an example of a computer that realizes the inspection device. DETAILED DESCRIPTION OF THE INVENTION

[0025] [Defects detected by inspection equipment] FIG. 1 is a schematic diagram showing an example of a configuration for conventional rule-based inspection of the appearance of an object. As shown in FIG. 1, when inspecting the appearance of an object, the object is photographed with a camera, and the resulting image data is processed. In image processing, thresholds are set for the components (hue, saturation, and brightness) of the photographed image to detect defects in appearance. Thresholds may also be set for the minimum detectable area of ​​a defect on the object. In this way, whether or not a defect has occurred on the object is determined based on artificially set criteria according to certain rules. In the rule-based inspection shown in FIG. 1, for example, upper and lower limits of hue, lower limits of saturation, lower limits of brightness, and a minimum detectable area are set as feature quantities indicating a defect. Other examples of feature quantities indicating a defect include area, length, width, width / length, angle, transparency, etc.

[0026] Then, in the captured image, any location that fits the above rules is determined to be a location where a defect has occurred. However, such rule-based defect determination may not be able to accurately determine defects. Figure 2 is a schematic diagram for explaining the elements of defect determination. As shown in Figure 2, defects that appear in images obtained by photographing an object to be inspected may not have a uniform shape or size, and their brightness and hue may also be irregular and vary depending on the type of defect. For this reason, it may be difficult to detect some defects using a rule-based method that artificially sets thresholds for single or multiple feature quantities.

[0027] [An example of a defect that is difficult to detect accurately using rules] Defects that are difficult to detect accurately using a rule-based approach include defects that occur in electronic components. FIG. 3A is a schematic perspective view of an electronic component as an inspection object 80, illustrating an example of a defect that is difficult to detect accurately using a rule-based approach. FIG. 3B is a schematic plan view of the inspection object 80. Note that FIG. 3A omits the defect F, while FIG. 3B illustrates the defect F. The inspection object 80 is a component used in an electrical product. The inspection object 80 is a semiconductor element, a resistor, a capacitor, or the like. The semiconductor element may be an integrated circuit such as an IC (Integrated Circuit) or an LSI (Large Scale Integration), or may be a discrete semiconductor such as a transistor, a diode, an LED (Light Emitting Diode), a capacitor, or a thyristor. The inspection object 80 is often small, with a side measuring several tens of micrometers to several centimeters.

[0028] The inspection target 80 has, for example, a structure in which electrodes are exposed from a block-shaped molded resin 81. As an example, the inspection target 80 has a structure in which two electrodes 82 and 83 are exposed. In such an inspection target 80, burrs F1, which are defects F, may occur at the ends of the electrodes 82 and 83. The feature values ​​that can be set by a rule-based method for such burrs F1 are indefinite, and accurate rule-based detection may be difficult. As shown in FIG. 3B, the burrs F1 are not uniform in shape and also have indefinite hue, saturation, and brightness. In FIG. 3B, the burr F1 on the left side is zigzag-shaped and low in brightness, while the burr F1 on the right side is arc-shaped and high in brightness. Therefore, it is difficult to accurately detect both burrs F1 using a rule-based method. However, it is clear that the burrs F1 occur at the ends of the electrodes 82 and 83. That is, the burr F1, which is a defect that has occurred on the inspection object 80, is difficult to detect using a rule-based method that assigns a threshold value to the feature amount on the image at first glance, but it can be said that the location where it occurs is consistent.

[0029] In this embodiment, defects that are difficult to detect using a rule base but occur at a fixed location can be detected with higher accuracy. A specific configuration will be described below.

[0030] [Outline of inspection equipment] Fig. 4 is a side view showing the appearance of the inspection device 1. Fig. 5 is a block diagram showing the configuration of the inspection device main body 10. As shown in Figs. 4 and 5, the inspection device 1 determines whether or not there is a defect in the inspection object 80 based on an inspection image P1 obtained by photographing the inspection object 80.

[0031] The inspection device 1 includes an inspection area setting unit 12 and a trained model 13 obtained by machine learning.

[0032] The inspection area setting unit 12 sets a specific part of the inspection image P1 as the inspection area P2. The trained model 13 reads the inspection area P2 and outputs a discrimination result for discriminating whether or not a defect F exists in the inspection area P2.

[0033] With this configuration, even if a defect F occurs in a specific location in the inspection object 80 and is difficult to detect on a rule basis, the defect F can be detected with high accuracy.

[0034] [Specific example of inspection equipment configuration] [Embodiment 1] As described above, the inspection device 1 inspects the appearance of the inspection object 80 to determine whether or not the inspection object 80 has a defect F. The inspection object 80 can be exemplified by the above-mentioned electronic component, but the specific component configuration is not limited thereto. Furthermore, the defect F can be exemplified by the above-mentioned burr F1.

[0035] The inspection device 1 detects defects F that are difficult to detect based on rules but occur in a fixed location. Note that the inspection device 1 does not exclude a configuration that determines whether or not there are defects F that can be detected based on rules.

[0036] The inspection device 1 comprises one or more lights 2 provided to illuminate the object to be inspected 80, an imaging unit 3 that photographs the object to be inspected 80 illuminated by the lights 2, and an inspection device main body 10 connected to the imaging unit 3.

[0037] The inspection object 80 is held by a holding member 4 such as a suction nozzle and is placed to the side of the light source 2. The holding member 4 is configured to rotate, for example, around a horizontal axis or a vertical axis. The inspection object 80 may be judged by the inspection device 1 for the presence or absence of defects F while being transported by the holding member 4.

[0038] The lighting 2 includes a light source such as an LED (light emitting diode), a fluorescent lamp, or an incandescent lamp, and illuminates at least the portion of the inspection object 80 that is to be photographed by the imaging unit 3 with light.

[0039] The imaging unit 3 is an imaging device such as a CCD (Charge Coupled Device) camera or a CMOS (Complementary Metal Oxide Semiconductor) camera, and creates image data that identifies the captured image. In this embodiment, the imaging unit 3 captures an image of the inspection object 80 illuminated by the illumination 2, and generates image data that identifies the image of the inspection object 80.

[0040] The image specified by the image data acquired by the imaging unit 3 is also called an "inspection image."

[0041] The inspection device main body 10 is formed using, for example, a computer, and operates by executing a predetermined program.

[0042] The inspection device main body 10 includes a pre-processing unit 11, an inspection area setting unit 12, a trained model 13, a notification control unit 14, a setting unit 15, a display control unit 16, a display unit 17, and an operation unit 18.

[0043] (Preprocessing section) The pre-processing unit 11 performs pre-processing on the image data obtained by the imaging unit 3 before the image data is processed by the inspection area setting unit 12. Examples of this pre-processing include resizing to standardize the size of the inspection images, normalization to scale pixel values ​​between 0 and 1 or between -1 and 1, noise removal using a filter or the like, and histogram equalization to adjust contrast, but the specific processing is not limited thereto. Furthermore, the pre-processing unit 11 may be omitted.

[0044] (Inspection area setting unit) 6(A) to 6(D) and 7 are diagrams for explaining the processing performed by the inspection area setting unit 12 on the inspection object 80. As shown in FIGS. 5 to 7, the inspection area setting unit 12 performs, for example, the following series of processing (i) to (iv). (i) Boundary detection processing to detect the boundary P5 (edge, outer periphery) between the inspection object 80 and the background in the inspection image P1. (ii) Processing for correcting the placement of the inspection object 80 whose boundary has been detected. (iii) A rule-based discrimination process for a reference area P3 that serves as a reference for setting an inspection area P2 in the inspection image P1. (iv) A setting process for setting the inspection area P2 based on the reference area P3.

[0045] The inspection area setting unit 12 sets the area and position by setting coordinates in the inspection image P1.

[0046] The boundary detection process (i) above can be performed on a rule basis. Specifically, a configuration in which a threshold value for the difference in brightness between the portion of the boundary P5 where the inspection object 80 exists and the background portion is preset in the inspection area setting unit 12 can be exemplified. In this case, the inspection area setting unit 12 can detect a set of portions where the difference in brightness exceeds the threshold value as the boundary P5. Alternatively, the boundary P5 may be detected by utilizing the difference in binarized data in the image data between the portion of the boundary P5 where the inspection object 80 exists and the background portion. Alternatively, the inspection area setting unit 12 may detect the boundary P5 using a segmentation function in AI (artificial intelligence). An example of such AI is AI equipped with a convolutional neural network (CNN). The specific configuration of the inspection area setting unit 12 is not limited as long as it can detect the boundary P5 of the inspection object 80.

[0047] The above-mentioned (ii) arrangement correction process is a process for adjusting the arrangement of the inspection object 80 in the inspection image P1 to an arrangement suitable for discrimination in the trained model 13. This arrangement correction process is, for example, a process for aligning the inspection object 80 vertically and horizontally in the inspection image P1 as shown in FIG. 6(B). Such a process may be performed on a rule basis or using AI. An example of the arrangement correction process is a configuration in which feature points of an image are extracted and matching is performed to correct the position, but the specific configuration is not limited thereto.

[0048] Regarding the discrimination process of the reference area P3 in (iii) above, the reference area P3 refers to an area that serves as a reference for setting the inspection area P2. As described above, the defects F detected in this embodiment are defects whose occurrence locations have been identified. Therefore, for example, by setting the area where the defects F originate as the reference area P3, the accuracy of detecting the defects F can be improved.

[0049] In the inspection object 80, which is an electronic component, burrs F1, which are defects F, occur around the electrodes 82 and 83. For this reason, in this embodiment, the inspection area setting unit 12 sets the range in which the electrodes 82 and 83 exist in the inspection image P1 as the reference area P3. The configuration (method) for detecting the outer peripheries (edges) of the electrodes 82 and 83 in the inspection image P1 uses the rule base described in the boundary detection process in (i) above, but an AI-based configuration (method) can also be used.

[0050] In the setting process (iv) above, in which the inspection area P2 (P21) is set based on the reference area P3, various setting methods can be used. The inspection area P21 is set according to the type of defect F. In this embodiment, the defect F occurs around the reference area P3. Therefore, the inspection area P2 is set around the reference area P3. In other words, the same inspection area P21 is set for all inspection objects 80 as long as they are the same type of inspection object 80 (for example, inductors with the same specifications). Therefore, the same inspection area P21 is set for multiple inspection objects 80 (for example, multiple electronic components of the same type) that are inspected consecutively.

[0051] In this embodiment, the inspection area P21 for inspecting for the presence or absence of burrs F1 is set, for example, so as to border the reference area P3. Note that the inspection area P21 may also be set apart from the reference area P3. The inspection area P21 only needs to be set in a location where defects F are expected to occur, and the specific position and range of the inspection area P21 relative to the reference area P3 are not limited.

[0052] In this embodiment, the inspection region P21 is set on a setting screen P7 (FIG. 10) described later. The number of inspection regions P21 in one inspection object 80 is not limited.

[0053] In this embodiment, the inspection area P21 for determining the presence or absence of a burr F1 includes an area surrounding the electrodes 82, 83 (specific components) of the inspection object 80 in the inspection image P1. More specifically, the inspection area P21 is set as a rectangular area between the electrodes 82, 83 and arranged along the longitudinal direction of the elongated electrodes 82, 83. In this embodiment, an inspection area P21 is set for each electrode 82 and electrode 83 (each reference area P3) on the inspection object 80. In this embodiment, the multiple inspection areas P21 are spaced apart from one another. The process for setting the inspection area P21 on the electrode 82 side and the process for setting the inspection area P21 on the electrode 83 side are similar, so the setting of the inspection area P21 on the electrode 82 (reference area P3) side will be described below.

[0054] The inspection area P21 may be a single area, but in this embodiment, the inspection area setting unit 12 configures the inspection area P21 by combining a plurality of small areas P25.

[0055] The small region P25 is a region consisting of a part of the inspection region P2 (P21). Within one inspection region P21, the small regions P25 may be set to the same size as shown in FIG. 7 or may be set to different sizes. That is, the size of the small regions P25 within the inspection region P21 is individually set. The small region P25 may be square, as shown as an example in this embodiment. In FIG. 7, a single small region P25 is indicated by an imaginary line (two-dot chain line) at a location outside the illustration of the inspection region object 80 so that it can be seen that one small region P25 is square. The small region P25 may also be rectangular, polygonal such as triangular, elliptical, or circular. The small region P25 may be formed across the entire width of the inspection region P21 and only part of its length, or may be formed across a part of the width of the inspection region P21 and only part of its length, or may be formed across a part of the width of the inspection region P21 and only part of its length.

[0056] The appropriate shape and size of the small region P25 are determined based on the relationship between the position, shape, and size of the component in the object under inspection 80 where defects are expected to occur and / or its surrounding area, and the shape, size, and appearance of the defects F that will occur in the inspection image P1. For example, if the size of the defects F is small, it is preferable to reduce the size of the small region P25 accordingly. Also, if the defects F appear intermittently in the inspection image P1, it is preferable to set the small region P25 so that it comprehensively includes a group of defects F. In this way, the optimal shape and size of the small region P25 are determined based on various factors. For example, in the inspection region P21, the small region P25 may be set so that it is equal to or larger than the area of ​​the smallest burr F1 that is expected to be detected.

[0057] In this embodiment, multiple small regions P25 are set in each inspection region P21 so that adjacent small regions P25 partially overlap each other. This setting allows the inspection device 1 to more accurately identify defects F. The overlapping range of adjacent small regions P25 is not specifically limited. For example, as shown in FIG. 7, adjacent small regions P25 may overlap by half their area. Alternatively, adjacent small regions P25 may overlap by one-third their area, or by one-quarter their area. Furthermore, three or more small regions P25 may overlap in one location.

[0058] The inspection area P2 may be linked to one reference area P3 and provided as one as in this embodiment, or two or more may be provided.

[0059] (Pre-trained model) As shown in FIGS. 5 and 7, the trained model 13 inspects the inspection area P21 to determine whether or not a defect F exists in the inspection area P21. The trained model 13 determines whether or not a defect F exists in the inspection area P21 with higher accuracy by determining only the inspection area P2 (P21) in the inspection image P1 as the discrimination target. The trained model 13 may be configured as an integrated device including the preprocessing unit 11, the inspection area setting unit 12, the notification control unit 14, the setting unit 15, the display control unit 16, the display unit 17, and the operation unit 18, or may be connected via a network such as the Internet or an intranet. The trained model 13 is a model trained using as training data an image of the inspection area P21 in the inspection image P1 of the inspection object 80 without a defect F and an image of the inspection area P21 in the inspection image P1 of the inspection object 80 with a defect F. As described above, the trained model 13 is a model obtained by machine learning.

[0060] An example of an image of a small area P25 in the inspection area P21 without a defect F and an example of an image of a small area P25 in the inspection area P21 with a defect F are shown in Fig. 8. Fig. 8 is a diagram showing an example of an image of a small area P25 with a defect F.

[0061] 8, in the small region P25 that does not have a burr F1, no configuration related to the burr F1 is displayed, whereas in the small region P25 that has a burr F1, various burrs F1 are displayed.

[0062] Machine learning refers to a process in which a computer learns patterns and regularities from data. The trained model 13 then determines whether or not a defect F exists in the inspection area P21 of a given inspection image P1 based on the patterns and regularities obtained through machine learning.

[0063] Examples of such trained model 13 include models using a neural network (NN) or a support vector machine (SVM). The neural network (NN) may be a deep neural network (DNN) or a convolutional neural network (CNN). As such, the trained model 13 in this embodiment may be configured to determine the presence or absence of a defect F according to a model (algorithm) obtained by machine learning, rather than an artificially set rule base, and the specific configuration of the machine learning is not limited.

[0064] When determining whether or not there is a defect F in the inspection area P21 in the inspection image P1, the trained model 13 extracts (calculates) elements necessary for determining whether or not there is a defect F, such as image feature amounts in the inspection area P21, using a model possessed by the trained model 13. The feature amounts at this time are not artificial feature amounts based on certain standards, such as those exemplified by a rule base, but are obtained by the trained model 13 obtained by training a predetermined model with training data. An example of such feature amounts is a feature amount obtained by vector-representing components in a small area P25 of the inspection area P21, but the specific configuration is not limited thereto.

[0065] The discrimination result of the trained model 13, that is, the output of the trained model 13, can be exemplified as follows. Specifically, the trained model 13 outputs the reliability when it is determined that the small area P25 is "good" in which no defect F is present, as a percentage, per thousand, or ten thousand, between 0 and 1. For example, when the trained model 13 determines that no defect F is present with high reliability, it outputs the reliability with a value close to 1. On the other hand, similarly, when the trained model 13 determines that no defect F is present with low reliability, it outputs the reliability with a value close to 0. When the reliability is low, it can be said that the small area P25 is "fail" in which a defect F is present.

[0066] In this embodiment, the setting unit 15 is configured to be able to change the threshold value Th, which is the reliability level at which the determination of "good" and "fail" is distinguished. That is, if the setting unit 15 sets the threshold value Th to, for example, 0.6, when the reliability level output by the trained model 13 is 0.6 or higher, the notification control unit 14 determines that the result is "good." On the other hand, when the reliability level output by the trained model 13 is less than 0.6, the notification control unit 14 determines that the result is "fail."

[0067] In this embodiment, the trained model 13 outputs the discrimination result as a heat map. The heat map is a diagram that highlights the points that the trained model 13 placed emphasis on when calculating the reliability for the small region P25 where the trained model 13 performed inference.

[0068] An example of a heat map M1 for a small region P25 without a defect F and an example of a heat map M1 for a small region P25 with a defect F are shown in Fig. 9. Fig. 9 is a diagram showing an example of a heat map M1 etc. showing the results of determining the presence or absence of a burr F1.

[0069] As shown in Figure 9, in the heat map M1 showing the discrimination results for the small region P25 that does not have a burr F1 and the heat map M1 showing the discrimination results for the small region P25 that has a burr F1, the more importance is placed on the area when calculating the reliability, the darker the color (e.g., red) is displayed.

[0070] In this embodiment, a plurality of trained models 13 are provided, with one provided for at least one inspection area P2. Note that the plurality of trained models are collectively referred to as trained models 13.

[0071] Referring to Figure 5, the trained model 13 does not determine whether or not there is a defect F in the entire inspection image P1, but determines whether or not there is a defect F only in the assigned portion of the inspection area P2 (assigned small area P25).

[0072] In this embodiment, a plurality of trained models 13 are provided corresponding to each of the plurality of inspection areas P21. Specifically, a trained model 13 is provided corresponding to each of the inspection areas P21 of the electrodes 82, 83 of the inspection object 80. In other words, different trained models 13 are used for different inspection areas P21. In this embodiment, a configuration is used to discriminate one type of defect F (burr), and one trained model 13 is used for each inspection area P21.

[0073] In this embodiment, the trained model 13 is constructed by causing the untrained model to perform learning in response to an instruction from the setting unit 15. In addition, the trained model 13 performs repeated learning (relearning) by issuing a learning instruction to the trained model 13 again in response to an instruction from the setting unit 15. Note that the trained model 13 may perform repeated learning only when instructed to do so by an instruction from the setting unit 15, or may perform repeated learning autonomously.

[0074] (Notification control unit) The notification control unit 14 is provided to notify the presence or absence of a defect F based on the discrimination result obtained by the trained model 13 in determining whether or not a defect F exists in the inspection object 80. Upon receiving the discrimination result, the notification control unit 14 causes the display unit 17 to display a predetermined result.

[0075] (Settings section) The setting unit 15 is configured to perform various settings by receiving operation instructions from the operation unit 18. Examples of settings performed by the setting unit 15 include a learning instruction setting, an inspection area setting command, and an inspection execution setting.

[0076] (Display control unit and display unit) The display unit 17 is an image display unit such as a liquid crystal display device or an organic EL display device, and performs a predetermined display under the control of the notification control unit 14.

[0077] The display content on the display unit 17 is controlled by the display control unit 16. As shown in Fig. 10, the display control unit 16 causes the display unit 17 to display a setting result for indicating that a part of the inspection image P1 has been set as the inspection area P21. Specifically, the display control unit 16 causes the display unit 17 to display the inspection image P1, a setting screen P6, and a setting screen P7. Fig. 10 is a diagram showing an example of the display content on the display unit 17.

[0078] The setting screen P6 on the display unit 17 displays, for example, a position correction field P61, an area setting field P62, an AI inspection field P63, and a rule-based inspection field P64.

[0079] The position correction field P61 is a field for issuing a command to manually or automatically correct the position of the inspection image P1 (the position of the inspection image P1 relative to the origin of the coordinates set by the inspection area setting unit 12) using a preset algorithm. An example of the preset algorithm is pattern matching, but the specific algorithm is not limited thereto.

[0080] The area setting field P62 is a field where an operator issues a command to manually set an inspection area P2 within the inspection image P1. When the operator performs an operation to select this area setting field P62, a setting screen P7 is displayed on the display unit 17.

[0081] The AI ​​inspection field P63 is a field for setting whether or not to inspect the inspection object 80 using the trained model 13. When this AI inspection field P63 is set by the setting unit 15, the inspection device 1 performs an inspection using the trained model 13. In this way, the display control unit 16 causes the display unit 17 to display the setting content for setting whether or not to use the trained model 13 to determine the presence or absence of a defect F in the inspection object 80 in the inspection area P2.

[0082] The rule-based inspection field P64 is provided to give instructions to the inspection device 1 to perform a rule-based inspection in which areas of the inspection image P1 that meet certain brightness, lightness, saturation, etc. are determined to be defective.

[0083] Note that when the AI ​​inspection field P63 is selected, both an inspection using the trained model 13 and a rule-based inspection may be performed, or an inspection using the trained model 13 may be performed but a rule-based inspection may not be performed. Similarly, when the rule-based inspection field P64 is selected, both an inspection using the trained model 13 and a rule-based inspection may be performed, or an inspection using the trained model 13 may not be performed but a rule-based inspection may be performed. In this way, both an inspection using the trained model 13 and a rule-based inspection may be performed, or one of them may be performed alternatively.

[0084] The setting screen P7 displays an area name field P71, a shape field P72, a coordinates / size field P73, an area shape setting field P74, an add button P75, and a delete button P76.

[0085] The area name field P71 displays the area name, such as a serial number. The shape field P72 displays the shape of the set area. The area shape setting field P74 is a tab for setting the area shape, and in this embodiment, a rectangle or ellipse can be set. The shape that can be set in the area shape setting field P74 may also be a circle or other shape.

[0086] The coordinates and size field P73 is a field showing the coordinates that identify one area (small area P25) and the size of this area. The values ​​displayed in this coordinates and size field P73 are set by operating the operation unit 18. The coordinates indicate the position from the origin when the inspection image P1 is placed at a predetermined coordinate on the computer. The size is a value that indicates, for example, the number of pixels in the image, and is shown as vertical x horizontal pixels.

[0087] In this way, by performing settings in response to an operator's instructions using the setting screen P7, the inspection device 1 can set one or more areas (small areas P25) in the inspection image P1. That is, when an inspection area P2 is set using one area name, an inspection area P21 consisting of a single area is set. On the other hand, an inspection area P21 consisting of a combination of small areas P25 is set by arranging multiple small areas P25 side by side without any gaps or so that some of the small areas P25 overlap. The display control unit 16 displays the setting results of the inspection area P21 on the display unit 17 together with the inspection image P1. FIGS. 7 and 10 illustrate a state in which small areas P25 of the same size are set in a vertical column in the inspection area P21.

[0088] When the add button P75 is operated, a new area name is added to the area name column P71, the shape linked to this area name is displayed in the shape column P72, and the coordinates and size are displayed in the coordinate / size column P73. Furthermore, when the delete button P76 is operated with a specific area name column P71 specified, the area name column P71, the shape column P72 linked to the area name column P71, and the coordinate / size column P73 are deleted.

[0089] The settings using the setting screens P6 and P7 are performed for each type of inspection object. That is, for example, area setting is performed using the setting screens for inspection object 80, which is a semiconductor component, and area setting is also performed using the setting screens for other inspection objects (for example, electronic components such as semiconductor elements exemplified by capacitors).

[0090] (Operation unit) The operation unit 18 is an input unit such as a keyboard, a mouse, a touch panel, etc., and is operated when an operator issues a predetermined operation command to the inspection device 1.

[0091] [Operation of inspection equipment] In the inspection device 1, a process of constructing a trained model 13 by training a model before training, or a process of training the trained model 13 again is performed. Hereinafter, the model before training and the trained model 13 may be referred to as the "trained model 13" without distinction.

[0092] As an example of the operation of the inspection device 1, the following operation will be described below. (Operation 1) The operation of constructing a trained model 13 by training the model. (Operation 2) Operation for setting inspection conditions in the inspection device 1. (Operation 3) Execution of inspection in the inspection device 1.

[0093] ((Action 1) Action of setting conditions for constructing a trained model 13 by training the model) 11 is a flowchart illustrating an example of an operation for setting conditions for constructing the trained model 13. In the following description, reference will be made to FIGS. 1 to 11 as appropriate.

[0094] As shown in FIG. 11, during this operation, first, in the inspection device 1, a plurality of inspection objects 80 (e.g., several tens to 1,000 or more) of the same type as the object on which the discrimination operation using the trained model 13 is to be performed are photographed, and an identification label is input for each inspection object 80 (step S1). Specifically, the inspection objects 80 are photographed one by one by the imaging unit 3 of the inspection device 1, thereby acquiring an inspection image P1. The identification label is a label containing correct answer data for the inspection object 80, and is input to the inspection device 1 by an operator operating the operation unit 18, for example. The identification label contains information specifying whether or not there is a defect F in the inspection object 80 being photographed.

[0095] Next, the inspection area P21 is set (step S2). In this embodiment, the inspection area P2 is set manually by an operator. Specifically, with the inspection image P1 displayed on the display unit 17, the operator operates the operation unit 18, whereby setting screens P6 and P7 shown in FIG. 10 are displayed on the display unit 17. Then, through the operator's operation, an area name field P71 is created, the shape of the inspection area P2 is set in a shape field P72 linked to this area name field P71, and the coordinates and size of the inspection area P2 (small area P25) are set in a coordinate / size field P73. At this time, multiple area name fields P71 are created, and small areas P25 are set in each area name field P71, thereby setting the inspection area P2 consisting of multiple small areas P25. That is, the setting unit 15 sets the inspection area P2.

[0096] Next, the inspection area setting unit 12 cuts out the set inspection area P21 from the inspection image P1 of each of the plurality of inspection objects 80 (step S3).

[0097] Next, the images of the multiple inspection areas P2 are learned by the trained model 13 (step S4). The trained model 13 at this time is a model configured using the machine learning method described above. In this learning step, the trained model 13 learns to classify the images of the inspection area P21 into two classes: good and bad. In the present embodiment in which one type of defect F is inspected, the presence or absence of a defect F in multiple small areas P25 of one inspection area P21 is configured to be determined by one trained model 13.

[0098] In this learning step, as shown in the example of FIG. 8, a plurality of images of the small region P25 without the defect F and a plurality of images of the small region P25 with the defect F are learned.

[0099] In the learning step (step S4), the trained model 13 compares the judgment result of the presence or absence of a defect F in the read inspection area P21 with the correct answer of the identification label linked to the inspection area P21, and if the answer is incorrect, it performs learning again. In this way, the trained model 13 repeats learning, inferring the presence or absence of a defect F, and comparing the inference result with the identification label (answer checking), and constructs a trained model for the corresponding inspection area P21. As a result, a trained model 13 for inspecting the inspection target 80 is constructed and output (step S5).

[0100] Through the above operations, a trained model 13 is constructed.

[0101] ((Operation 2) Operation of setting inspection conditions in inspection device 1) FIG. 12 is a flowchart for explaining an example of an operation for setting conditions for performing a discrimination operation of a defect F using the trained model 13.

[0102] As shown in FIG. 12, when inspecting the presence or absence of defects F in the inspection object 80, the inspection device 1 captures an image of one inspection object 80 to obtain an inspection image P1 (step S11).

[0103] Next, an inspection area P21 is set (step S12). It is preferable that the inspection area P21 is the same as the inspection area P21 set when the trained model 13 was constructed, since this can further increase the accuracy of discrimination of defects F by the trained model 13. It is also preferable that the shape and size of the small area P25 are the same as the small area P25 set when the trained model 13 was constructed, since this can further increase the accuracy of discrimination of defects F by the trained model 13. On the other hand, if a small area P25 different from that set when the trained model 13 was constructed is set, this small area P25 is set manually by an operator. Specifically, the operator operates the operation unit 18, and the inspection image P1 and setting screens P6 and P7 are displayed on the display unit 17 in the same manner as when the trained model 13 was constructed in (Operation 1) above. Then, an area name column P71 is created by an operator, and the shape of the inspection area P2 is set in a shape column P72 linked to this area name column P71, and the coordinates and size of the inspection area P2 are set in a coordinate / size column P73.

[0104] Next, the setting unit 15 calls the trained model 13 for the inspection object 80 to be discriminated (step S13). Then, the setting unit 15 sets a threshold value Th for discriminating whether or not there is a defect F (step S14). The threshold value Th is set by the operator operating the operation unit 18.

[0105] In this embodiment, the threshold value Th is a value that serves as a reference value for the reliability when the trained model 13 determines whether a defect F to be determined is "good" (no defect F is present) or "fail" (defect F is present), and is set between 0 and 1 as described above. If the reliability output by the trained model 13 is equal to or greater than the threshold value Th, it will be determined as "good." If the reliability is less than the threshold value Th, it will be determined as "fail."

[0106] The above is an example of the operation for setting inspection conditions.

[0107] ((Operation 3) Inspection execution operation in inspection device 1) Fig. 13 is a flowchart for explaining an example of the flow of an inspection execution operation in the inspection device 1. As shown in Fig. 13, when inspecting the inspection object 80, the inspection area setting unit 12 cuts out an inspection area P21 from the inspection image P1 obtained when the above-mentioned inspection conditions are set (step S21). If the inspection area P21 is made up of a plurality of small areas P25, the inspection area setting unit 12 cuts out the inspection area P21 for each small area P25.

[0108] Next, the trained model 13 outputs the reliability of the presence or absence of a defect F of the type to be discriminated for the corresponding small region P25 (step S22). All trained models 13 for the inspection object 80 perform this reliability output operation. Next, the notification control unit 14 determines whether the set reliability is equal to or greater than a set threshold value Th, thereby determining whether or not a defect F is present ("good" or "fail") in the small region P25 to be discriminated (step S23).

[0109] Next, the notification control unit 14 performs a comprehensive judgment (step S24). Specifically, if any one of the small areas P25 in the inspection area P2 is judged to be "fail," the notification control unit 14 instructs the display control unit 16 to display "fail" as shown in Fig. 14(A). As a result, the display unit 17 displays "fail," the small area P25 judged to be "fail," the heat map M1 of the small area P25, and the reliability.

[0110] On the other hand, if none of the small areas P25 in the inspection area P2 are judged as "fail" and all of the small areas P25 are judged as "good," the notification control unit 14 instructs the display control unit 16 to display "good" as shown in Fig. 14(B). As a result, the display unit 17 displays "good," the small areas P25 judged as good, the heat map M1 of the small areas P25, and the reliability.

[0111] In this way, the display control unit 16 causes the display unit 17 to display the discrimination result obtained by using the trained model 13 to discriminate whether or not the inspection object 80 in the inspection area P21 is defective.

[0112] [effect] As described above, according to this embodiment, even when a defect F occurs in a specific location in the inspection object 80 and is difficult to detect on a rule-based basis, the defect F can be detected with high accuracy.

[0113] Furthermore, according to this embodiment, the inspection area setting unit 12 sets the inspection area P21 based on the reference area P3. With this configuration, the inspection area setting unit 12 can set the inspection area P21, which is a location where a defect F may occur, with high accuracy and a simple configuration.

[0114] Furthermore, according to this embodiment, the inspection device 1 can determine the presence or absence of defects F by concentrating on the small area P25, which is a small area, and therefore can determine the defects F with higher accuracy.

[0115] Furthermore, according to this embodiment, the trained model 13 is a model trained using, as training data, an inspection image P1 of an inspection object 80 without a defect F and an inspection image P1 of an inspection object 80 having a defect F. With this configuration, the training data includes the inspection image P1 of an inspection object 80 having a defect F, so that the trained model 13 can detect the defect F with higher accuracy.

[0116] The above describes an embodiment of the present invention. However, the present invention is not limited to the above embodiment. Various modifications of the present invention are possible within the scope of the claims. Note that the following mainly describes configurations that differ from the above embodiment and modified examples, and similar configurations are designated by similar reference numerals and detailed description thereof is omitted.

[0117] [Embodiment 2] Fig. 15(A) is a schematic perspective view of an electronic component as an inspection object 90 in embodiment 2. Fig. 15(B) is a schematic plan view of the inspection object 90. The inspection object 90 is a component used in an electrical product, similar to the inspection object 80 in the embodiment.

[0118] The inspection target 90 has a structure in which an electrode is exposed from a block-shaped molded resin 91, for example, and two electrodes 92 and 93 are exposed. The electrode 92 has a lead wire 94, and a melted portion F2, which is a defect F, may occur on the lead wire 94. The feature values ​​that can be set by a rule-based method for such a melted portion F2 are indefinite, and accurate rule-based detection may be difficult. The melted portion F2 does not have a constant shape, and its hue, saturation, and brightness are also indefinite. Such a melted portion F2 occurs when a spark causes a dent in the lead wire 94. However, the degree of melting and the shape of the dent are also indefinite. However, it is clear that the melted portion F2 occurs on the lead wire 94 inside the electrode 92. That is, although the melted portion F2, which is a defect occurring in the inspection object 90, is difficult to detect at first glance using a rule-based method that assigns a threshold value to the feature amount on the image, it can be said that the location where it occurs is consistent.

[0119] In the second embodiment, the configuration when inspecting the inspection object 90 will be mainly described with respect to differences from the inspection object 80 of the embodiment, and a description of the similar configuration will be omitted.

[0120] (Area setting unit of the second embodiment) 16 is a diagram for explaining the processing performed by the inspection area setting unit 12 on the inspection object 90 according to the second embodiment. In the second embodiment as well, the inspection area setting unit 12 performs the series of processing steps (i) to (iv) described in the first embodiment. (i) Boundary detection processing for detecting the boundary P5 (edge, outer periphery) between the inspection object 90 and the background in the inspection image P1. (ii) Processing for correcting the placement of the inspection object 90 whose boundary has been detected. (iii) Rule-based discrimination processing of a reference area P3 that serves as a reference for setting an inspection area P2 (P22) in the inspection image P1. (iv) A setting process for setting the inspection area P22 based on the reference area P3.

[0121] In the second embodiment, the inspection area setting unit 12 sets the range in which the lead wires 94 of the electrodes 92 exist in the inspection image P1 as the reference area P3.

[0122] In the second embodiment, the defect F occurs within the reference area P3. Therefore, the inspection area P22 is set within the reference area P3. For the same type of inspection objects 90, the same inspection area P22 is set for each inspection object 90. Therefore, the same inspection area P22 is set for multiple inspection objects 90 (for example, multiple electronic components of the same type) that are inspected consecutively.

[0123] In this way, the inspection area P22 for determining the presence or absence of the melted portion F2 includes an area within the electrode 92 (specific component) of the inspection object 90 in the inspection image P1. More specifically, the inspection area P22 is set as a rectangular area in the lead wire 94, arranged along the longitudinal direction of the elongated lead wire 94.

[0124] The inspection area setting unit 12 configures the inspection area P22 by combining a plurality of small areas P25. For example, in the inspection area P22, the small areas P25 may be set so that they are equal to or larger than the area of ​​the smallest estimated melted portion F2 among the melted portions F2 that can be detected.

[0125] (Trained model of embodiment 2) The trained model 13 inspects the inspection area P22 of the inspection image P1 to determine whether or not a defect F exists in the inspection area P22. The trained model 13 is a model trained using an image of a small area P25 of the inspection object 90 without a defect F and an image of a small area P25 of the inspection object 90 with a defect F as training data.

[0126] An example of an image of a small area P25 in the inspection area P22 without a defect F and an example of an image of a small area P25 in the inspection area P22 with a defect F are shown in Fig. 17. Fig. 17 is a diagram showing an example of an image of a small area P25 with a defect F in the second embodiment.

[0127] 17, the small region P25 that does not have the erosion portion F2 does not display the erosion portion F2. On the other hand, the small region P25 that has the erosion portion F2 displays various erosion portions F2. The trained model 13 determines the presence or absence of defects F in the inspection region P22 of the given inspection image P1 based on the patterns and regularities obtained by machine learning.

[0128] The trained model 13 extracts (calculates) elements, such as image features, necessary for determining the presence or absence of a defect F for the small region P25. The notification control unit 14 determines the determination result of the inspection object 90 as "good" or "fail" based on the set threshold value Th.

[0129] Furthermore, in the second embodiment, the trained model 13 outputs the discrimination result as a heat map.

[0130] An example of a heat map M2 for a small region P25 without a defect F and an example of a heat map M2 for a small region P25 with a defect F are shown in Fig. 18. Fig. 18 is a diagram showing an example of a heat map M2 etc. showing the results of determining the presence or absence of a melted portion F2.

[0131] As shown in Figure 18, in the heat map M2 showing the discrimination results for the small region P25 that does not have the melted portion F2, and the heat map M2 showing the discrimination results for the small region P25 that has the melted portion F2, the more importance is placed on the area when calculating the reliability, the darker the color (e.g., red) is displayed.

[0132] (Settings section) The setting unit 15 performs learning instruction setting, inspection area setting command, and inspection execution setting by receiving operation instructions from the operation unit 18. As a result, the location where the lead wire 94 is arranged is set as the reference area P3, and the area inside the lead wire 94 is set as the inspection area P22 for the inspection object 90.

[0133] [Operation of the inspection device in embodiment 2] In the inspection device 1, the following operations are performed in the same manner as in the first embodiment. (Operation 1) The operation of constructing a trained model 13 by training the model. (Operation 2) Operation for setting inspection conditions in the inspection device 1. (Operation 3) Execution of inspection in the inspection device 1.

[0134] In the above (operation 1), a plurality of (e.g., several tens to 1000 or more) inspection objects 90 of the same type as the object to be discriminated by the trained model 13 are photographed, and an identification label is input for each inspection object 80 (step S1). The identification label at this time is a label containing the correct answer data for the inspection object 90, and the identification label contains information for specifying whether or not there is a defect F in the inspection object 90 being photographed.

[0135] [effect] In the second embodiment, the defect F (melted portion F2) can be detected with high accuracy, similarly to the embodiment.

[0136] [Variation 1] 19 is a schematic plan view of the main part for explaining Modification 1. The inspection device 1 of Modification 1 differs from Embodiment 1 in that it is configured to be able to distinguish between multiple types of defects F for one inspection object 80.

[0137] 19, in the first modification, in addition to a burr F1 as a defect F, other defects F (for example, a missing portion F3) may exist at the end of the electrode 82. In such a case, the inspection device 1 can determine whether or not there are multiple types of defects F (burr F1 and missing portion F3).

[0138] In this first modification, the trained model 13 determines whether or not there are multiple types of defects F (burrs F1 and missing portions F3). In order to further increase the accuracy of determining the various types of defects F, a trained model 13 is provided for each type of defect F. In this way, a trained model 13 is provided for each type of defect F. In this first modification, two trained models 13 are provided to determine two types of defects F. More specifically, if the number of inspection areas P2 is x1, the type of defect F is y1, and the number of trained models 13 is z1, then z1 = x1 × y1 trained models 13 are provided. In this way, in the same inspection area P2, a trained model 13 for detecting burrs F1 and a trained model 13 for detecting missing portions F3 are constructed in the inspection device main body 10.

[0139] When the trained model 13 is trained, the identification labels in the training data include information that identifies the type of defect F (for example, whether the defect F is a burr F1 or a missing part F3) in addition to the presence or absence of a defect F in the imaged inspection object 90. By setting such identification labels, a trained model 13 can be constructed for each type of defect F in the trained model 13.

[0140] According to the present modification 1, since it is possible to use a trained model 13 that is specialized for discriminating one type of defect, it is possible to discriminate the defect F with higher accuracy.

[0141] [Variation 2] 20 is a schematic plan view of the main parts for explaining Modification 2. The inspection device 1 of Modification 2 differs from Modification 1 in that multiple inspection areas P2 (P21, P23) linked to one reference area P3 are provided.

[0142] As shown in FIG. 20, in the first modification, inspection regions P21 and P23 are provided at the ends of the electrode 82.

[0143] In this modified example 2, the inspection areas P21 and P23 partially overlap each other. Note that the inspection areas P21 and P23 may be spaced apart from each other. The inspection areas P21 and P23 may be set according to the type of defect F that is likely to occur. For example, the inspection area P21 may be set in a location where burrs F1 are likely to occur, and the inspection area P23 may be set in a location where defects F3 are likely to occur.

[0144] Furthermore, the configuration conditions of the small regions P25 may be different between the inspection region P21 and the inspection region P23. For example, the shape of the small regions P25 in the inspection region P21 may be different from that of the small regions P25 in the inspection region P23, such as a square and a rectangle, respectively. In the inspection region P21, the small regions P25 may be set so that their area is equal to or larger than the smallest burr F1 that can be detected. Furthermore, in the inspection region P23, the small regions P25 may be set so that their area is equal to or larger than the smallest defect F3 that can be detected. In addition to the above examples, the configuration conditions of the small regions P25 in the inspection region P21 and the inspection region P23 may be different depending on the area of ​​each small region P25, the overlapping range of the small regions P25, etc., and the specific configuration is not limited.

[0145] In this modified example 2, a trained model 13 is also provided for each type of defect F. More specifically, if the total number of inspection areas P2 is x2, the number of types of defect F is y2, and the number of trained models 13 is z2, then z2 = x2 × y2 trained models 13 are provided.

[0146] [Effects of Modification 2] According to the configuration of Modification 2, the inspection area setting unit 12 can set the inspection areas P21, P23 at multiple locations so that they are separated from each other or partially overlap each other. This allows the inspection area setting unit 12 to set the inspection area P2 in a manner that can further increase the accuracy of identifying defects F.

[0147] Furthermore, according to the configuration of Modification 2, the inspection area setting unit 12 can set different configuration conditions for the small area P25 in one inspection area P21 and another inspection area P23. This allows the small area P25 to be set under more suitable conditions in each of the inspection areas P21 and P23.

[0148] [Other variations] (1) In the above-described embodiments, an example has been described in which, when there is one type of defect F to be identified, the presence or absence of a defect F in multiple small areas P25 within one inspection area P2 is identified using one trained model 13. However, this does not have to be the case. In one inspection area P2, a trained model 13 may be constructed for each small area P25, and the presence or absence of a defect F may be identified. Also, an example has been described in which a trained model 13 is set for each type of defect F. However, this does not have to be the case. The presence or absence of multiple types of defect F may be identified using one trained model 13.

[0149] (2) Furthermore, in the above-described embodiments, an example has been described in which at least one trained model 13 is provided for each inspection area P2. However, this is not necessarily the case. For example, if the electrodes 82 and 83 of the inspection object 80 are of the same type, shape, and size, even if the respective inspection areas P21 are set in different positions, the inspection areas P21 that can be considered to be identical can be considered to be arranged symmetrically. In such a case, a single trained model 13 may be applied to the multiple inspection areas P21, P21 that can be considered to be identical. In other words, if multiple inspection areas P2 have the same or substantially the same characteristics, the multiple inspection areas P2 may be inspected using a single trained model 13.

[0150] (3) Furthermore, in the above-described embodiments, the inspection area P2 of the object to be inspected 80 may include at least one of the area in the inspection image P1 where the electrodes 82, 83 of the object to be inspected 80 are present and the area surrounding the electrodes 82, 83, or may include both.

[0151] (4) In the above-described embodiment, the trained model 13 outputs the reliability and heat maps M1 and M2 regarding the presence or absence of a defect F, but this is not limited to this. The trained model 13 only needs to output a determination result regarding the presence or absence of a defect F, and the specific output content is not limited.

[0152] (5) In the above-described embodiment, the notification control unit 14 is configured to display the determination result, "good" or "fail," and the heat maps M1 and M2 on the display unit 17. However, this is not necessarily the case. The notification control unit 14 may be configured to control the notification of the presence or absence of defects F in the inspection objects 80 and 90. For example, the notification control unit 14 may output the feature amount calculated by the trained model 13 to determine the inspection area P2, or may be configured to notify the determination result by voice. The specific notification configuration is not limited.

[0153] (6) In the above-described embodiment, the burrs F1, the melted portions F2, and the missing portions F3 are described as examples of the defects F, but this is not necessarily the case. The defects F may be other defects that are difficult to detect based on rules but occur in a fixed location.

[0154] (7) In the above-described embodiments, the inspection objects 80 and 90 are electronic components, but this is not necessarily the case. The inspection objects 80 and 90 may be small items, other than electronic components, measuring a few millimeters in size, or large items.

[0155] [Computer configuration] An example of a program used in the inspection device 1 in this embodiment is a program that causes a computer to execute steps S1 to S5, S11 to S14, and S21 to S24 shown in Fig. 21. By installing and executing this program in a computer, the processor of the computer functions as a preprocessing unit 11, an inspection area setting unit 12, a trained model 13, a notification control unit 14, a setting unit 15, and a display control unit 16, and performs processing.

[0156] An example of a computer that realizes the inspection device 1 by executing a program in this embodiment will now be described with reference to Fig. 21. Fig. 21 is a block diagram showing an example of a computer that realizes the inspection device 1.

[0157] 21, a computer 110 includes a CPU (Central Processing Unit) 111, a main memory 112, a storage device 113, an input interface 114, a display controller 115, a data reader / writer 116, and a communication interface 117. These components are connected to each other via a bus 121 so as to be able to communicate data with each other.

[0158] Furthermore, the computer 110 may include a GPU (Graphics Processing Unit) or an FPGA (Field-Programmable Gate Array) in addition to or instead of the CPU 111. In this aspect, the GPU or FPGA can execute the programs in the embodiments.

[0159] The CPU 111 loads a program in the embodiment, which is composed of a group of codes and stored in the storage device 113, into the main memory 112 and executes each code in a predetermined order to perform various calculations. The main memory 112 is typically a volatile storage device such as a DRAM (Dynamic Random Access Memory).

[0160] The program in the embodiment is provided in a state stored in a computer-readable recording medium 120. The program in the embodiment may be distributed over the Internet connected via the communication interface 117.

[0161] Specific examples of the storage device 113 include a hard disk drive and a semiconductor storage device such as a flash memory. The input interface 114 mediates data transmission between the CPU 111 and input devices 118 such as a keyboard and a mouse. The display controller 115 is connected to a display device 119 and controls the display on the display device 119.

[0162] Data reader / writer 116 mediates data transmission between CPU 111 and recording medium 120, reads programs from recording medium 120, and writes processing results from computer 110 to recording medium 120. Communication interface 117 mediates data transmission between CPU 111 and other computers.

[0163] Specific examples of the recording medium 120 include general-purpose semiconductor storage devices such as CF (Compact Flash (registered trademark)) and SD (Secure Digital), magnetic recording media such as flexible disks, or optical recording media such as CD-ROMs (Compact Disk Read Only Memory).

[0164] The inspection device 1 in this embodiment can be realized by using hardware corresponding to each unit, such as an electronic circuit, instead of a computer on which a program is installed. In this embodiment, the computer is not limited to the computer shown in FIG. [Industrial Applicability]

[0165] The present invention can be applied as an inspection device. [Explanation of symbols]

[0166] 1. Inspection equipment 12 Inspection area setting section 13 Pre-trained model 14 Notification control section 16 Display control unit 17 Display section (display device) 80 Inspection object 82, 83 Electrodes (certain components of electronic components) 90 Inspection object 92 Electrodes (specific components of electronic components) F Bad P1 inspection image P2, P21, P22, P23 Inspection area P25 small area P3 reference area

Claims

1. An inspection device that determines the presence or absence of a specific defect in an inspection object based on an inspection image obtained by photographing the inspection object, An inspection area setting unit and a trained model obtained by machine learning, the inspection area setting unit sets a specific portion of the inspection image as an inspection area, and uniformly sets the range of the inspection area for the plurality of inspection objects; the trained model reads the inspection area for each of the plurality of inspection objects and outputs a discrimination result for discriminating the presence or absence of the specific defect in the inspection area; the inspection area setting unit sets the inspection area by selecting whether the inspection area is a single area or a combination of a plurality of small areas; The trained model is a model trained for the inspection area using, as training data, an image of the inspection object linked to data specifying the absence of the specific defect and an image of the inspection object linked to data specifying the presence of the specific defect, The trained model reads the inspection area, and if the inspection area is the single area, outputs a discrimination result for determining the presence or absence of the defect for the single area, and if the inspection area is an area combining multiple small areas, outputs a discrimination result for determining the presence or absence of the defect for each small area.

2. the inspection area setting unit sets the inspection area at a plurality of different locations; The inspection device according to claim 1, wherein the inspection area setting unit configures the inspection area by combining a plurality of the small areas for each of at least two of the inspection areas, and makes the configuration conditions of the small areas different between one of the inspection areas and the other of the inspection areas.

3. The inspection device according to claim 1 , wherein the single area or the multiple small areas of the inspection area in the training data and the single area or the multiple small areas of the inspection area set by the inspection area setting unit have the same shape and size.

4. The inspection device according to claim 1 , wherein the inspection area setting section sets a plurality of small areas such that adjacent small areas partially overlap each other.

5. The inspection device according to claim 1 , wherein the inspection area setting unit determines a reference area in the inspection image that serves as a reference for setting the inspection area, and sets the inspection area based on the reference area.

6. There are a plurality of types of defects in the inspection object, The inspection device according to claim 1 , wherein the trained model is provided for each type of defect.

7. the inspection area setting unit sets the inspection area at a plurality of different locations; The inspection device according to claim 1 , wherein the inspection areas are spaced apart from one another or partially overlap each other.

8. The inspection device according to claim 7 , wherein the trained model is provided for each of the plurality of inspection areas.

9. The inspection device according to claim 1 , further comprising a notification control unit that notifies the presence or absence of the defect based on the determination result.

10. The inspection device according to claim 9 , wherein the notification control unit notifies the defect when detecting the presence of the defect in at least one of the small regions in the trained model.

11. the inspection object includes an electronic component, The inspection device according to claim 1 , wherein the inspection area includes at least one of an area in the inspection image where a specific component of the electronic component is present and an area surrounding the component.

12. a display control unit that controls the display performed by the display device; the display control unit causes the display device to display setting content for setting the specific part of the inspection image as the inspection area; The inspection device according to any one of claims 1 to 11, wherein the display control unit causes the display device to display setting content for setting whether or not to use the trained model to determine whether or not the inspection object in the inspection area is defective.

13. a display control unit that controls the display performed by the display device; the display control unit causes the display device to display a setting result indicating that the specific portion of the inspection image has been set as the inspection area; The inspection device according to any one of claims 1 to 11, wherein the display control unit causes the display device to display a discrimination result obtained by using the trained model to determine whether or not the object to be inspected in the inspection area is defective.

Images