Inspection device

Abstract

Machine learning is performed using appropriate teacher data when the appearance of a substrate is inspected using an artificial intelligence technology. The inspection device is provided with: an input unit for inputting a maximum allowable value of a first offset amount indicating an offset amount in a plane direction between a substrate and a component mounted on the substrate, and a maximum allowable value of a second offset amount indicating an offset amount in a rotation direction between the substrate and the component; a correction unit that corrects one of the maximum permissible values of the first and second offset amounts input to the input unit, on the basis of the other; a teacher image creation unit that creates a plurality of teacher images having different first and second offset amounts on the basis of the maximum allowable value of the offset amount corrected by the correction unit; a label imparting unit that imparts different labels to the teacher image in which the combination of the first and second offsets satisfies the first condition and the teacher image in which the combination of the first and second offsets does not satisfy the first condition; a machine learning unit that generates, on the basis of the labeled teacher image, a learning model in which the image of the substrate on which the component is mounted is input and whether the mounting state of the component is qualified is output; a target image acquisition unit that acquires a target image, which is captured by the camera and represents an image of the substrate on which the component is mounted; a determination unit that inputs the target image to the learning model and determines whether or not the mounting state of the component is qualified; and an output unit that outputs a determination result of the determination unit.

Description

Technical Field

[0001] The technology disclosed in this specification relates to an inspection device. Background Art

[0002] As disclosed in Patent Document 1, the appearance of a substrate on which components are mounted is inspected by an appearance inspection device (AOI: Automated Optical Inspection).

[0003] Patent Document 1: Japanese Patent Application Publication No. 2021-150495

[0004] Consider generating a learning model through teacher learning when inspecting the appearance of a substrate using artificial intelligence (AI) technology. In teacher learning, machine learning is implemented using teacher data that is assigned an OK label and teacher data that is assigned an NG label. As teacher data, consider using image data of substrates actually produced in a production line. An OK label is assigned to the image data of a normal substrate, and an NG label is assigned to the image data of an abnormal substrate. However, abnormal substrates are rarely produced in actual production lines. Therefore, it is difficult to collect appropriate teacher data from actually produced substrates. Summary of the invention

[0005] The purpose of the technology disclosed in this specification is to perform machine learning using appropriate teacher data when inspecting the appearance of a substrate using artificial intelligence technology.

[0006] The inspection device disclosed in the present specification comprises: an input unit to which is input a maximum allowable value of a first offset representing an offset in a planar direction between a substrate and a component mounted on the substrate and a maximum allowable value of a second offset representing an offset in a rotational direction between the substrate and the component; a correction unit that corrects the maximum allowable value of the offset of one of the maximum allowable value of the first offset and the maximum allowable value of the second offset input to the input unit; and a teacher image creation unit that creates a plurality of teacher images that are different from each other for the first offset and the second offset based on the maximum allowable value of the offset corrected by the correction unit. image; a label assigning unit, which assigns different labels to the teacher image whose combination of the first offset and the second offset satisfies the first condition and the teacher image which does not satisfy the first condition; a machine learning unit, which generates a learning model based on the teacher image assigned with the label, which takes the image of the substrate with components mounted on it as input and takes the pass or fail of the installation state of the components as output; an object image acquiring unit, which acquires the object image of the substrate with components mounted on it captured by a camera; a determination unit, which inputs the object image into the learning model and determines whether the installation state of the components is pass or fail; and an output unit, which outputs the determination result of the determination unit.

[0007] Effects of the Invention

[0008] According to the technology disclosed in this specification, when inspecting the appearance of a substrate using artificial intelligence technology, machine learning can be performed using appropriate teacher data. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] Figure 1 It is a figure which shows schematically the production system involved in embodiment.

[0010] Figure 2 It is a figure which shows schematically the mounting device involved in embodiment.

[0011] Figure 3 It is a perspective view schematically showing a part of a substrate on which components according to the embodiment are mounted.

[0012] Figure 4 It is a plan view schematically showing a substrate on which components according to the embodiment are mounted.

[0013] Figure 5 It is a diagram schematically showing an inspection device according to an embodiment.

[0014] Figure 6 It is a diagram showing the hardware configuration of the control device involved in the embodiment.

[0015] Figure 7 This is a functional block diagram showing an inspection device according to an embodiment.

[0016] Figure 8 It is a diagram showing an input device and a display device when creating a teacher image involved in an implementation method.

[0017] Fig. 9 It is a diagram showing an input device and a display device when creating a teacher image involved in an implementation method.

[0018] Fig.10 It is a diagram showing an input device and a display device when creating a teacher image involved in an implementation method.

[0019] Fig.11 It is a diagram showing an input device and a display device when creating a teacher image involved in an implementation method.

[0020] Fig.12 It is a diagram for explaining relevant data involved in the implementation mode.

[0021] Fig.13 This is a flowchart showing a method for generating a learning model according to an embodiment.

[0022] Fig.14 This is a diagram for explaining a method for adjusting the assignment of labels according to the embodiment.

[0023] Fig.15 This is a flowchart showing a method for determining whether the mounting state of a component according to the embodiment is acceptable or not.

[0024] Fig.16 This is a diagram for explaining a method for adjusting the assignment of labels according to the embodiment. DETAILED DESCRIPTION

[0025] [Production system]

[0026] Figure 1 Schematically shows a production system 1 according to an embodiment. Figure 1 As shown, the production system 1 includes an inspection device 2, a mounting device 3, and an inspection device 4. The inspection device 2, the mounting device 3, and the inspection device 4 form a production line of electronic devices. In the embodiment, the starting device of the production line is the inspection device 2. The end device of the production line is the inspection device 4.

[0027] The substrate P is conveyed in the production line. Before the substrate P is carried into the inspection device 2 , a solder paste is printed on the substrate P by a printer (not shown). The substrate P printed with the solder paste is carried into the inspection device 2 .

[0028] After being carried into the inspection device 2, the substrate P is inspected in the inspection device 2. The inspection device 2 includes a solder print inspection device (SPI: Solder Paste Inspection) that inspects the printed state of the substrate P before the components C are mounted.

[0029] The substrate P inspected in the inspection device 2 is carried to the mounting device 3. The mounting device 3 mounts the component C on the substrate P printed with the solder paste. The substrate P with the component C mounted is heated in a reflow oven (not shown). The substrate P is heated in the reflow oven, whereby the solder paste melts. The molten solder paste is cooled, whereby the component C is soldered to the substrate P.

[0030] The substrate P unloaded from the reflow furnace is carried into the inspection device 4 and inspected therein. The inspection device 4 includes an appearance inspection device (AOI: Automated Optical Inspection) that inspects the appearance of the substrate P after mounting the components C. The substrate P inspected by the inspection device 4 is unloaded therefrom.

[0031] [Installation device]

[0032] Figure 2 Schematically showing a mounting device 3 according to an embodiment. The mounting device 3 includes a substrate support member 5 that supports a substrate P and a mounting head 6 that mounts a component C on the substrate P supported by the substrate support member 5. The mounting head 6 includes a suction nozzle 7 that releasably holds the component C.

[0033] [Substrate]

[0034] Figure 3 1 is a perspective view schematically showing a portion of a substrate P on which a component C according to an embodiment is mounted. The substrate P has a pad E. The component C has a body B and a lead L protruding from the body B. The body B has a housing made of synthetic resin and an element arranged inside the housing. A coil is shown as an example of the element. The lead L is a metal protrusion protruding from the body B. The lead L is connected to the element arranged inside the body B. The component C is mounted on the substrate P in a manner that the pad E and the lead L are connected via a paste solder. The component C is mounted on the substrate P in a manner that the lead L is mounted on the pad E.

[0035] Figure 4 2 is a top view schematically showing a substrate P on which a component C according to the embodiment is mounted. In the embodiment, a plurality of pads E are provided on the surface of the substrate P. Figure 4 In the example shown, the pads E include pads Ea, pads Eb, and pads Ec. Component C has a plurality of leads L. Component C has the same number of leads L as pads E. Figure 4In the example shown, the leads L include leads La, leads Lb, and leads Lc. The component C is mounted on the substrate P such that the leads La are mounted on the pads Ea, the leads Lb are mounted on the pads Eb, and the leads Lc are mounted on the pads Ec.

[0036] In the following description, the direction parallel to the surface of the substrate P is appropriately referred to as the plane direction, and the direction orthogonal to the surface of the substrate P is appropriately referred to as the vertical direction. In the XY plane parallel to the surface of the substrate P, the direction parallel to the X-axis is appropriately referred to as the X-axis direction, in the XY plane orthogonal to the X-axis, the direction parallel to the Y-axis is appropriately referred to as the Y-axis direction, and the rotation direction centered on the vertical axis orthogonal to the surface of the substrate P is appropriately referred to as the θ direction. The plane direction includes one or both of the X-axis direction and the Y-axis direction. The rotation direction is the θ direction.

[0037] The mounting device 3 mounts the component C on the substrate P in such a manner that the lead L is mounted on the pad E at a target relative position. In the embodiment, the case where the lead L is mounted on the pad E at a target relative position includes mounting the lead L on the pad E in such a manner that the center position of the pad E coincides with the center position of the lead L in the planar direction and the orientation of the pad E coincides with the orientation of the lead L in the rotational direction. Figure 4 The state where the lead L is mounted on the pad E at the target relative position is shown. In the embodiment, the state where the lead L is mounted on the pad E at the target relative position includes mounting the lead L on the pad E in such a manner that the center position of each of the plurality of pads E (Ea, Eb, Ec) coincides with the center position of each of the plurality of leads L (La, Lb, Lc) in the planar direction, and the orientation of each of the plurality of pads E (Ea, Eb, Ec) coincides with the orientation of each of the plurality of leads L (La, Lb, Lc) in the rotational direction. In the state where the lead L is mounted on the pad E at the target relative position, the lead L does not extend from the pad E.

[0038] [Inspection device]

[0039] Figure 5 : is a diagram schematically showing an inspection device 4 according to an embodiment. The inspection device 4 is an appearance inspection device for inspecting the appearance of a substrate P after a component C is mounted thereon. The inspection device 4 includes: a substrate support member 8 for supporting the substrate P after the component C is mounted thereon; a camera 9 for photographing the substrate P supported by the substrate support member 8; a control device 10; an input device 11; and a display device 12.

[0040] Figure 61 is a hardware structure diagram of the control device 10 involved in the embodiment. The control device 10 includes a computer system 1000. The control device 10 has a processor 1001 such as a CPU (Central Processing Unit), a main memory 1002 including a non-volatile memory such as a ROM (Read Only Memory) and a volatile memory such as a RAM (Random Access Memory), a storage 1003, and an interface 1004 including an input and output circuit. The functions of the control device 10 are stored in the storage 1003 as a computer program. The processor 1001 reads the computer program from the storage 1003 and expands it in the main memory 1002, and executes a prescribed process according to the computer program. In addition, the computer program can be transmitted to the control device 10 via a network.

[0041] Figure 7 1 is a functional block diagram of the inspection device 4 according to the embodiment. The input device 11 and the display device 12 are each connected to the control device 10. The input device 11 generates input data by being operated by the operator. As the input device 11, a touch sensor, a computer keyboard or a mouse is exemplified. The display device 12 displays the display data. As the display device 12, a flat panel display such as a liquid crystal display or an organic EL display is exemplified. The display device 12 may also be a touch screen including a touch sensor.

[0042] The control device 10 includes an input unit 13 , a correction unit 14 , a teacher image creation unit 15 , a label assignment unit 16 , a machine learning unit 17 , a target image acquisition unit 18 , a determination unit 19 , and an output unit 20 .

[0043] The input unit 13 obtains input data from the input device 11. The input data from the input device 11 is input to the input unit 13. In the embodiment, the operator operates the input device 11 and inputs the maximum allowable value of the offset between the substrate P and the component C mounted on the substrate P to the input unit 13. The offset between the substrate P and the component C includes an offset in the plane direction and an offset in the rotation direction. The offset in the plane direction includes an offset in the X-axis direction and an offset in the Y-axis direction. The offset in the rotation direction is an offset in the θ direction. In the following description, the offset in the X-axis direction is appropriately referred to as the X offset, the offset in the Y-axis direction is appropriately referred to as the Y offset, and the offset in the θ direction is appropriately referred to as the θ offset.

[0044] The input unit 13 inputs the respective maximum allowable values ​​of the X offset (first offset) and the Y offset (first offset) representing the offset in the planar direction between the substrate P and the component C, and the maximum allowable value of the θ offset (second offset) representing the offset in the rotational direction between the substrate P and the component C.

[0045] As reference Figure 3 and Figure 4 As described above, pads E are provided on the surface of substrate P. Component C has leads L. The offset between substrate P and component C includes the offset between pads E and leads L. The X offset between substrate P and component C is the X offset between pads E and leads L. The Y offset between substrate P and component C is the Y offset between pads E and leads L. The θ offset between substrate P and component C is the θ offset between pads E and leads L.

[0046] When the lead L is mounted on the pad E at the target relative position, the lead L is arranged at the reference position (target position) relative to the pad E in the plane direction, and at the reference angle (target angle) relative to the pad E in the rotation direction. The offset in the plane direction is the offset of the lead L from the reference position in the plane direction. The X offset is the offset of the lead L from the reference position in the X-axis direction. The Y offset is the offset of the lead L from the reference position in the Y-axis direction. The θ offset is the offset angle of the lead L from the reference angle in the θ direction.

[0047] The correction unit 14 corrects the maximum allowable value of the offset amount input to the input unit 13 based on the maximum allowable value of the offset amount in the plane direction and the maximum allowable value of the offset amount in the rotation direction. The correction unit 14 corrects the maximum allowable value of the offset amount input to the input unit 13 based on the correlation data indicating the relationship between the maximum allowable value of the offset amount in the plane direction and the maximum allowable value of the offset amount in the rotation direction. The correlation data is determined in advance and stored in the correction unit 14. The correction unit 14 corrects the maximum allowable value of the offset amount based on the correlation data in such a manner that the larger the maximum allowable value of the offset amount in the plane direction, the smaller the maximum allowable value of the offset amount in the rotation direction, and the larger the maximum allowable value of the offset amount in the rotation direction, the smaller the maximum allowable value of the offset amount in the plane direction.

[0048] The teacher image creation unit 15 creates a plurality of teacher images having different amounts of deviation in the planar direction and the rotational direction based on the maximum permissible value of the deviation corrected by the correction unit 14. In the embodiment, the teacher image is an image created by the computer system 1000.

[0049] The label assigning unit 16 assigns different labels to the teacher image whose combination of the offset in the planar direction and the offset in the rotational direction satisfies the first condition and the teacher image that does not satisfy the first condition. The label assigning unit 16 assigns an OK label (qualified label) to the teacher image that satisfies the first condition and an NG label (unqualified label) to the teacher image that does not satisfy the first condition.

[0050] The machine learning unit 17 generates a learning model that takes an image of the substrate P with the component C mounted thereon as input and outputs whether the mounting state of the component C on the substrate P is passable or not.

[0051] The object image acquisition unit 18 acquires an object image captured by the camera 9, which represents an image of the substrate P on which the component C is mounted. The component C is mounted on the surface of the substrate P. Figure 5 As shown, in the inspection device 4, the substrate P after the component C is mounted is supported by the substrate support member 8. The substrate support member 8 supports the substrate P in a manner that the surface of the substrate P faces upward. The camera 9 is arranged above the substrate support member 8 in a manner opposite to the surface of the substrate P. The camera 9 photographs the substrate P after the component C is mounted from above the substrate P. The camera 9 photographs the pad E and the lead L. The object image is an image photographed by the camera 9. The object image acquisition unit 18 acquires the object image including the pad E and the lead L.

[0052] The determination unit 19 inputs the object image acquired by the object image acquisition unit 18 into the learning model, and determines whether the mounting state of the component C relative to the substrate P is acceptable. The acceptance of the mounting state of the component C includes the acceptance of the relative position of the pad E and the lead L. The acceptance of the relative position of the pad E and the lead L includes the acceptance of the relative position of the pad E and the lead L in the planar direction and the acceptance of the relative angle of the pad E and the lead L in the rotation direction.

[0053] The output unit 20 outputs the determination result of the determination unit 19. In the embodiment, the output unit 20 outputs the determination result of the determination unit 19 to the display device 12. The display device 12 displays the determination result of the determination unit 19.

[0054] [Creation of teacher image]

[0055] Next, the method of creating a teacher image is described. Figure 8 , Fig. 9 , Fig.10 and Fig.11 Each of them is a diagram showing the input device 11 and the display device 12 when creating a teacher image involved in the embodiment.

[0056] When creating the teacher image 21, the operator operates the input device 11 to input the maximum allowable value of the offset into the input unit 13. The maximum allowable value of the offset may be a value determined according to a standard or may be a value determined independently by the electronic device manufacturer.

[0057] The maximum allowable value is determined for each of the X offset, the Y offset, and the θ offset. The operator operates the input device 11 to input the maximum allowable value of the X offset, the maximum allowable value of the Y offset, and the maximum allowable value of the θ offset into the input unit 13. As an example, in the embodiment, the maximum allowable value of the X offset is "0.26 mm", the maximum allowable value of the Y offset is "0.26 mm", and the maximum allowable value of the θ offset is "6.0 degrees".

[0058] When the offset between the pad E and the lead L is less than or equal to the maximum allowable value, the produced substrate P is judged to be normal, and the produced electronic device is judged to be a qualified product. When the offset between the pad E and the lead L exceeds the maximum allowable value, the produced substrate P is judged to be abnormal, and the produced electronic device is judged to be a defective product.

[0059] If the maximum allowable value of the X offset, the maximum allowable value of the Y offset, and the maximum allowable value of the θ offset are input to the input unit 13, the teacher image creation unit 15 creates a plurality of teacher images 21 based on the maximum allowable value of the X offset, the maximum allowable value of the Y offset, and the maximum allowable value of the θ offset input to the input unit 13. The teacher image creation unit 15 creates a plurality of teacher images to be assigned an OK label and a plurality of teacher images to be assigned an NG label, using the maximum allowable value of the X offset, the maximum allowable value of the Y offset, and the maximum allowable value of the θ offset as a reference. In the following description, the state of the normal substrate P to which the OK label should be assigned to the teacher image 21 is appropriately referred to as the OK state (normal state), and the state of the abnormal substrate P to which the NG label should be assigned to the teacher image 21 is appropriately referred to as the NG state (abnormal state). In addition, the teacher image 21 to which the OK label should be assigned is appropriately referred to as the OK image, and the teacher image 21 to which the NG label should be assigned is appropriately referred to as the NG image.

[0060] The OK state of the substrate P means that the relative position between the pad E and the lead L is normal, and the produced electronic device is judged to be a qualified product. The NG state of the substrate P means that the relative position between the pad E and the lead L is abnormal, and the produced electronic device is judged to be unqualified.

[0061] For example, if the maximum allowable value of the X offset, i.e., "0.26 mm", is input to the teacher image creation unit 15, the maximum allowable value of the X offset, i.e., "0.26 mm", is used as a boundary to create multiple teacher images with an X offset less than or equal to "0.26 mm" and multiple teacher images with an X offset exceeding "0.26 mm". If the maximum allowable value of the Y offset, i.e., "0.26 mm", is input to the teacher image creation unit 15, the maximum allowable value of the Y offset, i.e., "0.26 mm", is used as a boundary to create multiple teacher images with a Y offset less than or equal to "0.26 mm" and multiple teacher images with a Y offset exceeding "0.26 mm". If the maximum allowable value of the θ offset, i.e., "6.0 deg", is input to the teacher image creation unit 15, the maximum allowable value of the θ offset, i.e., "6.0 deg", is used as a boundary to create multiple teacher images with a θ offset less than or equal to "6.0 deg" and multiple teacher images with a θ offset exceeding "6.0 deg".

[0062] That is, in the embodiment, the teacher image creation unit 15 creates many OK images that should be given an OK label and many NG images that should be given an NG label, using the maximum allowable value of the offset in the plane direction and the maximum allowable value of the offset in the rotation direction input to the input unit 13 as references. The teacher image creation unit 15 automatically creates many OK images and many NG images while slightly offsetting the position of the lead L relative to the pad E.

[0063] Figure 8 An example is shown in which the teacher image 21 with an X offset of "0.00 mm", a Y offset of "0.00 mm", and a θ offset of "0.0 deg" is displayed on the display device 12. Since the X offset, the Y offset, and the θ offset are each zero, an OK image is created in which the lead L is mounted on the pad E at the target relative position and displayed on the display device 12.

[0064] Fig. 9 An example is shown in which the teacher image 21 with an X offset of "0.02 mm", a Y offset of "0.26 mm", and a θ offset of "0.0 deg" is displayed on the display device 12. The X offset is less than or equal to the maximum allowable value, the Y offset is less than or equal to the maximum allowable value, and the θ offset is less than or equal to the maximum allowable value, so that although the pad E and the lead L are slightly offset, an OK image is created and displayed on the display device 12, in which the lead L is normally mounted on the pad E.

[0065] Fig.10An example is shown in which the teacher image 21 with an X offset of "0.02 mm", a Y offset of "0.06 mm", and a θ offset of "6.0 degrees" is displayed on the display device 12. The X offset is less than or equal to the maximum allowable value, the Y offset is less than or equal to the maximum allowable value, and the θ offset is less than or equal to the maximum allowable value, so that although the pad E and the lead L are slightly offset, an OK image is created and displayed on the display device 12, in which the lead L is normally mounted on the pad E.

[0066] Fig.11 An example is shown in which the teacher image 21 with an X offset of "0.02 mm", a Y offset of "0.26 mm", and a θ offset of "6.0 deg" is displayed on the display device 12. Although the X offset is less than or equal to the maximum allowable value, the Y offset is less than or equal to the maximum allowable value, and the θ offset is less than or equal to the maximum allowable value, the pad E and the lead L are greatly deviated. Although the X offset, the Y offset, and the θ offset are each less than or equal to the maximum allowable value, it is inappropriate to set the teacher image 21 in which the pad E and the lead L are greatly deviated as an OK image. Although the X offset, the Y offset, and the θ offset are each less than or equal to the maximum allowable value, it is sometimes appropriate to set the created teacher image 21 as an NG image based on the combination of the X offset, the Y offset, and the θ offset.

[0067] That is, when the maximum allowable value of the X offset, the maximum allowable value of the Y offset, and the maximum allowable value of the θ offset are determined independently, even if the X offset, the Y offset, and the θ offset are each less than or equal to the maximum allowable value, it may be appropriate to set the created teacher image 21 as an NG image depending on the combination of the X offset, the Y offset, and the θ offset.

[0068] Therefore, in the embodiment, when the OK image and the NG image are created based on the maximum allowable value of the offset, the correction unit 14 corrects the maximum allowable value of the offset input to the input unit 13 when the offset in the plane direction is less than or equal to the maximum allowable value of the offset in the plane direction and the offset in the rotation direction is less than or equal to the maximum allowable value of the offset in the rotation direction so as to create a teacher image, that is, an OK image, to which a pass label should be assigned. That is, the correction unit 14 corrects the maximum allowable value of the offset in the plane direction and the maximum allowable value of the offset in the rotation direction based on the maximum allowable value of the offset in one of the maximum allowable values ​​of the offset in the plane direction and the maximum allowable value of the offset in the rotation direction input to the input unit 13 so that as long as the offset is less than or equal to the maximum allowable value, an OK image is generated regardless of any combination of the X offset, the Y offset, and the θ offset.

[0069] The correction unit 14 corrects the maximum allowable value of the offset input to the input unit 13 so that the larger the maximum allowable value of the offset in the plane direction, the smaller the maximum allowable value of the offset in the rotation direction, and the larger the maximum allowable value of the offset in the rotation direction, the smaller the maximum allowable value of the offset in the plane direction. The correction unit 14 corrects the maximum allowable value of the offset input to the input unit 13 based on predetermined correlation data.

[0070] Fig.12 It is a diagram used to explain the relevant data involved in the implementation method. Fig.12 As shown in FIG. 1 , the correlation data indicates the relationship between the maximum allowable value of the X offset or the Y offset and the maximum allowable value of the θ offset. With respect to the correlation data, the maximum allowable value of the offset input to the input unit 13 is corrected so that the larger the maximum allowable value of the X offset or the Y offset, the smaller the maximum allowable value of the offset in the θ direction becomes, and the larger the maximum allowable value of the θ offset, the smaller the maximum allowable value of the X offset or the Y offset becomes. Fig.12 In the example shown, the correlation data is determined so that the maximum allowable value of the X offset or the Y offset and the maximum allowable value of the θ offset are in a proportional relationship.

[0071] For example, when the operator inputs the Y offset as "0.26mm", the correction unit 14 corrects the θ offset input to the input unit 13 to "0.0deg" even if the θ offset is input as "6.0deg". When the operator inputs the Y offset as "0.13mm", the correction unit 14 corrects the θ offset input to the input unit 13 to "3.0deg" even if the θ offset is input as "6.0deg". When the operator inputs the Y offset as "0.13mm", if the θ offset is input as "2.0deg", for example, the correction by the correction unit 14 is not performed, and the θ offset input to the input unit 13, i.e., "2.0deg", is adopted as the reference for the creation of the teacher image 21. When the operator inputs "0.0 mm" for the Y offset, even if "6.0 degrees" is input for the θ offset, the correction by the correction unit 14 is not performed, and the θ offset "6.0 degrees" input to the input unit 13 is used as the basis for creating the teacher image 21. The same applies to the case where the operator inputs the X offset.

[0072] For example, when the operator inputs the θ offset as "6.0 deg", the correction unit 14 corrects the Y offset input to the input unit 13 to "0.00 mm" even if the Y offset is input as "0.26 mm". When the operator inputs the θ offset as "3.0 deg", the correction unit 14 corrects the Y offset input to the input unit 13 to "0.13 mm" even if the Y offset is input as "0.26 mm". When the operator inputs the θ offset as "3.0 deg", if the Y offset is input as "0.05 mm", for example, the correction by the correction unit 14 is not performed, and the Y offset input to the input unit 13, i.e., "0.05 mm", is adopted as the reference for creating the teacher image 21. When the operator inputs “0.0deg” as the θ offset, even if “0.26mm” is input as the Y offset, the correction by the correction unit 14 is not performed, and the Y offset “0.26mm” input to the input unit 13 is adopted as the basis for creating the teacher image 21 .

[0073] The teacher image creation unit 15 creates a plurality of teacher images 21 to be assigned an OK label and a plurality of teacher images 21 to be assigned an NG label, using the maximum allowable values ​​of the X offset and the Y offset and the maximum allowable value of the θ offset corrected by the correction unit 14 as references. Fig.12 In the example shown, a straight line SH indicating the relationship between the maximum allowable value of the X offset or the Y offset and the maximum allowable value of the θ offset is defined. The teacher image creation unit 15 creates a plurality of OK images that satisfy the first condition that the combination of the X offset, the Y offset, and the θ offset is smaller than the straight line SH. The teacher image creation unit 15 creates a plurality of OK images whose X offset, the Y offset, and the θ offset are different from each other. In addition, the teacher image creation unit 15 creates a plurality of NG images that satisfy the second condition that the combination of the X offset, the Y offset, and the θ offset is larger than the straight line SH. The teacher image creation unit 15 creates a plurality of NG images whose X offset, the Y offset, and the θ offset are different from each other.

[0074] [Generation of learning model]

[0075] Fig.13 1 is a flowchart showing a method for generating a learning model according to an embodiment. The operator operates the input device 11 and inputs the maximum allowable values ​​of the X offset, the Y offset, and the θ offset to the input unit 13. The input unit 13 obtains the input values ​​of the maximum allowable values ​​of the X offset, the Y offset, and the θ offset from the input device 11 (step SA1).

[0076] The correction unit 14 corrects the input values ​​of the maximum allowable values ​​of the X offset, the Y offset, and the θ offset input to the input unit 13 based on the correlation data (step SA2 ).

[0077] The teacher image creation unit 15 creates a plurality of teacher images 21 having different X offsets, Y offsets, and θ offsets based on the maximum allowable values ​​of the X offsets, Y offsets, and θ offsets corrected by the correction unit 14 (step SA3 ).

[0078] The label assigning unit 16 assigns an OK label to the teacher image 21 whose combination of X offset, Y offset, and θ offset satisfies the first condition, and assigns an NG label to the teacher image 21 whose combination of X offset, Y offset, and θ offset does not satisfy the first condition but satisfies the second condition (step SA4).

[0079] The machine learning unit 17 generates a learning model that takes the image of the substrate P with the component C mounted thereon as input and outputs whether the mounting state of the component C is acceptable based on the teacher image 21 to which the OK label or NG label is assigned (step SA5 ).

[0080] [Adjustment of label assignment]

[0081] Fig.14 1 is a diagram for explaining a method for adjusting the label assignment according to the embodiment. In the embodiment, when there is a teacher image 21 in which the combination of the X offset or the Y offset and the θ offset corrected by the correction unit 14 is consistent with the straight line SH, as shown in FIG. Fig.14 As shown, the output unit 20 displays the teacher image 21 whose Y offset or the combination of the Y offset and the θ offset is consistent with the straight line SH and a list of labels attached to the teacher image 21 on the display device 12. In addition, the output unit 20 displays the teacher image 21 selected by the operator from the list of teacher images 21 on the display device 12. The operator can select the teacher image 21 by operating the input device 11.

[0082] The teacher image 21 whose combination of X offset or Y offset and θ offset is consistent with the straight line SH can be regarded as the teacher image 21 existing at the boundary between the OK image and the NG image. Therefore, although the label assigning unit 16 assigns the OK label to the teacher image 21 whose combination of X offset or Y offset and θ offset is consistent with the straight line SH, it is sometimes appropriate to set it as an NG image. In addition, although the label assigning unit 16 assigns the NG label to the teacher image 21 whose combination of X offset or Y offset and θ offset is consistent with the straight line SH, it is sometimes appropriate to set it as an OK image. Regarding the teacher image 21 existing at the boundary between the OK image and the NG image, the operator may judge whether it is an OK image or an NG image. The teacher image 21 existing at the boundary between the OK image and the NG image is displayed on the display device 12, so that the operator can confirm the teacher image 21 displayed on the display device 12 and judge whether it is an OK image or an NG image. The operator can operate the input device 11 based on the judgment result of the operator to change the label assigned by the label assigning unit 16. The operator can operate the input device 11 based on the determination result of the operator to change the OK label assigned by the label assigning unit 16 to an NG label, or change the NG label assigned by the label assigning unit 16 to an OK label.

[0083] [Quality or Failure Judgment of Component Installation Status]

[0084] Fig.15 1 is a flowchart showing a method for determining whether the mounting state of a component C according to an embodiment is qualified or not. The substrate P on which the component C is mounted in the mounting device 3 is transported from the mounting device 3 to the inspection device 4. The substrate P on which the component C is mounted is supported by the substrate support member 8. In a state where the substrate P is supported by the substrate support member 8, the substrate P is photographed by the camera 9.

[0085] The object image acquisition unit 18 acquires an object image representing an image of the substrate P captured by the camera 9 (step SB1 ).

[0086] The determination unit 19 inputs the target image acquired by the target image acquisition unit 18 into the learning model (step SB2 ).

[0087] The determination unit 19 determines whether the mounting state of the component C is acceptable based on the output from the learning model (step SB3 ).

[0088] The output unit 20 outputs the determination result of the determination unit 19. The determination result of the determination unit 19 includes a determination result of whether the produced substrate P is normal. In the embodiment, the output unit 20 displays the determination result of the determination unit 19 on the display device 12 (step SB4).

[0089] [Effect]

[0090] As described above, according to the embodiment, when the appearance of the substrate P after the component C is mounted is inspected using artificial intelligence technology, an appropriate teacher image 21 is generated. Depending on the combination of the maximum allowable value of the X offset, the maximum allowable value of the Y offset, and the maximum allowable value of the θ offset input to the input unit 13, the teacher image 21 that should be set as an OK image may become an NG image, or the teacher image 21 that should be set as an NG image may become an OK image. In the embodiment, at least one of the maximum allowable value of the X offset, the maximum allowable value of the Y offset, and the maximum allowable value of the θ offset input to the input unit 13 is corrected, thereby generating an appropriate teacher image 21. Therefore, the pass / fail judgment of the produced substrate P is appropriately implemented.

[0091] [Other embodiments]

[0092] Fig.16 FIG. 2 is a diagram for explaining a method for adjusting the label assignment according to the embodiment. Fig.14 In the example described, the teacher image 21 existing at the boundary between the OK image and the NG image is displayed on the display device 12, and the label assigned by the label assigning unit 16 is changed based on the input data from the input device 11. Fig.16 As shown, for example, the determination unit 19 may change the label assigned by the label assigning unit 16 by performing image processing on the teacher image 21 existing at the boundary between the OK image and the NG image. The determination unit 19 calculates the area of ​​the overlapping range M between the pad E and the lead L. The determination unit 19 calculates the ratio of the area of ​​the overlapping range M to the area of ​​the lead L. When the ratio of the area of ​​the overlapping range M to the area of ​​the lead L is greater than or equal to a predetermined threshold, the determination unit 19 determines that the relative position between the pad E and the lead L is normal. The overlapping range M includes the overlapping range Ma of the pad Ea and the lead La, the overlapping range Mb of the pad Eb and the lead Lb, and the overlapping range Mc of the pad Ec and the lead Lc. When the ratio of all the overlapping ranges Ma, Mb, and Mc is greater than or equal to the threshold, the determination unit 19 determines that the relative position between the pad E and the lead L is normal, and assigns the OK label to the teacher image 21. When the ratio of at least one of the overlapping ranges Ma, Mb, and Mc is smaller than the threshold, the determination unit 19 determines that the relative position between the pad E and the lead L is not normal and gives the teacher image 21 an NG label.

[0093] Description of the label

[0094] 1…production system, 2…inspection device, 3…mounting device, 4…inspection device, 5…substrate supporting member, 6…mounting head, 7…sucking nozzle, 8…substrate supporting member, 9…camera, 10…control device, 11…input device, 12…display device, 13…input unit, 14…correction unit, 15…teacher image creating unit, 16…label assigning unit, 17…machine learning unit, 18…object image acquiring unit, 19…judgment unit, 20…output unit, 21…teacher image, 1000…computer system, 1001…processor, 1002…main memory, 1003…storage, 1004…interface, B…main body, C…component, E…pad, Ea…pad, Eb…pad, Ec…pad, L…lead, P…substrate.

Claims

1. An inspection device, comprising: an input unit to which is input a maximum allowable value of a first offset amount indicating an offset amount in a planar direction between a substrate and a component mounted on the substrate and a maximum allowable value of a second offset amount indicating an offset amount in a rotational direction between the substrate and the component; a correction unit that corrects a maximum allowable value of one of the first maximum allowable value and the second maximum allowable value of the offset input to the input unit based on the maximum allowable value of the offset of the other one; a teacher image creating unit that creates a plurality of teacher images having different first and second offset amounts based on a maximum permissible value of the offset amount corrected by the correcting unit; a label assigning unit that assigns different labels to the teacher image for which the combination of the first offset and the second offset satisfies a first condition and to the teacher image for which the combination does not satisfy the first condition; a machine learning unit that generates a learning model that takes the image of the substrate on which the component is mounted as input and outputs whether the mounting state of the component is acceptable based on the teacher image to which the label is assigned; a target image acquisition unit that acquires a target image captured by a camera and representing an image of the substrate on which the component is mounted; a determination unit that inputs the object image into the learning model and determines whether the installation state of the component is acceptable or not; as well as An output unit outputs the determination result of the determination unit.

2. The inspection device according to claim 1, wherein: A pad is provided on the surface of the substrate, The component has a lead placed on the pad, The offset includes an offset between the pad and the lead.

3. The inspection device according to claim 1, wherein: The correction unit corrects the maximum allowable value of the offset based on predetermined correlation data indicating a relationship between the maximum allowable value of the first offset and the maximum allowable value of the second offset.

4. The inspection device according to claim 1, wherein: The correction unit corrects the maximum allowable value of the offset so that the larger the maximum allowable value of the first offset is, the smaller the maximum allowable value of the second offset is, and the larger the maximum allowable value of the second offset is, the smaller the maximum allowable value of the first offset is.

5. The inspection device according to claim 1, wherein: The teacher image creation unit creates a plurality of teacher images to be assigned a pass label and a plurality of teacher images to be assigned a fail label, using the maximum allowable value of the first offset amount and the maximum allowable value of the second offset amount corrected by the correction unit as references.

6. The inspection device according to claim 5, wherein: When the first offset is less than or equal to the maximum allowable value of the first offset and the second offset is less than or equal to the maximum allowable value of the second offset, the correction unit corrects the maximum allowable value of the offset so as to create a teacher image that should be assigned the qualified label.

Images