Inspection device

Abstract

To perform machine learning by using appropriate teacher data in an inspection of an external appearance of a substrate by using an artificial intelligence.SOLUTION: An inspection device comprises: an input part to which a maximum permission value of a first deviation amount between a substrate and a mounting component in a plan surface direction and a maximum permission value of a second deviation amount between the substrate and the component in a rotational direction; a teacher image creation part that creates a teaching image of which the first deviation amount and the second deviation amount are different on the basis of the maximum permission value of one of the maximum permission values of the first and second deviation amounts, and on the basis of the maximum permission value of the other deviation amount corrected by a correction part; a label application part that applies a label to the teacher image in which a combination of the first deviation amount and the second deviation amount satisfies a first condition and the teacher image in which the combination does not satisfy it; a machine learning part that generates a learning model that outputs a quality of a mounting state on the basis of the teacher image to which the label is applied; an objective image acquisition part that acquires an image of a symmetrical substrate imaged; and an output part that inputs the objective image into the learning model, and outputs a determination result of the quality of the mounting state.SELECTED DRAWING: Figure 12

Description

[Technical field]

[0001] The technology disclosed in this specification relates to an inspection apparatus. [Background technology]

[0002] As disclosed in Patent Document 1, the appearance of the board after components are mounted is inspected by an appearance inspection device (AOI: Automated Optical Inspection). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2021-150495 A Summary of the Invention [Problem to be solved by the invention]

[0004] When inspecting the appearance of a circuit board using artificial intelligence (AI) technology, it is possible to generate a learning model through supervised learning. In supervised learning, machine learning is carried out using training data that has been given an OK label and training data that has been given an NG label. It is possible to use image data of circuit boards that have actually been produced on a production line as training data. An OK label is given to image data of normal circuit boards, and an NG label is given to image data of abnormal circuit boards. However, it is rare for abnormal circuit boards to be produced on an actual production line. Therefore, it is difficult to collect appropriate training data from circuit boards that have actually been produced.

[0005] The technology disclosed in this specification aims to perform machine learning using appropriate training data when inspecting the appearance of a substrate using artificial intelligence technology. [Means for solving the problem]

[0006] The inspection device disclosed in this specification includes an input unit to which a maximum allowable value of a first amount of misalignment indicating a misalignment amount in a planar direction between a board and a component mounted on the board and a maximum allowable value of a second amount of misalignment indicating a misalignment amount in a rotational direction between the board and the component are input, a correction unit that corrects the maximum allowable value of one of the first and second amounts of misalignment based on the maximum allowable value of the other amount of misalignment input to the input unit, and a teacher image creation unit that creates a plurality of teacher images each having a different first amount of misalignment and a different second amount of misalignment based on the maximum allowable value of the amount of misalignment corrected by the correction unit. The system includes a label assignment unit that assigns different labels to teacher images whose combination of the deviation amount and the second deviation amount satisfies a first condition and teacher images whose combination does not satisfy the first condition; a machine learning unit that generates a learning model based on the labeled teacher images, which takes as input an image of a board on which components are mounted and outputs a pass / fail state of the component mounting; a target image acquisition unit that acquires a target image showing an image of the board on which components are mounted, captured by a camera; a judgment unit that inputs the target image into the learning model and judges the pass / fail state of the component mounting; and an output unit that outputs the judgment result of the judgment unit. Effect of the Invention

[0007] According to the technology disclosed in this specification, when inspecting the appearance of a substrate using artificial intelligence technology, machine learning can be performed using appropriate training data. [Brief description of the drawings]

[0008] [Figure 1] FIG. 1 is a diagram illustrating a production system according to an embodiment. [Diagram 2] FIG. 2 is a diagram illustrating a mounting apparatus according to an embodiment. [Diagram 3] FIG. 3 is a perspective view that illustrates a part of a substrate on which components according to the embodiment are mounted. [Figure 4] FIG. 4 is a plan view illustrating a board on which components according to the embodiment are mounted. [Diagram 5] FIG. 5 is a diagram illustrating an inspection device according to an embodiment. [Figure 6]FIG. 6 is a hardware configuration diagram showing the control device according to the embodiment. [Figure 7] FIG. 7 is a functional block diagram showing an inspection device according to an embodiment. [Figure 8] FIG. 8 is a diagram showing an input device and a display device when a teacher image according to the embodiment is created. [Figure 9] FIG. 9 is a diagram showing an input device and a display device when a teacher image according to the embodiment is created. [Figure 10] FIG. 10 is a diagram showing an input device and a display device when a teacher image according to the embodiment is created. [Figure 11] FIG. 11 is a diagram showing an input device and a display device when a teacher image according to the embodiment is created. [Figure 12] FIG. 12 is a diagram for explaining correlation data according to the embodiment. [Figure 13] FIG. 13 is a flowchart showing a method for generating a learning model according to an embodiment. [Figure 14] FIG. 14 is a diagram for explaining a method for adjusting label assignment according to the embodiment. [Figure 15] FIG. 15 is a flowchart showing a method for determining whether or not a mounting state of a component is good according to the embodiment. [Figure 16] FIG. 16 is a diagram for explaining a method for adjusting label assignment according to the embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] [Production System] Fig. 1 is a diagram illustrating a schematic diagram of a production system 1 according to an embodiment. As shown in Fig. 1, the production system 1 includes an inspection apparatus 2, a mounting apparatus 3, and an inspection apparatus 4. The inspection apparatus 2, the mounting apparatus 3, and the inspection apparatus 4 form a production line for electronic devices. In the embodiment, the first apparatus in the production line is the inspection apparatus 2. The last apparatus in the production line is the inspection apparatus 4.

[0010] A substrate P is transported along a production line. Before the substrate P is carried into the inspection device 2, cream solder is printed on the substrate P by a printer (not shown). The substrate P on which the cream solder has been printed is then carried into the inspection device 2.

[0011] The substrate P is carried into the inspection device 2 and then inspected by the inspection device 2. The inspection device 2 includes a solder paste inspection (SPI) device that inspects the printing state of the substrate P before the components C are mounted thereon.

[0012] The board P inspected by the inspection device 2 is carried into the mounting device 3. The mounting device 3 mounts components C on the board P on which the cream solder has been printed. The board P on which the components C have been mounted is heated in a reflow furnace (not shown). The cream solder melts as the board P is heated in the reflow furnace. The melted cream solder is cooled, and the components C are soldered to the board P.

[0013] The substrate P removed from the reflow furnace is carried into an inspection device 4 and then inspected by the inspection device 4. The inspection device 4 includes an appearance inspection device (AOI: Automated Optical Inspection) that inspects the appearance of the substrate P after the components C are mounted thereon. The substrate P inspected by the inspection device 4 is then carried out from the inspection device 4.

[0014] [Mounting equipment] 2 is a diagram illustrating a mounting apparatus 3 according to an embodiment. The mounting apparatus 3 has a board support member 5 that supports a board P, and a mounting head 6 that mounts a component C on the board P supported by the board support member 5. The mounting head 6 has a nozzle 7 that releasably holds the component C.

[0015] [substrate] FIG. 3 is a perspective view showing a schematic view of a part of a substrate P on which a component C according to an embodiment is mounted. The substrate P has a land E. The component C has a body B and a lead L protruding from the body B. The body B has a housing made of synthetic resin and an element disposed inside the housing. An example of the element is a coil. The lead L is a metal protrusion protruding from the body B. The lead L is connected to the element disposed inside the body B. The component C is mounted on the substrate P such that the land E and the lead L are connected via cream solder. The component C is mounted on the substrate P such that the lead L is mounted on the land E.

[0016] FIG. 4 is a plan view showing a schematic diagram of a substrate P on which a component C according to an embodiment is mounted. In the embodiment, a plurality of lands E are provided on the surface of the substrate P. In the example shown in FIG. 4, the lands E include lands Ea, Eb, and Ec. The component C has a plurality of leads L. The component C has the same number of leads L as the lands E. In the example shown in FIG. 4, the leads L include leads La, Lb, and Lc. The component C is mounted on the substrate P such that the leads La are mounted on the lands Ea, the leads Lb are mounted on the lands Eb, and the leads Lc are mounted on the lands Ec.

[0017] In the following description, a direction parallel to the surface of the substrate P will be referred to as the planar direction, and a direction perpendicular to the surface of the substrate P will be referred to as the vertical direction. A direction parallel to the X-axis in an XY plane parallel to the surface of the substrate P will be referred to as the X-axis direction, a direction parallel to the Y-axis in an XY plane perpendicular to the X-axis will be referred to as the Y-axis direction, and a rotational direction around a vertical axis perpendicular to the surface of the substrate P will be referred to as the θ direction. The planar direction includes one or both of the X-axis direction and the Y-axis direction. The rotational direction is the θ direction.

[0018] The mounting device 3 mounts the component C on the substrate P so that the lead L is mounted on the land E at a target relative position. In the embodiment, mounting the lead L on the land E at the target relative position includes mounting the lead L on the land E so that the center position of the land E and the center position of the lead L coincide in the planar direction, and the orientation of the land E and the orientation of the lead L coincide in the rotational direction. FIG. 4 shows a state in which the lead L is mounted on the land E at the target relative position. In the embodiment, mounting the lead L on the land E at the target relative position includes mounting the lead L on the land E so that the center position of each of the lands E (Ea, Eb, Ec) and the center position of each of the leads L (La, Lb, Lc) coincide in the planar direction, and the orientation of each of the lands E (Ea, Eb, Ec) and the orientation of each of the leads L (La, Lb, Lc) coincide in the rotational direction. When the lead L is mounted on the land E at the target relative position, the lead L does not protrude from the land E.

[0019] [Inspection equipment] 5 is a diagram illustrating an inspection device 4 according to an embodiment. The inspection device 4 is an appearance inspection device that inspects the appearance of the board P after the components C are mounted thereon. The inspection device 4 has a board support member 8 that supports the board P after the components C are mounted thereon, a camera 9 that captures an image of the board P supported by the board support member 8, a control device 10, an input device 11, and a display device 12.

[0020] FIG. 6 is a hardware configuration diagram showing a control device 10 according to an embodiment. The control device 10 includes a computer system 1000. The control device 10 includes a processor 1001 such as a central processing unit (CPU), a main memory 1002 including a non-volatile memory such as a read only memory (ROM) and a volatile memory such as a random access memory (RAM), a storage 1003, and an interface 1004 including an input / output circuit. The functions of the control device 10 are stored in the storage 1003 as a computer program. The processor 1001 reads the computer program from the storage 1003, expands it in the main memory 1002, and executes a predetermined process according to the computer program. The computer program may be distributed to the control device 10 via a network.

[0021] 7 is a functional block diagram showing an inspection device 4 according to an embodiment. Each of the input device 11 and the display device 12 is connected to the control device 10. The input device 11 generates input data by being operated by an operator. Examples of the input device 11 include a touch sensor, a computer keyboard, and a mouse. The display device 12 displays display data. Examples of the display device 12 include a flat panel display such as a liquid crystal display or an organic EL display. The display device 12 may be a touch screen including a touch sensor.

[0022] The control device 10 has an input unit 13, a correction unit 14, a teacher image creation unit 15, a label assignment unit 16, a machine learning unit 17, a target image acquisition unit 18, a determination unit 19, and an output unit 20.

[0023] The input unit 13 acquires input data from the input device 11. The input data from the input device 11 is input to the input unit 13. In the embodiment, the operator operates the input device 11 to input to the input unit 13 a maximum allowable value of the amount of misalignment between the substrate P and the component C mounted on the substrate P. The amount of misalignment between the substrate P and the component C includes the amount of misalignment in a planar direction and the amount of misalignment in a rotational direction. The amount of misalignment in a planar direction includes the amount of misalignment in an X-axis direction and the amount of misalignment in a Y-axis direction. The amount of misalignment in a rotational direction is the amount of misalignment in a θ-direction. In the following description, the amount of misalignment in the X-axis direction is appropriately referred to as an X-misalignment amount, the amount of misalignment in the Y-axis direction is appropriately referred to as a Y-misalignment amount, and the amount of misalignment in the θ-direction is appropriately referred to as a θ-misalignment amount.

[0024] The input unit 13 receives the maximum allowable values ​​of the X-shift amount (first shift amount) and the Y-shift amount (first shift amount) which indicate the amount of shift in the planar direction between the substrate P and the component C, and the maximum allowable value of the θ-shift amount (second shift amount) which indicates the amount of shift in the rotational direction between the substrate P and the component C.

[0025] As described with reference to Figures 3 and 4, a land E is provided on the surface of the substrate P. The component C has leads L. The amount of misalignment between the substrate P and the component C includes the amount of misalignment between the land E and the leads L. The amount of X misalignment between the substrate P and the component C is the amount of X misalignment between the land E and the leads L. The amount of Y misalignment between the substrate P and the component C is the amount of Y misalignment between the land E and the leads L. The amount of θ misalignment between the substrate P and the component C is the amount of θ misalignment between the land E and the leads L.

[0026] When the lead L is mounted on the land E at a target relative position, the lead L is placed at a reference position (target position) relative to the land E in the planar direction, and at a reference angle (target angle) relative to the land E in the rotational direction. The deviation in the planar direction is the deviation of the lead L from the reference position in the planar direction. The X deviation is the deviation of the lead L from the reference position in the X-axis direction. The Y deviation is the deviation of the lead L from the reference position in the Y-axis direction. The θ deviation is the deviation angle of the lead L from the reference angle in the θ direction.

[0027] The correction unit 14 corrects the maximum allowable value of one of the maximum allowable value of the misalignment in the planar direction and the maximum allowable value of the misalignment in the rotational direction based on the maximum allowable value of the other misalignment input to the input unit 13. The correction unit 14 corrects the maximum allowable value of the misalignment input to the input unit 13 based on correlation data indicating the relationship between the maximum allowable value of the misalignment in the planar direction and the maximum allowable value of the misalignment in the rotational direction. The correlation data is determined in advance and is held in the correction unit 14. The correction unit 14 corrects the maximum allowable value of the misalignment based on the correlation data such that the larger the maximum allowable value of the misalignment in the planar direction is, the smaller the maximum allowable value of the misalignment in the rotational direction is, and vice versa.

[0028] The teacher image creation unit 15 creates a plurality of teacher images having different amounts of misalignment in the planar direction and in the rotational direction, based on the maximum allowable value of the amount of misalignment corrected by the correction unit 14. In the embodiment, the teacher images are images created by the computer system 1000.

[0029] The label assignment unit 16 assigns different labels to teacher images whose combination of the deviation amount in the planar direction and the deviation amount in the rotational direction satisfies a first condition and teacher images whose combination does not satisfy the first condition. The label assignment unit 16 assigns an OK label (good label) to teacher images whose combination satisfies the first condition, and an NG label (bad label) to teacher images whose combination does not satisfy the first condition.

[0030] The machine learning unit 17 generates a learning model that takes an image of a board P on which a component C is mounted as input, based on a teacher image to which a label has been added, and outputs the quality of the mounting state of the component C on the board P.

[0031] The target image acquisition unit 18 acquires a target image showing an image of the board P on which the component C is mounted, captured by the camera 9. The component C is mounted on the surface of the board P. As shown in FIG. 5, in the inspection device 4, the board P on which the component C is mounted is supported by a board support member 8. The board support member 8 supports the board P so that the surface of the board P faces upward. The camera 9 is disposed above the board support member 8 so as to face the surface of the board P. The camera 9 captures an image of the board P on which the component C is mounted from above the board P. The camera 9 captures an image of the land E and the lead L. The target image is an image captured by the camera 9. The target image acquisition unit 18 acquires a target image including the land E and the lead L.

[0032] The judgment unit 19 inputs the target image acquired by the target image acquisition unit 18 into a learning model to judge whether the mounted state of the component C on the board P is good or bad. The quality of the mounted state of the component C includes whether the relative position of the land E and the lead L is good or bad. The quality of the relative position of the land E and the lead L includes whether the relative position of the land E and the lead L is good or bad in the planar direction, and whether the relative angle between the land E and the lead L is good or bad in the rotational direction.

[0033] The output unit 20 outputs the determination result of the determination unit 19. In the embodiment, the output unit 20 outputs the determination result of the determination unit 19 to the display device 12. The display device 12 displays the determination result of the determination unit 19.

[0034] [Creating teacher images] A method for creating a teacher image will be described below. Figures 8, 9, 10, and 11 are diagrams showing the input device 11 and the display device 12 when a teacher image according to the embodiment is created.

[0035] When creating the teacher image 21, the operator operates the input device 11 to input the maximum allowable value of the deviation amount to the input unit 13. The maximum allowable value of the deviation amount may be, for example, a value determined by a standard, or may be a value determined independently by an electronic device manufacturer.

[0036] The maximum allowable values ​​are determined for each of the X deviation, the Y deviation, and the θ deviation. The operator operates the input device 11 to input the maximum allowable value of the X deviation, the maximum allowable value of the Y deviation, and the maximum allowable value of the θ deviation into the input unit 13. As an example, in this embodiment, the maximum allowable value of the X deviation is "0.26 mm", the maximum allowable value of the Y deviation is "0.26 mm", and the maximum allowable value of the θ deviation is "6.0 deg."

[0037] If the amount of misalignment between the land E and the lead L is equal to or less than the maximum allowable value, the substrate P produced is judged to be normal, and the electronic device produced is judged to be a non-defective product. If the amount of misalignment between the land E and the lead L exceeds the maximum allowable value, the substrate P produced is judged to be abnormal, and the electronic device produced is judged to be a defective product.

[0038] When the maximum allowable value of the X deviation, the maximum allowable value of the Y deviation, and the maximum allowable value of the θ deviation are input to the input unit 13, the teacher image creation unit 15 creates multiple teacher images 21 based on the maximum allowable value of the X deviation, the maximum allowable value of the Y deviation, and the maximum allowable value of the θ deviation input to the input unit 13. The teacher image creation unit 15 creates multiple teacher images to which an OK label should be assigned and multiple teacher images to which an NG label should be assigned, based on the maximum allowable value of the X deviation, the maximum allowable value of the Y deviation, and the maximum allowable value of the θ deviation. In the following description, the state of a normal substrate P to which an OK label should be assigned to the teacher image 21 is appropriately referred to as an OK state (normal state), and the state of an abnormal substrate P to which an NG label should be assigned to the teacher image 21 is appropriately referred to as an NG state (abnormal state). In addition, the teacher image 21 to which an OK label should be assigned is appropriately referred to as an OK image, and the teacher image 21 to which an NG label should be assigned is appropriately referred to as an NG image.

[0039] When the substrate P is in an OK state, it means that the relative positions of the lands E and the leads L are normal, and the electronic device to be produced will be judged as a non-defective product. When the substrate P is in an NG state, it means that the relative positions of the lands E and the leads L are abnormal, and the electronic device to be produced will be judged as a defective product.

[0040] For example, when "0.26 mm" which is the maximum allowable value of the X deviation is input, the teacher image creation unit 15 creates multiple teacher images whose X deviation is "0.26 mm" or less and multiple teacher images whose X deviation exceeds "0.26 mm" with the maximum allowable value of the X deviation "0.26 mm" as the boundary. When "0.26 mm" which is the maximum allowable value of the Y deviation is input, the teacher image creation unit 15 creates multiple teacher images whose Y deviation is "0.26 mm" or less and multiple teacher images whose Y deviation exceeds "0.26 mm" with the maximum allowable value of the Y deviation "0.26 mm" as the boundary. When "6.0 deg" which is the maximum allowable value of the θ deviation is input, the teacher image creation unit 15 creates multiple teacher images whose θ deviation is "6.0 deg" or less and multiple teacher images whose θ deviation exceeds "6.0 deg" with the maximum allowable value of the θ deviation "6.0 deg".

[0041] That is, in the embodiment, the teacher image creation unit 15 creates many OK images to which an OK label should be assigned and many NG images to which an NG label should be assigned, based on the maximum allowable deviation in the planar direction and the maximum allowable deviation in the rotational direction input to the input unit 13. The teacher image creation unit 15 automatically creates many OK images and many NG images while shifting the position of the lead L little by little with respect to the land E.

[0042] 8 shows an example in which a teacher image 21 in which the X-shift amount is "0.00 mm", the Y-shift amount is "0.00 mm", and the θ-shift amount is "0.0 deg" is displayed on the display device 12. Since the X-shift amount, the Y-shift amount, and the θ-shift amount are all zero, an OK image in which the lead L is mounted on the land E at the target relative position is created and displayed on the display device 12.

[0043] 9 shows an example in which a teacher image 21 in which the X-deviation amount is "0.02 mm", the Y-deviation amount is "0.26 mm", and the θ-deviation amount is "0.0 deg" is displayed on the display device 12. Because the X-deviation amount is below the maximum allowable value, the Y-deviation amount is below the maximum allowable value, and the θ-deviation amount is below the maximum allowable value, an OK image in which the lead L is normally mounted on the land E is created and displayed on the display device 12, although there is a slight deviation between the land E and the lead L.

[0044] 10 shows an example in which a teacher image 21 with an X-deviation amount of "0.02 mm", a Y-deviation amount of "0.06 mm", and a θ-deviation amount of "6.0 deg" is displayed on the display device 12. Because the X-deviation amount, the Y-deviation amount, and the θ-deviation amount are below the maximum allowable value, and therefore, although there is a slight deviation between the land E and the lead L, an OK image in which the lead L is normally mounted on the land E is created and displayed on the display device 12.

[0045] FIG. 11 shows an example in which a teacher image 21 with an X deviation of "0.02 mm", a Y deviation of "0.26 mm", and a θ deviation of "6.0 deg" is displayed on the display device 12. Although the X deviation is equal to or less than the maximum allowable value, the Y deviation is equal to or less than the maximum allowable value, and the θ deviation is equal to or less than the maximum allowable value, the land E and the lead L are significantly misaligned. Although the X deviation, the Y deviation, and the θ deviation are each equal to or less than the maximum allowable value, it is not appropriate to determine the teacher image 21 in which the land E and the lead L are significantly misaligned as an OK image. Although the X deviation, the Y deviation, and the θ deviation are each equal to or less than the maximum allowable value, it may be appropriate to determine the created teacher image 21 as an NG image depending on the combination of the X deviation, the Y deviation, and the θ deviation.

[0046] That is, when the maximum allowable value of the X-shift amount, the maximum allowable value of the Y-shift amount, and the maximum allowable value of the θ-shift amount are determined separately and independently, although each of the X-shift amount, the Y-shift amount, and the θ-shift amount is below the maximum allowable value, depending on the combination of the X-shift amount, the Y-shift amount, and the θ-shift amount, it may be appropriate to treat the created teacher image 21 as an NG image.

[0047] Therefore, in the embodiment, when an OK image and an NG image are created based on the maximum allowable deviation value, the correction unit 14 corrects the maximum allowable deviation value input to the input unit 13 so that an OK image, which is a teacher image to be labeled as good, is created when the deviation in the planar direction is equal to or less than the maximum allowable deviation value in the planar direction and the deviation in the rotational direction is equal to or less than the maximum allowable deviation value in the rotational direction. That is, the correction unit 14 corrects the maximum allowable deviation value of one of the maximum allowable deviation values ​​in the planar direction and the maximum allowable deviation value in the rotational direction input to the input unit 13 based on the maximum allowable deviation value of the other one of the maximum allowable deviation values ​​in the planar direction and the maximum allowable deviation value in the rotational direction so that an OK image is generated for any combination of the X deviation, the Y deviation, and the θ deviation as long as the deviation is equal to or less than the maximum allowable value.

[0048] Correction unit 14 corrects the maximum allowable value of the deviation input to input unit 13 so that the larger the maximum allowable value of the deviation in the planar direction, the smaller the maximum allowable value of the deviation in the rotational direction, and the larger the maximum allowable value of the deviation in the rotational direction, the smaller the maximum allowable value of the deviation in the planar direction. Correction unit 14 corrects the maximum allowable value of the deviation input to input unit 13 based on predetermined correlation data.

[0049] Fig. 12 is a diagram for explaining correlation data according to an embodiment. As shown in Fig. 12, the correlation data shows the relationship between the maximum allowable value of the X-shift amount or the Y-shift amount and the maximum allowable value of the θ-shift amount. The correlation data corrects the maximum allowable value of the shift amount input to the input unit 13 so that the larger the maximum allowable value of the X-shift amount or the Y-shift amount, the smaller the maximum allowable value of the shift amount in the θ direction, and the larger the maximum allowable value of the θ-shift amount, the smaller the maximum allowable value of the X-shift amount or the Y-shift amount. In the example shown in Fig. 12, the correlation data is set so that the maximum allowable value of the X-shift amount or the Y-shift amount and the maximum allowable value of the θ-shift amount are proportional to each other.

[0050] For example, when the operator inputs the Y deviation amount of "0.26 mm", even if the θ deviation amount is input as "6.0 deg", the correction unit 14 corrects the θ deviation amount input to the input unit 13 to "0.0 deg". When the operator inputs the Y deviation amount of "0.13 mm", even if the θ deviation amount is input as "6.0 deg", the correction unit 14 corrects the θ deviation amount input to the input unit 13 to "3.0 deg". When the operator inputs the Y deviation amount of "0.13 mm", if the θ deviation amount is input as, for example, "2.0 deg", no correction is performed by the correction unit 14, and the θ deviation amount of "2.0 deg" input to the input unit 13 is adopted as the standard for creating the teacher image 21. When the operator inputs the Y deviation amount as "0.0 mm", even if the θ deviation amount is input as "6.0 deg", no correction is performed by the correction unit 14, and the θ deviation amount of "6.0 deg" input to the input unit 13 is used as the standard for creating the teacher image 21. The same applies when the operator inputs the X deviation amount.

[0051] For example, when the operator inputs the θ deviation amount as "6.0 deg", even if the operator inputs the Y deviation amount as "0.26 mm", the correction unit 14 corrects the Y deviation amount inputted to the input unit 13 to "0.00 mm". When the operator inputs the θ deviation amount as "3.0 deg", even if the operator inputs the Y deviation amount as "0.26 mm", the correction unit 14 corrects the Y deviation amount inputted to the input unit 13 to "0.13 mm". When the operator inputs the θ deviation amount as "3.0 deg", if the Y deviation amount is inputted as, for example, "0.05 mm", no correction is performed by the correction unit 14, and the Y deviation amount "0.05 mm" inputted to the input unit 13 is adopted as the standard for creating the teacher image 21. If the operator inputs the θ shift amount as "0.0 deg," even if the Y shift amount is input as "0.26 mm," no correction is performed by the correction unit 14, and the Y shift amount of "0.26 mm" input to the input unit 13 is adopted as the standard for creating the teacher image 21.

[0052] The teacher image creation unit 15 creates multiple teacher images 21 to which an OK label should be assigned and multiple teacher images 21 to which an NG label should be assigned, based on the maximum allowable values ​​of the X and Y deviations and the maximum allowable value of the θ deviation corrected by the correction unit 14. In the example shown in FIG. 12, a straight line SH is defined that indicates the relationship between the maximum allowable value of the X deviation or the Y deviation and the maximum allowable value of the θ deviation. The teacher image creation unit 15 creates multiple OK images that satisfy a first condition in which the combination of the X deviation, the Y deviation, and the θ deviation is smaller than the straight line SH. The teacher image creation unit 15 creates multiple OK images that have different X deviations, Y deviations, and θ deviations. The teacher image creation unit 15 also creates multiple NG images that satisfy a second condition in which the combination of the X deviation, the Y deviation, and the θ deviation is larger than the straight line SH. The teacher image creation unit 15 creates multiple NG images that have different X deviations, Y deviations, and θ deviations.

[0053] [Generating learning models] 13 is a flowchart showing a method for generating a learning model according to an embodiment. An operator operates the input device 11 to input the maximum allowable values ​​of the X-shift amount, the Y-shift amount, and the θ-shift amount to the input unit 13. The input unit 13 acquires the input values ​​of the maximum allowable values ​​of the X-shift amount, the Y-shift amount, and the θ-shift amount from the input device 11 (step SA1).

[0054] The correction unit 14 corrects the input values ​​of the maximum allowable values ​​of the X deviation amount, the Y deviation amount, and the θ deviation amount input to the input unit 13 based on the correlation data (step SA2).

[0055] The teacher image creation unit 15 creates a plurality of teacher images 21 having mutually different X-shift amounts, Y-shift amounts, and θ-shift amounts based on the respective maximum allowable values ​​of the X-shift amount, the Y-shift amount, and the θ-shift amount corrected by the correction unit 14 (step SA3).

[0056] The label assignment unit 16 assigns an OK label to the teacher image 21 whose combination of the X-shift amount, the Y-shift amount, and the θ-shift amount satisfies the first condition, and assigns an NG label to the teacher image 21 whose combination of the X-shift amount, the Y-shift amount, and the θ-shift amount does not satisfy the first condition but satisfies the second condition (step SA4).

[0057] The machine learning unit 17 generates a learning model that takes an image of the board P on which the component C is mounted as input, based on the teacher image 21 to which an OK or NG label has been assigned, and outputs the quality of the mounting state of the component C (step SA5).

[0058] [Adjustment of label assignment] FIG. 14 is a diagram for explaining a method for adjusting label assignment according to the embodiment. In the embodiment, when there is a teacher image 21 whose combination of the X-shift amount or the Y-shift amount and the θ-shift amount after correction by the correction unit 14 matches the straight line SH, as shown in FIG. 14, the output unit 20 displays a list of teacher images 21 whose Y-shift amount or the combination of the Y-shift amount and the θ-shift amount matches the straight line SH and the labels assigned to the teacher images 21 on the display device 12. In addition, the output unit 20 displays a teacher image 21 selected by the operator from the list of teacher images 21 on the display device 12. The operator can select a teacher image 21 by operating the input device 11.

[0059] The teacher image 21 in which the combination of the X-shift amount or the Y-shift amount and the θ-shift amount coincides with the straight line SH can be regarded as the teacher image 21 existing on the boundary between the OK image and the NG image. Therefore, for the teacher image 21 in which the combination of the X-shift amount or the Y-shift amount and the θ-shift amount coincides with the straight line SH, it may be appropriate to treat it as an NG image even though the labeling unit 16 has given an OK label to it. Also, for the teacher image 21 in which the combination of the X-shift amount or the Y-shift amount and the θ-shift amount coincides with the straight line SH, it may be appropriate to treat it as an OK image even though the labeling unit 16 has given an NG label to it. For the teacher image 21 existing on the boundary between the OK image and the NG image, the operator may determine whether it is an OK image or an NG image. By displaying the teacher image 21 existing on the boundary between the OK image and the NG image on the display device 12, the operator can check the teacher image 21 displayed on the display device 12 and determine whether it is an OK image or an NG image. The operator can change the label given by the labeling unit 16 based on his / her own judgment result by operating the input device 11. Based on the result of the operator's own judgment, the operator can operate the input device 11 to change the OK label assigned by the label assignment unit 16 to an NG label, or change the NG label assigned by the label assignment unit 16 to an OK label.

[0060] [Determining whether a component is properly mounted] 15 is a flowchart showing a method for determining the quality of the mounting state of a component C according to an embodiment. The board P on which the component C has been mounted by the mounting device 3 is transported from the mounting device 3 to the inspection device 4. The board P on which the component C has been mounted is supported by a board support member 8. With the board P supported by the board support member 8, an image of the board P is captured by a camera 9.

[0061] The target image acquisition unit 18 acquires a target image indicating an image of the substrate P captured by the camera 9 (step SB1).

[0062] The determination unit 19 inputs the target image acquired by the target image acquisition unit 18 into the learning model (step SB2).

[0063] The judgment unit 19 judges whether the mounting state of the component C is good or bad based on the output from the learning model (step SB3).

[0064] The output unit 20 outputs the judgment result of the judgment unit 19. The judgment result of the judgment unit 19 includes a judgment result as to whether or not the produced substrate P is normal. In the embodiment, the output unit 20 causes the display device 12 to display the judgment result of the judgment unit 19 (step SB4).

[0065] [effect] As described above, according to the embodiment, when inspecting the appearance of the substrate P after the components C are mounted using artificial intelligence technology, an appropriate teacher image 21 is generated. Depending on the combination of the maximum allowable value of the X-shift amount, the maximum allowable value of the Y-shift amount, and the maximum allowable value of the θ-shift amount input to the input unit 13, the teacher image 21 that should be an OK image may become an NG image, or the teacher image 21 that should be an NG image may become an OK image. In the embodiment, at least one of the maximum allowable value of the X-shift amount, the maximum allowable value of the Y-shift amount, and the maximum allowable value of the θ-shift amount input to the input unit 13 is corrected, thereby generating an appropriate teacher image 21. Therefore, the quality of the produced substrate P is judged appropriately.

[0066] [Other embodiments] FIG. 16 is a diagram for explaining a method for adjusting the label assignment according to the embodiment. In the example described with reference to FIG. 14, the teacher image 21 existing on the boundary between the OK image and the NG image is displayed on the display device 12, and the label assigned by the label assignment unit 16 is changed based on the input data from the input device 11. As shown in FIG. 16, for example, the determination unit 19 may change the label assigned by the label assignment unit 16 by performing image processing on the teacher image 21 existing on the boundary between the OK image and the NG image. The determination unit 19 calculates the area of ​​the overlapping range M between the land E and the lead L. The determination unit 19 calculates the ratio of the area of ​​the overlapping range M to the area of ​​the lead L. When the ratio of the area of ​​the overlapping range M to the area of ​​the lead L is equal to or greater than a predetermined threshold, the determination unit 19 determines that the relative position between the land E and the lead L is normal. The overlapping range M includes an overlapping range Ma between the land Ea and the lead La, an overlapping range Mb between the land Eb and the lead Lb, and an overlapping range Mc between the land Ec and the lead Lc. When the ratios for all of the overlap ranges Ma, Mb, and Mc are equal to or greater than the threshold, the determination unit 19 determines that the relative positions of the land E and the lead L are normal, and assigns an OK label to the teacher image 21. When the ratios for at least one of the overlap ranges Ma, Mb, and Mc are less than the threshold, the determination unit 19 determines that the relative positions of the land E and the lead L are not normal, and assigns an NG label to the teacher image 21. [Explanation of symbols]

[0067] 1...production system, 2...inspection device, 3...mounting device, 4...inspection device, 5...substrate support member, 6...mounting head, 7...nozzle, 8...substrate support member, 9...camera, 10...control device, 11...input device, 12...display device, 13...input section, 14...correction section, 15...teacher image creation section, 16...label assignment section, 17...machine learning section, 18...target image acquisition section, 19...judgment section, 20...output section, 21...teacher image, 1000...computer system, 1001...processor, 1002...main memory, 1003...storage, 1004...interface, B...body, C...component, E...land, Ea...land, Eb...land, Ec...land, L...lead, P...substrate.

Claims

1. an input unit for inputting a first maximum allowable amount of misalignment indicating a misalignment amount in a planar direction between a board and a component mounted on the board, and a second maximum allowable amount of misalignment indicating a misalignment amount in a rotational direction between the board and the component; a correction unit that corrects the maximum allowable value of one of the first and second deviation amounts based on the maximum allowable value of the other deviation amount, the first and second deviation amounts being input to the input unit; a teacher image creating unit that creates a plurality of teacher images each having a different first deviation amount and a different second deviation amount based on the maximum allowable value of the deviation amount corrected by the correcting unit; a label assignment unit that assigns different labels to the teacher image in which a combination of the first shift amount and the second shift amount satisfies a first condition and to the teacher image in which a combination of the first shift amount and the second shift amount does not satisfy the first condition; a machine learning unit that generates a learning model based on the teacher image to which the label has been assigned, using an image of the board on which the components are mounted as an input and outputting a quality of the mounting state of the components; an object image acquisition unit that acquires an object image showing an image of the board on which the component is mounted, the image being captured by a camera; a judgment unit that inputs the target image into the learning model and judges whether the mounting state of the component is good or bad; An output unit that outputs the judgment result of the judgment unit. Inspection equipment.

2. A land is provided on a surface of the substrate, The component has leads mounted on the lands, the amount of misalignment includes an amount of misalignment between the land and the lead, 2. The inspection device according to claim 1.

3. the correction unit corrects the maximum allowable value of the deviation based on correlation data indicating a relationship between a predetermined maximum allowable value of the first deviation amount and a predetermined maximum allowable value of the second deviation amount.

2. The inspection device according to claim 1.

4. the correction unit corrects the maximum allowable value of the deviation such that the maximum allowable value of the second deviation becomes smaller as the maximum allowable value of the first deviation becomes larger, and the maximum allowable value of the first deviation becomes smaller as the maximum allowable value of the second deviation becomes larger.

2. The inspection device according to claim 1.

5. the teacher image creation unit creates a plurality of teacher images to which good labels should be assigned and a plurality of teacher images to which bad labels should be assigned, based on the maximum allowable value of the first deviation amount and the maximum allowable value of the second deviation amount corrected by the correction unit, respectively.

2. The inspection device according to claim 1.

6. the correction unit corrects the maximum allowable value of the amount of deviation when the first amount of deviation is equal to or less than a maximum allowable value of the amount of deviation and the second amount of deviation is equal to or less than a maximum allowable value of the amount of deviation so that a teacher image to which the good label should be assigned is created.

6. The inspection apparatus according to claim 5.

Images