Inspection system

Abstract

This inspection system for a mounting substrate includes: a learning unit that generates an inference model for evaluating an abnormality of the mounting substrate by machine learning using a pass / fail determination result of the mounting substrate and a substrate image as a data set; an evaluation unit that derives an abnormality evaluation value of the mounting substrate using the inference model; a determination display unit that displays determination reference data including a substrate image and the abnormality evaluation value; a determination input unit that receives an input of the result of pass / fail determination of the mounting substrate performed using the determination reference data; and a monitoring unit that monitors the abnormality evaluation value and the pass / fail determination result. The monitoring unit outputs re-determination notification information when the pass / fail determination result indicates a non-defective product despite the abnormality evaluation value being equal to or greater than a predetermined upper threshold, or when the pass / fail determination result indicates a defective product despite the abnormality evaluation value being equal to or less than a predetermined lower threshold.

Description

Inspection System

[0001] The present invention relates to an inspection system for a mounting board on which components are mounted.

[0002] In circuit board production lines where electronic components are mounted on boards, visual inspection machines are installed to inspect whether the required components have been mounted on the board in the correct condition after component mounting. Visual inspection machines take an image of the mounted board and perform the required image processing on the acquired image to automatically make a primary judgment as to whether the mounted board is pass or fail. For mounted boards that are judged as failing in this primary judgment, an inspector visually judges the pass or fail of the mounted board as the final judgment. A mounted board that is judged as failing in the primary judgment may be judged as pass in the secondary judgment.

[0003] Patent Literature 1 discloses an inspection system that uses AI (artificial intelligence) to determine whether an item is good or bad, and that determines whether the item is good or bad according to a learning model in which a threshold for determining whether the item is good or bad is set so that an overdetection rate, which indicates the degree of overdetection that causes an item that should be determined to be good to be determined to be defective, is a set value. Introducing such an AI-based determination technology into a circuit board production line to support secondary determination is preferable, as it reduces the burden on inspectors.

[0004] However, if the accuracy of the pass / fail judgment using AI is low, or if there is an error in the pass / fail judgment made by the inspector, there may be a discrepancy between the pass / fail judgment made by AI and the pass / fail judgment made by the inspector. In this case, there is a risk that the pass / fail judgment of the mounted board cannot be made with high accuracy.

[0005] JP 2022-38020 A

[0006] An object of the present invention is to provide an inspection system that can accurately determine whether a mounting board is good or bad.

[0007] An inspection system according to one aspect of the present invention is an inspection system for a mounting board on which components are mounted, and includes: a learning unit that generates an inference model for evaluating an abnormality of the mounting board based on the board image by machine learning using a data set that includes a pass / fail judgment result indicating whether the mounting board is a good or defective product and a board image of the mounting board; an evaluation unit that uses the inference model to evaluate an abnormality of the mounting board based on the board image and derives a numerical abnormality evaluation value that becomes larger the higher the degree of abnormality of the mounting board; a judgment display unit that displays judgment reference data including the board image and the abnormality evaluation value; a judgment input unit that accepts input of the pass / fail judgment result of the mounting board using the judgment reference data; and a monitoring unit that monitors the abnormality evaluation value included in the judgment reference data and the pass / fail judgment result input to the judgment input unit. The monitoring unit outputs re-evaluation notification information prompting a re-evaluation using the judgment reference data when the abnormality evaluation value is above a predetermined upper threshold and the pass / fail judgment result indicates the product is good, or when the abnormality evaluation value is below a predetermined lower threshold and the pass / fail judgment result indicates the product is defective.

[0008] Fig. 1 is a block diagram showing a simplified configuration of a production line for mounting substrates to which an inspection system according to an embodiment of the present invention is applied. Fig. 2 is a diagram showing a schematic configuration of the inspection system. Fig. 3 is a block diagram showing a configuration of the inspection system. Fig. 4 is a schematic diagram for explaining the processing of an AI processing device provided in the inspection system. Fig. 5 is a schematic diagram for explaining the processing of a determination terminal provided in the inspection system. Fig. 6 is a flowchart showing an example of secondary determination processing by the determination terminal. Fig. 7 is a schematic diagram for explaining the processing of a selection terminal provided in the inspection system.

[0009] An inspection system according to an embodiment of the present invention will be described below with reference to the drawings. The inspection system according to this embodiment is a system for inspecting a mounting board on which components are mounted. The mounting board is a product in which electronic components such as chip components such as chip resistors and chip capacitors, ball bump components, packaged components such as ICs, power supply related components such as capacitors and transformers, connectors, heat sinks, etc. are mounted on a printed circuit board on which a circuit pattern is printed.

[0010] 1 is a block diagram showing a simplified configuration of a mounted board production line 1 to which an inspection system according to this embodiment is applied. The mounted board production line 1 is a line in which electronic components and the like are mounted on printed circuit boards. The mounted board production line 1 includes a board transport path TR that transports boards on which electronic components are mounted, a plurality of work devices that are arranged on the board transport path TR and perform predetermined operations on the boards, and a line control device 10 that provides overall control of these work devices.

[0011] 1 shows the working devices arranged in tandem from upstream to downstream in the board transport direction: a printer 11, a print inspection machine 12, a mounting machine 13, a board inspection machine 14, a reflow oven 15, and an appearance inspection machine 16. A loader 17 that carries boards into the printer 11 is located at the upstream end of the mounted board production line 1, and an unloader 18 that removes produced boards from the appearance inspection machine 16 is located at the downstream end.

[0012] The printer 11 has a working section that applies solder to the pads of the printed circuit board. For example, the printer 11 places a mask with openings where solder should be applied on the printed circuit board, and applies cream solder through the mask. The print inspection machine 12 takes an image of the printed circuit board with the solder applied, determines the shape of the solder, and inspects whether the position, amount, and height of the solder applied to the printed circuit board are appropriate.

[0013] The mounter 13 is equipped with a component mounting head and produces mounted boards by mounting required electronic components on printed circuit boards. The mounted board production line 1 shown in FIG. 1 shows an example in which three mounters 13 are arranged in series. That is, in the mounted board production line 1, a first mounter 13A, a second mounter 13B, and a third mounter 13C are arranged in tandem from the upstream side. The board inspection machine 14 takes images of the mounted boards that have passed the mounters 13 and inspects the mounted boards for misalignment of electronic components, lead misalignment, component lift, missing components, soldering defects, and the presence of foreign matter. The reflow furnace 15 heats the mounted boards to melt the solder and fix the electronic components to the mounted boards.

[0014] The visual inspection machine 16 takes an image of the mounted board after heat treatment in the reflow furnace 15, and similarly to the board inspection machine 14, inspects for misalignment of electronic components, lead misalignment, component lift, missing mounting, soldering defects, the presence of foreign matter, etc. In other words, the visual inspection machine 16 takes an image of the mounted board of the finished product, and automatically judges whether there are any abnormalities in the mounted board based on the image. In this embodiment, an example is shown in which the inspection system according to the present invention is applied for secondary judgment as to whether a mounted board that has been automatically judged as a "defective product" by the visual inspection machine 16 is truly a "defective product" or whether it has been over-judged as a "defective product" despite being a "good product."

[0015] The line control device 10 controls the production work of mounted boards by comprehensively controlling the work devices that are provided on the mounted board production line 1, which are the printing machine 11, the print inspection machine 12, the mounting machine 13, the board inspection machine 14, the reflow oven 15, and the appearance inspection machine 16. The line control device 10 also collects various types of information output from the work devices. This various information includes notification information such as status information that indicates the operating status of the work devices, error information that is output when a malfunction occurs in the operation, information requesting the replenishment of consumable materials such as electronic components, and information requesting some kind of response.

[0016] [Overall Configuration of Inspection System] Fig. 2 is a diagram schematically showing the configuration of the inspection system IS. Fig. 3 is a block diagram showing the configuration of the inspection system IS. The inspection system IS includes the above-mentioned appearance inspection machine 16, an AI station 2, and a repair station 3.

[0017] The appearance inspection machine 16 is a primary inspection device that makes a primary judgment as to whether a mounted board produced on the mounted board production line 1 is a pass or fail. The appearance inspection machine 16 has a camera 161 as an imaging unit that acquires a board image SI that shows an image of the mounted board. The appearance inspection machine 16 automatically judges whether the inspected mounted board satisfies the standard by comparing a reference evaluation value that allows a primary judgment of the mounted board with an evaluation value extracted from the board image SI. If the board image SI of a mounted board is judged to be a defective product (hereinafter referred to as an NG judgment) because it does not satisfy the standard as a result of the automatic judgment by the appearance inspection machine 16, the board image SI is sent to the AI ​​station 2 and the repair station 3.

[0018] The repair station 3 is a work site where a secondary judgment is made, through visual inspection by operators OP working on the mounted board production line 1, as to whether a mounted board that has been judged as NG in the primary judgment by the visual inspection machine 16 is a pass or fail product. The operators OP include a first operator OP1 as an inspector who judges the pass or fail of the mounted board, and a second operator OP2 as a manager who manages the first operator OP1.

[0019] The repair station 3 includes a judgment terminal 30 operated by a first operator OP1 or a second operator OP2. The judgment terminal 30 is configured, for example, by a personal computer. A board image SI of a mounting board judged as NG is displayed on the judgment terminal 30. The first operator OP1 visually checks the board image SI and performs a secondary judgment to finally determine whether the mounting board is a "defective" mounting board as judged in the primary judgment, or whether the primary judgment was an overjudgment and the mounting board should be judged as a "good" mounting board. If there is an error in the quality judgment of the mounting board made by the first operator OP1, the second operator OP2 corrects the result of the quality judgment. The judgment terminal 30 sends a quality judgment result JR of the mounting board based on the board image SI to the AI ​​station 2 through operation by the first operator OP1 or the second operator OP2. Details of the judgment terminal 30 will be described later.

[0020] The AI ​​station 2 uses machine learning that utilizes AI (artificial intelligence) to construct an inference model for evaluating abnormalities in the mounting board based on the board image SI. The AI ​​station 2 includes an AI processing device 21, an over-determined image DB 22, a selection terminal 23, and a learning DB 24.

[0021] The AI ​​processing device 21 is configured, for example, with a processor capable of information processing. The AI ​​processing device 21 generates an inference model IM by machine learning using a data set DS that includes the pass / fail judgment result JR of the mounted board sent from the judgment terminal 30 and the board image SI sent from the visual inspection machine 16. Furthermore, when a board image SI of a newly-targeted mounted board is input, the AI ​​processing device 21 evaluates an abnormality in the mounted board using the generated inference model IM and outputs abnormality evaluation data AD that indicates the evaluation result.

[0022] The over-determined image DB 22 is a database that accumulates board images SI of over-determined mounting boards that have been judged as NG in the primary judgment by the visual inspection machine 16 but have been judged as passable in the secondary judgment using the judgment terminal 30. Note that the over-determined image DB 22 may also accumulate board images SI of mounting boards that have been judged as NG in the primary judgment and also in the secondary judgment.

[0023] The selection terminal 23 is configured by, for example, a personal computer. The selection terminal 23 is a terminal through which the first operator OP1 or the second operator OP2 selects the board images SI to be added to the data set DS that is to be learned by the AI ​​processing device 21. Details of the selection terminal 23 will be described later.

[0024] The learning DB 24 is a database that accumulates board images SI of existing mounting boards that have been determined to be non-defective. Board images SI selected by the selection terminal 23 are also stored in the learning DB 24. A data set DS of the board images SI and the pass / fail determination results JR stored in the learning DB 24 is subjected to machine learning, and an inference model IM is generated by the AI ​​processing device 21, and the inference model IM is updated as needed.

[0025] The AI ​​station 2, which is now equipped with an inference model IM with improved evaluation accuracy for abnormalities in mounted boards through machine learning in the AI ​​processing device 21, functions as a support device that supports secondary evaluation using the evaluation terminal 30 in the repair station 3. It may be difficult to determine the pass / fail of a mounted board when the first operator OP1 visually performs secondary evaluation using the evaluation terminal 30. Furthermore, even if an experienced first operator OP1 is able to determine the pass / fail status, an inexperienced first operator OP1 may have difficulty making the same determination. In consideration of these factors, a support request is sent from the repair station 3 to the AI ​​station 2 for a mounted board that has been judged as "NG" in the primary evaluation. In response to this support request, the AI ​​processing device 21 in the AI ​​station 2 evaluates the abnormality of the mounted board using the inference model IM obtained through machine learning and inputs abnormality evaluation data AD indicating the evaluation results to the repair station 3. The first operator OP1 makes a final judgment on the pass / fail status of the mounted board by referring to the input abnormality evaluation data AD.

[0026] [Regarding the AI ​​Processing Device] The AI ​​processing device 21 will be described in detail with reference to Fig. 4 in addition to Fig. 3. Fig. 4 is a schematic diagram for explaining the processing of the AI ​​processing device 21. Functionally, the AI ​​processing device 21 includes a feature amount calculation unit 211, a learning unit 212, and an evaluation unit 213.

[0027] The feature calculation unit 211 extracts feature data FD indicating feature amounts of the mounting board based on the board image SI, and generates a feature map FM by mapping the feature data FD. The learning unit 212 generates an inference model IM for evaluating an abnormality in the mounting board based on the board image SI through machine learning using a data set DS that includes the board image SI of the mounting board to be subjected to abnormality evaluation and the pass / fail judgment result JR for the board image SI. When a new board image SI of a mounting board to be judged is input, the evaluation unit 213 evaluates the mounting board for an abnormality using the inference model IM and outputs abnormality evaluation data AD indicating the evaluation result.

[0028] 4, first, in step #1, a database DB1 is prepared that accumulates a large number of datasets DS. These datasets DS are used for machine learning by the learning unit 212 to generate an inference model IM.

[0029] In the following step #2, the feature calculation unit 211 performs predetermined image processing on the board image SI to calculate feature data FD for the board image SI. Examples of feature data FD obtained from the board image SI include color features, edge features, texture features, and local features. Color features are features that use the RGB values ​​of the board image SI to express color information such as hue, saturation, and brightness. Edge features are features that express edges such as contours and boundaries within the board image SI. Texture features are features that express textures within the board image SI. Local features are features that focus on specific regions within the board image SI and express patterns and shapes within the specific regions. Note that the features included in the feature data FD may be features derived by machine learning in the learning unit 212, such as features indicated by output values ​​of an intermediate layer (convolutional layer) of a convolutional neural network.

[0030] In step #3, the learning unit 212 generates an inference model IM by machine learning of the data set DS accumulated in the database DB1. The learning unit 212 generates the inference model IM by using the feature amount data FD calculated by the feature amount calculation unit 211. The inference model IM has the meaning of an evaluation formula for evaluating an abnormality in the mounting board based on the board image SI.

[0031] In step #4, the feature calculation unit 211 generates a feature map FM by mapping the feature data FD calculated in step #2. The feature map FM has the purpose of visualizing the feature data FD. When a wide variety of feature data FD is calculated and therefore there are many variables, it becomes difficult to represent all of the feature amounts in a two-dimensional or three-dimensional graph or the like for visualization. For this reason, the feature calculation unit 211 generates the feature map FM according to a dimensionality reduction algorithm. Examples of dimensionality reduction algorithms that can be used include t-SNE, which compresses data to preserve local similarities, and UMAP, which compresses data to preserve the local structure of high-dimensional data.

[0032] 4 illustrates an example of a graph in which the feature data FD of good and defective products are plotted two-dimensionally as the feature map FM. By using the inference model IM generated by the learning unit 212, it is possible to display a good product distribution model that reflects the inference model IM in the feature map FM. The good product distribution model expresses representative points that represent the characteristics of good products and approximates the average and spread (variance) of the characteristics of good products.

[0033] The feature data FD including the feature map FM generated by the feature calculation unit 211 is input to the selection terminal 23 .

[0034] When the accuracy of the inference model IM generated by the learning unit 212 reaches a required level, the evaluation unit 213 uses the inference model IM to evaluate an abnormality of the mounting board. When a board image SI of a mounting board to be newly determined is input, the evaluation unit 213 evaluates an abnormality of the mounting board using the inference model IM and outputs abnormality evaluation data AD indicating the evaluation result. The abnormality evaluation data AD includes an abnormality evaluation value EV, which is quantified so that the numerical value increases as the degree of abnormality of the mounting board increases, an evaluation result ER indicating whether the mounting board is a good product with no abnormalities or a defective product with an abnormality, and an abnormality degree map AM, which is a map obtained by superimposing the degree of abnormality of the mounting board on the board image SI.

[0035] The evaluation unit 213 derives a larger anomaly evaluation value EV as the degree of abnormality of the mounting board based on the board image SI increases. In other words, the larger the anomaly evaluation value EV, the higher the degree of abnormality of the mounting board. The evaluation unit 213 derives the anomaly evaluation value EV within a numerical range of, for example, "0.000 to 100.000." Furthermore, if the anomaly evaluation value EV is equal to or greater than a predetermined upper threshold, the evaluation unit 213 derives an evaluation result ER of "NG," indicating that the mounting board is a defective product with an abnormality. On the other hand, if the anomaly evaluation value EV is equal to or less than a predetermined lower threshold, the evaluation unit 213 derives an evaluation result ER of "OK," indicating that the mounting board is a non-defective product with no abnormalities. Furthermore, the evaluation unit 213 generates an anomaly map AM by superimposing the degree of abnormality of the mounting board on the board image SI using a color index that represents the degree of abnormality using a single color shade or color change.

[0036] The abnormality evaluation data AD including the abnormality evaluation value EV, the evaluation result ER, and the abnormality degree map AM are input to the determination terminal 30 .

[0037] [Regarding the Determination Terminal] The determination terminal 30 will be described in detail with reference to Fig. 5 in addition to Fig. 3. Fig. 5 is a schematic diagram for explaining the processing of the determination terminal 30. As described above, the determination terminal 30 is a terminal operated by the first operator OP1 or the second operator OP2 when a secondary determination of a mounted board is performed in the repair station 3. The determination terminal 30 includes a determination display unit 31, a determination input unit 32, and a determination processing unit 33.

[0038] The judgment display unit 31 is composed of a display capable of displaying various types of data. The judgment display unit 31 displays judgment reference data JD that is used as a reference when making a secondary judgment on a mounting board. The judgment reference data JD includes a board image SI of the mounting board to be judged and abnormality evaluation data AD for the board image SI. By visually checking the judgment reference data JD displayed on the judgment display unit 31, the first operator OP1 can make a pass / fail judgment on whether the mounting board is a good product or a defective product based on the board image SI while referring to the abnormality evaluation data AD.

[0039] The judgment input unit 32 is composed of a keyboard or the like that accepts input operations by the first operator OP1 or the second operator OP2. The judgment input unit 32 accepts input of a pass / fail judgment result JR, indicating whether the mounting board is a pass or a fail product, by the first operator OP1 who is visually checking the judgment reference data JD displayed on the judgment display unit 31.

[0040] The judgment processing unit 33 is configured, for example, by a processor capable of information processing. Functionally, the judgment processing unit 33 includes a judgment display control unit 331 and a monitoring unit 332. The judgment display control unit 331 controls the judgment display unit 31. The judgment display control unit 331 causes the judgment display unit 31 to display the judgment reference data JD.

[0041] The monitoring unit 332 monitors the abnormality evaluation value EV of the abnormality evaluation data AD included in the judgment reference data JD and the pass / fail judgment result JR input to the judgment input unit 32. When the abnormality evaluation value EV is equal to or greater than a predetermined upper threshold and the pass / fail judgment result JR indicates a pass product, the monitoring unit 332 outputs re-judgment notification information RJI that prompts a re-judgment using the judgment reference data JD. The monitoring unit 332 also outputs re-judgment notification information RJI when the abnormality evaluation value EV is equal to or less than a predetermined lower threshold and the pass / fail judgment result JR indicates a fail product.

[0042] If the anomaly evaluation value EV is equal to or greater than a predetermined upper threshold, the evaluation unit 213 of the AI ​​processing device 21 evaluates the mounting board as a defective product with an abnormality. If the pass / fail judgment result JR input to the judgment input unit 32 for such an evaluation result of the evaluation unit 213 indicates a pass / fail product, a discrepancy occurs between the evaluation result of the evaluation unit 213 using the inference model IM and the pass / fail judgment result JR of the first operator OP1. Also, if the anomaly evaluation value EV is equal to or less than a predetermined lower threshold, the evaluation unit 213 evaluates the mounting board as a non-defective product with no abnormalities. Even if the pass / fail judgment result JR input to the judgment input unit 32 for such an evaluation result of the evaluation unit 213 indicates a failing product, a discrepancy occurs between the evaluation result of the evaluation unit 213 using the inference model IM and the pass / fail judgment result JR of the first operator OP1.

[0043] As described above, when a discrepancy occurs between the evaluation result of the evaluation unit 213 and the pass / fail judgment result JR of the first operator OP1, the monitoring unit 332 outputs re-judgment notification information RJI to prompt a re-judgment using the judgment reference data JD. When the re-judgment notification information RJI is output from the monitoring unit 332, for example, the second operator OP2, who serves as a manager higher than the first operator OP1 who made the pass / fail judgment, can efficiently determine whether the accuracy of the anomaly evaluation using the inference model IM by the evaluation unit 213 is low or whether there is an error in the pass / fail judgment by the first operator OP1 by checking the judgment reference data JD displayed on the judgment display unit 31. In this case, the second operator OP2 can take measures to update the inference model IM to improve the accuracy of the anomaly evaluation or to correct the pass / fail judgment result JR. This enables accurate pass / fail judgment of the mounted board.

[0044] After the monitoring unit 332 outputs the re-determination notification information RJI, the determination input unit 32 accepts input of correction information RI or erroneous evaluation information MEI by the second operator OP2. The correction information RI is information for correcting the pass / fail determination result JR for the determination reference data JD displayed on the determination display unit 31. The erroneous evaluation information MEI is information indicating that there is an error in the abnormality evaluation value EV included in the determination reference data JD. In other words, after the monitoring unit 332 outputs the re-determination notification information RJI, the correction information RI or erroneous evaluation information MEI is input to the determination input unit 32 by the second operator OP2.

[0045] When correction information RI is input to the judgment input unit 32, the pass / fail judgment result JR for the judgment reference data JD displayed on the judgment display unit 31 can be corrected in accordance with the correction information RI. On the other hand, when erroneous evaluation information MEI is input to the judgment input unit 32, the monitoring unit 332 records the board image SI included in the judgment reference data JD as an erroneously evaluated board image MSI to which an erroneous evaluation label MEL has been assigned. Such an erroneously evaluated board image MSI can be used to update the inference model IM to improve the accuracy of abnormality evaluation.

[0046] When the number of recorded erroneous evaluation board images MSI exceeds a predetermined number, the monitoring unit 332 outputs model update notification information MUI to prompt updating of the inference model IM. In this case, the timing when the model update notification information MUI is output from the monitoring unit 332 can be set as the timing for updating the inference model IM.

[0047] Next, the secondary determination process by the determination terminal 30 will be described in more detail with reference to the flowchart of FIG.

[0048] First, in step S1, the judgment display control unit 331 displays judgment reference data JD including the board image SI and the abnormality evaluation data AD on the judgment display unit 31. With the judgment reference data JD displayed on the judgment display unit 31, the judgment input unit 32 accepts input of the pass / fail judgment result JR by the first operator OP1.

[0049] In the following step S2, the monitoring unit 332 determines whether the pass / fail judgment result JR input to the judgment input unit 32 is an "OK" judgment result indicating a good product, or an "NG" judgment result indicating a defective product.

[0050] If the pass / fail judgment result JR in step S2 is a judgment result of "OK," the monitoring unit 332 determines in step S3 whether the abnormality evaluation value EV included in the judgment reference data JD is equal to or greater than a predetermined upper threshold. If the pass / fail judgment result JR is a judgment result of "OK" and the abnormality evaluation value EV is equal to or greater than a predetermined upper threshold (YES in step S3), a discrepancy occurs between the evaluation result of the evaluation unit 213 using the inference model IM and the pass / fail judgment result JR of the first operator OP1. In such a case, the monitoring unit 332 outputs re-judgment notification information RJI in step S5. On the other hand, if the pass / fail judgment result JR is a judgment result of "OK" and the abnormality evaluation value EV is not equal to or greater than the predetermined upper threshold (NO in step S3), no discrepancy occurs between the evaluation result of the evaluation unit 213 and the pass / fail judgment result JR of the first operator OP1. In such a case, in step S10, the monitoring unit 332 registers the pass / fail judgment result JR input to the judgment input unit 32 in step S2 as the final secondary judgment result.

[0051] If the pass / fail judgment result JR in step S2 is a judgment result of "NG," the monitoring unit 332 determines in step S4 whether the abnormality evaluation value EV included in the judgment reference data JD is equal to or less than a predetermined lower threshold. If the pass / fail judgment result JR is a judgment result of "NG" and the abnormality evaluation value EV is equal to or less than the predetermined lower threshold (YES in step S4), a discrepancy occurs between the evaluation result of the evaluation unit 213 using the inference model IM and the pass / fail judgment result JR of the first operator OP1. In such a case, the monitoring unit 332 outputs re-judgment notification information RJI in step S5. On the other hand, if the pass / fail judgment result JR is a judgment result of "NG" and the abnormality evaluation value EV is not equal to or less than the predetermined lower threshold (NO in step S4), a discrepancy does not occur between the evaluation result of the evaluation unit 213 and the pass / fail judgment result JR of the first operator OP1. In such a case, in step S10, the monitoring unit 332 registers the pass / fail judgment result JR input to the judgment input unit 32 in step S2 as the final secondary judgment result.

[0052] After the monitoring unit 332 outputs the re-evaluation notification information RJI in step S5, the evaluation input unit 32 accepts input of the correction information RI in step S6, or accepts input of the erroneous evaluation information MEI in step S7.

[0053] If correction information RI is input to the judgment input unit 32 in step S6, the monitoring unit 332 registers the pass / fail judgment result JR after correction in accordance with the correction information RI as the final secondary judgment result in step S10.

[0054] If erroneous evaluation information MEI is input to the judgment input unit 32 in step S7, the monitoring unit 332 records, in step S8, the board image SI included in the judgment reference data JD displayed on the judgment display unit 31 in step S1 as an erroneously evaluated board image MSI to which an erroneous evaluation label MEL has been assigned. In step S9 following step S8, if the number of recorded erroneously evaluated board images MSI exceeds a predetermined number, the monitoring unit 332 outputs model update notification information MUI to prompt an update of the inference model IM.

[0055] [Regarding the Selection Terminal] The selection terminal 23 will be described in detail with reference to FIG. 7 in addition to FIG. 3 . FIG. 7 is a schematic diagram for explaining the processing of the selection terminal 23. As described above, the selection terminal 23 is a terminal operated by an operator OP, either the first operator OP1 or the second operator OP2, when selecting a board image SI to be added to a dataset DS to be learned by the learning unit 212 in the AI ​​station 2. The selection terminal 23 allows the operator OP to arbitrarily select which board image SI to add as a dataset DS for generating an inference model IM by the learning unit 212. The selection terminal 23 includes a feature display unit 231, a selection input unit 232, and a selection processing unit 233.

[0056] The feature display unit 231 is configured with a display capable of displaying various types of data. The feature display unit 231 displays a feature map FM included in the feature data FD calculated by the feature calculation unit 211. The operator OP can select a board image SI to be added to the data set DS while visually checking the feature map FM displayed on the feature display unit 231.

[0057] The selection input unit 232 is configured with a keyboard or the like that accepts input operations from the operator OP. The selection input unit 232 accepts input of the selection of the substrate image SI to be added to the data set DS in a state in which the feature amount map FM is displayed in the feature display unit 231.

[0058] The selection processing unit 233 is configured by, for example, a processor capable of information processing. The selection processing unit 233 functionally includes a feature display control unit 2331 and a selection management unit 2332.

[0059] The feature display control unit 2331 controls the feature display unit 231. The feature display control unit 2331 displays a feature map FM on the feature display unit 231. As shown in Fig. 7 , the feature map FM includes an existing feature plot PG that shows feature amounts related to a board image SI that has been determined to be a non-defective product based on an existing inference model IM. The feature map FM also displays the boundary line of a non-defective product distribution model M that reflects the existing inference model IM and that covers the existing feature plot PG.

[0060] When model update notification information MUI is output from the monitoring unit 332 of the judgment terminal 30, the feature display control unit 2331 displays a feature map FM on the feature display unit 231 in which the mis-evaluation feature plots PT1, PT2, and PT3 showing the features related to the mis-evaluated substrate image MSI are distinguished from the existing feature plots PG related to other substrate images SI other than the mis-evaluated substrate image MSI.

[0061] When the feature map FM including the erroneous evaluation feature plots PT1, PT2, and PT3 for the erroneous evaluation board image MSI is displayed on the feature display unit 231, the selection input unit 232 accepts input of a selection command SC indicating whether or not to add the erroneous evaluation board image MSI to the data set DS. The selection management unit 2332 manages the selection command SC input to the selection input unit 232.

[0062] As described above, in the selection terminal 23, when model update notification information MUI is output from the monitoring unit 332 of the determination terminal 30, a feature value map FM in which the erroneous evaluation feature value plots PT1, PT2, PT3 for the erroneous evaluation board image MSI are distinguished from the existing feature value plots PG for the other board images SI is displayed on the feature display unit 231. By visually checking the feature value map FM displayed on the feature display unit 231, the operator OP can easily select, by inputting a selection command SC to the selection input unit 232, whether to add the erroneous evaluation board image MSI to the dataset DS that is the target of machine learning by the learning unit 212.

[0063] When a selection command SC for adding an erroneously evaluated board image MSI to the data set DS is input to the selection input unit 232, the selection management unit 2332 inputs the selection command SC to the AI ​​processing device 21. When the selection command SC is input to the AI ​​processing device 21, the learning unit 212 updates the inference model IM by machine learning using the data set DS to which the erroneously evaluated board image MSI has been added. This makes it possible to update the inference model IM to one with improved accuracy in anomaly evaluation.

[0064] [Inventions Included in the Above-Described Embodiments] The above-described embodiments include inventions having the following configurations.

[0065] An inspection system according to one aspect of the present invention is an inspection system for a mounting board on which components are mounted, and includes: a learning unit that generates an inference model for evaluating an abnormality of the mounting board based on the board image by machine learning using a data set that includes a pass / fail judgment result indicating whether the mounting board is a good or defective product and a board image of the mounting board; an evaluation unit that uses the inference model to evaluate an abnormality of the mounting board based on the board image and derives a numerical abnormality evaluation value that becomes larger the higher the degree of abnormality of the mounting board; a judgment display unit that displays judgment reference data including the board image and the abnormality evaluation value; a judgment input unit that accepts input of the pass / fail judgment result of the mounting board using the judgment reference data; and a monitoring unit that monitors the abnormality evaluation value included in the judgment reference data and the pass / fail judgment result input to the judgment input unit. The monitoring unit outputs re-evaluation notification information prompting a re-evaluation using the judgment reference data when the abnormality evaluation value is above a predetermined upper threshold and the pass / fail judgment result indicates the product is good, or when the abnormality evaluation value is below a predetermined lower threshold and the pass / fail judgment result indicates the product is defective.

[0066] According to this inspection system, the judgment display unit displays judgment reference data including an abnormality evaluation value of the mounting board derived by the evaluation unit using an inference model generated by machine learning in the learning unit, and a board image. In this case, an operator can visually check the judgment reference data displayed on the judgment display unit and make a pass / fail judgment of whether the mounting board is a good or defective product based on the board image while referring to the abnormality evaluation value based on the inference model. The operator's pass / fail judgment result for the mounting board is input to the judgment input unit. The abnormality evaluation value included in the judgment reference data and the pass / fail judgment result input to the judgment input unit are then monitored by a monitoring unit.

[0067] Since the anomaly evaluation value indicates a larger value as the degree of anomaly of the mounting board increases, when the anomaly evaluation value is equal to or greater than a predetermined upper threshold, the evaluation unit evaluates the mounting board as a defective product with an abnormality. When the pass / fail judgment result input to the judgment input unit for such an evaluation result by the evaluation unit indicates a pass / fail product, a discrepancy occurs between the evaluation result by the evaluation unit using the inference model and the pass / fail judgment result by the operator. Also, when the anomaly evaluation value is equal to or less than a predetermined lower threshold, the evaluation unit evaluates the mounting board as a non-fail product with no abnormalities. When the pass / fail judgment result input to the judgment input unit for such an evaluation result by the evaluation unit indicates a failing product, a discrepancy occurs between the evaluation result by the evaluation unit using the inference model and the pass / fail judgment result by the operator.

[0068] As described above, when a discrepancy occurs between the evaluation result of the evaluation unit and the pass / fail judgment result of the operator, the monitoring unit outputs re-judgment notification information that prompts a re-judgment using the judgment reference data. When the re-judgment notification information is output from the monitoring unit, for example, a manager or other person higher in rank than the operator who made the pass / fail judgment can efficiently determine whether the accuracy of the anomaly evaluation using the inference model by the evaluation unit is low or whether the operator's pass / fail judgment is incorrect. In this case, the manager or other person can update the inference model to improve the accuracy of the anomaly evaluation or take measures to correct the pass / fail judgment result. This enables the pass / fail judgment of the mounted board to be performed with high accuracy.

[0069] In the above inspection system, the judgment input unit may, after outputting the re-judgment notification information from the monitoring unit, accept input of correction information for correcting the pass / fail judgment result for the judgment reference data, or accept input of erroneous evaluation information indicating that the abnormality evaluation value included in the judgment reference data is incorrect. When the erroneous evaluation information is input to the judgment input unit, the monitoring unit records the board image included in the judgment reference data as an erroneously evaluated board image to which an erroneous evaluation label has been assigned.

[0070] In this aspect, after the monitoring unit outputs the re-evaluation notification information, correction information or erroneous evaluation information is input to the judgment input unit. When correction information is input to the judgment input unit, the pass / fail judgment result for the judgment reference data can be corrected in accordance with the correction information. On the other hand, when erroneous evaluation information is input to the judgment input unit, the monitoring unit records the board image included in the judgment reference data as an erroneously evaluated board image to which an erroneous evaluation label has been assigned. Such erroneously evaluated board images can be used to update the inference model to improve the accuracy of abnormality evaluation.

[0071] In the above inspection system, the monitoring unit may output model update notification information prompting an update of the inference model when the number of recorded erroneously evaluated board images exceeds a predetermined number.

[0072] In this aspect, the timing when the model update notification information is output from the monitoring unit can be used as the timing for updating the inference model.

[0073] The inspection system may further include a feature calculation unit that extracts features of the mounted board based on the board image and generates a feature map that maps the feature, a feature display unit that displays the feature map, and a feature display control unit that controls the feature display unit, and when the model update notification information is output from the monitoring unit, the feature display control unit causes the feature display unit to display the feature map in an aspect in which the feature related to the erroneous evaluation board image is distinguished from the feature related to other board images other than the erroneous evaluation board image.

[0074] The inspection system may further include a selection input unit that accepts a selection input as to whether or not to add the erroneously evaluated substrate image to the data set while the feature map is displayed on the feature display unit.

[0075] In this aspect, when model update notification information is output from the monitoring unit, a feature amount map in which the feature amounts related to the erroneously evaluated board image are distinguished from the feature amounts related to the other board images is displayed on the feature display unit. By visually checking the feature amount map displayed on the feature display unit, the operator can easily select, by inputting to the selection input unit, whether to add the erroneously evaluated board image to the data set that is the target of machine learning by the learning unit.

[0076] In the above-mentioned inspection system, when an input to add the incorrectly evaluated substrate image to the dataset is made to the selection input unit, the learning unit may update the inference model by machine learning using the dataset to which the incorrectly evaluated substrate image has been added.

[0077] In this aspect, when an input for adding an erroneously evaluated board image to a dataset is made to the selection input unit, the learning unit updates the inference model by machine learning using the dataset to which the erroneously evaluated board image has been added, thereby enabling the inference model to be updated to one with improved accuracy in anomaly evaluation.

[0078] According to the present invention as described above, it is possible to provide an inspection system that can accurately determine whether a mounting board is good or bad.

Claims

1. An inspection system for a mounting board on which components are mounted, comprising: a learning unit that generates an inference model for evaluating an abnormality of the mounting board based on a board image by machine learning using a data set consisting of a pass / fail judgment result as to whether the mounting board is a good or defective product and a board image of the mounting board; an evaluation unit that uses the inference model to evaluate an abnormality of the mounting board based on the board image and derives a numerical abnormality evaluation value that becomes larger the higher the degree of abnormality of the mounting board; a judgment display unit that displays judgment reference data including the board image and the abnormality evaluation value; a judgment input unit that receives input of the pass / fail judgment result of the mounting board using the judgment reference data; and a monitoring unit that monitors the abnormality evaluation value included in the judgment reference data and the pass / fail judgment result input to the judgment input unit, The monitoring unit outputs re-evaluation notification information to prompt a re-evaluation using the judgment reference data when the abnormality evaluation value is equal to or greater than a predetermined upper threshold and the pass / fail judgment result indicates the good product, or when the abnormality evaluation value is equal to or less than a predetermined lower threshold and the pass / fail judgment result indicates the defective product.

2. The inspection system described in claim 1, wherein, after the re-evaluation notification information is output from the monitoring unit, the judgment input unit accepts input of correction information to correct the pass / fail judgment result for the judgment reference data, or accepts input of erroneous evaluation information indicating that there is an error in the abnormality evaluation value included in the judgment reference data, and when the erroneous evaluation information is input to the judgment input unit, the monitoring unit records the board image included in the judgment reference data as an erroneous evaluation board image with an erroneous evaluation label attached.

3. The inspection system according to claim 2, wherein the monitoring unit outputs model update notification information prompting an update of the inference model when the number of recorded erroneously evaluated board images exceeds a predetermined number.

4. The inspection system of claim 3, further comprising: a feature calculation unit that extracts features of the mounted board based on the board image and generates a feature map that maps the feature values; a feature display unit that displays the feature map; and a feature display control unit that controls the feature display unit, wherein when the model update notification information is output from the monitoring unit, the feature display control unit causes the feature display unit to display the feature map in a manner in which the feature values ​​related to the erroneous evaluation board image are distinguished from the feature values ​​related to other board images other than the erroneous evaluation board image.

5. The inspection system according to claim 4, further comprising a selection input section that accepts a selection input as to whether or not to add the erroneously evaluated board image to the data set while the feature map is displayed in the feature display section.

6. The inspection system described in claim 5, wherein the learning unit updates the inference model by machine learning using the dataset to which the incorrectly evaluated substrate image has been added when an input to add the incorrectly evaluated substrate image to the dataset is made to the selection input unit.

Images