Quality evaluation equipment and inspection management system

Abstract

To grasp a change in the quality of an object to be inspected and encourage re-learning of a learned learner through supervised machine learning used for inspection processing to achieve inspection with high accuracy.SOLUTION: A quality evaluation device comprises: a non-defective article image acquisition unit that acquires a non-defective article image being an inspection image determined to be a non-defective article in inspection processing using a learned first learner generated through machine learning with the inspection image created by picking up an image of an object to be inspected as learning data and an inspection result as training data; a quality change evaluation unit that evaluates a change in the quality of the object to be inspected; and a quality evaluation information generation unit that generates quality evaluation information including a result of the evaluation.SELECTED DRAWING: Figure 3

Description

[Technical field]

[0001] The present invention relates to a quality evaluation device and an inspection management system for evaluating the quality of an inspection object such as a printed circuit board. [Background technology]

[0002] In the field of FA (Factory Automation), a process is realized in which an image of an object to be inspected, such as a workpiece, is captured and input into a pre-trained supervised AI model to inspect the quality of the workpiece. The quality of the workpieces mass-produced at production sites is constantly changing, and as the change becomes greater, the supervised AI model will no longer be able to make correct judgments, increasing the risk of overlooking or overlooking. Therefore, it is necessary to retrain the supervised AI model using newly acquired learning data and use the most appropriate supervised AI model according to the change in quality. For example, if the workpiece is a printed circuit board, examples of the quality of the workpiece include the appearance of the solder and the results of visual inspection of the component positions, etc.

[0003] For example, in Patent Document 1, a model for image judgment is trained on new images of good and defective products, making it possible to generate a model that can appropriately judge data with new features that have not been seen before.

[0004] However, because the characteristics of a workpiece can change due to various factors such as the temperature at the time, variations in the material, and the condition of the inspection equipment and the equipment used in the upstream process, it is difficult for users to determine when the characteristics of the workpiece will change and when the supervised AI model should be retrained. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] JP 2020-107104 A Summary of the Invention [Problem to be solved by the invention]

[0006] The present invention has been made in consideration of the above-mentioned problems, and the problem that the present invention aims to solve is to realize highly reliable inspection by capturing changes in the quality of the object to be inspected and encouraging re-learning of a trained learning device using supervised machine learning that is used in the inspection process. [Means for solving the problem]

[0007] To solve the above problems, the present invention provides: a good-quality image acquiring unit that acquires a good-quality image, which is an inspection image determined to be a good-quality image in an inspection process using a trained first learning device generated by machine learning using an inspection image generated by capturing an image of an inspection object as learning data and the inspection results as teacher data; a quality change evaluation unit for evaluating a change in the quality of the inspection object; a quality evaluation information generating unit that generates quality evaluation information including a result of the evaluation; The quality evaluation device is provided with the above.

[0008] According to this, the user can recognize the change in the quality of the inspection object by the quality evaluation information including the result of the evaluation by the quality change evaluation unit that evaluates the change in the quality of the inspection object. The user can grasp the change in the quality of the inspection object by the quality evaluation information and determine the need for re-learning the first learning device used in the inspection process. This encourages re-learning and enables highly reliable testing. Here, the good product images include inspection images generated by imaging an inspection object that has been determined to be good by the trained first learning device, and inspection images generated by imaging an inspection object that has been determined to be defective by the trained first learning device but is subsequently determined to be good in a visual inspection process.

[0009] In the present invention, The quality change evaluation unit a quality evaluation unit that outputs a quality evaluation index for evaluating the quality based on the non-defective product image; a quality evaluation index storage unit that stores the quality evaluation index; a change determination unit that determines a change in the quality based on a change over a predetermined period of time of the quality evaluation index stored in the quality evaluation index storage unit; The above formula may be used.

[0010] According to this, based on the change over a specified period of time in the quality evaluation index that changes over time, it is possible to grasp the change in the quality of the object to be inspected and determine the need for re-learning of the first learning device used in the inspection process, thereby encouraging the user to re-learn the first learning device and achieving highly reliable inspection.

[0011] In the present invention, the quality evaluation index is an anomaly degree output by a trained second learning device generated by machine learning through unsupervised learning using the non-defective product image as training data; The change determination unit may calculate an average value and a standard deviation of the degree of abnormality over a predetermined period of time, and compare the average value and the standard deviation with a first threshold and a second threshold, respectively, to determine the change in the quality.

[0012] According to this, the degree of abnormality output by the trained second learning device based on images of good products can be used to capture changes in the quality of the inspection object and determine the need for re-learning of the first learning device used in the inspection process, thereby encouraging the user to re-learn the first learning device and achieving highly reliable inspection.

[0013] The device may further include a setting unit that automatically sets at least one of the first threshold value and the second threshold value.

[0014] According to this, the first threshold value and the second threshold value are automatically set by the setting unit, so that the user does not need to set them, and appropriate setting using machine learning or the like is possible.

[0015] In the present invention, The quality change evaluation unit Clustering may be performed using a trained third learning device generated by unsupervised machine learning using the good product images as training data, and the number of outliers may be compared with a third threshold to evaluate the change in quality.

[0016] According to this, by using clustering performed by the trained third learning device based on images of good products, changes in the quality of the object to be inspected can be captured and the need for re-learning of the first learning device used in the inspection process can be determined, thereby encouraging the user to re-learn the first learning device and achieving highly reliable inspection.

[0017] In the present invention, The device may further include a third threshold setting unit that automatically sets the third threshold.

[0018] Since the third threshold is automatically set by the third threshold setting unit, the user does not need to set it, and appropriate setting using machine learning or the like becomes possible.

[0019] In the present invention, The quality evaluation information may include information recommending relearning of the first learning device.

[0020] This allows the user to recognize the results of the quality evaluation of the test object, and the information recommending re-learning of the first learning device directly urges the user to re-learn the first learning device, which can lead to more reliable re-learning of the first learning device and enable highly reliable testing.

[0021] In the present invention, The quality evaluation information may include information recommending improvements to a front-end process related to the inspection target.

[0022] This allows the user to recognize the evaluation results of the quality of the inspection object and is also recommended to improve the upstream process related to the inspection object, thereby enabling improvement in the quality of the inspection object to be achieved by improving the upstream process.

[0023] In the present invention, The apparatus may further include a display unit for displaying the quality evaluation information.

[0024] This allows the user to visually recognize the contents of the quality evaluation information via the display unit.

[0025] The present invention also provides a method for producing a semiconductor device comprising the steps of: The quality evaluation device; an inspection device including an inspection processing unit that performs an inspection process on the inspection object using the trained first learning device, and a storage unit that stores the inspection image provided to the inspection process and the inspection result that is a result of the inspection process; It is an inspection management system including:

[0026] According to this, The quality evaluation information from the quality evaluation device allows the user to grasp changes in the quality of the object to be inspected and determine the need for re-learning the first learning device used in the inspection process, thereby making it possible to configure an inspection management system that encourages the user to re-learn the first learning device and achieves highly reliable inspections.

[0027] In the present invention, The system may include a learning device that retrains the first learning device.

[0028] According to this, by re-learning the first learning device at an appropriate timing prompted by the quality evaluation information, it is possible to configure an inspection management system that can realize highly reliable inspection. Effect of the Invention

[0029] According to the present invention, by capturing changes in the quality of an object to be inspected and encouraging re-learning of a trained learning device using supervised machine learning used in the inspection process, highly reliable inspection can be achieved. [Brief description of the drawings]

[0030] [Figure 1] 1 is a diagram showing a schematic configuration of a manufacturing facility according to a first embodiment of the present invention. [Diagram 2] 1 is a functional block diagram of an inspection device according to a first embodiment of the present invention. [Diagram 3] FIG. 2 is a functional block diagram of a management device according to the first embodiment of the present invention. [Figure 4] FIG. 4 is a diagram showing an example of quality evaluation information according to the first embodiment of the present invention. [Diagram 5] FIG. 11 is a diagram showing another example of the quality evaluation information according to the first embodiment of the present invention. [Figure 6] FIG. 11 is a diagram showing another example of the quality evaluation information according to the first embodiment of the present invention. [Figure 7] FIG. 11 is a diagram showing another example of the quality evaluation information according to the first embodiment of the present invention. [Figure 8] FIG. 1 is a functional block diagram of a supervised learning device according to a first embodiment of the present invention. [Figure 9] FIG. 1 is a functional block diagram of an unsupervised learning device according to a first embodiment of the present invention. [Figure 10] FIG. 11 is a functional block diagram of a management device according to a second embodiment of the present invention. [Figure 11] FIG. 11 is a functional block diagram of an unsupervised learning device according to a second embodiment of the present invention. [Figure 12] FIG. 11 is a diagram illustrating a quality evaluation according to the second embodiment of the present invention. [Figure 13] FIG. 11 is a diagram showing an example of quality evaluation information according to Application Example 2 of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0031] [Application example] Hereinafter, application examples of the present invention will be described with reference to the drawings.

[0032] Fig. 1 shows a schematic configuration of a management system 1 including a management device 10 to which the present invention is applied. In Fig. 1, the management system 1 includes manufacturing equipment in a surface mounting line for printed circuit boards, that is, a solder printing device X1, a mounter X2, and a reflow furnace X3, and inspection equipment, that is, a solder printing inspection device Y1, a component inspection device Y2, an appearance inspection device Y3, and an X-ray inspection device Y4.

[0033] The function of the visual inspection device Y3 regarding the inspection process is realized by the camera 31, the information processing device 32, and the display device 33 as shown in FIG. 2. Here, the inspection program read from the inspection program storage unit 324 is executed by the CPU, and the inspection unit 323 performs the inspection process on the inspection image. As the inspection program, a learning device 215 (see FIG. 8) that has been machine-learned by supervised learning using the inspection image as learning data and the inspection result as teacher data is used. Since the quality of the mass-produced printed circuit boards to be inspected is constantly changing, if the change in quality becomes large, the learning device using supervised learning cannot correctly judge whether the board is good or bad, and the risk of overlooking or overlooking increases. For this reason, it is desirable to re-learn the learning device using newly obtained inspection images and perform the inspection using an inspection program that corresponds to the changing quality. The inspection unit 323 corresponds to the inspection processing unit of the present invention.

[0034] In the management device 10 shown in Fig. 3, the quality evaluation unit 103 performs quality evaluation using a quality evaluation program on non-defective images, which are inspection images of inspection objects determined to be non-defective among the inspection images acquired from the inspection device. Furthermore, the quality change determination unit 105 evaluates the change in quality of the inspection object based on the accumulated quality evaluation results. Then, the quality evaluation information generation unit 106 generates quality evaluation information including the evaluation results regarding the quality change, and causes the display unit 107 to display it.

[0035] Specifically, the quality evaluation program is a learning device that is trained by unsupervised learning using a learning device 22 as shown in FIG. 9, and is an unsupervised AI model such as a VAE that uses images of good products for learning as learning data.

[0036] The quality evaluation information 71 includes, for example, information as shown in Fig. 4. The control chart 711 plots the average value of the degree of abnormality over a predetermined period of time, which is output by inputting images of non-defective products, which are mass production data, into a quality evaluation program, and shows it with a dashed line, and the degree of abnormality over a predetermined period of time with a solid vertical bar. It shows the standard deviation. The threshold value (degree of abnormality) displayed in the display area 712 of the quality evaluation information 71 and the threshold value (variation) displayed in the display area 714 are set to values ​​of "0.5" and "0.4", respectively. In the control chart 711, the degree of abnormality value exceeds the threshold value (degree of abnormality) shown by the solid line parallel to the horizontal axis, and the standard deviation shown by the vertical bar also exceeds the threshold value (variation).

[0037] The quality evaluation information 71 displays an overall evaluation of the quality of the inspection target based on the set threshold (degree of abnormality) and threshold (variation) and the change in the degree of abnormality shown in the control chart 711. Here, the message "There is variation and change in quality. We recommend reviewing the process and re-learning the AI ​​model" is displayed. This allows the user to recognize that there is a large change in the quality of mass-produced printed circuit boards and that it is necessary to review the process and re-learn the learning device of the inspection program. Therefore, by re-learning the learning device of the inspection program in a timely manner, highly reliable inspection is possible. Here, the above message corresponds to the information of the present invention recommending improvement of the front-end process related to the inspection target.

[0038] Example 1 The management system 1 according to the first embodiment of the present invention will be described in more detail below with reference to the drawings. (System Configuration) 1 is a schematic diagram showing an example of the configuration of manufacturing equipment in a surface mounting line for printed circuit boards according to this embodiment. Surface mounting technology (SMT) is a technology for soldering electronic components to the surface of a printed circuit board, and the surface mounting line mainly consists of three processes: solder printing, component mounting, and reflow (solder deposition).

[0039] As shown in FIG. 1, the surface mounting line is provided with, in order from the upstream side, a solder printing device X1, a mounter X2, and a reflow furnace X3 as manufacturing equipment. The solder printing device X1 is a device that prints solder paste on electrode portions (called lands) on a printed circuit board by screen printing. The mounter X2 is a device that picks up electronic components to be mounted on the board and places the components on the solder paste at the corresponding locations, and is also called a chip mounter. The reflow furnace X3 is a heating device that heats and melts the solder paste, cools it, and solders the electronic components onto the board. When the number and types of electronic components to be mounted on the board are large, multiple mounters X2 may be provided on the surface mounting line.

[0040] In addition, the surface mounting line is equipped with a system that inspects the condition of the board at the exit of each process, from solder printing to component mounting to reflow, and automatically detects defects or potential defects. In addition to automatically separating good and bad products, the system also has the function of feeding back the results of inspection and analysis to the operation of each manufacturing device (for example, changing the mounting program).

[0041] The solder print inspection device Y1 is a device for inspecting the printing condition of the solder paste on the board discharged from the solder printing device X1. The solder print inspection device Y1 measures the solder paste printed on the board in two or three dimensions, and judges whether or not various inspection items are normal (within the allowable range) based on the measurement results. Inspection items include, for example, the volume, area, height, positional deviation, and shape of the solder. An image sensor (camera) can be used for two-dimensional measurement of the solder paste, and a laser displacement meter, phase shift method, spatial coding method, light section method, etc. can be used for three-dimensional measurement.

[0042] The component inspection device Y2 is a device for inspecting the placement of electronic components on the board carried out from the mounter X2. This method measures the product (the component body, or a part of the component such as an electrode) in two or three dimensions, and judges whether various inspection items are within normal values ​​(tolerance ranges) based on the measurement results. Inspection items include, for example, misalignment of components, misalignment of angles (rotation), missing components (a component not placed), wrong components (different components placed), wrong polarity (different polarity of electrodes on the component side and board side), inversion (a component placed upside down), component height, etc. As with solder printing inspection, image sensors (cameras) can be used for two-dimensional measurement of electronic components, and laser displacement meters, phase shifting method, spatial coding method, light section method, etc. can be used for three-dimensional measurement.

[0043] The visual inspection device Y3 is a device for inspecting the quality of soldering on the boards removed from the reflow furnace X3. The visual inspection device Y3 measures the soldered part after reflow in two or three dimensions, and judges whether various inspection items are normal (within the allowable range) based on the measurement results. Inspection items include the same items as in component inspection, as well as the quality of the solder fillet shape. In addition to the above-mentioned laser displacement meter, phase shift method, spatial coding method, and light section method, the so-called color highlight method (a method in which R, G, and B lighting are applied to the solder surface at different angles of incidence, and the reflected light of each color is photographed with a zenith camera to detect the 3D shape of the solder as 2D hue information) can be used to measure the solder shape.

[0044] 2 is a block diagram showing a schematic configuration of the inspection processing functions of the appearance inspection device Y3. The inspection processing functions of the appearance inspection device Y3 are mainly realized by a camera 31, an information processing device 32, and a display device 33. The information processing device 32 is configured by a general-purpose computer system equipped with a CPU (processor), a main storage device (memory), an auxiliary storage device (hard disk, etc.), an input device (keyboard, mouse, controller, touch panel, etc.), etc.

[0045] The image acquisition unit 321 acquires an image of the substrate to be inspected taken by the camera 31. The inspection image generation unit 322 performs a predetermined process on the image acquired by the image acquisition unit 321 to generate an inspection image. The inspection unit 323 executes an inspection program to measure (calculate) a predetermined index based on the inspection image, and uses these measurement values ​​to inspect the state of the inspection object and judge whether it is good or bad. Here, the inspection program executed is a program including a learning device learned by supervised learning. The inspection program storage unit 324 stores the inspection program executed by the inspection unit 323. The result output unit 325 outputs the inspection result by the inspection unit 323 to a screen and displays it on the display device 33. The inspection result storage unit 326 stores the inspection image and the inspection result in association with each other. The communication interface 327 is an interface for communicating with the management device 10, the program management server 20, etc. connected via a network (LAN). Here, the inspection images and inspection results stored in the inspection result memory unit 326 also include those for inspection images that were judged to be defective by the inspection program but were judged to be good in a subsequent visual inspection process (over-detected inspection images).

[0046] The inspection images and inspection results stored in the inspection result storage unit 326 are transmitted to the management device 10 via the communication interface 327. The inspection program is re-learned by the program management server 20, and transmitted from the program management server 20 to the inspection program storage unit 324 via the communication interface 327 and stored therein. Here, an overview of the functions related to the inspection process has been described for the appearance inspection device Y3, but the other inspection devices have similar functional configurations.

[0047] The X-ray inspection device Y4 is a device for inspecting the state of soldering on a circuit board using X-ray images. For example, in the case of packaged components such as BGA (Ball Grid Array) and CSP (Chip Size Package) and multi-layer circuit boards, the solder joints are Since the solder is hidden under the board or the like, the appearance inspection device Y3 (i.e., the appearance image) cannot inspect the state of the solder. The X-ray inspection device Y4 is a device to supplement such weaknesses of appearance inspection. Items inspected by the X-ray inspection device Y4 include, for example, component positional deviation, solder height, solder volume, solder ball diameter, back fillet length, and the quality of the solder joint. Note that as the X-ray image, an X-ray transmission image may be used, and it is also preferable to use a CT (Computed Tomography) image.

[0048] (Management device) The manufacturing devices X1 to X3 and the inspection devices Y1 to Y4 described above are connected to a management device 10 via a network (LAN). The management device 10 is a system that manages and controls the manufacturing devices X1 to X3 and the inspection devices Y1 to Y4, and is configured by a general-purpose computer system including a CPU (processor), a main storage device (memory), an auxiliary storage device (hard disk, etc.), an input device (keyboard, mouse, controller, touch panel, etc.), a display device, etc., although not shown. The functions of the management device 10 described below are realized by the CPU reading and executing a program stored in the auxiliary storage device.

[0049] The management device 10 may be configured with one computer or multiple computers. Alternatively, all or part of the functions of the management device 10 may be implemented in a computer built into any of the manufacturing devices X1 to X3 or the inspection devices Y1 to Y4. Alternatively, part of the functions of the management device 10 may be realized by a server on a network (such as a cloud server).

[0050] A program management server 20 is connected to the management device 10 via a network (LAN). The program management server 20 is a server that manages the inspection program and the quality evaluation program. The program management server 20 is configured by a general-purpose computer system equipped with a CPU (processor), a main storage device (memory), an auxiliary storage device (hard disk, etc.), an input device (keyboard, mouse, controller, touch panel, etc.), a display device, etc. The inspection program is a program that realizes the inspection process in the inspection devices Y1 to Y4, and is stored in a predetermined storage area of ​​the program management server 20, while being downloaded to each of the inspection devices Y1 to Y4 as necessary, stored in a predetermined storage area of ​​each device, and executed in each device. The quality evaluation program is a program that realizes the quality evaluation process in the management device 10, and is stored in a predetermined storage area of ​​the program management server 20, while being downloaded to the management device 10 as necessary, stored in a quality evaluation program storage unit 109, and executed in the management device. The program management server 20 may manage various programs that control not only the inspection program and the quality evaluation program but also the manufacturing devices X1 to X3 and the inspection devices Y1 to Y4.

[0051] The management device 10 of this embodiment has a functional unit for realizing a function that enables a manager of a manufacturing facility to efficiently perform maintenance and quality control of the facility. Fig. 2 shows a block diagram of each functional unit related to quality evaluation possessed by the management device 10. Here, the management device 10 corresponds to the quality evaluation device of the present invention, and the management system 1 corresponds to the inspection management system of the present invention.

[0052] As shown in Fig. 3, the management device 10 has an inspection image / inspection result acquisition unit 101, an inspection image / inspection result database (DB), a quality evaluation unit 103, an evaluation result accumulation unit 104, a quality change determination unit 105, a quality evaluation information generation unit 106, a display unit 107, an input unit 108, and a quality evaluation program storage unit 109. Here, the quality evaluation unit 103, the evaluation result accumulation unit 104, and the quality change determination unit 105 correspond to the quality change evaluation unit of the present invention. Also, the quality evaluation unit 103, the evaluation result accumulation unit 104, and the quality change determination unit 105 correspond to the quality evaluation unit, the quality evaluation accumulation unit, and the change determination unit of the present invention, respectively. Also, the quality evaluation information generation unit 106 corresponds to the quality evaluation information generation unit of the present invention.

[0053] The inspection image and inspection result acquisition unit 101 acquires the inspection images by the inspection devices Y1 to Y4 and the inspection results associated therewith from the inspection devices Y1 to Y4. The inspection images and inspection results acquired by the inspection image and inspection result acquisition unit 101 are stored in an inspection image and inspection result DB. Here, the inspection images are also associated with information such as the inspection date and time and the line-symmetric product number, and this information is also acquired and stored.

[0054] The quality evaluation unit 103 has a function of evaluating the quality of the printed circuit board by the output obtained by inputting an inspection image to a trained learning device such as an autoencoder that has been trained by unsupervised learning. Here, what is input to the quality evaluation unit 103 is an inspection image of a printed circuit board that has been determined to be a good product among the inspection images recorded in the inspection image / inspection result DB 102. Note that the inspection images that have been determined to be good products also include inspection images of printed circuit boards that have been determined to be defective by the inspection program in the inspection device but have been determined to be good products in a subsequent visual inspection process. The trained learning device used here outputs a one-dimensional numerical value, which is called an abnormality degree. The inspection image used for evaluating the quality can be sampled as appropriate. For example, the sampling results once per hour are accumulated for a predetermined period of time, and the abnormality degree is calculated. Here, the abnormality degree corresponds to the quality evaluation index of the present invention.

[0055] The degree of abnormality calculated in the quality evaluation unit 103 is stored in the evaluation result storage unit 104. Here, the evaluation result storage unit 104 corresponds to the quality evaluation index storage unit of the present invention.

[0056] The quality change determination unit 105 analyzes the change over time in the degree of abnormality accumulated in the evaluation result accumulation unit 104, and evaluates the change in the quality of the printed circuit board. Specifically, the change in the degree of abnormality is compared with a set threshold value to make a judgment. As described above, the average value of the degrees of abnormality accumulated over a predetermined period is taken as the degree of abnormality for that period, and the standard deviation is taken as the variation in the degree of abnormality. Here, two threshold values ​​are set: a threshold value (degree of abnormality) related to the value of the degree of abnormality, and a threshold value (variation) related to the variation in the degree of abnormality. The threshold value (degree of abnormality) and the threshold value (variation) are input in advance by the user via the input unit 108. The threshold value (degree of abnormality) and the threshold value (variation) correspond to the first threshold value and the second threshold value of the present invention, respectively.

[0057] The quality evaluation information generating unit 106 generates quality evaluation information based on the evaluation result by the quality change determining unit 105, and causes the display unit 107 to display the information. 4 shows an example of display of the quality evaluation information 71. Frames and leader lines indicating each piece of information in the quality evaluation information 71 are displayed with dotted lines to distinguish them from the elements of the quality evaluation information 71.

[0058] A control chart 711 is displayed in the center of the quality evaluation information 71. Here, a control chart for part number A is displayed. In the control chart 711, the horizontal axis represents the period during which the inspection images were acquired, and the vertical axis represents the degree of abnormality. The average value for the target period is plotted and shown as a dashed line for the degree of abnormality, and the standard deviation is shown as a solid vertical bar. Also, the vertical axis displays the threshold value of the degree of abnormality as a solid line.

[0059] Below the control chart 711, a threshold (degree of abnormality) display area 712 is arranged, and the currently set value "0.5" is displayed. The threshold (degree of abnormality) display area 712 is an input field, and the threshold (degree of abnormality) can be changed by inputting a numerical value and specifying the setting button 713 on the right side by clicking or the like. Below the threshold (degree of abnormality) display area 712, a threshold (variation) display area 714 is arranged, and the currently set value "0.4" is displayed. The threshold (variation) display area 712 is an input field, and the threshold (variation) can be changed by inputting a numerical value and pressing the setting button 713 on the right side by clicking or the like.

[0060] As mentioned above, the user can input and set the threshold (degree of anomaly) and threshold (variation). Alternatively, a history of the thresholds (degree of abnormality) and thresholds (variation) may be stored in a predetermined storage area, and appropriate thresholds (degree of abnormality) and thresholds (variation) may be set by the quality change determination unit 105 using machine learning or the like based on the history of the thresholds (degree of abnormality) and thresholds (variation) and the history of evaluation results stored in the evaluation result storage unit 104, or a recommended value may be taught to the user. It is also possible for the user to input only one of the thresholds (degree of abnormality) and thresholds (variation), and for the other to be automatically set or a recommended value to be taught. Here, the quality change determination unit 105 corresponds to the setting unit of the present invention.

[0061] An overall evaluation display area 716 is disposed above the control chart 711. A message regarding quality is displayed in the overall evaluation display area 716. This overall evaluation is based on the change in the degree of anomaly displayed in the control chart 711. In the control chart 711, the degree of anomaly exceeds the threshold (degree of anomaly), and the variation also exceeds the threshold (variation). For this reason, the overall evaluation display area 716 displays a message as an overall evaluation of quality, saying, "There is variation and change in quality. We recommend that you review the process and retrain the AI ​​model.", informing the user that it is time to retrain the learning device of the inspection program used in the inspection.

[0062] In this way, changes in the quality of the test object are captured, and quality evaluation information 71, etc. are displayed on the display unit 107, and a message is displayed encouraging the user to re-learn the learning device of the test program, thereby enabling a highly reliable test to be achieved.

[0063] A part number display area 717 is arranged on the left side of the quality evaluation information 71. The inspection program generates an AI model for each component type, and one component type includes multiple part numbers. Here, it is shown that the QFP AI model used for the inspection includes those for part numbers A, B, and C. When an alarm for reviewing the process or relearning the AI ​​model is issued, the part number display is displayed in a color different from white, such as red. Here, the display 717a of part number A is displayed in a different color. With such a display, the user can clearly recognize the part number that needs to be addressed. In addition, each part number display 717a, etc. is a button, and by clicking or pressing these buttons, the display switches to the control chart of the corresponding part number. In addition, by clicking or pressing the display 717b of the QFP AI model, the entire control chart can be confirmed.

[0064] A representative data display area 718 is located to the right of the quality evaluation information 71. Here, representative data for each period of the inspection images sampled by the quality change determination unit 105 is displayed in chronological order. This allows the validity of the overall evaluation to be visually confirmed. By pressing a reselect button 719 located below the representative data display area 718, for example by clicking, the representative data can be switched to another inspection image for the relevant period.

[0065] 5 shows a display example of other quality evaluation information 72. Explanation of information common to the quality evaluation information 71 will be omitted. Here, a control chart 721 and an overall evaluation display area 716 are different from the quality evaluation information 71. In the control chart 721, the degree of abnormality does not exceed the threshold (degree of abnormality), but exceeds the threshold (variation). For this reason, in the overall evaluation display area 726, a message is displayed as the overall evaluation of quality, saying "There is variation in quality. We recommend that you review the process," to notify the user that the process should be reviewed.

[0066] FIG. 6 shows a display example of other quality evaluation information 73. Explanation of information common to the quality evaluation information 71 will be omitted. Explanation of information common to the quality evaluation information 71 will be omitted. Here, a control chart 731 and an overall evaluation display area 736 are different from the quality evaluation information 71. In the control chart 731, the degree of abnormality exceeds the threshold (degree of abnormality), but does not exceed the threshold (variation). For this reason, the overall evaluation display area 736 displays the following as the overall evaluation of quality: The system then displays the message, "There has been a change in quality. We recommend retraining the AI ​​model," informing the user that it is time to retrain the inspection program's learning device.

[0067] FIG. 7 shows a display example of other quality evaluation information 74. Explanation of information common to quality evaluation information 71 will be omitted. Explanation of information common to quality evaluation information 71 will be omitted. Here, a control chart 731, an overall evaluation display area 736, and a product number display area 747 are different from the quality evaluation information 71. In the control chart 741, the degree of abnormality does not exceed the threshold (degree of abnormality) or the threshold (variation). Therefore, the overall evaluation display area 746 displays a message saying "Quality is stable" as the overall evaluation of quality. In addition, a display 747a of product number A in the product number display area 747 is displayed in white, indicating that no alarm has been issued.

[0068] In this manner, the management device 10 can instruct the user on the timing to re-learn the learning module of the inspection program.

[0069] When the quality change determination unit 105 recommends re-learning the learning device of the inspection program and the user instructs re-learning of the learning device of the inspection program, the program management server 20, which manages the inspection program, re-learns the inspection program. At this time, the program management server 20 also re-learns the learning device of the quality evaluation program. In this way, by re-learning the quality evaluation program together with the inspection program, inspection and quality evaluation that promptly respond to quality changes become possible.

[0070] Fig. 8 illustrates a learning device 21 that learns an inspection program. This learning device 21 is configured in a program management server 20. Learning device 21 includes a learning test image acquisition unit 211, a learning test result acquisition unit 212, a learning data storage unit 213, a learning processing unit 214, and a learning device 215. In the case of re-learning, the subject of the learning data is the test images and test results from the most recent period up to the time of re-learning. Here, learning device 215 corresponds to the first learning device of the present invention. Learning device 21 corresponds to the learning device of the present invention.

[0071] The learning test image acquiring unit 211 acquires the test image for re-learning from the test image and test result DB 102. The learning test result acquiring unit 212 acquires the test results of the test image from the test image and test result DB 102.

[0072] The learning test images and learning test results acquired via the learning test image acquisition unit 211 and the learning test result acquisition unit 212 are stored in the learning data memory unit 213, which is a specified area of ​​the auxiliary storage device of the program management server 20.

[0073] The learning processing unit 214 performs machine learning of a learning device so as to output a test result when a learning test image stored in the learning data storage unit 213 is input. The learning processing unit 214 is realized by the CPU of the program management server 20 reading and executing a learning model generation program stored in a predetermined area of ​​the auxiliary storage device. Here, the learning device is, for example, a program that calculates a model by a neural network using the learning test image as learning data and the learning test result as teacher data, but is not limited to this.

[0074] The machine learning of the learning device in the learning processing unit 214 is repeated for a large amount of learning data to obtain a trained learning device 215. The learning device 215 thus obtained is stored in a learning result data storage unit 216, which is a predetermined area of ​​the auxiliary storage device. The trained learning device 215 stored in the learning result data storage unit 216 is transmitted to and stored in, for example, the inspection program storage unit 324 of the inspection machine Y3.

[0075] 9 illustrates a learning device 22 that learns a quality evaluation program. This learning device 22 is configured in the program management server 20. The learning device 22 includes a learning-use good product image acquisition unit 221, a learning data storage unit 222, a learning processing unit 214, and a learning device 224. In the case of re-learning, the subject of the learning data is good product images from the most recent period up to the time of re-learning. Here, the learning device 224 corresponds to the second learning device of the present invention.

[0076] The learning-use non-defective product image acquisition unit 221 acquires, from the inspection image / inspection result DB 102, inspection images that have been determined to be non-defective products.

[0077] The learning-use good-quality images acquired via the learning-use good-quality image acquisition unit 221 are stored in a learning data storage unit 222, which is a predetermined area of ​​the auxiliary storage device of the program management server 20.

[0078] The learning processing unit 223 performs machine learning of a learning device so as to output an anomaly level when a learning image of a good product stored in the learning data storage unit 222 is input. The learning processing unit 223 is realized by the CPU of the program management server 20 reading and executing a learning model generation program stored in a predetermined area of ​​the auxiliary storage device. Here, the learning device is an unsupervised AI model such as a VAE that uses learning images of good products as learning data.

[0079] The machine learning of the learning device in the learning processing unit 223 is repeated for a large amount of learning data to obtain a trained learning device 224. The learning device 224 obtained in this manner is stored in a learning result data storage unit 225, which is a predetermined area of ​​the auxiliary storage device. The trained learning device 224 stored in the learning result data storage unit 225 is transmitted to the quality evaluation program storage unit 109 and stored therein.

[0080] Example 2 A management system 2 according to a second embodiment of the present invention will be described below. Configurations common to the first embodiment will be designated by the same reference numerals and detailed description will be omitted.

[0081] The overall configuration of the management system 2 according to the second embodiment and the functional configurations of the inspection devices Y1 to Y4 and the learning device 21 are the same as those of the first embodiment. Fig. 10 shows a block diagram of each functional unit related to quality evaluation possessed by the management device 11 according to the second embodiment. In the second embodiment, the quality evaluation program is different from that of the first embodiment, and therefore the functions of the quality evaluation unit 203, evaluation result accumulation unit 204, quality change determination unit 205 and quality evaluation information generation unit 206 of the management device 11, the configuration of the quality evaluation information, and the configuration of the learning device 23 are different from those of the first embodiment. Here, the management device 11 corresponds to the quality evaluation device of the present invention, and the management system 2 corresponds to the inspection management system of the present invention.

[0082] 11 shows a schematic functional configuration of a learning device 23 that performs machine learning on the quality evaluation program according to the second embodiment. The learning device 23 is configured in the program management server 20. The learning device 23 includes a learning-use good product image acquisition unit 231, a learning data storage unit 232, a learning processing unit 233, and a learning device 234. In the case of re-learning, the subject of the learning data is good product images from the most recent period up to the time of re-learning. Here, the learning device 234 corresponds to the third learning device of the present invention.

[0083] The learning-use non-defective product image acquisition unit 231 acquires, from the inspection image / inspection result DB 102, inspection images that have been determined to be non-defective products.

[0084] The learning-use good-quality images acquired via the learning-use good-quality image acquisition unit 231 are stored in a learning data storage unit 232 which is a predetermined area of ​​the auxiliary storage device of the program management server 20.

[0085] The learning processing unit 233 performs machine learning of a learning device by inputting a learning image of a good product stored in the learning data storage unit 232, mapping the good product image on a two-dimensional plane by dimensionally compressing the feature amount of the good product image, and clustering each mapped good product image. The learning processing unit 223 is realized by the CPU of the program management server 20 reading and executing a learning model generation program stored in a predetermined area of ​​the auxiliary storage device. Here, the learning device is an unsupervised AI model such as a principal component analysis (PCA) that uses the learning image of a good product as learning data.

[0086] Machine learning of the learner in the learning processing unit 233 is repeated for a large amount of learning data to obtain a trained learner 234. The learner 234 obtained in this manner is stored in a learning result data storage unit 235, which is a predetermined area of ​​the auxiliary storage device. The trained learner 234 stored in the learning result data storage unit 235 is transmitted to and stored in the quality evaluation program storage unit 109. Fig. 12(A) shows an example of output of the trained learner 234. Points L1, L2, etc., which represent trained images of non-defective products, are mapped onto a two-dimensional surface S, and are clustered by a circle C surrounding these points L1, etc.

[0087] In the quality evaluation unit 203 of the second embodiment, the quality evaluation program generated in this way maps the non-defective images of mass-produced data acquired from the inspection images and inspection results. FIG. 12(B) shows a two-dimensional map in which non-defective images, which are mass-produced data, are input to the trained learning device 234 and mapped onto a two-dimensional surface S. Here, non-defective images, which are training data, are displayed as black points, and non-defective images, which are mass-produced data, are displayed as gray points. In the state shown in FIG. 12(B), points P1, P2, etc., which indicate non-defective images, which are mass-produced data, are mapped onto the two-dimensional surface S, but points P1, etc., which indicate non-defective images, which are mass-produced data, are all included in the cluster indicated by a circle C1.

[0088] In the quality evaluation unit 203 of the second embodiment, as described above, the non-defective product images, which are mass-produced data acquired from the inspection image / inspection result DB, are sampled and mapped, and a two-dimensional map is output. The two-dimensional map is stored in the evaluation result storage unit 204.

[0089] In the quality change determination unit 205 of the second embodiment, the number of points indicating good quality images that are outliers outside of the clusters is counted in the two-dimensional map generated by the quality evaluation unit 203. As shown in FIG. 11B, when good quality images that are mass-produced data are sequentially mapped, data that is mapped to a position outside of cluster C as an outlier appears when the quality changes. In FIG. 12C, point O1 indicating a good quality image that is mass-produced data is mapped to a position outside of cluster C that includes point L1 indicating a good quality image for learning, mass-produced data, and the like. The quality change determination unit 205 counts the number of such outliers, compares the number of outliers with a threshold value, and judges whether the threshold value is exceeded.

[0090] In the second embodiment as well, the quality evaluation information generating unit 206 generates quality evaluation information based on the evaluation result by the quality change determining unit 205, and causes the display unit 107 to display it. 13 shows an example of display of the quality evaluation information 75 according to Example 2. Frames and leader lines indicating each piece of information in the quality evaluation information 75 are displayed with dotted lines to distinguish them from the elements of the quality evaluation information 75.

[0091] A 2D map 751 is displayed on the right side of the center of the quality evaluation information 75. The 2D map 751 also displays the product number that is the subject of the quality evaluation (here, "product number A") and the date (evaluation date) indicating when the evaluation data was collected. The evaluation date of the displayed quality evaluation information 75 can be specified by the user as appropriate. As described above, the 2D map is generated by inputting images of good products, which are mass-produced data, into the quality evaluation program, and displaying the 2D map generated from the images of good products for learning. The 2D map 751 is a map of the non-defective image data that is mapped onto a cluster. Cluster C of the 2D map 751 includes points L1 and the like that indicate non-defective images for learning, and points P1 and the like that indicate non-defective images that are mass-produced data. In the 2D map 751, four points O1, O2, O3, and O4 that indicate non-defective images that are mass-produced data are further arranged as outliers outside of cluster C. When the cursor is placed on point O4 that indicates a non-defective image that is an outlier, a pop-up is displayed with the non-defective image data corresponding to that point and the inspection date when that non-defective image Im4 was inspected.

[0092] A threshold value (number of alarms) display area 752 is arranged below the 2D map 751 of the quality evaluation information 75, and the currently set "4" is displayed. The threshold value (number of alarms) display area 752 is an input field, and the threshold value (number of alarms) can be changed by inputting a numerical value and specifying it by clicking or the like on the setting button 753 on the right side. This threshold value (number of alarms) is intended to issue an alarm recommending the user to review the process or re-learn the learning device of the inspection program when the number of outliers in the mass production data arranged on the 2D map exceeds the threshold value (number of alarms). Here, the threshold value (number of alarms) corresponds to the third threshold value of the present invention.

[0093] As described above, the threshold (number of alarms) may be input and set by the user, but the history of the threshold (degree of abnormality) and threshold (number of alarms) may be stored in a predetermined storage area, and an appropriate threshold (number of alarms) may be set by the quality change determination unit 205 through machine learning or the like based on the history of the threshold (number of alarms) and the history of the evaluation results accumulated in the evaluation result accumulation unit 204, or a recommended value may be instructed to the user. Here, the quality change determination unit 205 corresponds to the threshold setting unit of the present invention.

[0094] An overall evaluation display area 754 is disposed above the quality evaluation information 75. A message regarding quality is displayed in the overall evaluation display area 754. This overall evaluation is based on the number of outliers in the 2D map 751. As displayed in the threshold (number of alarms) display area 752, the current threshold (number of alarms) is 4. In contrast, four points O1, O2, O3, and O4 are mapped as outliers in the 2D map 751, so the number of outliers is equal to the threshold (number of alarms). Therefore, the quality change determination unit 105 evaluates that the quality has changed, and a message saying "The quality has changed. We recommend that you review the process or retrain the AI ​​model" is displayed in the overall evaluation display area 754, informing the user that they should review the process or retrain the learning device of the inspection program used in the inspection.

[0095] In this way, changes in the quality of the test object are captured, quality evaluation information 75 is displayed on display unit 107, and a message is displayed to the user encouraging re-learning of the learning device of the test program, thereby enabling highly reliable testing to be achieved.

[0096] A part number display area 755 is arranged on the left side of the quality evaluation information 75. The inspection program generates an AI model for each component type, and one component type includes multiple part numbers. Here, it is shown that the QFP AI model used for the inspection includes those for part numbers A, B, and C. When an alarm for reviewing the process or relearning the AI ​​model is issued, the part number display is displayed in a color different from white, such as red. Here, the display 755a of part number A is displayed in a different color. With such a display, the user can clearly recognize the part number that needs to be addressed. In addition, each part number display 755a, etc. is a button, and by clicking or pressing these buttons, the display switches to a 2D map of the corresponding part number. In addition, by clicking or pressing the display 755b of the QFP AI model, the entire 2D map can be confirmed.

[0097] <Appendix 1> a good-quality image acquiring unit (101) for acquiring a good-quality image, which is an inspection image determined to be a good-quality image in an inspection process using a trained first learning device (215) generated by machine learning using an inspection image generated by capturing an image of an inspection object as learning data and an inspection result as teacher data; A quality change evaluation unit (103, 104, 105) for evaluating changes in the quality of the inspection object; a quality evaluation information generating unit (106) for generating quality evaluation information including the result of the evaluation; A quality evaluation device (10) comprising: [Explanation of symbols]

[0098] 10, 11: Management device 103: Quality Evaluation Department 104: Evaluation result storage unit 105: Quality Change Evaluation Department 106: Quality evaluation information generation unit 215: Learning device

Claims

1. a good-quality image acquiring unit that acquires a good-quality image, which is an inspection image determined to be a good-quality image in an inspection process using a trained first learning device generated by machine learning the inspection results as teacher data, using an inspection image generated by imaging an inspection object as learning data; a quality change evaluation unit for evaluating a change in the quality of the inspection object; a quality evaluation information generating unit that generates quality evaluation information including a result of the evaluation; Equipped with The quality change evaluation unit a quality evaluation unit that outputs a quality evaluation index for evaluating the quality based on the non-defective product image; a quality evaluation index storage unit that stores the quality evaluation index; a change determination unit that determines a change in the quality based on a change over a predetermined period of time of the quality evaluation index stored in the quality evaluation index storage unit; A quality evaluation device comprising:

2. the quality evaluation index is an anomaly degree output by a trained second learning device generated by machine learning through unsupervised learning using the non-defective product image as training data; The quality evaluation device according to claim 1, wherein the change determination unit calculates an average value and a standard deviation of the degree of anomaly over a predetermined period of time, and compares the average value and the standard deviation with a first threshold value and a second threshold value, respectively, to determine the change in the quality.

3. 3. The quality evaluation device according to claim 2, further comprising a setting unit that automatically sets at least one of the first threshold value and the second threshold value.

4. A good-quality image acquisition unit acquires a good-quality image, which is an inspection image determined to be a good product in an inspection process using a trained first learning device generated by machine learning the inspection results as teacher data, using an inspection image generated by imaging an inspection object as learning data, and a quality change evaluation unit for evaluating a change in the quality of the inspection object; a quality evaluation information generating unit that generates quality evaluation information including a result of the evaluation; Equipped with The quality change evaluation unit A quality evaluation device characterized by performing clustering using a trained third learning device generated by unsupervised machine learning using the good product images as learning data, and evaluating the change in quality by comparing the number of outliers with a third threshold.

5. 5. The quality evaluation device according to claim 4, further comprising a threshold setting unit that automatically sets the third threshold.

6. The quality evaluation device according to claim 1 , wherein the quality evaluation information includes information recommending relearning of the first learning device.

7. 7. The quality evaluation device according to claim 1, wherein the quality evaluation information includes information recommending an improvement of a pre-process related to the inspection target.

8. 8. The quality evaluation device according to claim 1, further comprising a display unit for displaying the quality evaluation information.

9. A quality evaluation device according to any one of claims 1 to 8, an inspection device including: an inspection processing unit that performs an inspection process on the inspection object using the trained first learning device; and a storage unit that stores the inspection image provided to the inspection process and the inspection result that is a result of the inspection process; Inspection management system including.

10. The inspection management system according to claim 9 , further comprising a learning device for re-training the first learning device.

Images