Manufacturing process analysis method, analysis apparatus, analysis program, and computer-readable storage medium storing analysis program

Abstract

To appropriately retrain two or more machine learning models related to product quality determination so as to handle rare events.SOLUTION: A manufacturing process analysis method comprises: estimating a specific process Qs that is a manufacturing process Q estimated to contribute to quality determination, based on a specific feature quantity Ps that is estimated to contribute to quality determination among a plurality of feature quantities P generated from inspection data 31 and used for the quality determination, and on two or more types of process estimation models 51; when estimation results of the specific process Qs differ among the types of process estimation models 51, newly generating a plurality of manufacturing conditions R by inputting the inspection data 31 used to generate the plurality of feature quantities P into a generative AI model M5; and updating the two or more types of process estimation models 51 respectively based on each inspection data 31 that is obtained by inspecting products W manufactured under each of the manufacturing conditions R.SELECTED DRAWING: Figure 5B

Description

[Technical Field]

[0001] The present disclosure relates to a manufacturing process analysis method, an analysis device, an analysis program, and a computer-readable storage medium storing the analysis program. [Background technology]

[0002] For example, Patent Document 1 discloses a measurement system that utilizes machine learning. This measurement device includes a predicted value estimation unit, an abnormality determination unit, and a machine learning unit.

[0003] Here, the predicted value estimation unit estimates the predicted value of the second sensor from the actual measured value of the first sensor using a machine learning model that has learned the relationship between the actual measured value of the first sensor and the actual measured value of the second sensor. The abnormality determination unit compares the predicted value of the second sensor with the actual measured value of the second sensor to determine whether an abnormality exists in the machine learning model or the measuring device. If the abnormality determination unit determines that the machine learning model is abnormal, the machine learning unit re-learns the machine learning model. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] International Publication No. 2023 / 038022 Summary of the Invention [Problem to be solved by the invention]

[0005] The inventors of the present application have focused on inspection data of products manufactured through multiple manufacturing processes and developed a first method for determining the quality of a product based on multiple feature quantities generated from that inspection data.

[0006] The inventors of the present application have also developed a second method for estimating the manufacturing process that contributed to quality judgment based on the feature that contributed to the judgment, among a plurality of feature values.

[0007] As a result of further intensive research, the inventors of the present application have come up with the idea of ​​using two or more types of machine learning models when making a judgment in at least one of the first and second methods in order to improve the judgment accuracy of each method.

[0008] However, when using two or more machine learning models related to product quality judgment, as in the first and second methods, cases may arise in which the judgment results of each model conflict with each other. In such cases, retraining each model is an option, but as mentioned above, cases in which the judgment results conflict are considered to be rare events in the first place. Therefore, it is not easy to prepare multiple data sets for retraining.

[0009] On the other hand, if the amount of data available for retraining is limited, the discrepancy described above may not be resolved, which may cause inconvenience when using judgments based on two or more machine learning models.

[0010] The present disclosure has been made in consideration of these points, and its purpose is to appropriately retrain two or more machine learning models related to product quality judgment so as to respond to rare events. [Means for solving the problem]

[0011] A first aspect of the present disclosure relates to a method for analyzing a manufacturing process, which is performed using a computer having a memory unit and a calculation unit, and a quality assessment result of a product obtained by inspecting a product manufactured through multiple manufacturing processes.

[0012] According to the first aspect, among a plurality of feature amounts generated from inspection data obtained by inspecting the product and used in the quality determination, one or more feature amounts estimated to contribute to the quality determination are defined as specific feature amounts, and among the plurality of manufacturing processes, one or more manufacturing processes estimated to contribute to the quality determination are defined as specific processes. In the analysis method, the calculation unit reads from the storage unit two or more types of process estimation models that have been machine-learned in advance to associate each of the plurality of feature amounts with one or more of the plurality of manufacturing processes, the calculation unit reads the specific feature amount selected in advance from the plurality of feature amounts, and the calculation unit associates the specific feature amount with the two or more process amounts. and a process estimation model, and estimates the specified process for each type of process estimation model based on the process estimation model; and if the estimation result of the specified process differs between the types of process estimation models, the calculation unit acquires the inspection data used to generate the plurality of feature quantities, and generates a plurality of manufacturing conditions for manufacturing the product by inputting the inspection data into a generation AI. The calculation unit newly acquires the plurality of inspection data based on products manufactured under each of the plurality of manufacturing conditions, and newly generates the plurality of feature quantities based on each of the newly acquired plurality of inspection data, and updates the two or more types of process estimation models based on the newly generated plurality of feature quantities.

[0013] According to the first aspect, when a discrepancy occurs in the estimation results for a specific process obtained from two or more process estimation models, the calculation unit inputs the inspection data that is considered to be the origin of the estimation results to the generation AI, and the calculation unit then generates multiple manufacturing conditions for obtaining other inspection data based on the inspection data.

[0014] The calculation unit then acquires new inspection data based on a product newly manufactured under the generated manufacturing conditions, and updates the process estimation model based on the newly acquired inspection data.

[0015] In this way, even if the event is rare, new inspection data related to the event can be generated, and the process estimation model, which is a machine learning model related to product quality judgment, can be appropriately retrained.

[0016] Furthermore, according to a second aspect of the present disclosure, the calculation unit may be configured to determine that the estimation results of the specific process are different when one or more of the one or more manufacturing processes constituting the specific process are different when the specific process is estimated using the two or more process estimation models.

[0017] According to the second aspect, the calculation unit executes re-learning of the process estimation model when there is even a slight discrepancy between the estimation results of the process estimation model. This configuration is particularly effective when similar estimation results are expected, such as when the two process estimation models use machine learning models of the same system (for example, when both are decision tree systems).

[0018] Furthermore, according to a third aspect of the present disclosure, the calculation unit may be configured to determine that the estimation results of the specific process differ when the specific process is estimated using the two or more process estimation models and a predetermined percentage or more of one or more manufacturing processes constituting the specific process differ.

[0019] According to the third aspect, the calculation unit executes re-learning of the process estimation model depending on the degree of difference between the estimation results of the process estimation model. This configuration is particularly effective when different types of machine learning models are used for the two process estimation models (for example, when a decision tree system and a non-decision tree system are used), in which different estimation results are allowed to a certain extent.

[0020] Furthermore, according to a fourth aspect of the present disclosure, when the calculation unit determines that the estimation results of the specified process differ, the calculation unit may increase the number of times the inspection data is re-acquired as the number of manufacturing processes that differ between the types of the process estimation models among one or more manufacturing processes that make up the specified process increases.

[0021] According to the fourth aspect, the number of times test data is reacquired is changed depending on the degree of discrepancy between the two models, thereby optimizing the number of times test data is reacquired and realizing more appropriate re-learning.

[0022] Furthermore, a fifth aspect of the present disclosure relates to a method for analyzing a manufacturing process, which is executed using a computer having a memory unit and a calculation unit, and a quality assessment result of a product obtained by inspecting a product manufactured through multiple manufacturing processes.

[0023] According to the fifth aspect, among a plurality of feature quantities generated from inspection data obtained by inspecting the product and used in the quality judgment, one or more feature quantities estimated to contribute to the quality judgment are defined as specific feature quantities, and among the plurality of manufacturing processes, one or more manufacturing processes estimated to contribute to the quality judgment are defined as specific processes. In the analysis method, the calculation unit reads from the storage unit two or more quality judgment models, each of which includes a first quality judgment model that has been machine-learned in advance to associate the plurality of feature quantities with the quality judgment result, and a second quality judgment model that has been machine-learned in advance to associate the inspection data with the quality judgment result; the calculation unit acquires the plurality of feature quantities and the inspection data; and the calculation unit executes the quality judgment based on the plurality of feature quantities and the first quality judgment model, and the inspection data and the second quality judgment model; and the calculation unit calculates the quality judgment based on each of the plurality of feature quantities and the plurality of manufacturing processes. The method reads from the storage unit two or more types of process estimation models that have been machine-trained in advance to associate the specific process with one or more processes, the calculation unit reads the specific feature from the plurality of feature values, the calculation unit estimates the specific process for each type of process estimation model based on the specific feature value and the two or more types of process estimation models, and if the quality judgment results differ between the types of quality judgment models, the calculation unit acquires the inspection data used to generate the plurality of feature values, and inputs the inspection data to a generation AI to generate a plurality of manufacturing conditions for manufacturing the product, the calculation unit newly acquires a plurality of pieces of inspection data based on products manufactured under each of the plurality of manufacturing conditions, and the calculation unit newly generates the plurality of feature values ​​based on each of the newly acquired plurality of pieces of inspection data, and updates the two or more types of quality judgment models based on the newly generated inspection data and the plurality of feature values.

[0024] According to the fifth aspect, when a discrepancy occurs between quality assessment results obtained from two or more quality assessment models, the calculation unit inputs the inspection data that is considered to be the origin of the assessment result to the generation AI, and the calculation unit then generates multiple manufacturing conditions for obtaining other inspection data based on the inspection data.

[0025] The calculation unit then acquires new inspection data based on a product newly manufactured under the generated manufacturing conditions, and updates the quality determination model based on the newly acquired inspection data.

[0026] In this way, even if the event is rare, new inspection data related to that event can be generated, and the quality assessment model, which is a machine learning model related to product quality assessment, can be appropriately retrained.

[0027] Furthermore, according to a sixth aspect of the present disclosure, the second quality assessment model may be configured to include a generation AI that receives information corresponding to the inspection data as input and outputs information corresponding to the quality assessment result.

[0028] Here, one of the two or more quality judgment models is required to use a plurality of feature quantities in order to be used in the process estimation model.

[0029] In contrast, according to the sixth aspect, the second quality assessment model receives information related to the inspection data as input, rather than a plurality of feature quantities, which enables more accurate assessment to be made from a different perspective than the first quality assessment model.

[0030] Furthermore, according to a seventh aspect of the present disclosure, the generation AI may be an image generation AI that receives image data representing the inspection data as input, converts the input image data, and outputs the converted image data so as to reflect the quality of the product.

[0031] According to the seventh aspect, an image generation AI is used as the generation AI that configures the second quality assessment model, which enables more accurate assessment to be made from a different perspective than the first quality assessment model.

[0032] Furthermore, according to an eighth aspect of the present disclosure, the second quality assessment model may assess the quality of the product based on the difference between the brightness of the image data before conversion by the generation AI and the brightness of the image data after conversion by the generation AI.

[0033] According to the eighth aspect, the second quality assessment model performs assessment based on the luminance of the image data, which makes it possible to make a more accurate assessment from a different perspective than the first quality assessment model.

[0034] Furthermore, according to a ninth aspect of the present disclosure, the generation AI may restore and output the input image data when the quality of the product is desired, and the generation AI may be pre-trained by using, as training data, the inspection data when the quality of the product is desired, among the inspection data obtained multiple times.

[0035] According to the ninth aspect, the image generation AI is machine-trained in advance using image data corresponding to a desired quality, which enables more accurate judgments to be made from a different perspective than the first quality judgment model.

[0036] Furthermore, according to a tenth aspect of the present disclosure, the two or more process estimation models may include a first process estimation model based on a decision tree algorithm and a second process estimation model based on a non-decision tree algorithm.

[0037] According to the tenth aspect, by using models constructed from different viewpoints, it is possible to realize analysis from more multifaceted viewpoints.

[0038] Furthermore, according to an eleventh aspect of the present disclosure, the second process estimation model may be a Bayesian network in which each of the plurality of feature quantities is a child node and each of the plurality of manufacturing processes is a parent node, and the calculation unit may determine, when the Bayesian network is visualized as a directed graph structure, a manufacturing process that is connected to the specific feature quantity via one edge as the specific process.

[0039] According to the eleventh aspect, the manufacturing process is determined based on probabilistic connections (dependencies), making it possible to realize analysis from more multifaceted perspectives.

[0040] Furthermore, a twelfth aspect of the present disclosure relates to a manufacturing process analysis device that uses a computer having a memory unit and a calculation unit, and a quality assessment result of a product obtained by inspecting a product manufactured through multiple manufacturing processes.

[0041] According to the twelfth aspect, among a plurality of feature quantities generated from inspection data obtained by inspecting the product and used in the quality judgment, one or more feature quantities estimated to contribute to the quality judgment are defined as specific feature quantities, and among the plurality of manufacturing processes, one or more manufacturing processes estimated to contribute to the quality judgment are defined as specific processes. The analysis device includes: means for reading from the storage unit two or more process estimation models that have been machine-trained in advance to associate each of the plurality of feature quantities with one or more of the plurality of manufacturing processes; means for reading the specific feature quantities selected in advance from the plurality of feature quantities; and means for associating the specific feature quantities with the two or more process quantities. and means for, when the estimation results of the specified process differ between the types of process estimation models, acquiring the inspection data used to generate the plurality of feature quantities and inputting the inspection data into a generation AI to generate a plurality of manufacturing conditions for manufacturing the product, newly acquiring a plurality of pieces of inspection data based on products manufactured under each of the plurality of manufacturing conditions, newly generating the plurality of feature quantities based on each of the newly acquired plurality of pieces of inspection data, and updating the two or more types of process estimation models based on the newly generated plurality of feature quantities.

[0042] According to the twelfth aspect, two or more types of machine learning models related to product quality judgment can be appropriately retrained to deal with rare events.

[0043] Furthermore, a thirteenth aspect of the present disclosure relates to a manufacturing process analysis program executed using a computer having a memory unit and a calculation unit, and quality assessment results of a product obtained by inspecting a product manufactured through multiple manufacturing processes.

[0044] According to the thirteenth aspect, among a plurality of feature amounts generated from inspection data obtained by inspecting the product and used in the quality judgment, one or more feature amounts estimated to contribute to the quality judgment are defined as specific feature amounts, and among the plurality of manufacturing processes, one or more manufacturing processes estimated to contribute to the quality judgment are defined as specific processes. The analysis program includes the steps of: a process in which the calculation unit reads from the storage unit two or more types of process estimation models that have been machine-learned in advance to associate each of the plurality of feature amounts with one or more of the plurality of manufacturing processes; a process in which the calculation unit reads the specific feature amount selected in advance from the plurality of feature amounts; and a process in which the calculation unit performs a process of associating the specific feature amount with the two or more types of process estimation models. and a process in which, when the estimation results of the specific process differ between the types of process estimation models, the calculation unit acquires the inspection data used to generate the plurality of feature quantities, inputs the inspection data into a generation AI, and generates a plurality of manufacturing conditions for manufacturing the product. The calculation unit newly acquires the plurality of inspection data based on the products manufactured under each of the plurality of manufacturing conditions, and newly generates the plurality of feature quantities based on each of the newly acquired plurality of inspection data, and updates the two or more types of process estimation models based on the newly generated plurality of feature quantities.

[0045] According to the thirteenth aspect, two or more types of machine learning models related to product quality judgment can be appropriately retrained to deal with rare events.

[0046] A fourteenth aspect of the present disclosure relates to a computer-readable storage medium, which stores the analysis program according to the thirteenth aspect.

[0047] Furthermore, a fifteenth aspect of the present disclosure relates to a manufacturing process analysis device that uses a computer having a memory unit and a calculation unit, and a quality assessment result of a product obtained by inspecting a product manufactured through multiple manufacturing processes.

[0048] According to the fifteenth aspect, when one or more of a plurality of feature quantities generated from inspection data obtained by inspecting the product and used in the quality judgment are assumed to contribute to the quality judgment as specific feature quantities, and one or more of the manufacturing processes are assumed to contribute to the quality judgment as specific processes, the analysis device includes: means for reading from the storage unit two or more quality judgment models, the two or more quality judgment models including a first quality judgment model that has been machine-learned in advance to associate the plurality of feature quantities with the quality judgment result, and a second quality judgment model that has been machine-learned in advance to associate the inspection data with the quality judgment result; means for acquiring the plurality of feature quantities and the inspection data; means for executing the quality judgment based on the plurality of feature quantities and the first quality judgment model, and the inspection data and the second quality judgment model; and means for executing the quality judgment based on each of the plurality of feature quantities and the plurality of feature quantities. means for reading from the storage unit two or more types of process estimation models that have been machine-learned in advance to associate the specific process with one or more of the manufacturing processes of the manufacturing process; means for reading the specific feature from the plurality of feature values; means for estimating the specific process for each type of process estimation model based on the specific feature value and the two or more types of process estimation models; and means for, when the quality judgment results differ between the types of quality judgment models, acquiring the inspection data used to generate the plurality of feature values ​​and inputting the inspection data into a generation AI to generate a plurality of manufacturing conditions for manufacturing the product, newly acquiring a plurality of inspection data based on products manufactured under each of the plurality of manufacturing conditions, newly generating the plurality of feature values ​​based on each of the newly acquired plurality of inspection data, and updating the two or more types of quality judgment models based on the newly generated inspection data and the plurality of feature values.

[0049] According to the fifteenth aspect, two or more types of machine learning models related to product quality judgment can be appropriately retrained to deal with rare events.

[0050] Furthermore, a sixteenth aspect of the present disclosure relates to a manufacturing process analysis program executed using a computer having a memory unit and a calculation unit, and quality assessment results of a product obtained by inspecting a product manufactured through multiple manufacturing processes.

[0051] According to the sixteenth aspect, when one or more feature quantities that are generated from inspection data obtained by inspecting the product and that are used in determining the quality are assumed to contribute to the quality determination as specific feature quantities, and one or more manufacturing processes that are assumed to contribute to the quality determination are assumed to be specific processes among the plurality of manufacturing processes, the analysis program includes the following steps in the computer: a process in which the calculation unit reads from the storage unit two or more quality determination models, each of which includes a first quality determination model that has been machine-learned in advance to associate the plurality of feature quantities with the quality determination result, and a second quality determination model that has been machine-learned in advance to associate the inspection data with the quality determination result; a process in which the calculation unit acquires the plurality of feature quantities and the inspection data; a process in which the calculation unit executes the quality determination based on the plurality of feature quantities and the first quality determination model, and the inspection data and the second quality determination model; and a process in which the calculation unit executes the quality determination based on each of the plurality of feature quantities and the plurality of manufacturing processes. the calculation unit to estimate the specific process for each type of process estimation model based on the specific feature and the two or more types of process estimation models; and, when the quality judgment results differ between the types of quality judgment models, the calculation unit to acquire the inspection data used to generate the plurality of feature values ​​and input the inspection data to a generation AI to generate a plurality of manufacturing conditions for manufacturing the product. The calculation unit to newly acquire a plurality of pieces of inspection data based on products manufactured under each of the plurality of manufacturing conditions, and the calculation unit to newly generate the plurality of feature values ​​based on each of the newly acquired pieces of inspection data and update the two or more types of quality judgment models based on the newly generated inspection data and the plurality of feature values.

[0052] According to the sixteenth aspect, two or more types of machine learning models related to product quality judgment can be appropriately retrained to deal with rare events.

[0053] A seventeenth aspect of the present disclosure relates to a computer-readable storage medium, which stores the analysis program according to the fifteenth aspect. [Effects of the Invention]

[0054] As described above, according to the present disclosure, two or more types of machine learning models related to product quality judgment can be appropriately retrained to deal with rare events. [Brief explanation of the drawings]

[0055] [Figure 1] FIG. 1 is a system diagram illustrating the configuration of a manufacturing management system. [Figure 2] FIG. 2 is a diagram illustrating the configuration of a measurement device in a manufacturing system. [Figure 3] FIG. 3 is a diagram illustrating an example of a hardware configuration of the analysis device. [Figure 4A] FIG. 4A is a diagram illustrating an example of the software configuration of the analysis device. [Figure 4B] FIG. 4B is a diagram illustrating an example of the software configuration of the analysis device. [Figure 5A] FIG. 5A is a flowchart illustrating the steps of the analysis method. [Figure 5B] FIG. 5B is a flowchart illustrating the procedure of the analysis method. [Figure 6] FIG. 6 is a table showing examples of control factors as manufacturing conditions. [Figure 7] FIG. 7 is a diagram illustrating an example of a change in current over time and a feature quantity obtained from the waveform. [Figure 8] FIG. 8 is a flowchart illustrating the measurement process. [Figure 9] FIG. 9 is a flow chart illustrating the test data analysis process. [Figure 10] FIG. 10 is a diagram illustrating the first quality determination model. [Figure 11] FIG. 11 is a diagram for explaining the second quality determination model. [Figure 12] FIG. 12 is a flowchart illustrating a quality determination process using the first quality determination model. [Figure 13] FIG. 13 is a flowchart illustrating a quality determination process using the second quality determination model. [Figure 14] FIG. 14 is a flowchart illustrating the feature selection process. [Figure 15] FIG. 15 is a diagram illustrating the feature estimation model. [Figure 16A] FIG. 16A is a diagram for explaining the first process estimation model. [Figure 16B] FIG. 16B is a diagram for explaining the specific feature amount and the specifying step. [Figure 17A] FIG. 17A is a diagram for explaining the second process estimation model. [Figure 17B] FIG. 17B is a diagram for explaining the specific feature amount and the specifying step. [Figure 18] FIG. 18 is a flowchart illustrating a specific process estimation process. [Figure 19] FIG. 19 is an image diagram for explaining a generation AI for generating a plurality of manufacturing conditions. [Figure 20] FIG. 20 is a diagram for explaining the number of generated manufacturing conditions. [Figure 21] FIG. 21 is a display screen showing an example of the GUI of the generation AI. DETAILED DESCRIPTION OF THE INVENTION

[0056] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. Note that the following description is for illustrative purposes only.

[0057] <1. System configuration> 1 is a system diagram illustrating the configuration of a manufacturing management system 100 according to the present disclosure. This manufacturing management system 100 is composed of one or more systems. For example, in this embodiment, the manufacturing management system 100 includes an analysis system 101 and a manufacturing system 102.

[0058] The manufacturing system 102 includes a manufacturing device 121, a measuring device 122, and an information processing device 123. The manufacturing device 121 manufactures a product W by executing a plurality of manufacturing processes Q. The measuring device 122 inspects the product W manufactured through the plurality of manufacturing processes Q, thereby acquiring one or more feature quantities P that characterize the quality of the product W.

[0059] The analysis system 101 is configured with a computer 1 that functions as a first analysis device. The analysis system 101 analyzes a manufacturing process Q in a manufacturing system 102 by using the computer 1 and the quality determination result of a product W. In this embodiment, the quality of the product W is determined by the computer 1 of the analysis system 101 based on a feature quantity P acquired by the manufacturing system 102.

[0060] The quality inspection of the product W (in other words, the process of acquiring one or more feature quantities P) may be performed by the analysis system 101 instead of the manufacturing system 102. The quality of the product W may be determined by the manufacturing system 102 instead of the analysis system 101.

[0061] More generally, training data for various machine learning, such as the first training data D1 described below, may be data acquired in the manufacturing system 102, or may be data acquired in a facility other than the manufacturing system 102, such as a development facility, research facility, or experimental facility for product W.

[0062] Although the following description mainly relates to the former data, the present disclosure can also be applied to the latter data. When applied to the latter data, the "manufacturing conditions" detailed below may be read as "development conditions," "research conditions," "experimental conditions," etc.

[0063] In addition, even when experimental data of product W is used as inspection data 31, various pieces of information that characterize the production of product W (so to speak, ``experimental conditions'') can be considered as production conditions R, not limited to the production of product W.

[0064] In this embodiment, the product W is a metal product. The product W as a metal product includes steel materials used in various parts that make up a vehicle. When the product W is a metal product, the analysis system 101 may determine the rust-preventive performance of the metal product as the quality. In this case, the metal product 201 may have an insulating layer 203 on its surface. Below, the rust-preventive performance of the metal product 201 will be described as an example of quality. In the following description, the word "quality" may be replaced with "performance" such as rust-preventive performance, as appropriate.

[0065] In this case, the manufacturing management system 100 can be regarded as a rust prevention performance management system that manages the rust prevention performance of metal products manufactured through multiple manufacturing processes Q. Note that even when a metal product is used for the product W, it is not essential that the product W be provided with the insulating layer 203.

[0066] Also, if the product W is a metal product, the multiple manufacturing processes Q performed by the manufacturing equipment 121 of the manufacturing system 102 may include one or more processes related to the formation of the insulating layer 203, such as an electroplating process, a water washing process, a drying process, etc.

[0067] For the sake of comprehensive discussion, the manufacturing processes Q are assumed to be a total of M processes (M≧2). Accordingly, for example, the Mth process Q is referred to as the “Mth process Q.” M " is sometimes called.

[0068] Below, we will first explain an example of the configuration of the measuring device 122 when a metal product 201 having an insulating layer 203 on its surface is used as the product W. Then, we will explain a method for analyzing the results obtained by the measuring device 122 through an explanation of the computer 1 in the analysis system 101.

[0069] <2. Example of measurement device configuration> 2 is a diagram showing an example of the configuration of the measuring device 122 according to this embodiment. As described above, in this embodiment, the product W inspected by this measuring device 122 is a metal product 201. The metal product 201 has a base material 202 such as a steel plate, and an insulating layer 203 located on the surface of the base material 202.

[0070] Although not shown in the drawings, a chemical conversion coating may be formed on the surface of the substrate 202, and an insulating coating may be provided on the surface of the chemical conversion coating. In this case, the insulating layer 203 as in this embodiment is formed by the insulating coating.

[0071] The measuring device 122 is configured to apply a voltage to the surface of the metal product 201 as the product W, and acquire the change over time in the current caused by the voltage.

[0072] Specifically, the measuring device 122 applies a voltage to the surface of the metal product 201 while the corrosion factor 205 is in contact with the surface of the metal product 201. The corrosion factor 205 may be an electrolyte material containing a supporting electrolyte such as water or sodium chloride, and a clay mineral such as kaolinite.

[0073] Specifically, the measuring device 122 includes a container 220 , an electrode 221 , a power source 222 , a sealing material 223 , and wiring 224 .

[0074] Of these elements, the container 220 is placed on the surface of the metal product 201 (for example, the surface of the insulating layer 203) via a sealing material 223 to prevent liquid leakage. The corrosion factor 205 is contained in the container 220 and is in contact with the surface of the insulating layer 203. The shape and material of the container 220 are not particularly limited. The container 220 has a tubular shape and is made of a resin material such as acrylic resin or epoxy resin. The term "tubular shape" used here includes tubular shapes with any cross-sectional shape, such as a cylindrical shape or a polygonal cylindrical shape.

[0075] The sealing material 223 is a sheet-like sealing material made of, for example, silicone resin. When the container 220 is placed on the metal product 201, the sealing material 223 improves the adhesion between the container 220 and the insulating layer 203 and fills the gap between them. This effectively prevents the corrosion factor 205 from leaking from between the container 220 and the insulating layer 203.

[0076] The electrode 221 is used to apply a voltage between the substrate 202 and the insulating layer 203. The electrode 221 is configured so that at least its tip is embedded in the corrosion factor 205 in the container 220 and comes into contact with the corrosion factor 205. The electrode 221 may be, for example, an electrode that can be used for electrochemical measurement. The electrode 221 may be, for example, a carbon electrode or a platinum electrode.

[0077] The power supply 222 is connected to the electrode 221 and the substrate 202 via wiring 224, and applies a voltage between the electrode 221 and the substrate 202. At the same time, the power supply 222 measures the change over time in the current flowing between the electrode 221 and the substrate 202 as the voltage is applied. The application of the voltage and the measurement of the current by the power supply 222 are controlled by the information processing device 123.

[0078] "Measurement by the power source 222" is an example of "inspection" in this embodiment.

[0079] The measurement data (test data 31) measured by the power supply 222 is output from the power supply 222 to the information processing device 123. The information processing device 123 transmits the test data 31 to the analysis system 101 by wireless or wired communication.

[0080] The measurement data may be data in which the detected current value is plotted against time, or in the case of applying a gradually increasing voltage, data in which the detected current value is plotted against the applied voltage value. In addition to the detected current value, applied voltage value, and measurement time, the measurement data also includes information identifying the production lot of the metal product 201 on which the measurement was performed and the measurement position on the metal product 201.

[0081] 2 is merely an example of analyzing the rust prevention performance of a metal product (product W) having an insulating layer 203. The configuration of the measuring device 122 may vary depending on the type of product W and the type of quality to be analyzed.

[0082] Furthermore, the present disclosure does not necessarily require the measurement device 122. For example, the analysis system 101 may be configured to input the quality assessment results by a craftsman together with the data obtained by the craftsman's inspection.

[0083] The inspection data 31 includes measurement data obtained by the measuring device 122 as well as data obtained through methods other than measurement, such as inspection by a craftsman. In this disclosure, the inspection data 31 includes one or more general combinations of data obtained by inspecting the product W.

[0084] In this embodiment, the test data 31 is composed of a combination of the measurement data and image data 312 corresponding to the measurement data. The image data 312 is data obtained by visualizing a graph or plot showing the change over time in the detected current value, for example, so as to reflect the brightness of each pixel showing the graph or plot (see FIG. 11).

[0085] <3. Analysis System> Fig. 3 is a diagram illustrating an example of the hardware configuration of an analysis device (computer 1) according to the present disclosure, and Figs. 4A and 4B are diagrams illustrating an example of the software configuration of computer 1. This analysis device 1 is configured by a computer including a CPU 3 as a calculation unit, and a RAM 7 and an SSD 9 as storage units.

[0086] 3, the computer 1 includes a central processing unit (CPU) 3 that controls the entire computer 1, a read only memory (ROM) 5 that stores a boot program and the like, a random access memory (RAM) 7 that functions as a main memory, and a solid state drive (SSD) 9 that serves as a secondary storage device. Note that a hard disk drive (HDD) or the like can also be used as the secondary storage device instead of the SSD 9.

[0087] Of these elements, the CPU 3 executes various programs. The CPU 3 functions as a calculation unit in this embodiment. The RAM 7 and SSD 9 temporarily or continuously store the programs executed by the CPU 3. The RAM 7 and SSD 9 each function as a storage unit in this embodiment.

[0088] The computer 1 also includes a display 11, a graphics memory (Video RAM: VRAM) 13 that stores image data to be displayed on the display 11, and a keyboard 15 and a mouse 17 as man-machine interfaces. The keyboard 15 and the mouse 17 function as a reception unit that receives input from an operator. The display 11 functions as a display unit that displays a screen based on the results of calculations by the CPU 3. The computer 1 according to this embodiment can also send and receive data to and from external devices via a communication interface 19.

[0089] As shown in FIG. 4A, the program memory of the SSD 9 stores an analysis program 21 according to this embodiment, an operating system (OS) and application programs (not shown).

[0090] Here, the analysis method according to this embodiment is executed using a computer (analysis device 1) configured as described above and the quality judgment results of the product W, and analyzes the manufacturing process Q.

[0091] The analysis program 21 is a program coded to realize such an analysis, and is configured to cause the computer 1 serving as an analysis device to execute each process constituting the analysis method sequentially or simultaneously. The analysis program 29 is pre-stored in a computer-readable storage medium 18. This storage medium 18 is a tangible storage medium such as a disk medium.

[0092] Specifically, the analysis program 21 according to this embodiment is made up of a quality determination program 23, a process estimation program 25, and a relearning program 27.

[0093] Here, the quality determination program 23 is made up of an inspection data analysis program 231 , a quality determination model reading program 233 , a quality determination program 235 , and an update necessity determination program 237 .

[0094] The process estimation program 25 is made up of a feature selection program 251 , a process estimation model reading program 253 , and a process estimation program 255 .

[0095] The re-learning program 27 is composed of an inspection data acquisition program 271, a manufacturing condition generation program 273, an inspection data addition program 275, a feature addition program 277, and a machine learning model update program 279.

[0096] These programs constituting the analysis program 21 are merely grouped for convenience, categorizing the analysis program 21 by function. One or more of these programs may constitute one program.

[0097] 1 and 3 are merely examples of the single computer 1. The programs constituting the analysis program 21 may be executed by two or more computers 1.

[0098] In the program memory of the SSD 9, each program constituting the analysis program 21 is started in response to a command input from the keyboard 15, the mouse 17, etc. At that time, each program is loaded from the SSD 9 into the RAM 7 and executed by the CPU 3.

[0099] 4B, the data memory of SSD 9 stores inspection data 31 to be analyzed and feature data 33. The feature data 33 is a data set generated from the inspection data 31 and composed of a plurality of feature amounts P used to determine quality. One piece of feature data 33 corresponds to one piece of inspection data 31 obtained by one inspection.

[0100] The test data 31 and the feature amount data 33 are each stored in the data memory in CSV format (except for image data 312, which will be described later, for the former). This improves the usability of the test data 31 and the feature amount data 33 in various processes, such as the re-learning process (step S3 in FIG. 5B).

[0101] The term "test data 31" used here includes pre-learning test data 31, test data 31 to be analyzed, and re-learning test data 31. The pre-learning and re-learning test data 31 constitute various training data, which will be described later.

[0102] Similarly, the term "feature amount data 33" used here includes feature amount data 33 for pre-learning, feature amount data 33 to be analyzed, and feature amount data 33 for re-learning.

[0103] It is not essential to store the test data 31 and feature amount data 33 for pre-learning in the computer 1. These data sets are introduced for the purpose of explaining the pre-learning described later, and are not necessarily used in the following processing.

[0104] Note that "relearning" here refers to the sequential execution of the following processes: adding other learning data to previously set learning data (training data); resetting one or more learned parameters in a machine learning model such as the one exemplified below; and re-machining each reset parameter based on the added learning data (i.e., learning data with an increased amount of data).

[0105] The data memory of the SSD 9 also stores two or more types of quality assessment models 41 and two or more types of process estimation models 51. The two or more types of quality assessment models 41 and the two or more types of process estimation models 51 are all machine learning models generated by prior machine learning.

[0106] In particular, in this embodiment, the two or more types of quality judgment models 41 include a first quality judgment model 411 and a second quality judgment model 412. The two or more types of process estimation models 51 include a first process estimation model 511 and a second process estimation model 512. Details of these machine learning models will be described later.

[0107] In addition, various data generated by each program and each process that constitutes the analysis program 29, as well as the execution results of application programs, etc., are stored in the data memory of the SSD 9 or in the RAM 7 as the main memory, as necessary.

[0108] <4. Overview of analysis method> 5A and 5B are flowcharts illustrating the steps of the analysis method. As shown in Fig. 5A and 5B, the analysis method is implemented by sequentially executing a quality determination process (step S1), a process estimation process (step S2), and a relearning process (step S3).

[0109] In detail, the quality assessment process (step S1) is carried out by sequentially performing an inspection data analysis process (step S11), a quality assessment model reading process (step S12), a quality assessment process (step S13), and a first update necessity determination process (step S14).

[0110] The process estimation process (step S2) is performed by sequentially performing a feature selection process (step S21), a process estimation model reading process (step S22), a specific process estimation process (step S23), and a second update necessity determination process (step S24).

[0111] In addition, the re-learning process (step S3) is performed by sequentially performing an inspection data acquisition process (step S31), a manufacturing condition generation process (step S32), an inspection data addition process (step S33), a feature addition process (step S34), and a machine learning model update process (step S35).

[0112] The analysis program 29 is configured to cause the computer 1 to execute these processes. That is, the quality assessment process (step S1) is performed by the CPU 3 executing the quality assessment program 23. The process estimation process (step S2) is performed by the CPU 3 executing the process estimation program 25. The relearning process (step S3) is performed by the CPU 3 executing the relearning program 27.

[0113] The same applies to each process constituting the quality determination process (step S1), the process estimation process (step S2), and the relearning process (step S3). For example, the inspection data analysis process (step S11) is performed by the CPU 3 executing the inspection data analysis program 231.

[0114] The same applies to other processes. When the CPU 3 executes a program with a predetermined name, such as the "read quality determination model" program 233, a process with the same name, such as the "read quality determination model" process (step S22), is executed.

[0115] When the CPU 3 executes the analysis program 21, the computer 1 including the CPU 3 constitutes an analysis device. That is, the computer 1 functions as an analysis device including a quality determination means that executes a quality determination process, a process estimation means that executes a process estimation process, and a relearning means that executes a relearning process.

[0116] Below, the process (measurement process) performed prior to the analysis of the product W will be described with reference to FIGS. 7 and 8, and then the analysis method according to the present disclosure will be described in detail with reference to FIGS. 5A and 5B.

[0117] The measurement processing and analysis method described below are performed periodically, for example, for each production lot of product W. A production lot generally refers to a unit of production or order. In this embodiment, a production lot can be a unit separated by, for example, a shipping unit, a production unit, a production month / date, a raw material (e.g., paint) lot, a raw material change point, or a process change point (e.g., when replacing the cutting blade of the metal product 201). It is not essential to perform the processing for each production lot. The measurement processing and analysis method may be performed for each product one by one.

[0118] <5. Measurement processing> FIG. 7 is a diagram illustrating a change in current over time and a feature value P obtained from the waveform. FIG. 8 is a flowchart illustrating the measurement process. First, as shown in step S101 of FIG. 8, the measurement device 122 extracts one or more products W for each production lot. The number of products to be extracted is adjusted appropriately based on the size of the production lot, the expected probability of abnormality occurrence, etc.

[0119] In the following step S102, the measuring device 122 inspects each product W extracted in step S101. The measuring device 122 inspects each product W, thereby acquiring inspection data 31 for each product W that has been inspected.

[0120] The inspection data 31 acquired by the measurement process may be used for pre-learning, for product analysis corresponding to the processes in steps S1 and S2, or for relearning corresponding to the process in step S3. In other words, the measurement process illustrated in Fig. 8 can be used in each of pre-learning, product analysis, and relearning.

[0121] Specifically, in step S102, the measurement device 122 applies a voltage between the insulating layer 203 and the substrate 202, for example, with the corrosion factor 205 in contact with the surface of the insulating layer 203. The manufacturing system 102 measures the change over time in the current caused by the applied voltage. The measurement locations include the main surfaces, edges, and welds of the metal product 201.

[0122] In the following step S103, the measuring device 122 converts the changes over time measured in step S102 into test data 31 and transmits it to the computer 1 of the analysis system 101. The transmitted test data 31 is stored in the RAM 7, SSD 9, or the like of the computer 1.

[0123] For a more general product W, the inspection data 31 is not limited to changes in current over time. The inspection data 31 may be at least one of image data, audio data, text data, and mechanical or electrical data that characterize the product W.

[0124] Image data is data obtained, for example, by capturing an image of the product W. Audio data is data obtained, for example, by recording sounds generated when the product W is in operation or when a strike is applied to the product W. Text data is data written related to the quality of the product W, such as inspection records by craftsmen. Mechanical or electrical data includes the weight and dimensions of the product W, as well as voltage and current values, as in this embodiment.

[0125] The following explanation will basically focus on the case where "test data 31 = change in current over time", but it can be replaced with other data as mentioned above as appropriate.

[0126] <6. Analysis Method (1): Quality Judgment Processing> Next, we will explain the processes constituting the analysis method, particularly the quality determination process. The following process is executed for each manufactured product W. It may also be executed for products W extracted for each manufacturing lot.

[0127] (6-1. Test data analysis process) Fig. 9 is a flowchart showing the details of the test data analysis process. When the control process proceeds to step S11 in Fig. 5A, the CPU 3 executes the flow shown in Fig. 9 in order from step S111.

[0128] 9, the CPU 3 reads the inspection data 31 from the RAM 7, the SSD 9, etc. In the following step S112, the CPU 3 extracts a plurality of types of feature amounts P from the read inspection data 31.

[0129] Specifically, the CPU 3 acquires, as the plurality of feature quantities P, a plurality of parameters characterizing the change over time of the current (current generated due to the applied voltage) indicated by the test data 31. Here, in this embodiment, the "change over time of the current" refers to the "change over time of the current measured as a waveform." In other words, it refers to the "current value corresponding to the voltage value changed over time."

[0130] More specifically, the CPU 3 extracts, as the plurality of types of feature quantities P, a group of three or more parameters including the number Np of peaks in the current, the height Ip of each peak, and the gradient S of the current (see FIG. 7).

[0131] For example, if there is no abnormality or sign of an abnormality in the insulating layer 203 of the metal product 201, when a DC voltage that increases (gradually increases) over time is applied as described above, almost no current flows until the voltage (applied voltage) reaches the breakdown voltage, and once the applied voltage reaches the breakdown voltage, the current increases rapidly.

[0132] The insulating layer 203 maintains its blocking performance against corrosion factors 205 until the applied voltage reaches the breakdown voltage. As a result, almost no current flows. On the other hand, when the applied voltage reaches the breakdown voltage, the corrosion factors 205 are encouraged to penetrate the insulating layer 203, and the corrosion factors 205 may reach the surface of the substrate 202 at the weakest points of the insulating layer 203, such as points with relatively few cross-linked resin structures. As a result, the current may increase rapidly. In other words, a sudden increase in the detected current value indicates that the corrosion factors 205 have reached the surface of the substrate 202, causing the insulating layer 203 to lose its rust-preventing performance.

[0133] On the other hand, consider a case where there is a local defect in the insulating layer 203 (for example, foreign matter such as gas pins, welding spatter and slag, burrs, iron powder, or unevenness on the surface of the substrate 202), causing a localized area where the effective film thickness is small in the insulating layer 203. In this case, when a gradually increasing DC voltage is applied between the electrode 204 and the substrate 202, the current generated between them is considered to change over time as shown in FIG.

[0134] That is, if a local defect exists in the insulating layer 203, the corrosion factor 205 penetrates locally at the defect location. When the corrosion factor 205 penetrates the insulating layer 203 at a certain defect location and reaches the substrate 202, conduction occurs, causing an instantaneous increase in the current value. If a voltage equal to or greater than the voltage at which water electrolysis occurs is applied between the electrode and the substrate 202 at this point, electrochemical reactions such as water electrolysis proceed on the surface of the substrate 202 due to the conduction. As a result, generated gas and electrolytic products accumulate in the defect, interrupting the conduction and causing a decrease in the current value. That is, if a local defect exists, the conduction and subsequent interruption at the defect location cause peaks indicating instantaneous increases and decreases in the current value in the waveform of the time-dependent change data of the detected current value.

[0135] When multiple defects exist, a number Np of peaks corresponding to the number of defects occurs, as shown in Figure 7. The current value at the peak, i.e., the peak height Ip, is related to the size of the conduction path through which the current flows, i.e., the size, type, and conductivity of the defect. Furthermore, since conduction occurs at a lower applied voltage in areas where the thickness of insulating layer 203 is thinner due to defects, the applied voltage that gives the peak is correlated with the film thickness of the area where the defect exists.

[0136] Furthermore, if the insulating layer 203 has a general defect (for example, a low cross-link density due to a lack of catalyst in the paint, or an oxide film on the surface of the substrate 202), corrosion factors 205 will penetrate the entire insulating layer 203 and reach the substrate 202. As a result, a gradual increase in the current value occurs in the waveform of the time-dependent change data of the detected current value. In other words, the slope S of the waveform of the time-dependent change data when the detected current value increases is related to the insulating quality (film quality) of the insulating layer 203.

[0137] The number, size, type, conductivity, film thickness, and film quality of the above defects are all factors that affect the rust-preventive performance of the insulating layer 203. Therefore, in this embodiment, the number Np of peaks in the current value, the height Ip of the peaks, and the slope S of the waveform of the change in current over time, which are related to these factors, are acquired as feature quantities P that characterize the rust-preventive performance of the insulating layer 203. Furthermore, since the likelihood of peaks occurring in the current varies depending on the position (including the main surface, edge, and weld of the metal product 201) at which the change in current over time is measured, position information indicating the position at which the change in current over time is measured may also be acquired from the measurement data (inspection data 31) as a parameter for evaluating the rust-preventive performance of the insulating layer 203.

[0138] Note that the parameters that can be used as the feature quantity P are not limited to the number of current peaks Np, the height of each peak Ip, and the current slope S. Any parameter can be used as long as it is related to the rust prevention performance of the metal product 201. When targeting other products W or other qualities (performances), more general parameters can be used.

[0139] Additionally, as described above, when image data, voice data, text data, and mechanical or electrical data are used as the inspection data 31, it is only necessary to extract feature quantities according to the type of data.

[0140] For comprehensive discussion, the plurality of feature quantities P are assumed to be N in total (N is a natural number equal to or greater than 2). Accordingly, the Nth feature quantity P is referred to as the “Nth feature quantity P N In the case of the metal product 201, the first feature amount P1 corresponds to the number of peaks Np, the second feature amount P2 corresponds to the peak height Ip, and the third feature amount P3 corresponds to the slope S (see FIG. 7).

[0141] Thereafter, in step S113, the CPU 3 converts each of the multiple feature quantities P into a dimensionless risk score between 0 and 1, and collectively sets the converted feature quantities P as feature quantity data 33. The CPU 3 transmits the feature quantity data 33 to the computer 1 of the analysis system 101, and stores it in the RAM 7, SSD 9, or the like. Thereafter, the control process returns from the flow in FIG. 8 to the flow in FIG. 5A, and proceeds to step S12.

[0142] The feature data 33 thus generated may be used for pre-learning, for product analysis exemplified in steps S1 and S2, or for relearning exemplified in the processing of step S3. In other words, the inspection data analysis process exemplified in Fig. 9 can be used for each of pre-learning, product analysis, and relearning.

[0143] In this embodiment, the risk score indicates the level of risk of abnormalities occurring in the quality (rust prevention performance) of the metal product 201. In other words, it can be interpreted that the higher the risk score, the more likely abnormalities will occur in the metal product 201.

[0144] The inventors of the present application attempted to determine whether the quality of the metal product 201 is good or bad based on the level of the risk score corresponding to each of the multiple feature quantities P and the level of the average value of the multiple feature quantities P. Furthermore, rather than simply dividing the quality of the metal product 201 into two categories, either "normal" or "abnormal," they attempted to achieve three or more classifications, including a gray area (an intermediate state) between white (normal) and black (abnormal), such as "there is a sign of an abnormality occurring."

[0145] However, such a judgment is not easy. Therefore, we decided to use a pre-generated machine learning model in the quality judgment process (step S13) that follows the analytical data analysis process (step S11) via the quality judgment model reading process (step S12). By making a judgment using a machine learning model, even if you are not an experienced craftsman, you can make a judgment with a certain degree of accuracy.

[0146] (6-2. Quality judgment model loading process) In step S12, a quality determination model reading process, the CPU 3 reads first and second quality determination models 411 and 412, which are two or more types of quality determination models 41 (two in this embodiment), from the SSD 9 serving as a storage unit.

[0147] When the loading of the first and second quality determination models 411 and 412 is completed, the control process proceeds from step S12 to step S13 in FIG. 5A.

[0148] Here, the first quality judgment model 411 is machine-trained in advance to associate a plurality of feature quantities P with quality judgment results. The second quality judgment model 412 is machine-trained in advance to associate the inspection data 31 used to generate the plurality of feature quantities P with quality judgment results. The first and second quality judgment models 411, 412 have different inputs but share a common output. The quality of the product W is judged from different perspectives (inputs).

[0149] The first and second quality judgment models 411, 412 each output a "degree of quality pass / fail" as a "quality judgment result." The "degree of quality pass / fail" is the degree of quality pass / fail of the product W. The term "degree of quality pass / fail" includes not only the two states of "normal" and "abnormal," but also the intermediate state of "there is a sign of abnormality" as described above.

[0150] In the case of the metal product 201, the quality referred to here is "rust prevention performance." The degree of quality (hereinafter simply referred to as "quality") is quantified so that it increases or decreases depending on whether the quality is good or bad. For example, in this embodiment, the quality exemplified as rust prevention performance is quantified as, for example, "normal = 2," "sign of abnormality present = 1," and "abnormal = 0." Hereinafter, the quantified quality is also referred to as "quality data 35."

[0151] (6-2-1. First quality judgment model) 10 is a diagram for explaining the first quality judgment model 411. As shown in the figure, the first quality judgment model 411 is a machine learning model that receives a plurality of feature quantities P as input and outputs a quality judgment result (quality data 35), such as an estimated value of the degree of pass / fail of rust prevention performance. The first quality judgment model 411 has been machine-learned in advance by supervised learning.

[0152] During pre-learning, the first quality judgment model 411 receives as input feature data 33 based on inspection data 31 acquired at a development facility, an experimental facility, etc. During relearning, the first quality judgment model 411 receives as input feature data 33 acquired by the relearning process. During quality judgment for process estimation processing, the first quality judgment model 411 receives as input feature data 33 based on inspection data 31 acquired by the manufacturing system 102.

[0153] For example, in this embodiment, the first quality assessment model 411 is trained using first training data D1. As shown in Fig. 10, the first training data D1 is configured by linking a plurality of feature amount data 33 with quality data 35 indicating a quality assessment result that has been acquired in advance for each feature amount data 33. Data sets configured with the plurality of feature amount data 33 and the quality data 35 linked thereto are created in advance, the number of which is equal to the number of samples of the first training data.

[0154] The first quality assessment model 411 may be a nonlinear regression model or a linear regression model. The first quality assessment model 411 as a nonlinear regression model may be a machine learning model based on a decision tree algorithm. The first quality assessment model 511 related to the decision tree may be a Random Forest (RF) model.

[0155] The first quality judgment model 411, like the RF model, is only required to judge quality based on a plurality of feature amounts P, and in addition, to be able to determine the importance of each feature amount P when making the judgment (see the first importance I1 in FIG. 10). This importance is a parameter given to each feature amount P, and indicates the degree of contribution of each feature amount P to the quality judgment.

[0156] 10, in this embodiment, an RF model M1 is used as the first quality judgment model 411. When the RF model M1 is used, the contribution of each feature amount P (so-called RF importance) can be used as the importance. The importance may be determined based on, for example, the Gini coefficient of a decision tree constituting a random forest model.

[0157] Hereinafter, among the multiple feature amounts P used to determine quality, one or more feature amounts P that are estimated to contribute to the determination of quality will be referred to as specific feature amounts Ps (see FIG. 10 and FIG. 16B described later, etc.).

[0158] The specific feature quantity Ps is a parameter used in the process estimation process S2. In this embodiment, the specific feature quantity Ps is determined in the feature quantity selection process (step S22), but this configuration is merely an example. The specific feature quantity Ps may also be determined during the quality assessment process S1, such as in the quality assessment process (step S13).

[0159] To determine the specific feature Ps, the importance of each of the multiple feature Ps is determined, and one or more feature Ps are selected in descending order of importance as the specific feature Ps. When the RF model M1 is used, the importance can be determined based on the Gini coefficient, as described above.

[0160] Alternatively, so-called SHAP (Shapley Additive exPlanations) processing may be used. In this case, based on the first quality judgment model 411 and a plurality of feature quantities P acquired from the measurement data 49, the first quality judgment model 411 is locally approximated around the feature quantities P. The model (approximation model) generated by approximating the first quality judgment model 411 may be, for example, a simple model that easily explains the contribution of each feature quantity P. Then, based on the approximation model, the contribution of each feature quantity P to the predicted value output from the approximation model is expressed by a so-called Shapley value used in cooperative game theory, etc. The contribution may be used as the importance for selecting a specific feature quantity Ps. In addition, various techniques in image processing for two-dimensional images, such as gradient processing, may be applied.

[0161] 2 and other specific examples, the specific feature quantity Ps is one or more feature quantities P that are estimated to have contributed to the determination of the rust-preventing performance. In this case, which of the three parameters, the number of peaks Np as the first feature quantity P1, the peak height Ip as the second feature quantity P2, and the slope S as the third feature quantity P3, contributed to the determination of the rust-preventing performance, including abnormalities and their precursors, is selected.

[0162] In the specific example of Fig. 2 etc., an RF model M1 is used as illustrated in Fig. 10. In this example, the specific feature Ps is composed of three items: the number of peaks Np as the first feature P1, and the peak height Ip as the second feature P2.

[0163] (6-2-2. Second quality judgment model) 11 is a diagram for explaining the second quality judgment model 412. As shown in the figure, the second quality judgment model 412 is a machine learning model that receives as input inspection data 31 corresponding to a plurality of feature quantities P and outputs a quality judgment result, such as an estimated value of the degree of pass / fail of rust prevention performance. The second quality judgment model 412 has been machine-learned in advance by supervised learning.

[0164] In detail, the second quality judgment model 412 receives the image data 312 of the inspection data 31 as input during pre-learning, quality judgment, and re-learning. During pre-learning and re-learning, of the various image data 312, the "image data 312 when the quality of the product W is desired" is used as input.

[0165] As shown in FIG. 11, the second quality assessment model 412 includes a generation AI model (generation AI) M2, a difference calculation unit 412a, and a brightness assessment unit 412b.

[0166] The generative AI model M2 receives information corresponding to the inspection data 31 as input and outputs information corresponding to the quality assessment result.

[0167] Specifically, the generative AI model M2 is an image generation AI that receives image data 312 representing each piece of inspection data 31 as input, converts the input image data 312, and outputs the converted image data 312 so as to reflect the quality of the product W corresponding to each piece of inspection data 31. The output of the generative AI model M2 changes depending on the quality of the product W from which the inspection data 31 was obtained.

[0168] The generative AI model M2 has been trained in advance by machine learning to restore and output an input image. This generative AI model M2 has been trained in advance by using, as training data (second training data D2), image data 312 obtained multiple times when the quality of the product W is desired.

[0169] By performing pre-learning in this manner, the generative AI model M2 will restore and output the input image data 312 when the quality of the product W is desired. On the other hand, when the quality of the product W is undesired, the generative AI model M2 will be unable to restore the input image data 312 and will output it. In the latter case, the generative AI model M2 will output image data different from the input image data 312. Note that the "desired case" here corresponds to the "normal" case in the specific example described above. In order to perform processing according to quality, each image data 312 constituting the second training data D2 may be associated with quality data 35 indicating the quality assessment result corresponding to each image data 312.

[0170] Specifically, the generative AI model M2 may be an image generation AI that combines CNN and AE, or may use VAE. CNN stands for Convolutional Neural Network. AE stands for AutoEncoder. VAE stands for Variational AutoEncoder. Any other machine learning model applicable to image processing can be used.

[0171] In this way, the generation AI model M2 according to this embodiment is configured to receive image data 312 as input, convert and output the input image data 312. Image data 312 contains a larger amount of information than plots, etc., and therefore contributes to improving the accuracy of anomaly detection.

[0172] However, the generation AI model M2 may also receive input of information other than image data 312. The generation AI model M2 may also receive input of waveform data that indicates changes over time in current, etc., such as the measurement data. This waveform data is a data set configured by arranging the waveform values ​​at each time in chronological order.

[0173] In this case, the generation AI model M2 may be configured, as in the above embodiment, to attempt restoration of the input information (waveform data) and output a restoration result (result of waveform data restoration) according to the quality of the product W. As in the above embodiment, the generation AI model M2 in this case may be configured to succeed in restoration if the quality of the product W is desired, and to fail in restoration if it is not.

[0174] In this case, the generative AI model M2 may be a generative AI that uses an LSTM autoencoder that takes into account time context. LSTM autoencoders excel at extracting features from time-series data and other sequential data. In this way, the generative AI model M2 can be configured using various encoders depending on the type of input information.

[0175] The difference calculation unit 412a outputs the difference between the brightness of the image data 312 input to the second quality determination model 412, i.e., the image data 312 before conversion by the generation AI model M2, and the brightness of the image data 312 after conversion by the generation AI model M2. The difference calculation unit 412a adds up the brightness differences for all pixels in the image. If the quality of the product W is desired, the total sum of the differences is expected to be small.

[0176] Therefore, the brightness determination unit 412b compares the magnitude of the input difference with a predetermined threshold value. If the magnitude of the difference is less than the threshold value, the brightness determination unit 412b determines that "the quality is desirable," for example, "normal," and if the magnitude is equal to or greater than the threshold value, the brightness determination unit 412b determines that "the quality is undesirable," for example, "abnormal" or "intermediate."

[0177] In this embodiment, two or more threshold values ​​are prepared. The brightness determining unit 412b determines one of "normal", "abnormal" and "intermediate state".

[0178] (6-3. Quality assessment process) Fig. 12 is a flowchart illustrating a quality determination process using the first quality determination model 411. Fig. 13 is a flowchart illustrating a quality determination process using the second quality determination model 412. When the control process proceeds to step S13 in Fig. 5A, the CPU 3 executes the flow shown in Fig. 12 and the flow shown in Fig. 13 in order or in parallel.

[0179] In the quality assessment process, the CPU 3 acquires a plurality of feature quantities P and the inspection data 31. The CPU 3 performs quality assessment based on the plurality of feature quantities P and the first quality assessment model 411, and based on the inspection data 31 and the second quality assessment model 512, respectively.

[0180] 12, the CPU 3 first reads a plurality of feature amounts P scored as risk scores, that is, the feature amount data 33. In the specific example described above, the feature amounts P read here are the number of peaks Np as the first feature amount P1, the peak height Ip as the second feature amount P2, and the slope S as the third feature amount P3.

[0181] In the following step S122, the CPU 3 determines the quality of the product W (for example, the degree of quality of rust prevention performance) based on the plurality of feature amounts P and the first quality determination model 411 already described.

[0182] Specifically, the CPU 3 inputs the multiple feature quantities P read in step S121, i.e., the feature quantities P obtained from the inspection data 31 of the product W to be analyzed, into the trained first quality judgment model 411. The CPU 3 causes the model 411 to output a quality judgment result (e.g., an estimated result of the pass / fail degree). When quantified as described above, this estimated result is output as quality data 35.

[0183] In the following step S123, the CPU 3 stores the output result of step S122 in the RAM 7, the SSD 9, etc. As a result, the control process returns from the flow of Fig. 12. At that time, the CPU 3 stores the importance of each of the multiple feature amounts P together with the quality determination result.

[0184] 13, the CPU 3 reads the visualized test data 31, i.e., image data 312 corresponding to the test data 31. In the specific example described above, the image data 312 read here is image data obtained by visualizing a current waveform.

[0185] In the following step S132, the CPU 3 determines the quality of the product W (for example, the degree of quality of rust prevention performance) based on the inspection data 31 and the second quality determination model 412 already described.

[0186] Specifically, the CPU 3 inputs the image data 312 read in step S131, i.e., the image data 312 obtained from the inspection data 31 of the product W to be analyzed, into the trained second quality determination model 412. The CPU 3 causes the model 512 to output a quality determination result (e.g., an estimated result of the pass / fail degree). When quantified as described above, this estimated result is output as quality data 35.

[0187] In the following step S133, the CPU 3 stores the output result of step S132 in the RAM 7 or SSD 9. Then, the control process returns from the flow in FIG.

[0188] When both the flows of FIG. 12 and FIG. 13 are completed, the CPU 3 advances the control process from step S13 to step S14.

[0189] (6-4. Update necessity determination process) In the update necessity determination process of step S14, the CPU 3 determines whether or not the quality determination results differ between the types of quality determination models 41. Specifically, the CPU 3 compares the determination result output from the first quality determination model 411 with the determination result output from the second quality determination model 412.

[0190] If the two determination results are significantly consistent, the CPU 3 determines "YES" and advances the control process to step S21 of the process estimation process. If the two determination results are not significantly consistent (different), the CPU 3 determines "NO" and advances the control process to step S31 of the re-learning process shown in FIG. 5B.

[0191] The term "significantly consistent" does not necessarily mean that the two judgment results must be completely consistent. If one judgment result is "normal" and the other is "abnormal," it is determined that they "do not significantly coincide," while if one is "intermediate" and the other is "normal" or "abnormal," it may be determined that they "significantly coincide."

[0192] <7. Analysis Method (2): Process Estimation Processing> Next, we will explain the process estimation process among the processes constituting the analysis method. The following process is executed for each manufactured product W. It may also be executed for products W extracted for each manufacturing lot.

[0193] (7-1. Feature selection process) Fig. 14 is a flowchart showing the details of the feature selection process. When the control process proceeds to step S21 in Fig. 5A, the CPU 3 executes the flow shown in Fig. 14 in order from step S211.

[0194] In the feature selection process, the CPU 3 selects the specific feature Ps from among the multiple feature P used in the quality assessment process (step S1). This selection can be made based on the importance of each of the multiple feature P corresponding to the quality assessment result (for example, the degree of quality).

[0195] Specifically, in step S211, the CPU 3 determines the importance of each of the plurality of feature amounts P. In the following step S212, the CPU 3 selects one or more feature amounts P in descending order of importance as specific feature amounts Ps.

[0196] Here, the importance of each feature P may be scored as a dimensionless number between 0 and 1, and if the score is equal to or greater than a predetermined value (e.g., 0.5), it may be considered to be "highly important." Furthermore, if there are multiple feature Ps with scores thus defined that are equal to or greater than a predetermined value, only the top multiple feature Ps (e.g., three feature Ps) may be selected in descending order of importance.

[0197] In the following step S213, the CPU 3 stores the specific feature amount Ps selected in step S212 in the RAM 7, the SSD 9, etc. Thereafter, the control process returns from the flow in Fig. 14 to the flow in Fig. 5A, and proceeds to step S22.

[0198] (7-2. Process estimation model loading process) In the process estimation model reading process, the CPU 3 reads two or more types (two in this embodiment) of process estimation models 511 and 512 from the SSD 9 serving as a storage unit. Both of the two process estimation training models 511 and 512 associate each of a plurality of feature quantities P with the above-mentioned one or more manufacturing processes Q. Both of the two process estimation training models 511 and 512 have been machine-learned in advance.

[0199] Each of the multiple manufacturing processes Q is quantified to characterize the content of the manufacturing process Q. Each manufacturing process Q includes one or more control factors. Each control factor characterizes the content of the control process Q. These control factors are used as the "manufacturing conditions R" illustrated in FIG. 6 and are quantified as "manufacturing data 39."

[0200] As an example, suppose the first process Q1 is an electrodeposition coating process. In this case, the control factors of the first process Q1 include the electrical conductivity (μS / cm) during electrodeposition coating, the paint temperature, the voltage value during electrodeposition, the ion concentration (MEQ) of the electrodeposition solution, the acid concentration of the electrodeposition solution, the distribution of the electrodeposition solution, the amount and type of solvent in the electrodeposition solution, the presence or absence of foreign matter in the electrodeposition solution, and the takt time.

[0201] The numerical data (manufacturing data 39) quantified corresponding to each manufacturing process Q may be a control target for each control factor, such as a target paint temperature. When the control process Q includes multiple control factors, the numerical data obtained by quantifying each control process Q may be numerical data corresponding to each of the multiple control factors.

[0202] As another example, suppose the second process Q2 is a drying process using a drying oven. In this case, the control factors of the second process Q2 include the amount of moisture in the drying oven, the set temperature of the drying oven, the baking time in the drying oven, the capacity of the drying oven, etc.

[0203] In this embodiment, one of the two process estimation models 51 corresponds to the first process estimation model 511 illustrated in Fig. 4B. The first process estimation model 511 is generated in advance by supervised learning.

[0204] The first process estimation model 511 is trained using third training data. The third training data is composed of numerical data corresponding to each manufacturing process Q (e.g., manufacturing conditions R associated with each manufacturing process Q) and each of a plurality of feature quantities P, which are measured in advance for each of a large number of sample data.

[0205] Here, the first process estimation model 511 is generated using a feature estimation model 513 shown in Fig. 15 etc. The feature estimation model 513 is machine-learned in advance by using the above-mentioned third training data D3 as teacher data, and is a machine learning model that receives numerical data (manufacturing data 39) corresponding to each of a plurality of manufacturing processes Q as input and outputs one feature P.

[0206] By generating the feature estimation model 513, it is possible to determine one or more manufacturing processes Q that contributed to the output of one feature P among the multiple manufacturing processes Q. This selection can be made based on the importance of each of the multiple manufacturing processes Q (see the second importance I2 in FIG. 15). Specifically, the importance of each of the multiple manufacturing processes Q is determined, and one or more manufacturing processes Q are selected in descending order of importance.

[0207] Specifically, the feature estimation model 513 according to this embodiment is a model based on a decision tree algorithm. The feature estimation model 513 based on the decision tree is, for example, an RF model. When an RF model is used, the contribution of each manufacturing process Q (so-called RF importance) can be used as the importance. The importance may be determined based on, for example, the Gini coefficient of a decision tree constituting a random forest model. The first process estimation model 511 created based on the feature estimation model 513 can also be considered to be based on a decision tree algorithm.

[0208] Alternatively, so-called SHAP (Shapley Additive exPlanations) processing may be used. In this case, based on numerical data quantifying each of the multiple manufacturing processes Q and the feature quantity estimation model 513, the feature quantity estimation model 513 is locally approximated around the numerical data corresponding to each manufacturing process Q. A model (second approximation model) generated by approximating the feature quantity estimation model 513 may be, for example, a simple model that easily explains the contribution of the numerical data corresponding to each manufacturing process Q. Then, based on the second approximation model, the contribution of the numerical data corresponding to each manufacturing process Q to the predicted value output from the second approximation model is expressed using a so-called Shapley value, which is used in cooperative game theory, etc. The contribution may be used as the importance for selecting one or more manufacturing processes Q.

[0209] Then, similarly, the manufacturing processes Q with high importance are determined for the other feature quantities P. By performing this for all feature quantities P, it is possible to generate a first process estimation model 511 in which each of the multiple feature quantities P is associated with one or more of the multiple manufacturing processes Q, as shown in FIG.

[0210] Here, the importance of each manufacturing process Q may be scored as a dimensionless number between 0 and 1, and if the score is equal to or greater than a predetermined value (e.g., 0.5), it may be considered to be "highly important." Furthermore, if there are multiple manufacturing processes Q with scores thus defined that are equal to or greater than a predetermined value, only the top three may be selected in descending order of importance.

[0211] 16A, the first process estimation model 511 associates each feature amount P with one or more manufacturing processes Q. Each square in FIG. 16A is associated with the importance of each manufacturing process Q for each feature amount P. The first process estimation model 511 can be considered as a model or map that uses each feature amount P as an input and one or more manufacturing processes Q as an output.

[0212] The manufacturing process Q output from the first process estimation model 511 is a process that has a relatively high importance with respect to the feature value P input to the model 511. In the example of FIG. 16A, the first process estimation model 511 estimates the number of peaks as the feature value P between the first process Q1 and the Mth process Q M and for the peak height as the feature amount P, the Mth step QM is output.

[0213] The other of the two process estimation models 51 is a Bayesian network in which each of the plurality of feature quantities P is a child node and each of the plurality of manufacturing processes Q is a parent node. This Bayesian network can be visualized as a directed graph structure as shown in FIG. 17A, for example. This model corresponds to the second process estimation model 512 illustrated in FIG. 4B. The second process estimation model 512 has been machine-learned in advance by unsupervised learning.

[0214] The second process estimation model 512 is trained using fourth training data D4 (shown only in FIG. 4B ). This fourth training data D4 is composed of waveforms of current changes over time corresponding to each feature value P, which are measured in advance for each of a total of L sample data (measurement data for advance learning, not measurement data for product analysis), and the contents of each of the multiple manufacturing processes Q (numerical data quantified by factor, such as each manufacturing condition R described below).

[0215] Specifically, the second-step estimation model 512 may be a model based on a non-decision tree algorithm. The second-step estimation model 512 related to the non-decision tree may have a graph structure determined by so-called graph structured analysis (hereinafter referred to as "GSA").

[0216] GSA is a big data analysis method proposed by the inventors of the present application that combines probability theory (Bayesian estimation) and graph theory. In this embodiment, GSA is used to determine the graph structure. However, it is not necessary to use GSA to determine the graph structure. For details of GSA, please refer to JP 2021-111063 A.

[0217] In the example of Figure 17A, N feature quantities P and M manufacturing processes Q are nodes, and the dependency between one feature quantity P and one or more manufacturing processes Q is graphed as an edge.

[0218] A probability distribution function corresponding to a Bayesian network is usually expressed by multiplying multiple conditional probabilities. For example, in FIG. 17A, the multiple conditional probabilities include a probability distribution function with the first process Q1 and the second process Q2 as conditions and the first feature P1 as a variable. The former variable is visualized as a child node, and the latter condition is visualized as a first-level parent node. This suggests that the first feature P1 is more strongly dependent on the first process Q1 and the second process Q2 than on the other manufacturing processes Q.

[0219] The graph structure of Fig. 17A can be replaced with a map such as that shown in Fig. 17B. A check mark in each square in Fig. 17B indicates that one feature (e.g., the number of peaks) P is connected to one or more manufacturing processes (e.g., the first process and the Mth process) Q via one edge. A square without a check mark indicates that no such connection exists.

[0220] When a feature value P is input, the second process estimation model 512 outputs one or more manufacturing processes Q connected via an edge to the feature value P. The manufacturing processes Q output from the second process estimation model 512 are processes that have a relatively strong dependency on the feature value P input to the second process estimation model 512.

[0221] Once the first and second process estimation models 511 and 512 are loaded, the control process proceeds from step S22 in FIG. 5A to step S23 in the same figure.

[0222] (7-3. Specific process estimation process) Fig. 18 is a flowchart showing the details of the specific process estimation process. When the control process proceeds to step S23 in Fig. 5A, the CPU 3 executes the flow shown in Fig. 18 in order from step S501.

[0223] Hereinafter, one or more manufacturing processes Q that are estimated to contribute to the assessment of quality among the multiple manufacturing processes Q are referred to as specific processes Qs. In the specific examples shown in Figures 2 and 7, the specific processes Qs are one or more manufacturing processes Q that are estimated to have contributed to the assessment of rust-preventing performance.

[0224] First, in step S231, the CPU 3 reads a specific feature quantity Ps that has been selected in advance from among a plurality of feature quantities P in a feature quantity selection process.

[0225] In the following steps S232 to S234, the CPU 3 estimates a specific process Qs from among the multiple manufacturing processes Q. This estimation is performed based on the specific feature amount Ps and two or more types of process estimation models 51, for each type of the process estimation model 51.

[0226] Specifically, in step S232, the CPU 3 inputs the specific feature quantity Ps to the first process estimation model 511. The first process estimation model 511 outputs a manufacturing process Q that has a high contribution to the specific feature quantity Ps, that is, a specific process Qs. For example, as shown in FIG. 16B, it is assumed that the "number of peaks" is selected as the specific feature quantity Ps. In this case, the specific process Qs corresponds to the first process and the Mth process in the same figure.

[0227] In the following step S233, the CPU 3 inputs the specific feature quantity Ps to the second process estimation model 512. The second process estimation model 512 outputs a manufacturing process Q that has a strong dependency on the specific feature quantity Ps, i.e., a specific process Qs. For example, assume that the second feature quantity P2 in FIGS. 17A and 17B is selected as the specific feature quantity Ps. The specific process Qs in this case corresponds to the second process in these figures.

[0228] In the following step S234, the CPU 3 stores the specific process Qs output for each model in the RAM 7, the SSD 9, etc. Thereafter, the control process returns from the flow in Fig. 18 to the flow in Fig. 5A, and proceeds to step S24.

[0229] (7-4. Update necessity determination process) In the update necessity determination process of step S24, the CPU 3 determines whether or not the estimation results of the specific process Qs differ between the types of estimation determination models 51. Specifically, the CPU 3 compares the estimation result output from the first process estimation model 511 with the estimation result output from the second process estimation model 512.

[0230] If the two estimation results significantly match, the CPU 3 determines "YES" and ends the process illustrated in Fig. 5A. At that time, the CPU 3 digitizes the content of the specific process Qs obtained by each estimation determination model 51 and stores it in the RAM 7, SSD 9, or the like.

[0231] On the other hand, if the two estimation results do not significantly match (are different), the CPU 3 determines "NO" and proceeds to step S31 of the re-learning process shown in Fig. 5B. Note that the term "significantly match" means that it is not necessarily required that the two estimation results completely match.

[0232] In this embodiment, when a specific process Qs is estimated using two or more process estimation models 51, the CPU 3 determines that the estimation results of the specific process Qs differ if a predetermined percentage or more of one or more manufacturing processes Q that make up the specific process Qs differ (first case).

[0233] In this case, instead of the "predetermined ratio," the determination may be made based on whether the number of different manufacturing processes Q is equal to or greater than a "predetermined number." The CPU 3 quantifies the ratio or number of different manufacturing processes Q between the two models as a generation number setting index 37, and stores it in the RAM 7, SSD 9, or the like (see FIGS. 4B and 19).

[0234] In this embodiment, the total number of specific processes Qs estimated by the first and second process estimation models 51 is used as the denominator (overlapping manufacturing processes Q are counted as one), and the ratio (matching rate) of the number of overlapping specific processes Qs among the specific processes Qs estimated by the first and second process estimation models 51 is used as the generation number setting index 37. A parameter that increases or decreases according to the ratio, such as the reciprocal of this ratio, may also be used.

[0235] Furthermore, in this embodiment, when attempting to estimate a specific process Qs using two or more process estimation models 51, if the specific process Qs cannot be estimated using at least one of the process estimation models 51, the CPU 3 determines that the estimation results of the specific process Qs are different.

[0236] For example, in the case of FIG. 17A, the Nth feature amount P N In this case, the Nth feature P N The specific process Qs corresponding to the above cannot be estimated using the model visualized in Fig. 17A. In such a case, the CPU 3 determines that the estimation result of the specific process Qs is incorrect.

[0237] Alternatively to the above configuration, when a specific process Qs is estimated using two or more process estimation models 51, the CPU 3 may determine that the estimation results of the specific process Qs differ if one or more of the one or more manufacturing processes Q that make up the specific process Qs differ (second case).

[0238] In the second case, the CPU 3 determines "NO" in step S24 if the specific process Qs estimated by the first process estimation model 511 and the specific process Qs estimated by the second process estimation model 512 do not completely match. For example, if the specific process Qs estimated by the former is composed of a first process Q1, a second process Q2, and a third process Q3, and the specific process Qs estimated by the latter is composed of only the first process Q1 and the second process Q2, the determination in step S24 will be "NO."

[0239] <8. Analysis Method (3-1): First Re-learning Process> Next, the re-learning process (step S3) in Fig. 5B will be described among the processes constituting the analysis method. The following description is executed when the determination in step S14 or step S24 in Fig. 5A is "NO."

[0240] If the determination in step S14 is "NO," the CPU 3 performs a re-learning process to update the first and second quality determination models 411 and 412. In this case, the CPU 3 may additionally update the first and second process estimation models 511 and 512, respectively, through the re-learning process.

[0241] If the determination in step S24 is "NO," the CPU 3 performs a re-learning process to update the first and second process estimation models 511 and 512. In this case, the CPU 3 may additionally update the first and second quality determination models 411 and 412 by the re-learning process.

[0242] Here, first, a detailed description will be given of the case where the determination in step S24 is "NO." For convenience of explanation, the relearning process performed in this case will be referred to as a "first relearning process."

[0243] (8-1. Test data acquisition process) First, in step S31, the CPU 3 executes an inspection data acquisition process. In this inspection data acquisition process, the CPU 3 acquires the inspection data 31 used to generate the plurality of feature quantities P. The plurality of feature quantities P referred to here refers to the estimation results used in the determination in step S24 (particularly, the estimation results when the determination in step S24 is NO), that is, the plurality of feature quantities P input to each of the first and second process estimation models 511 and 512 during the estimation in step S23.

[0244] The CPU 3 acquires the test data 31 from which the plurality of feature quantities P are obtained. This test data 31 is considered to be data that reflects a "rare event" and has properties different from those of the test data 31 used in pre-learning.

[0245] (8-2. Manufacturing condition generation process) In the following step S32, the CPU 3 executes a manufacturing condition generation process. In this manufacturing condition generation process, the CPU 3 inputs the inspection data 31 acquired in step S31 into a generative AI model (generative AI) M5 to generate a plurality of manufacturing conditions R for manufacturing the product W.

[0246] Here, the generation AI model M5 may input only the test data 31, or may input the multiple feature quantities P referenced in step S31 in addition to the test data 31. The test data 31 may be data showing graphs or plots that show the changes over time of various measurement values, such as the image data 312.

[0247] Furthermore, the manufacturing conditions R output from the generative AI model M5 include, for example, control factors that characterize each manufacturing process Q, as shown in FIG. 6. As described below, the manufacturing conditions R may also include factors that characterize the manufacturing environment of the product W, such as the climate and manufacturing location. In addition, the manufacturing conditions R may also include information that characterizes the worker himself (e.g., the worker's years of experience), work time (the time required for each process), bolt tightening torque applied by the worker, etc. Including information that characterizes the worker himself, work time, and bolt tightening torque in the manufacturing conditions R is useful for processes performed by workers, such as assembly processes. The generative AI model M5 may also be configured to generate natural language that describes each manufacturing condition R.

[0248] Furthermore, the generation AI model M5 used in the manufacturing condition generation process is configured as a model that outputs multiple manufacturing conditions R (e.g., multiple manufacturing data 39) for one piece of inspection data 31 used to analyze the product W, as illustrated in FIG. 19. This model can use various pre-trained machine learning models such as neural networks. For example, the generation AI model M5 may be a large language model (LLM) configured by a neural network.

[0249] The generation AI model M5 is machine-trained in advance, for example, using a plurality of manufacturing conditions R acquired in advance and inspection data 31 corresponding to each manufacturing condition R (see fifth training data D5 in FIG. 19 ). When input and output are performed with reference to a plurality of feature quantities P, the fifth training data D5 may include feature quantity data 33.

[0250] As described above, the generation AI model M5 generates multiple manufacturing conditions R for one input. The CPU 3 is configured to change the number of manufacturing conditions R generated by the generation AI based on the generation number setting index 37.

[0251] 20 is a diagram for explaining the number of generated manufacturing conditions R. As indicated by the line L1 in the figure, the CPU 3 sets a larger number of generated manufacturing conditions R as the generation number setting index 37 (matching rate of the specific process Qs) increases.

[0252] More generally, when the CPU 3 determines that the estimation results of a specific process Qs differ, it increases the number of times that the inspection data 31 is re-acquired as the number of manufacturing processes Q that differ between the types of process estimation models 51 increases among one or more manufacturing processes Q that make up the specific process Qs.

[0253] On the other hand, as in the second case described above, if the re-learning process is always performed unless one or more manufacturing processes Q constituting the specific process Qs are completely identical, the CPU 3 may set the number of generations of the manufacturing conditions R regardless of the size of the generation number setting indicator 37, as shown by the dashed line L2 in Figure 20.

[0254] Furthermore, it is advantageous if the multiple manufacturing conditions R output from the generative AI model M5, when the product W is manufactured under each manufacturing condition R, produce inspection results that reflect rare events, similar to the inspection data 31 input into the generative AI model M5.

[0255] Therefore, by using a plurality of feature quantities P as input as described above, it becomes possible to generate manufacturing conditions R that take into consideration each feature quantity P. This makes it convenient to generate manufacturing conditions R that can obtain similar inspection data 31 when the goal is to obtain inspection data similar to the input inspection data 31 (inspection data that reflects a rare event).

[0256] Furthermore, in order to generate the desired manufacturing conditions R, it would be even more convenient if various instructions related to the feature quantity P could be input to the generative AI model M5. Therefore, the generative AI model M5 according to this embodiment is configured as a so-called interactive AI, and various instructions can be input through its prompts.

[0257] FIG. 21 shows a display screen Sc illustrating an example of the GUI of the generated AI model M5. The display screen Sc is displayed, for example, on a display 11 serving as a display unit. This display screen Sc has a first display item C1 and a second display item C2 showing details of input data, and a third display item C3 and a third display item C4 for entering settings related to the new generation of manufacturing conditions R. Note that the contents of FIG. 21 are merely an example. A general command prompt-like GUI may also be used.

[0258] The first display item C1 displays the identification number of the test data 31 corresponding to the feature data P for which the determination in step S14 or step S24 was NO, that is, the test data 31 that may be considered to be a rare event. The identification number may be, for example, a data value for identifying the CSV format file described above.

[0259] The second display item C2 displays details of the feature data P for which the determination in step S14 or step S24 was NO. All of the multiple feature data P may be displayed, or only the feature data P determined to be specific feature data Ps may be displayed.

[0260] The third display item C3 is a GUI for specifying the number of new production conditions R to be generated. When the check box marked "Automatic" is selected, the CPU 3 determines the number of new productions based on the generation number setting indicator 37. On the other hand, when the check box marked "Manual" is selected as in the illustrated example, the CPU 3 sets the number of new productions to the value entered in the input field C31.

[0261] The fourth display item C4 is a GUI for specifying the target value of each feature P to be acquired from new inspection data 31 acquired from product W manufactured under manufacturing conditions R. If the check box marked "Automatic" is selected, the CPU 3 automatically determines each feature P. On the other hand, if the check box marked "Manual" is selected as in the illustrated example, the CPU 3 sets the numerical value entered in the input field C41 as the target value of each feature P.

[0262] In addition to the feature quantity P, control factors for each manufacturing process Q may be input, or factors that characterize the manufacturing environment of the product W, such as the climate and manufacturing location, may be input. By including these as input targets, it becomes possible to generate additional manufacturing conditions R from a more multifaceted perspective. As mentioned above, the input targets may include information that characterizes the worker himself (for example, the worker's years of experience), work time (the time required for each process), bolt tightening torque applied by the worker, etc.

[0263] The input items Pr set through the third and fourth display items C3 and C4 are input to the generated AI model M5, as shown in FIG.

[0264] It is not necessary to use various check boxes and input fields. By creating a setting file in CSV format in advance and loading the setting file, it is possible to select "automatic" or "manual" and set various values ​​when "manual" is selected. A setting file in CSV format may be created through the GUI shown in FIG. 21 and loaded into the CPU 3.

[0265] (8-3. Test data addition process) In the next step S33, the CPU 3 executes an inspection data addition process, in which the CPU 3 newly acquires a plurality of pieces of inspection data 31 based on the products W manufactured under each of the plurality of manufacturing conditions R generated in step S32.

[0266] Similar to the pre-learning and product analysis, the plurality of pieces of inspection data 31 are newly acquired by the CPU 3 executing the measurement process illustrated in Fig. 8. As described above, the plurality of pieces of inspection data 31 are acquired by inspecting each product W. The inspection may be performed by an operator or automatically by various devices.

[0267] Although not shown in the figure, before proceeding from step S32 to step S33, the manufacturer manufactures a plurality of products W under each of a plurality of manufacturing conditions R. The number of products W to be manufactured is equal to the number of products W under the plurality of manufacturing conditions R.

[0268] Therefore, the number of pieces of inspection data 31 newly acquired in step S33 is the same as the number of manufacturing conditions R generated in step S32. Acquiring a plurality of pieces of inspection data 31, rather than a single piece of inspection data 31, is advantageous for updating various machine learning models.

[0269] (8-4. Feature addition process) In the next step S34, the CPU 3 executes a feature amount addition process, in which the CPU 3 newly generates a plurality of feature amounts P based on each of the inspection data 31 newly acquired in step S33.

[0270] Similar to the pre-learning and product analysis, the CPU 3 newly acquires the plurality of feature quantities P by executing the inspection data analysis process illustrated in FIG.

[0271] The CPU 3 updates the third training data D3 based on the newly acquired plurality of feature quantities P. The third training data D3 is updated by adding, to the third training data D3 referred to during pre-learning, combinations of each manufacturing condition R generated in step S32 and the plurality of feature quantities P newly generated corresponding to each manufacturing condition R. As a result, the number of data items in the third training data D3 is amplified by the number of inspection data 31 added in step S33.

[0272] The CPU 3 updates the fourth training data D4 based on the newly acquired plurality of feature amounts P. The fourth training data D4 is updated by adding, to the fourth training data D4 referred to during pre-learning, combinations of each manufacturing condition R generated in step S32, a plurality of feature amounts P corresponding to each manufacturing condition R, and inspection data 31 corresponding to each feature amount P. As a result, the number of data items in the fourth training data D4 is amplified by the number of inspection data items 31 added in step S33.

[0273] The number of feature amounts P newly acquired in step S34 is equal to the number of pieces of inspection data 31 generated in step S33. In other words, the number of feature amount data 33 newly acquired in step S34 is equal to the number of pieces of inspection data 31 newly generated. Newly acquiring multiple pieces of feature amount data 33, rather than one piece of feature amount data 33, is advantageous for updating various machine learning models.

[0274] (8-5. Machine learning model update process) In the following step S35, the CPU 3 executes a machine learning model update process. In this machine learning model update process, the CPU 3 executes updates of two or more types of process estimation models 51 based on the newly generated multiple feature quantities P.

[0275] Specifically, the CPU 3 updates the feature estimation model 513 with the third training data D3 updated in step S34. The CPU 3 updates the first process estimation model 511 based on the updated feature estimation model 513. The CPU 3 updates the second process estimation model 512 with the fourth training data D4 also updated in step S34.

[0276] <9. Analysis Method (3-2): Second Re-learning Process> Next, a case where the determination in step S14 is "NO" will be described with regard to the relearning process (step S3) in Fig. 5B. For convenience of explanation, the relearning process performed in this case will be referred to as a "second relearning process."

[0277] (9-1. Common part with the first re-learning process) First, the details of the inspection data acquisition process (step S31), the manufacturing condition generation process (step S32), and the inspection data addition process (step S33) are the same as those of the first re-learning process.

[0278] That is, the CPU 3 acquires the inspection data 31 used to generate the plurality of feature quantities P (step S31), and inputs the acquired inspection data 31 into the generative AI model M5 to generate a plurality of manufacturing conditions R for manufacturing the product W (step S33). The CPU 3 newly acquires a plurality of inspection data 31 based on the product W manufactured under each manufacturing condition R (step S33).

[0279] (9-2. Feature addition process) In the next step S34, the CPU 3 executes a feature amount addition process, in which the CPU 3 newly generates a plurality of feature amounts P based on each of the inspection data 31 newly acquired in step S33.

[0280] Similar to the pre-learning and product analysis, the CPU 3 newly acquires the plurality of feature quantities P by executing the inspection data analysis process illustrated in FIG.

[0281] The CPU 3 updates the first training data D1 based on the newly acquired plurality of feature quantities P. The first training data D1 is updated by adding, to the first training data D1 referred to during pre-learning, a combination of quality assessment results acquired by inspecting the product W manufactured under the new manufacturing conditions R and a plurality of feature quantities P newly generated corresponding to each manufacturing condition R. As a result, the number of data items in the first training data D1 is amplified by the number of inspection data 31 added in step S33.

[0282] The CPU 3 updates the second training data D2 based on the newly acquired inspection data 31. The second training data D2 is updated by adding, to the second training data D2 referred to during pre-learning, inspection data 31 acquired by inspecting the product W manufactured under the new manufacturing conditions R, particularly inspection data 31 for when the quality of the product W is desired.

[0283] (9-3. Machine learning model update process) In the next step S35, the CPU 3 executes a machine learning model update process. In this machine learning model update process, the CPU 3 executes updates of two or more types of quality determination models 41 based on the newly generated multiple feature quantities P.

[0284] Specifically, the CPU 3 updates the first quality judgment model 411 with the first training data D1 updated in step S34, and the CPU 3 updates the second quality judgment model 412 with the second training data D2 also updated in step S34.

[0285] <10. Significance of re-learning process> As described above, according to this embodiment, when a discrepancy occurs in the estimation results for a specific process Qs obtained by two or more process estimation models 51, the CPU 3 inputs the inspection data 31 that is considered to be the origin of the estimation result and that is considered to correspond to a rare event into the generation AI (generation AI model M5), as illustrated in Figures 5B and 19. As a result, the CPU 3 generates multiple manufacturing conditions (specifically, manufacturing data 39) R for acquiring other inspection data 31 based on the inspection data 31.

[0286] 5B, the CPU 3 acquires new inspection data 31 based on the product W newly manufactured under the generated manufacturing conditions R. As illustrated in step S35 of the same figure, the CPU 3 updates the process estimation model 51 based on the newly acquired inspection data 31.

[0287] In this way, even if the event is rare, new inspection data 31 related to the event can be generated, and the process estimation model 51, which is a machine learning model related to the quality judgment of product W, can be appropriately retrained.

[0288] 20, if there is even a slight discrepancy in the estimation results of the process estimation model 51, the CPU 3 executes re-learning of the process estimation model 51. This configuration is particularly effective when similar estimation results are expected, such as when the same type of machine learning model is used for the two process estimation models 51 (for example, when both are decision tree-based).

[0289] 20, the CPU 3 executes re-learning of the process estimation model 51 depending on the degree of difference between the estimation results of the process estimation model 51. This configuration is particularly effective when different types of machine learning models are used for the two process estimation models 51 (for example, when a decision tree system and a non-decision tree system are used), in which different estimation results are allowed to a certain extent.

[0290] 20, the number of times inspection data 31 is reacquired (i.e., the number of times new manufacturing conditions R are generated) is changed depending on the degree of discrepancy between the two models (the level of generation number setting index 37). This makes it possible to optimize the number of times inspection data 31 is reacquired, thereby realizing more appropriate re-learning.

[0291] 5B and 19, even if there is a discrepancy in the quality judgment results obtained by two or more types of quality judgment models 41, the CPU 3 inputs the inspection data 31 that is considered to be the origin of the judgment result into the generation AI (generation AI model M5). As a result, the CPU 3 generates multiple manufacturing conditions R for obtaining other inspection data 31 based on the inspection data 31.

[0292] 5B, the CPU 3 acquires new inspection data 31 based on the product W newly manufactured under the generated manufacturing conditions R. As illustrated in step S35 of the same figure, the CPU 3 updates the quality determination model 41 based on the newly acquired inspection data 31.

[0293] In this way, even if the event is rare, new inspection data 31 related to the event can be generated, and the quality judgment model 41, which is a machine learning model related to the quality judgment of product W, can be appropriately retrained.

[0294] In this embodiment, as illustrated in FIG. 10, one of the two or more types of quality determination models 41 is required to use a plurality of feature quantities P in order to be used in the process estimation model 51.

[0295] 11, the second quality judgment model 412 receives image data 312 related to the inspection data 31 as input, rather than a plurality of feature quantities P. This enables a more accurate judgment to be made from a different perspective than that of the first quality judgment model 411.

[0296] 11, the second quality assessment model 412 is configured using an image generation AI (generation AI model M2), which enables more accurate assessment to be made from a different perspective than the first quality assessment model 411.

[0297] 11, the second quality determination model 412 makes a determination based on the luminance of the image data 312. This makes it possible to make a more accurate determination from a different perspective than the first quality determination model 411.

[0298] Furthermore, the generation AI model M2 constituting the second quality judgment model 412 is machine-trained in advance using image data 312 corresponding to the desired quality, as described with reference to Fig. 11. This makes it possible to make a more accurate judgment from a different perspective than the first quality judgment model 411.

[0299] 16A and 17A, models constructed from different perspectives are used for the first process estimation model 511 and the second process estimation model 512. This makes it possible to realize analysis from more multifaceted perspectives.

[0300] 17A, the second process estimation model 512 determines the manufacturing process Q based on a probabilistic connection (dependence relationship), thereby enabling analysis from more multifaceted perspectives.

[0301] <11. Other embodiments> In the embodiment, as illustrated in Fig. 5A, a method is disclosed that includes both step S14 for updating the quality determination model 41 and step S24 for updating the process estimation model 51, but in the present disclosure, it is not essential to include both of these two steps. It is sufficient to include either one of them.

[0302] That is, the analysis method according to the present disclosure only needs to include at least one of a process for updating the quality determination model 41 and a process for updating the process estimation model 51.

[0303] Furthermore, in the above embodiment, the output data (quality data 35) of the quality judgment program 235 is used as the quality judgment result, but the present disclosure is not limited to such a configuration.

[0304] In particular, when the process for updating the quality determination model 41 is omitted, at least the quality determination process (step S13) in the flow shown in FIG. 5A may be omitted.

[0305] The "quality judgment result" in the present disclosure may be, instead of or in addition to the output from a pre-generated machine learning model, a judgment result by a factory worker or a judgment result by rule-based judgment using multiple feature quantities P as input. For example, when using the judgment result by a factory worker, as described above, the configuration of the process estimation model 51 can be reconfigured so that the text data recorded by the factory worker is used as input.

[0306] The feature value P used in this process may be, for example, the index (color, shape, texture, etc. of the product) used by a worker when he or she judges the quality based on experience. By quantifying these indexes, the process similar to that of the above embodiment can be performed.

[0307] In addition, in the above embodiment, a machine learning model based on a decision tree algorithm is exemplified as an example of a nonlinear regression model, but the present disclosure is not limited to such a configuration. Instead of the decision tree algorithm, a so-called support vector machine may also be used.

[0308] Additionally, although the above embodiment has shown an example in which the analysis device is configured using one computer 1, the present disclosure is not limited to this example. The analysis method and analysis program 29 according to the present disclosure may be executed using multiple computers 1, such as by having a first computer execute one or more processes related to the quality assessment process, while having a second computer execute one or more processes constituting the process estimation process and / or the relearning process. Furthermore, the computer 1 in the present disclosure also includes parallel computers such as supercomputers and PC clusters.

[0309] Furthermore, the screens on which various types of information can be displayed are not limited to the display screen on the display 11 of the computer 1. Various types of screens may be displayed on a display device prepared separately from the computer 1. [Explanation of symbols]

[0310] 1. Computer 3 CPU (arithmetic unit) 7 RAM (memory section) 9 SSD (storage unit) 11 Display (display unit) 15 Keyboard (reception area) 17 Mouse (Reception) 18 Storage medium 21 Analysis Program 23 Quality Assessment Program 25 Process Estimation Program 27 Relearning Program 31 Test Data 33 Feature Data 35 Generation number setting index 41 Quality Judgment Model 411 First quality judgment model 412 Second quality judgment model 51 Process Estimation Model 511 First Process Estimation Model 512 Second Process Estimation Model 513 Feature Estimation Model D1 First training data D2 Second training data D3 Third training data D4 Fourth training data D5 5th training data M2 Generative AI Model M5 Generative AI Model (Generative AI) P feature Ps specific feature Q Manufacturing process Qs specific process R Manufacturing conditions W Products

Claims

1. A manufacturing process analysis method executed using a computer including a storage unit and a calculation unit, and a quality assessment result of a product obtained by inspecting a product manufactured through a plurality of manufacturing processes, comprising: Among a plurality of feature quantities generated from inspection data obtained by inspecting the product and used for determining the quality, one or more feature quantities estimated to contribute to the determination of the quality are defined as specific feature quantities, and among the plurality of manufacturing processes, one or more manufacturing processes estimated to contribute to the determination of the quality are defined as specific processes. the calculation unit reads from the storage unit two or more types of process estimation models that have been machine-learned in advance so as to associate each of the plurality of feature quantities with one or more of the plurality of manufacturing processes; the calculation unit reads the specific feature quantity selected in advance from the plurality of feature quantities; the calculation unit estimates the specific process for each type of process estimation model based on the specific feature amount and the two or more types of process estimation models; If the estimation results of the specific process differ between the types of the process estimation models, the calculation unit acquires the inspection data used to generate the plurality of feature amounts; The calculation unit inputs the inspection data into a generation AI to generate a plurality of manufacturing conditions for manufacturing the product; the calculation unit newly acquires a plurality of pieces of inspection data based on products manufactured under each of the plurality of manufacturing conditions; The calculation unit newly generates the plurality of feature quantities based on each of the plurality of newly acquired inspection data, and updates the two or more types of process estimation models based on the newly generated plurality of feature quantities. A manufacturing process analysis method comprising:

2. 2. The manufacturing process analysis method according to claim 1, When the specific process is estimated using the two or more process estimation models, if one or more of one or more manufacturing processes constituting the specific process are different, the calculation unit determines that the estimation results of the specific process are different. A manufacturing process analysis method comprising:

3. 2. The manufacturing process analysis method according to claim 1, The calculation unit determines that the estimation results of the specific process are different when one or more manufacturing processes constituting the specific process differ by a predetermined percentage or more when the specific process is estimated using the two or more process estimation models. A manufacturing process analysis method comprising:

4. 4. The manufacturing process analysis method according to claim 2 or 3, When it is determined that the estimation results of the specific process differ, the calculation unit increases the number of re-acquisitions of the inspection data as the number of manufacturing processes that differ between the types of the process estimation models increases among one or more manufacturing processes that constitute the specific process. A manufacturing process analysis method comprising:

5. A manufacturing process analysis method executed using a computer including a storage unit and a calculation unit, and a quality assessment result of a product obtained by inspecting a product manufactured through a plurality of manufacturing processes, comprising: Among a plurality of feature quantities generated from inspection data obtained by inspecting the product and used for determining the quality, one or more feature quantities estimated to contribute to the determination of the quality are defined as specific feature quantities, and among the plurality of manufacturing processes, one or more manufacturing processes estimated to contribute to the determination of the quality are defined as specific processes. the calculation unit reads from the storage unit two or more types of quality judgment models, each of which includes a first quality judgment model that has been machine-learned in advance to associate the plurality of feature quantities with the quality judgment result, and a second quality judgment model that has been machine-learned in advance to associate the inspection data with the quality judgment result; the calculation unit acquires the plurality of feature amounts and the inspection data, the calculation unit executes the quality judgment based on the plurality of feature amounts and the first quality judgment model, and based on the inspection data and the second quality judgment model; the calculation unit reads from the storage unit two or more types of process estimation models that have been machine-learned in advance so as to associate each of the plurality of feature quantities with one or more of the plurality of manufacturing processes; the calculation unit reads the specific feature from among the plurality of feature amounts, the calculation unit estimates the specific process for each type of process estimation model based on the specific feature amount and the two or more types of process estimation models; If the quality assessment results differ between the types of quality assessment models, the calculation unit acquires the inspection data used to generate the plurality of feature amounts; The calculation unit inputs the inspection data into a generation AI to generate a plurality of manufacturing conditions for manufacturing the product; the calculation unit newly acquires a plurality of pieces of inspection data based on products manufactured under each of the plurality of manufacturing conditions; The calculation unit newly generates the plurality of feature quantities based on each of the plurality of newly acquired inspection data, and updates the two or more types of quality determination models based on the newly generated inspection data and the plurality of feature quantities. A manufacturing process analysis method comprising:

6. 6. The manufacturing process analysis method according to claim 5, The second quality judgment model includes a generation AI that receives information corresponding to the inspection data as an input and outputs information corresponding to the quality judgment result. A manufacturing process analysis method comprising:

7. 7. The manufacturing process analysis method according to claim 6, The generation AI is an image generation AI that receives image data representing the inspection data as input, converts the input image data so as to reflect the quality of the product, and outputs the converted image data. A manufacturing process analysis method comprising:

8. 8. The manufacturing process analysis method according to claim 7, The second quality determination model determines the quality of the product based on a difference between the luminance of the image data before conversion by the generation AI and the luminance of the image data after conversion by the generation AI. A manufacturing process analysis method comprising:

9. 9. The manufacturing process analysis method according to claim 7 or 8, The generating AI restores and outputs the input image data if the quality of the product is desired, The generation AI is pre-trained by using, as training data, inspection data obtained multiple times when the product quality is desired. A manufacturing process analysis method comprising:

10. 10. The manufacturing process analysis method according to claim 1, further comprising: The two or more process estimation models are a first-stage estimation model based on a decision tree algorithm; a second step estimation model based on a non-decision tree algorithm; A manufacturing process analysis method comprising:

11. 11. The manufacturing process analysis method according to claim 10, the second process estimation model is a Bayesian network in which each of the plurality of feature quantities is a child node and each of the plurality of manufacturing processes is a parent node; The calculation unit determines, as the specified process, a manufacturing process that is connected to the specific feature via one edge when the Bayesian network is visualized as a directed graph structure. A manufacturing process analysis method comprising:

12. An apparatus for analyzing a manufacturing process using a computer having a storage unit and a calculation unit, and a result of product quality assessment obtained by inspecting a product manufactured through a plurality of manufacturing processes, the apparatus comprising: Among a plurality of feature quantities generated from inspection data obtained by inspecting the product and used for determining the quality, one or more feature quantities estimated to contribute to the determination of the quality are defined as specific feature quantities, and among the plurality of manufacturing processes, one or more manufacturing processes estimated to contribute to the determination of the quality are defined as specific processes. a means for reading from the storage unit two or more types of process estimation models that have been machine-learned in advance so as to associate each of the plurality of feature quantities with one or more of the plurality of manufacturing processes; means for reading the specific feature quantity selected in advance from the plurality of feature quantities; a means for estimating the specific process for each type of process estimation model based on the specific feature and the two or more types of process estimation models; When the estimation results of the specific process differ between the types of the process estimation models, acquiring the inspection data used to generate the plurality of feature amounts; The inspection data is input into a generation AI to generate a plurality of manufacturing conditions for manufacturing the product; newly acquiring a plurality of pieces of inspection data based on products manufactured under each of the plurality of manufacturing conditions; and means for newly generating the plurality of feature quantities based on each of the plurality of newly acquired inspection data, and updating the two or more process estimation models based on the newly generated plurality of feature quantities. A manufacturing process analysis device characterized by:

13. A manufacturing process analysis program executed using a computer including a storage unit and a calculation unit, and a quality assessment result of a product obtained by inspecting a product manufactured through a plurality of manufacturing processes, the program comprising: Among a plurality of feature quantities generated from inspection data obtained by inspecting the product and used for determining the quality, one or more feature quantities estimated to contribute to the determination of the quality are defined as specific feature quantities, and among the plurality of manufacturing processes, one or more manufacturing processes estimated to contribute to the determination of the quality are defined as specific processes. The computer, a process in which the calculation unit reads from the storage unit two or more types of process estimation models that have been machine-learned in advance so as to associate each of the plurality of feature quantities with one or more of the plurality of manufacturing processes; a process in which the calculation unit reads the specific feature quantity selected in advance from the plurality of feature quantities; a process in which the calculation unit estimates the specific process for each type of process estimation model based on the specific feature and the two or more types of process estimation models; When the estimation results of the specific process differ between the types of the process estimation models, the calculation unit acquires the inspection data used to generate the plurality of feature amounts; The calculation unit inputs the inspection data into a generation AI to generate a plurality of manufacturing conditions for manufacturing the product; the calculation unit newly acquires a plurality of pieces of inspection data based on products manufactured under each of the plurality of manufacturing conditions; the calculation unit executes a process of newly generating the plurality of feature quantities based on each of the plurality of newly acquired inspection data, and updating the two or more types of process estimation models based on the newly generated plurality of feature quantities.

14. The analysis program according to claim 13 is stored. A computer-readable storage medium comprising:

15. An apparatus for analyzing a manufacturing process using a computer having a storage unit and a calculation unit, and a result of product quality assessment obtained by inspecting a product manufactured through a plurality of manufacturing processes, the apparatus comprising: Among a plurality of feature quantities generated from inspection data obtained by inspecting the product and used for determining the quality, one or more feature quantities estimated to contribute to the determination of the quality are defined as specific feature quantities, and among the plurality of manufacturing processes, one or more manufacturing processes estimated to contribute to the determination of the quality are defined as specific processes. a means for reading from the storage unit two or more types of quality judgment models, each of which includes a first quality judgment model that has been machine-learned in advance to associate the plurality of feature quantities with the quality judgment result, and a second quality judgment model that has been machine-learned in advance to associate the inspection data with the quality judgment result; means for acquiring the plurality of feature amounts and the inspection data; means for executing the quality judgment based on the plurality of feature amounts and the first quality judgment model, and the inspection data and the second quality judgment model; a means for reading from the storage unit two or more types of process estimation models that have been machine-learned in advance so as to associate each of the plurality of feature quantities with one or more of the plurality of manufacturing processes; means for reading the specific feature from among the plurality of feature amounts; a means for estimating the specific process for each type of process estimation model based on the specific feature and the two or more types of process estimation models; When the quality determination results differ between the types of the quality determination models, acquiring the inspection data used to generate the plurality of feature amounts; The inspection data is input into a generation AI to generate a plurality of manufacturing conditions for manufacturing the product; newly acquiring a plurality of pieces of inspection data based on products manufactured under each of the plurality of manufacturing conditions; and means for newly generating the plurality of feature quantities based on each of the plurality of newly acquired inspection data, and updating the two or more types of quality determination models based on the newly generated inspection data and the plurality of feature quantities. A manufacturing process analysis device characterized by:

16. A manufacturing process analysis program executed using a computer including a storage unit and a calculation unit, and a quality assessment result of a product obtained by inspecting a product manufactured through a plurality of manufacturing processes, the program comprising: Among a plurality of feature quantities generated from inspection data obtained by inspecting the product and used for determining the quality, one or more feature quantities estimated to contribute to the determination of the quality are defined as specific feature quantities, and among the plurality of manufacturing processes, one or more manufacturing processes estimated to contribute to the determination of the quality are defined as specific processes. The computer, a process in which the calculation unit reads from the storage unit two or more types of quality judgment models, each of which includes a first quality judgment model that has been machine-learned in advance to associate the plurality of feature quantities with the quality judgment result, and a second quality judgment model that has been machine-learned in advance to associate the inspection data with the quality judgment result; a process in which the calculation unit acquires the plurality of feature amounts and the inspection data; a process in which the calculation unit executes a quality judgment based on the plurality of feature amounts and the first quality judgment model, and based on the inspection data and the second quality judgment model; a process in which the calculation unit reads from the storage unit two or more types of process estimation models that have been machine-learned in advance so as to associate each of the plurality of feature quantities with one or more of the plurality of manufacturing processes; a process in which the calculation unit reads the specific feature from among the plurality of feature amounts; a process in which the calculation unit estimates the specific process for each type of process estimation model based on the specific feature and the two or more types of process estimation models; When the quality determination results differ between the types of the quality determination models, the calculation unit acquires the inspection data used to generate the plurality of feature amounts; The calculation unit inputs the inspection data into a generation AI to generate a plurality of manufacturing conditions for manufacturing the product; the calculation unit newly acquires a plurality of pieces of inspection data based on products manufactured under each of the plurality of manufacturing conditions; the calculation unit executes a process of newly generating the plurality of feature quantities based on each of the plurality of newly acquired inspection data, and updating the two or more types of quality determination models based on the newly generated inspection data and the plurality of feature quantities. A manufacturing process analysis program characterized by:

17. The analysis program according to claim 16 is stored. A computer-readable storage medium comprising:

Images