Estimation program, information processing device, and estimation method

Abstract

An information processing device (1): identifies, from a prediction result from an XAI (40) that predicts a replacement part, two parts for which the incidence of misprediction exceeds a predetermined value, the misprediction such that if one part is a correct answer then another part is taken to be a prediction result; acquires BOM data (11) related to the two identified parts that have had mispredictions, as well as diagnostic data (33) indicating diagnosis contents for a worker corresponding to the BOM data (11); and uses the diagnostic data (33) to estimate a degree of improvement in prediction accuracy for a case where a feature amount is added to training data for the XAI (40), with regard at least one feature amount that is a candidate for addition to the BOM data (11) that is identified on the basis of link data (34) linking together the diagnosis contents included in the diagnostic data (33) and the feature amount. The information processing device (1) is thereby enabled to select a feature necessary in order to improve the accuracy of prediction if a replacement part is to be predicted.

Description

Estimation program, information processing device, and estimation method

[0001] The present invention relates to an estimation program and the like.

[0002] In the manufacturing process of a product, machine learning is used to predict replacement parts. For example, in the manufacturing process of a measuring instrument, a technology is disclosed that predicts defective parts of the measuring instrument from result data obtained from environmental testing using a defective part prediction model (see, for example, Patent Document 1). Such machine learning often cannot explain the basis of the prediction.

[0003] In recent years, predictive models using AI have been developed. Such predictive models are called Explainable AI (XAI), a technology that enables the explanation of the basis for AI predictions. XAI can explain the degree to which each feature contributes to the prediction result. Furthermore, XAI can be used to predict replacement parts in the product manufacturing process.

[0004] Japanese Patent Application Laid-Open No. 2022-22499

[0005] However, when predicting replacement parts using XAI or machine learning, there is a problem in that it is difficult to select features necessary to improve the accuracy of the prediction. It is unavoidable to select features necessary to maintain high accuracy of the prediction. In particular, in XAI, the basis of the prediction is explained based on the input features, so it is unavoidable to select features that are easy to use in explaining the basis. However, indiscriminately converting features into data is costly.

[0006] In one aspect, the present invention aims to select a feature amount necessary to improve the accuracy of prediction when predicting replacement parts.

[0007] In one aspect, the estimation program causes a computer to execute the following process: identify two parts from the prediction results of a prediction model that predicts replacement parts, for which the occurrence rate of prediction errors, in which one part is predicted as the other part when the other part is correct, exceeds a predetermined value; acquire input data related to the two parts with the identified prediction errors and diagnostic data corresponding to the input data that indicates the diagnosis content of the worker; and, for one or more feature values ​​that are candidates for addition to the input data, identified based on the correspondence between the diagnostic content and feature values ​​included in the diagnostic data, estimate the degree of improvement in prediction accuracy when the feature value is added to the training data of the prediction model, using the diagnostic data.

[0008] According to one embodiment, when predicting replacement parts, it is possible to select feature amounts necessary to improve the accuracy of prediction.

[0009] FIG. 1 is a diagram illustrating an example of a functional configuration of an information processing apparatus according to an embodiment. FIG. 2 is a diagram illustrating an example of a functional configuration of a learning unit according to an embodiment. FIG. 3 is a diagram illustrating an example of BOM data according to an embodiment. FIG. 4 is a diagram illustrating an example of defect phenomenon data according to an embodiment. FIG. 5 is a diagram illustrating an example of repair correspondence data according to an embodiment. FIG. 6 is a diagram illustrating an example of a functional configuration of a prediction unit according to an embodiment. FIG. 7 is a diagram illustrating an example of prediction result data according to an embodiment. FIG. 8 is a diagram illustrating an example of a functional configuration of an estimation unit according to an embodiment. FIG. 9 is a diagram illustrating an example of diagnostic data according to an embodiment. FIG. 10 is a diagram illustrating an example of linked data according to an embodiment. FIG. 11 is a diagram illustrating an example of prediction error evaluation according to an embodiment. FIG. 12 is a diagram illustrating an example of additional feature search according to an embodiment. FIG. 13 is a diagram illustrating an example of calculation of an accuracy improvement degree according to an embodiment. FIG. 14 is a diagram illustrating priority of feature selection. FIG. 15 is a diagram illustrating an example of a flowchart of prediction error evaluation processing according to an embodiment. FIG. 16 is a diagram illustrating an example of a flowchart of additional feature search processing according to an embodiment. FIG. 17 is a diagram illustrating a modified example of additional feature search according to an embodiment. FIG. 18 is a diagram illustrating an example of a hardware configuration.

[0010] Hereinafter, embodiments of the estimation program, information processing device, and estimation method disclosed in the present application will be described in detail with reference to the accompanying drawings. However, the present invention is not limited to the embodiments.

[0011] 1 is a diagram illustrating an example of a functional configuration of an information processing device according to an embodiment. As illustrated in FIG. 1, the information processing device 1 includes a learning unit 10, a prediction unit 20, an estimation unit 30, and an XAI 40.

[0012] XAI40 is an AI model that uses explainable AI (XAI). That is, XAI40 is explainable AI that allows the basis for predictions made by the AI ​​to be explained. Examples of XAI40 include random forest and wide learning. In the embodiment, a case where XAI40 is wide learning will be described, but the present invention is not limited to this.

[0013] The learning unit 10 learns the XAI 40. For example, the learning unit 10 uses the group of parts constituting a product, the defect phenomenon, and the replacement parts actually replaced as training data to train the XAI 40 (AI model) in the manufacturing process of a product. As an example, the learning unit 10 uses the group of parts constituting a product, the defect phenomenon, and the replacement parts actually replaced as training data to learn rules for predicting replacement parts using the rule-based model indicated by the XAI 40. The rules include a combination of two feature quantities and also include the class (e.g., CPU / memory, etc.). The individual parts constituting a product, the defect phenomenon, and the replacement parts are each feature quantities. Details of the learning unit 10 will be described later.

[0014] The prediction unit 20 predicts replacement parts using the XAI 40. For example, the prediction unit 20 inputs the group of parts that make up the product detected as abnormal and the defect phenomenon that indicates the abnormality into the XAI 40, and outputs replacement parts from the XAI 40. Specifically, the prediction unit 20 inputs the group of parts that make up the product detected as abnormal and the defect phenomenon that indicates the abnormality into a rule-based model indicated by the XAI 40, and predicts replacement parts that fit the rules. The prediction unit 20 then accumulates the prediction results. Details of the prediction unit 20 will be described later.

[0015] The estimation unit 30 estimates feature quantities to be added to the training data. For example, the estimation unit 30 evaluates prediction errors for each class (e.g., CPU / memory, etc.) using the rules and prediction results learned by the XAI 40. The estimation unit 30 then compares the difficult-to-predict classes and the rules at that time obtained from the evaluation of prediction errors with the diagnostic data to estimate feature quantities to be added. The diagnostic data here refers to information that has been digitized from the diagnostic content performed by on-site workers to determine replacement parts. As an example, the diagnostic data may include information indicating that "X" was checked to determine whether the replacement part is "A" or "B." Details of the estimation unit 30 will be described later.

[0016] Here, the functional configuration of the learning unit 10 according to the embodiment will be described with reference to FIG. 2 . FIG. 2 is a diagram illustrating an example of the functional configuration of the learning unit according to the embodiment. The functional configuration of FIG. 2 is also illustrated along with a flow chart of a product manufacturing process. As illustrated in FIG. 2 , the learning unit 10 includes bill of materials (BOM) data 11, defect phenomenon data 12, repair correspondence data 13, and an XAI 40. The learning unit 10 uses the BOM data 11, defect phenomenon data 12, and repair correspondence data 13 to learn the XAI 40. As an example, the learning unit 10 uses the BOM data 11, defect phenomenon data 12, and repair correspondence data 13 to learn rules for predicting replacement parts based on a rule-based model indicated by the XAI 40.

[0017] The BOM data 11 is information indicating, for each product, the group of components that make up each product. An example of a product is a personal computer (PC). The BOM data 11 is stored during the manufacturing phase of a production flow. An example of the BOM data 11 will now be described with reference to FIG. 3 . FIG. 3 is a diagram illustrating an example of BOM data according to an embodiment. The BOM data 11 stores a device identification (ID), a CPU, a memory, and so on, in association with each other. The device ID is an identifier that uniquely identifies the device to be manufactured. The CPU, memory, and so on are information about each component that makes up the device to be manufactured. The information about each component stores a type. For example, the information about the CPU may store a classification such as Intel, AMD, or Arm, or a classification of the CPU architecture. Note that the information about each component is not limited to a type and may be a detection value detected during the manufacturing phase. The information about each component is used as a feature.

[0018] As an example, when the device id is "1", "A" (for example, AMD) is stored as the CPU, "X" (for example, DDR4) is stored as the memory, and so on.

[0019] Returning to FIG. 2 , the defect phenomenon data 12 is information that associates a product with a defect phenomenon. The defect phenomenon data 12 is stored in the test phase of the on-site flow. An example of the defect phenomenon data 12 will now be described with reference to FIG. 4 . FIG. 4 is a diagram showing an example of defect phenomenon data according to an embodiment. As shown in FIG. 4 , the defect phenomenon data 12 stores a device ID and a defect phenomenon in association with each other. The device ID is an identifier that uniquely identifies a manufacturing device, and is linked to the device ID of the BOM data 11. A defect phenomenon is a phenomenon that indicates a malfunction.

[0020] For example, when the device ID is "1", "screen does not light up" is stored as the malfunction phenomenon.

[0021] Returning to Fig. 2, the repair correspondence data 13 is data that stores replacement parts as a result of actual repair work. The repair correspondence data 13 is stored in the repair phase of the on-site flow. An example of the repair correspondence data 13 will now be described with reference to Fig. 5. Fig. 5 is a diagram showing an example of repair correspondence data according to an embodiment. As shown in Fig. 5, the repair correspondence data 13 stores replacement parts as a result of actual repair work.

[0022] As an example, "CPU replacement" is stored as a repair response, and "memory replacement" is also stored as a repair response.

[0023] Next, the functional configuration of the prediction unit 20 according to the embodiment will be described with reference to FIG. 6 . FIG. 6 is a diagram illustrating an example of the functional configuration of the prediction unit according to the embodiment. The functional configuration of FIG. 6 is shown together with the on-site flow of the product manufacturing process. As shown in FIG. 6 , the prediction unit 20 has BOM data 11, defect phenomenon data 12, prediction result data 21, and XAI 40. XAI 40 is a trained AI model trained by the learning unit 10.

[0024] The prediction unit 20 inputs the BOM data 11 and defect phenomenon data 12 of the prediction target into the trained AI model XAI 40 to predict a replacement part corresponding to the prediction target. As an example, the prediction unit 20 inputs the BOM data 11 and defect phenomenon data 12 of the prediction target into a rule-based model indicated by the trained XAI 40 to predict a replacement part that satisfies the rule. The prediction unit 20 then orders the predicted replacement part. That is, the prediction unit 20 outputs an order request for the predicted replacement part to the parts ordering phase in the on-site flow.

[0025] In addition, the prediction unit 20 stores the prediction result in the prediction result data 21. For example, the prediction unit 20 adds a record associating the BOM data 11 and defect phenomenon data 12 to be predicted with the replacement part indicating the prediction result to the prediction result data 21. Thereafter, the prediction unit 20 simply updates the record with the replacement part indicating the correct answer.

[0026] Returning to FIG. 6 , the prediction result data 21 is data storing prediction results. Here, an example of the prediction result data 21 will be described with reference to FIG. 7 . FIG. 7 is a diagram showing an example of prediction result data according to an embodiment. As shown in FIG. 7 , the prediction result data 21 stores a device ID, a CPU, a memory, ..., a faulty phenomenon, a correct answer, and a prediction in association with each other. The device ID is the device ID of the prediction target, and corresponds to the device IDs in the BOM data 11 and the faulty phenomenon data 12. The CPU, memory, ... are the BOM data 11 of the prediction target, and are information about the device corresponding to the device ID in the BOM data 11. The faulty phenomenon is the faulty phenomenon data 12 of the prediction target, and are information about the faulty phenomenon corresponding to the device ID in the faulty phenomenon data 12. The correct answer indicates the correct replacement part. The prediction indicates the replacement part predicted by the XAI 40.

[0027] As an example, when the device ID is "10", "A" is stored as the CPU, "X" as the memory, "xxx" as the faulty phenomenon, "CPU" as the correct answer, and "CPU" as the prediction. When the device ID is "24", "A" is stored as the CPU, "X" as the memory, "yyy" as the faulty phenomenon, "CPU" as the correct answer, and "memory" as the prediction.

[0028] Next, the functional configuration of the estimation unit 30 according to the embodiment will be described with reference to FIG. 8 . FIG. 8 is a diagram illustrating an example of the functional configuration of the estimation unit according to the embodiment. The functional configuration of FIG. 8 is also illustrated along with the on-site flow of the product manufacturing process. As shown in FIG. 8 , the estimation unit 30 includes BOM data 11, prediction result data 21, a prediction error evaluation unit 31, an additional feature search unit 32, diagnostic data 33, linking data 34, and an XAI 40. The prediction error evaluation unit 31 is an example of an identification unit. The additional feature search unit 32 is an example of an acquisition unit and an estimation unit. The XAI 40 is an example of a prediction model. The BOM data 11 is an example of input data.

[0029] XAI40 is a trained AI model trained by the training unit 10. BOM data 11 is the same information as the BOM data 11 used by the training unit 10. Prediction result data 21 is information generated by the prediction unit 20.

[0030] The diagnostic data 33 is information that represents the diagnostic content performed by an on-site worker to determine the replacement part. In other words, the diagnostic data 33 can also be said to be on-site knowledge data. An example of the diagnostic data 33 will now be described with reference to FIG. 9.

[0031] FIG. 9 is a diagram illustrating an example of diagnostic data according to an embodiment. As illustrated in FIG. 9, the diagnostic data 33 stores a device ID, a diagnostic class a, a diagnostic class b, a ZZ voltage check, an LED power check, and so on, in association with each other. The device ID is the device ID of the prediction target, and corresponds to the device ID in the BOM data 11 and the defect phenomenon data 12. The diagnostic class a is one of the components of the diagnostic target. The diagnostic class b is the other component of the diagnostic target. The ZZ voltage check and the LED power check are diagnostic items performed by an on-site worker. For each diagnostic item, an identifier indicating whether or not the diagnostic item has been diagnosed is set. If the diagnostic item has been diagnosed, "0" is set as an example. If the diagnostic item has not been diagnosed, "X" is set as an example.

[0032] As an example, when the device ID is "10", "memory" is stored as diagnostic class a, "CPU" as diagnostic class b, "◯" as ZZ voltage check, and "X" as LED power check. In other words, it shows that in a case where a malfunction occurred in the device indicated by device ID "10", an on-site worker performed a "ZZ voltage check" to determine whether "memory" or "CPU" should be replaced, but did not perform an "LED power check".

[0033] Returning to FIG. 8 , the linking data 34 is information linking diagnostic items with feature amounts. An example of the linking data 34 will now be described with reference to FIG. 10 . FIG. 10 is a diagram showing an example of linking data according to an embodiment. As shown in FIG. 10 , the linking data 34 stores diagnostic items and feature amounts in association with each other. Diagnostic items are names indicating the diagnostic content performed by on-site workers. Feature amounts are names indicating the diagnostic content used when training the XAI 40.

[0034] For example, when the diagnostic item is "ZZ voltage check," "ZZ voltage" is stored as the feature amount. When the diagnostic item is "LED power check," "LED light intensity" is stored as the feature amount.

[0035] Returning to FIG. 8 , the prediction error evaluation unit 31 evaluates prediction errors. For example, the prediction error evaluation unit 31 evaluates prediction errors for each of two classes using rules learned by the XAI 40 and the prediction result data 21. For example, the prediction error evaluation unit 31 acquires multiple rules learned by the rule-based model of the XAI 40. For each rule, the prediction error evaluation unit 31 calculates the prediction error rates of the two classes using the prediction result data 21. Then, based on the calculation results, the prediction error evaluation unit 31 identifies two classes corresponding to rules whose occurrence rates of prediction errors exceed a predetermined value. The identified two classes are two parts that are difficult to predict as replacement parts. The predetermined value may be 0.5 or 0.7. The predetermined value can be changed as appropriate.

[0036] The additional feature search unit 32 searches for additional features by comparing the diagnostic data 33 with the two classes in which prediction errors frequently occur (are difficult to predict) and the rules at that time identified by the prediction error evaluation unit 31. For example, the additional feature search unit 32 acquires BOM data 11 that applies to the identified rules and diagnostic data 33 corresponding to the BOM data 11. The additional feature search unit 32 then collects diagnostic data 33 having the two identified classes from the acquired diagnostic data 33. The additional feature search unit 32 then identifies, based on the linking data 34, features corresponding to diagnostic items with one or more "o" marks included in the collected diagnostic data 33. The additional feature search unit 32 then uses the collected diagnostic data 33 and prediction result data 21 to estimate the rate at which prediction accuracy can be improved by adding the identified features to training data for learning. That is, the additional feature search unit 32 uses the collected diagnostic data 33 to estimate the degree of improvement in prediction accuracy when one or more candidate features to be added to the BOM data 11, which are identified based on the linking data 34, are added to the training data of XAI40.

[0037] The additional feature search unit 32 then outputs the estimated degree of improvement in prediction accuracy, along with candidate features to add, two classes in which prediction errors occur frequently, and the rules at that time. In this way, the additional feature search unit 32 estimates the degree of improvement in prediction accuracy for a feature using the diagnostic data 33, and can select a feature that has a greater effect on improving prediction accuracy while checking the knowledge of on-site workers.

[0038] When two or more feature quantities to be added are output, the additional feature quantity search unit 32 selects, from the feature quantities to be added, a feature quantity to be recommended for addition to the BOM data 11 based on the estimated degree of improvement in prediction accuracy. In this way, the additional feature quantity search unit 32 adds the selected feature quantity to the BOM data 11, thereby making it possible to improve the prediction accuracy of the XAI 40.

[0039] [Example of Prediction Mistake Evaluation] Here, an example of prediction mistake evaluation performed by the prediction mistake evaluation unit 31 according to the embodiment will be described with reference to FIG. 11. FIG. 11 is a diagram showing an example of prediction mistake evaluation according to the embodiment. FIG. 11 shows a prediction mistake evaluation result 31A. A plurality of rules obtained by the XAI 40 are set in the prediction mistake evaluation result 31A. The plurality of rules are learned by the XAI 40 of the learning unit 10. Each rule is represented by a combination of features. For example, one rule is "CPU=A and memory=X". Another rule is "LCD=B and KBD=Y".

[0040] The prediction error evaluation unit 31 calculates the prediction error rates of the two classes for each rule using the prediction result data 21. For example, the prediction error evaluation unit 31 acquires records corresponding to a specific rule from the prediction result data 21. The prediction error evaluation unit 31 calculates the confusion rate (prediction error rate) between the two classes using the correct answer and predicted component contents of the acquired records. As an example, a case where the two classes are "CPU" and "memory" will be described. From the record set corresponding to a specific rule, the prediction error evaluation unit 31 calculates the number of records whose component contents in (correct answer, prediction) are (CPU, CPU), (memory, memory), (CPU, memory), and (memory, CPU), and the calculation result is designated A. In addition, from the record set corresponding to the same specific rule, the prediction error evaluation unit 31 calculates the number of records whose component contents in (correct answer, prediction) are (CPU, memory), (memory, CPU), and the calculation result is designated B. Then, the prediction error evaluation unit 31 can calculate the confusion rate (prediction error rate) when the two classes are "CPU" and "memory" as B / A.

[0041] Here, for example, when the rule is "CPU = A and memory = X" and the two classes are "memory" and "CPU," the confusion rate is calculated to be 0.87 (d0). That is, under the condition "CPU = A and memory = X," the rate at which the prediction unit 20 predicts that an exchange target that is a memory is a CPU and that an exchange target that is a CPU is a memory is calculated to be 0.87.

[0042] Similarly, when the rule is "CPU=A and memory=X" and the two classes are "memory" and "LCD," the confusion rate is calculated to be 0.08. Also, when the rule is "LCD=B and KBD=Y" and the two classes are "memory" and "CPU," the confusion rate is calculated to be 0.11. Also, when the rule is "LCD=B and KBD=Y" and the two classes are "memory" and "LCD," the confusion rate is calculated to be 0.03.

[0043] Then, based on the calculation results, the misprediction evaluation unit 31 identifies two classes corresponding to rules that frequently cause mispredictions. That is, the misprediction evaluation unit 31 identifies two classes that are difficult to separate under a specific rule. Here, when the rule is "CPU=A and memory=X" and the two classes are "memory" and "CPU," the confusion rate is "0.87," which is the highest confusion rate. As a result, the rule is "CPU=A and memory=X" and the two classes are "memory" and "CPU." That is, when the rule is "CPU=A and memory=X," it is most difficult to determine whether to replace the CPU or the memory.

[0044] [Example of Additional Feature Search] Here, an example of additional feature search performed by the additional feature search unit 32 according to the embodiment will be described with reference to Fig. 12. Fig. 12 is a diagram showing an example of additional feature search according to the embodiment. In Fig. 12, it is assumed that the prediction error evaluation unit 31 has identified "CPU" and "memory" as two classes that are difficult to separate from each other under the rule "CPU = A and memory = X" from the prediction error evaluation result 31A.

[0045] As shown in FIG. 12 , the additional feature search unit 32 collects BOM data 11 that matches the specified rule and is related to the two specified classes, and diagnostic data 33 that corresponds to the BOM data 11. Here, the BOM data 11 and diagnostic data 33 include BOM data 11 that matches the specified rule "CPU=A and memory=X" and is related to the predictions of the two specified classes "CPU" and "memory." In addition, diagnostic data 33 that corresponds to the collected BOM data 11 is collected. In the diagnostic data 33, an identifier indicating whether or not a diagnostic item has been diagnosed is set in association with the two specified classes "CPU" and "memory." If a diagnostic item has been diagnosed, "◯" is set. If a diagnostic item has not been diagnosed, "X" is set.

[0046] Then, based on the linking data 34, the additional feature search unit 32 identifies a feature corresponding to a diagnostic item that has one or more "◯"s in the collected diagnostic data 33. Then, using the collected diagnostic data 33, the additional feature search unit 32 estimates the rate at which prediction accuracy can be improved when the identified feature is added to the training data for learning. Here, a "ZZ voltage check" is one of the diagnostic items that has one or more "◯"s in the collected diagnostic data 33. The "ZZ voltage check" is identified by being linked to "ZZ voltage" based on the linking data 34. Then, using the collected diagnostic data 33, the additional feature search unit 32 calculates the rate at which prediction accuracy can be improved when the identified feature "ZZ voltage" is added to the training data for learning as "+0.25".

[0047] An example of calculation of the degree of improvement in accuracy will now be described with reference to FIG. 13. FIG. 13 is a diagram showing an example of calculation of the degree of improvement in accuracy according to an embodiment. The right side of FIG. 13 represents collected diagnostic data 33'. The left side of FIG. 13 represents prediction result data 21' associated with the collected diagnostic data 33. That is, for a device in which a defect has occurred, prediction result data 21' obtained by prediction using XAI 40 is associated with diagnostic data 33' obtained by on-site diagnosis.

[0048] Under these circumstances, the diagnostic data 33' contains eight pieces of data for the two identified classes, "CPU" and "memory." The prediction result data 21' contains three pieces of data in which the correct and predicted component details differ. Of these, in two cases, the correct replacement parts were found in the diagnostic data 33' by an on-site worker diagnosing the ZZ voltage check. As a result, it is estimated that by incorporating the ZZ voltage as an additional feature into the AI ​​model, the prediction accuracy will improve by 2 / 8 (0.25). This "0.25" is the degree of accuracy improvement.

[0049] Returning to FIG. 12 , the additional feature search unit 32 outputs information 32A including the estimated degree of accuracy improvement, additional features, two classes where prediction errors are frequent, and the rules at that time. Then, a worker on-site refers to the output information 32A, confirms whether the output information matches their knowledge, and selects features that have a high accuracy improvement effect and low implementation costs. The selected features are acquired in the manufacturing phase. The features acquired in the manufacturing phase are added to the BOM data 11. The learning unit 10 then uses the BOM data 11 including the newly added features to train the XAI 40.

[0050] Here, the priority of features verified when the additional feature search unit 32 selects additional features will be described with reference to Fig. 14. Fig. 14 is a diagram illustrating the priority of feature selection. As shown in Fig. 14, when the diagnostic data 33 has many diagnostic items, it may be difficult for the additional feature search unit 32 to verify the feature corresponding to the diagnostic item.

[0051] In such a case, for example, the additional feature search unit 32 uses the prediction error evaluation result 31A shown on the left side of FIG. １ , and select the rule with the highest rate of misprediction from among the multiple rules. The degree of misprediction may be determined, for example, by using the average rate of confusion between multiple classes corresponding to the rule as the misprediction rate. Alternatively, the number of confusions between multiple classes corresponding to the rule may be used as the number of mispredictions. The additional feature search unit 32 then verifies the feature corresponding to the diagnostic item obtained from the selected rule. That is, the additional feature search unit 32 selects the most closely related diagnostic items in order from the top of the rules, and selects additional features.

[0052] As another example, the additional feature search unit 32 may use the prediction error evaluation result 31A shown on the right side of FIG. ２, and select classes in descending order of the number of prediction errors among the multiple classes. The degree of prediction error can be determined, for example, by using the confusion rate of the two classes. As another example, the number of confusions of the two classes can be used. Then, the additional feature search unit 32 can verify the features corresponding to the diagnostic items obtained from the two selected classes. That is, the additional feature search unit 32 can select the most closely related diagnostic items from the top of the two classes and select the additional features.

[0053] 15 is a diagram illustrating an example of a flowchart of a prediction error evaluation process according to an embodiment. It is assumed that a plurality of rules are acquired from the XAI 40.

[0054] 15 , the prediction error evaluation unit 31 selects one rule from the acquired multiple rules (step S11). Then, the prediction error evaluation unit 31 collects data corresponding to the selected rule (step S12). For example, the prediction error evaluation unit 31 collects records corresponding to the selected rule from the prediction result data 21.

[0055] The prediction error evaluation unit 31 then selects two classes (step S13). The prediction error evaluation unit 31 then calculates the confusion rate (prediction error rate) between the two selected classes (step S14). For example, the prediction error evaluation unit 31 calculates the confusion rate between the two selected classes using the correct answers and predicted component contents of the collected records.

[0056] Then, the prediction error evaluation unit 31 determines whether all classes have been calculated (step S15). If it determines that all classes have not been calculated (step S15; No), the prediction error evaluation unit 31 proceeds to step S13 to select another two classes.

[0057] On the other hand, if it is determined that all classes have been calculated (step S15; Yes), the prediction error evaluation unit 31 determines whether all rules have been calculated (step S16).If it is determined that all rules have not been calculated (step S16; No), the prediction error evaluation unit 31 proceeds to step S11 to select the next rule.

[0058] On the other hand, if it is determined that all rules have been calculated (step S16; Yes), the prediction error evaluation unit 31 stores the confusion rate between the two classes calculated for each rule as the prediction error evaluation result 31A and terminates the prediction error evaluation process.

[0059] [Flowchart of Additional Feature Search Process] FIG. 16 is a diagram illustrating an example of a flowchart of additional feature search process according to the embodiment.

[0060] As shown in FIG. 16, the additional feature search unit 32 selects a rule and a pair of classes (two classes) in descending order of confusion rate from the prediction error evaluation result 31A (step S21).

[0061] Then, the additional feature search unit 32 collects the corresponding BOM data 11 and diagnostic data 33 based on the selected rule (step S22). Furthermore, the additional feature search unit 32 collects diagnostic data 33 corresponding to the selected pair of classes (two classes) from the collected diagnostic data 33 (step S23).

[0062] Then, the additional feature search unit 32 selects one diagnostic item from the collected diagnostic data 33 (step S24). For example, the additional feature search unit 32 selects a diagnostic item having one or more "◯" marks from the collected diagnostic data 33.

[0063] The additional feature search unit 32 then calculates the accuracy improvement effect from the number of data items when the selected diagnostic item is implemented (step S25). For example, the additional feature search unit 32 counts the collected diagnostic data 33 having two classes (X). In addition, for the prediction result data 21 associated with the diagnostic data 33 to be counted, the additional feature search unit 32 counts the data items for which the diagnostic data 33 diagnostic item indicates "o" among the data items for which the correct answer and the predicted part contents differ (Y), for the prediction result data 21 associated with the diagnostic data 33 to be counted. The additional feature search unit 32 then calculates Y / X and determines the result as the accuracy improvement effect.

[0064] The additional feature search unit 32 then determines whether or not calculations have been completed for all diagnostic items (step S26). If it is determined that calculations have not been completed for all diagnostic items (step S26; No), the additional feature search unit 32 proceeds to step S24 to select the next diagnostic item.

[0065] On the other hand, if it is determined that calculations have been performed for all diagnostic items (step S26; Yes), the additional feature search unit 32 determines whether calculations have been performed for all rules and all classes (step S27).If it is determined that calculations have not been performed for all rules and all classes (step S27; No), the additional feature search unit 32 proceeds to step S21 to select the next rule and two classes.

[0066] On the other hand, if it is determined that calculations have been performed for all rules and all classes (step S27; Yes), the additional feature search unit 32 sorts and displays the results by accuracy improvement effect (step S28), and then ends the additional feature search process.

[0067] This allows the additional feature search unit 32 to select diagnostic items (features) necessary to improve the accuracy of prediction when predicting replacement parts.

[0068] In the embodiment, the prediction error evaluation unit 31 evaluates prediction errors for each of the two classes using the rules learned by the XAI 40 and the prediction result data 21. The additional feature search unit 32 searches for additional features by comparing the two classes with high occurrences of prediction errors (difficult to predict) and the rules used at that time with the diagnostic data 33 as a result of evaluation by the prediction error evaluation unit 31. However, the prediction error evaluation unit 31 is not limited to the XAI 40, and may also use machine learning that does not learn rules.

[0069] When machine learning that does not learn rules is applied, the prediction error evaluation unit 31 utilizes the prediction result data 21 to evaluate the prediction error for each of the two classes. For example, the prediction error evaluation unit 31 sequentially selects two classes from the prediction result data 21. Then, the prediction error evaluation unit 31 calculates the prediction error rates of the two selected classes using the prediction result data 21. Then, based on the calculation results, the prediction error evaluation unit 31 identifies two classes whose occurrence rate of prediction errors exceeds a predetermined value. The predetermined value may be 0.5 or 0.7. The predetermined value can be changed as appropriate.

[0070] The additional feature search unit 32 searches for features to be added by comparing the two classes identified by the prediction error evaluation unit 31 that frequently cause prediction errors (are difficult to predict) with the diagnostic data 33. For example, the additional feature search unit 32 collects diagnostic data 33 that fit into the two identified classes and the BOM data 11 corresponding to the diagnostic data 33. Then, based on the linking data 34, the additional feature search unit 32 identifies features corresponding to diagnostic items that have one or more "o" marks included in the collected diagnostic data 33. Then, using the collected diagnostic data 33 and the prediction result data 21, the additional feature search unit 32 estimates the rate at which prediction accuracy can be improved by adding the identified features to training data for learning. That is, for one or more candidate features to be added to the BOM data 11 identified based on the linking data 34, the additional feature search unit 32 uses the collected diagnostic data 33 to estimate the degree of improvement in prediction accuracy by adding the features to training data.

[0071] The additional feature search unit 32 then outputs the estimated degree of improvement in prediction accuracy, along with additional candidate feature values ​​and two classes in which prediction errors occur frequently. In addition, the additional feature search unit 32 may display statistics for parts whose predictions are likely to improve, using prediction result data 21 corresponding to diagnostic data 33 that falls into the two classes.

[0072] 17 is a diagram illustrating a modified example of the additional feature search according to the embodiment. In FIG. 17, it is assumed that the prediction error evaluation unit 31 identifies “CPU” and “memory” as two classes that are difficult to separate from each other based on the prediction error evaluation result 31B.

[0073] As shown in FIG. 17 , the additional feature search unit 32 collects diagnostic data 33 that applies to the two identified classes and BOM data 11 that corresponds to the diagnostic data 33. Here, the diagnostic data 33 includes diagnostic data 33 that applies to the two identified classes, "CPU" and "memory." Furthermore, the BOM data 11 includes BOM data 11 that corresponds to the collected diagnostic data 33. The diagnostic data 33 includes an identifier indicating whether or not each diagnostic item has been diagnosed, associated with the two identified classes, "CPU" and "memory." If a diagnostic item has been diagnosed, a "◯" is set. If a diagnostic item has not been diagnosed, an "X" is set. Note that the diagnostic data 33 and BOM data 11 are not collected using rules, and therefore all data that applies to the two classes, "CPU" and "memory," is collected.

[0074] The additional feature search unit 32 then identifies, based on the linking data 34, features corresponding to diagnostic items that have one or more "o"s in the collected diagnostic data 33. The additional feature search unit 32 then uses the collected diagnostic data 33 to estimate the percentage improvement in prediction accuracy when the identified features are added to the training data for learning. Here, a "ZZ voltage check" is one diagnostic item that has one or more "o"s in the collected diagnostic data 33. The "ZZ voltage check" is identified by being linked to "ZZ voltage" based on the linking data 34. The additional feature search unit 32 then uses the collected diagnostic data 33 to calculate "+0.11" as the percentage improvement in prediction accuracy when the identified feature "ZZ voltage" is added to the training data for learning. The calculation method for the percentage improvement in prediction accuracy may be the same as the method described with reference to FIG. 13 .

[0075] The additional feature search unit 32 then outputs information 32B including the estimated degree of accuracy improvement, the additional feature, and two classes in which prediction errors occur frequently. Additionally, the additional feature search unit 32 uses the prediction result data 21 corresponding to the diagnostic data 33 that falls into the two classes to display statistics for parts for which prediction is likely to improve. Here, for example, it is displayed that, among product groups for which prediction accuracy can be improved, 80% of the product groups have CPUs that are "A." In other words, among product groups for which prediction accuracy can be improved, 80% of the product groups have CPUs that are "A." This indicates that 80% of the product groups have prediction errors.

[0076] Then, a worker on-site refers to the output information 32B, checks whether the information matches their knowledge, and selects feature quantities that have a high effect on improving accuracy and feature quantities with low implementation costs. The selected feature quantities are acquired in the manufacturing phase. The feature quantities acquired in the manufacturing phase are added to the BOM data 11. The learning unit 10 then uses the BOM data 11 including the newly added feature quantities to train a machine learning learning model that does not learn rules.

[0077] As a result, when predicting replacement parts, the additional feature search unit 32 can select diagnostic items (features) necessary to improve the accuracy of predictions even when applying machine learning that does not learn rules.

[0078] Effects of the Embodiment According to the above embodiment, the information processing device 1 identifies two parts from the prediction results of a prediction model that predicts replacement parts, for which the occurrence rate of prediction errors, in which one part is predicted as a correct answer and the other part is predicted as a correct answer, exceeds a predetermined value. The information processing device 1 acquires BOM data 11 related to the two identified parts with prediction errors and diagnostic data 33 that indicates the diagnosis content of the worker corresponding to the BOM data 11. The information processing device 1 uses the diagnostic data 33 to estimate the degree of improvement in prediction accuracy when one or more feature values ​​that are candidates for addition to the BOM data 11, which are identified based on the correspondence between the diagnostic content and the feature values ​​included in the diagnostic data 33, are added to the training data of the prediction model. With this configuration, the information processing device 1 can select feature values ​​that improve the prediction accuracy of the prediction model that predicts replacement parts.

[0079] Furthermore, according to the above embodiment, the information processing device 1 determines, based on the estimated degree of improvement in prediction accuracy, from among one or more candidate features for addition, a feature to be recommended for addition to the BOM data 11. With this configuration, the information processing device 1 can improve the prediction accuracy of the prediction model by adding the determined feature to the BOM data 11.

[0080] Furthermore, according to the above embodiment, the information processing device 1 counts a first number indicating the number of diagnostic data 33 corresponding to two mispredicted components. The information processing device 1 extracts, from among the prediction results corresponding to the diagnostic data 33, diagnostic data 33 corresponding to prediction results in which the prediction differs from the correct answer, and counts a second number indicating the number of diagnostic data 33 in which the diagnostic content was implemented, from among the extracted diagnostic data 33. The information processing device 1 estimates a value calculated using the first number and the second number as the degree of improvement in prediction accuracy for the feature corresponding to the diagnostic content. According to this configuration, the information processing device 1 calculates the degree of improvement in prediction accuracy using diagnostic data 33 in which the diagnostic items have been diagnosed by a worker, thereby enabling the selection of additional feature values ​​while taking into account the knowledge of the on-site worker.

[0081] Furthermore, according to the above embodiment, the information processing device 1 uses a rule indicating the results of learning feature combinations using a prediction model and prediction results to identify a rule and two components whose prediction error rate exceeds a predetermined value. The information processing device 1 acquires BOM data 11 corresponding to the identified rule and diagnostic data 33 corresponding to the BOM data 11 and related to the two identified components. The information processing device 1 then uses the acquired diagnostic data 33 to estimate the degree of improvement in prediction accuracy for feature parameters that are candidates for addition to the BOM data 11. With this configuration, the information processing device 1 can accurately estimate the degree of improvement in prediction accuracy for feature parameters that are candidates for addition by using the rules obtained by the prediction model. As a result, the information processing device 1 can more reliably select feature parameters that improve the prediction accuracy of the prediction model.

[0082] [Others] In the above embodiment, the learning unit 10, the prediction unit 20, and the estimation unit 30 are described as being mounted on the information processing device 1. However, the learning unit 10, the prediction unit 20, and the estimation unit 30 do not have to be mounted on a single information processing device 1, and may be mounted on each information processing device. Furthermore, the learning unit 10 may be mounted on one information processing device 1, and the prediction unit 20 and the estimation unit 30 may be mounted on another information processing device.

[0083] In addition, in the above embodiment, the components of the information processing device 1 shown in the drawings do not necessarily have to be physically configured as shown. In other words, the specific form of distribution and integration of the information processing device 1 is not limited to that shown in the drawings, and all or part of it may be functionally or physically distributed and integrated in any unit depending on various loads, usage conditions, etc. Furthermore, a storage unit that stores data such as diagnostic data 33 may be connected to the information processing device 1 as an external device via a network.

[0084] Furthermore, all or any part of the processing functions performed by the information processing device 1 shown in Figure 1 can be realized by a CPU and a program analyzed and executed by the CPU, or can be realized as hardware using wired logic.

[0085] Fig. 18 is a diagram illustrating an example of a hardware configuration. As shown in Fig. 18, an information processing device 1 includes a communication device 100, a hard disk drive (HDD) 110, a memory 120, and a processor 130. The components shown in Fig. 18 are connected to each other via a bus or the like.

[0086] The communication device 100 is a network interface card or the like, and communicates with other devices. The HDD 110 stores programs and databases that operate the functions shown in FIG.

[0087] The processor 130 is equipped with a cache memory, and operates a process that executes each function described in FIG. 1 and the like by reading a program that executes the same processing as each processing unit shown in FIG. 1 from the HDD 110 or the like and expanding the program in the memory 120. For example, this process executes the same functions as each processing unit of the information processing device 1. Specifically, the processor 130 reads a program having the same functions as the learning unit 10, the prediction unit 20, the estimation unit 30, and the like from the HDD 110 or the like. The processor 130 then executes a process that executes the same processing as the learning unit 10, the prediction unit 20, the estimation unit 30, and the like.

[0088] In this way, the information processing device 1 operates as a device that executes information processing by reading and executing a program. The information processing device 1 can also realize functions similar to those of the above-described embodiment by reading the program from a recording medium using a media reader and executing the read program. Note that the program in these other embodiments is not limited to being executed by the information processing device 1. For example, the present invention can also be applied in the same way to cases where another computer or server executes a program, or where these execute a program in cooperation with each other.

[0089] This program can be distributed via a network such as the Internet. In addition, this program can be recorded on a computer-readable recording medium such as a hard disk, a flexible disk (FD), a CD-ROM, an MO (Magneto-Optical disk), or a DVD (Digital Versatile Disc), and can be read out from the recording medium and executed by a computer.

[0090] Furthermore, the information processing device 1 described in the above embodiment can be applied to, for example, selecting replacement parts that are defective in the manufacturing process of products such as not only computers but also automobiles.

[0091] REFERENCE SIGNS LIST 1 Information processing device 10 Learning unit 11 BOM data 12 Defective phenomenon data 13 Repair correspondence data 20 Prediction unit 21 Prediction result data 30 Estimation unit 31 Prediction error evaluation unit 32 Additional feature search unit 33 Diagnostic data 34 Linking data 40 XAI

Claims

1. An estimation program that causes a computer to execute the following processes: identify two parts from the prediction results of a prediction model that predicts replacement parts, for which the occurrence rate of prediction errors in which one part is predicted as the other part exceeds a predetermined value when the other part is correct; acquire input data related to the two identified parts and diagnostic data that indicates the diagnosis of the worker corresponding to the input data; and estimate, using the diagnostic data, the degree of improvement in prediction accuracy when one or more feature values ​​that are candidates for addition to the input data, which are identified based on the correspondence between the diagnostic content and feature values ​​contained in the diagnostic data, are added to the training data of the prediction model.

2. The estimation program according to claim 1, further comprising: determining a feature to be recommended for addition to the input data from among one or more candidate features for addition based on the estimated degree of improvement in prediction accuracy.

3. The estimation program according to claim 1, characterized in that the estimating process counts a first number indicating the number of diagnostic data corresponding to the two components of the misprediction, extracts the diagnostic data corresponding to the prediction results in which the prediction differs from the correct answer from the prediction results corresponding to the diagnostic data, counts a second number indicating the number of diagnostic data in which the diagnostic content was implemented from the extracted diagnostic data, and estimates a value calculated using the first number and the second number as a degree of improvement in the prediction accuracy for the feature corresponding to the diagnostic content.

4. The estimation program of claim 1, characterized in that the identification process uses a rule indicating the result of learning a combination of features by the predictive model and the prediction result to identify a rule and two parts whose prediction error rate exceeds a predetermined value; the acquisition process acquires input data corresponding to the identified rule and the diagnostic data corresponding to the input data and related to the two identified parts; and the estimation process uses the acquired diagnostic data to estimate the degree of improvement in prediction accuracy for features that are candidates for addition to the input data.

5. An information processing device comprising: an identification unit that identifies two parts for which the occurrence rate of prediction errors, in which one part is predicted as the correct answer when the other part is predicted, exceeds a predetermined value from the prediction results of a prediction model that predicts replacement parts; an acquisition unit that acquires input data related to the two parts identified by the identification unit and diagnostic data indicating the diagnostic content of the worker corresponding to the input data; and an estimation unit that uses the diagnostic data to estimate a degree of improvement in prediction accuracy when one or more feature values ​​that are candidates for addition to the input data, which feature value is identified based on the correspondence between the diagnostic content and feature values ​​contained in the diagnostic data acquired by the acquisition unit, are added to the teacher data of the prediction model.

6. An estimation method in which a computer executes the following steps: identify two parts for which the occurrence rate of prediction errors, in which one part is predicted as the replacement part result when the other part is correct, exceeds a predetermined value from the prediction results of a prediction model that predicts replacement parts; acquire input data related to the two identified parts and diagnostic data that indicates the diagnosis of the worker corresponding to the input data; and estimate, using the diagnostic data, the degree of improvement in prediction accuracy when one or more feature values ​​that are candidates for addition to the input data, which are identified based on the correspondence between the diagnostic content and feature values ​​contained in the diagnostic data, are added to the training data of the prediction model.

Images