Classification class inference device, classification class inference method, and classification class inference program

The classification class inference device automates the construction of accurate AI models by generating pseudo data and refining rules, addressing slow adaptation to new data formats and reducing manual effort, thus enhancing model efficiency and accuracy.

JP2024162867APending Publication Date: 2024-11-21HITACHI LTD
View PDF 0 Cites 3 Cited by

Patent Information

Application Number
JP2023078813
Authority / Receiving Office
JP · JP
Patent Type
Applications
Current Assignee / Owner
Filing Date
2023-05-11
Publication Date
2024-11-21

AI Technical Summary

Technical Problem

Existing AI model adaptation to new data formats is slow due to the need for extensive data accumulation and manual feature enhancement, limiting the efficiency of model construction and accuracy.

Method used

A classification class inference device that generates pseudo learning data and employs empirical rule models to infer classification classes, utilizing automatic rule extraction, data complementation, and erroneous inference analysis to enhance model accuracy with reduced data and manual effort.

Benefits of technology

Enables rapid construction of accurate classification models from minimal data with fewer man-hours by automating feature extraction and rule refinement, improving inference accuracy through pseudo data generation and rule correction.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 2024162867000001_ABST
    Figure 2024162867000001_ABST
Patent Text Reader

Abstract

To provide a classification class inference device, method, and program for building, from a smaller amount of learning data, a model for classifying target data into an appropriate class with smaller man-hours.SOLUTION: A classification class inference device builds a model for inferring a classification class from target data, and comprises: a storage that stores a plurality of rules for determining, from a condition related to business operations, a classification class corresponding to the condition, a plurality of pieces of target data, and correct labels of the pieces of target data; and an arithmetic unit that generates learning data corresponding to the condition related to the business operations, and executes model building processing of building the model on the basis of the learning data and the rules, model inference processing of inputting the target data to the model to infer the classification class corresponding to the target data, and erroneous inference cause analysis processing of comparing the inferred classification class and the correct label of the target data to generate information for specifying the rule that is the cause of a reduction in the inference accuracy of the model, and outputting the information.SELECTED DRAWING: Figure 3
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present invention relates to a classification class inference device, a classification class inference method, and a classification class inference program. [Background technology]

[0002] When operating a business system that uses an AI (Artificial Intelligence) model, there are cases where it is necessary to handle business data in a new format that differs from the format previously assumed. Normally, to handle this, a sufficient amount of business data in the new format is accumulated, and the AI ​​model is reconstructed using that business data as learning data. However, there is a problem in that it takes years to accumulate a sufficient amount of business data in a business system, making it difficult to quickly handle the new format.

[0003] To address such problems, the technology described in Non-Patent Document 1 uses feature engineering to extract features that reflect human knowledge, thereby enabling the construction of highly accurate models even with a small amount of training data. [Prior art documents] [Non-patent literature]

[0004] [Non-Patent Document 1] Dan Roth et. al, “Interactive Feature Space Construction using Semantic Information,” 13th CoNLL, pp 64-74, 2009. Summary of the Invention [Problem to be solved by the invention]

[0005] However, in the technology described in Non-Patent Document 1, even if the feature values ​​are appropriate, a certain amount of training data and correct labels are required to build a highly accurate model. Therefore, the effect of reducing the labor required to prepare training data is limited. Furthermore, in the technology described in Non-Patent Document 1, only a portion of the feature values ​​can be customized, so the other feature values ​​must be manually enhanced, and the effect of reducing the labor required to build a highly accurate model is also limited.

[0006] The present invention has been made in consideration of the above circumstances, and an object of the present invention is to provide a classification class inference device, a classification class inference method, and a classification class inference program that are capable of constructing a model for classifying target data into appropriate classes from a smaller amount of training data with fewer man-hours. [Means for solving the problem]

[0007] One aspect of the present invention for solving the above-mentioned problems is a classification class inference device that constructs a model for inferring a classification class from target data, the classification class inference device comprising: a storage device that stores a plurality of rules for determining a classification class corresponding to a business-related condition from the condition, a plurality of the target data, and a correct label for the target data; and a calculation device that executes a model construction process that generates pseudo learning data corresponding to the business-related conditions and constructs a model based on the pseudo learning data and the rules, a model inference process that infers a classification class corresponding to the target data by inputting the target data into the model, and an erroneous inference factor analysis process that generates and outputs information for identifying the rule that is a cause of a decrease in the inference accuracy of the model by comparing the inferred classification class with the correct label for the target data. Effect of the Invention

[0008] According to the present invention, a model that classifies target data into appropriate classes can be constructed with fewer steps from a smaller amount of training data. Configurations and effects other than those described above will become apparent from the following description of the embodiments. [Brief description of the drawings]

[0009] [Figure 1] 1 is a diagram illustrating an example of a configuration of a classification class inference device according to a first embodiment. [Diagram 2] FIG. 13 is a diagram illustrating an example of target data. [Diagram 3] FIG. 2 is a diagram for explaining an overview of processing performed by the classification class inference device in the first embodiment. [Figure 4] FIG. 13 is a diagram showing an example of a correct answer label. [Diagram 5] FIG. 11 is a diagram illustrating an example of important feature amount information. [Figure 6] FIG. 2 is a diagram showing various data input and output to an automatic empirical rule extraction unit that executes an automatic empirical rule extraction process. [Figure 7] FIG. 11 is a process flow diagram illustrating details of an empirical rule automatic extraction process. [Figure 8] FIG. 1 is a diagram illustrating an overview of a process for dividing target data into segments based on a feature amount of interest. [Figure 9] FIG. 13 illustrates an example of a segment management table. [Figure 10] FIG. 13 is a diagram illustrating an example of an experience rule. [Figure 11] FIG. 2 is a diagram showing various data input and output to a learning data complementing means that executes a learning data complementing process. [Figure 12] FIG. 11 is a process flow diagram illustrating details of a learning data complementation process. [Figure 13] FIG. 11 is a diagram illustrating an example of pseudo data. [Figure 14] FIG. 2 is a diagram showing various data input and output to an empirical rule model construction means that executes an empirical rule model construction process. [Figure 15] FIG. 13 is a process flow diagram illustrating details of the empirical rule model construction process S6. [Figure 16] FIG. 13 is a diagram showing an example of a result of applying an experience rule. [Figure 17] FIG. 13 is a diagram showing an example of a learning label. [Figure 18] FIG. 13 is a diagram illustrating an example of parameters in an empirical rule model. [Figure 19] 11 is a diagram showing various data input / output to / from an inference execution means for executing an erroneous inference cause analysis process and an erroneous inference cause analysis means; FIG. [Figure 20] FIG. 11 is a process flow diagram illustrating details of an erroneous inference cause analysis process. [Figure 21] FIG. 13 illustrates an example of a SHAP value management table. [Figure 22] FIG. 13 is a diagram illustrating an example of an erroneous inference factor. [Figure 23] FIG. 11 is a process flow diagram illustrating details of an empirical rule correction process. [Figure 24] FIG. 11 is a diagram illustrating an example of a configuration of a classification class inference device according to a second embodiment. [Diagram 25] FIG. 11 is a diagram showing various data input and output to and from each function of the classification class inference device in the second embodiment. [Figure 26] FIG. 11 is a diagram for explaining an overview of a process performed by a classification class inference device in the second embodiment. [Figure 27] FIG. 13 is a diagram illustrating an example of an additional target feature amount. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. The following description and drawings are examples for explaining the present invention, and appropriate omissions and simplifications have been made for clarity of explanation. The present invention can be implemented in various other forms. Unless otherwise specified, each component may be singular or plural. In order to facilitate understanding of the invention, the position, size, shape, range, etc. of each component shown in the drawings may not represent the actual position, size, shape, range, etc. Therefore, the present invention is not necessarily limited to the position, size, shape, range, etc. disclosed in the drawings. In the following explanation, various information may be explained using expressions such as "table," "list," and "queue," but the various information may be expressed in other data structures. To indicate independence of data structure, "XX table," "XX list," and the like may be referred to as "XX information." When explaining identification information, expressions such as "identification information," "identifier," "name," "ID," and "number" are used, but these are interchangeable. When there are multiple components having the same or similar functions, they may be described by using the same reference numerals with different subscripts. However, when there is no need to distinguish between these multiple components, the subscripts may be omitted. In addition, in the following description, the processing performed by executing a program may be described, but the program is executed by a processor (e.g., CPU, GPU) to perform a defined process using storage resources (e.g., memory) and / or interface devices (e.g., communication ports) as appropriate, so the subject of the processing may be the processor. Similarly, the subject of the processing performed by executing a program may be a controller, device, system, computer, or node having a processor. The subject of the processing performed by executing a program may be a calculation unit, and may include a dedicated circuit (e.g., FPGA or ASIC) that performs a specific process. A program may be installed in a device such as a computer from a program source. The program source may be, for example, a program distribution server or a computer-readable storage medium. When the program source is a program distribution server, the program distribution server may include a processor and a storage resource for storing a program to be distributed, and the processor of the program distribution server may distribute the program to be distributed to other computers. In the following description, two or more programs may be realized as one program, and one program may be realized as two or more programs.

[0011] [First embodiment] First, a classification class inference device, a classification class inference method, and a classification class inference program according to a first embodiment of the present invention will be described.

[0012] 1 is a diagram showing an example of the configuration of a classification class inference device 1 in this embodiment. The classification class inference device 1 is an information processing device that constructs an empirical rule model 125 (model) that is a model different from an AI model constructed based on learning data and is capable of inference similar to that of an AI model. The empirical rule model 125 infers a classification class from target data 121.

[0013] In this embodiment, a classification class inference device 1 that infers to which of a predetermined number of classes a company belongs (a classification class of the company) from activity data of the company (for example, financial indicators such as sales, net profit, or total assets) in a financial institution or the like will be described as an example. In this example, the activity data of the company is the target data 121.

[0014] The classification class inference device 1, as an example, includes an arithmetic device 101 composed of a CPU (Central Processing Unit) and memories such as RAM (Random Access Memory) and ROM (Read Only Memory), a storage device 102 such as an HDD (Hard Disk Drive) or SSD (Solid State Drive), an input device 103 such as a keyboard, a mouse, or a touch panel, an output device 104 such as a display or a printer, and a communication device 105 composed of a NIC (Network Interface Card), a wireless communication module, a USB (Universal Serial Interface) module, a serial communication module, or the like.

[0015] The calculation device 101 executes the programs stored in the storage device 102 to realize each of the functions of an automatic empirical rule extraction means 111, a learning data supplementation means 112, an empirical rule model construction means 113, an inference execution means 114, and an erroneous inference cause analysis means 115.

[0016] The empirical rule automatic extraction means 111 automatically generates empirical rules 123 for constructing an empirical rule model 125, based on target data 121 and a correct answer label 122 of the target data 121. The empirical rules 123 are rules for determining a classification class corresponding to a condition related to a business of a company from the condition.

[0017] The learning data complementing means 112 generates pseudo data 124 from the target data 121 as pseudo learning data for constructing the empirical rule model 125. The pseudo data 124 corresponds to pseudo learning data corresponding to business-related conditions.

[0018] The empirical rule model construction means 113 constructs an empirical rule model 125 based on the target data 121 , the correct label 122 , the pseudo data 124 , and the empirical rules 123 .

[0019] The inference execution means 114 executes empirical rule model inference processing for inferring a classification class corresponding to the target data 121 by inputting the target data 121 into the empirical rule model 125 .

[0020] The error inference cause analysis means 115 generates and outputs information (error inference cause 126) for identifying empirical rules 123 that may be a cause of an error in inference (a decrease in the inference accuracy of the empirical rule model 125) based on the target data 121 and the empirical rule model 125.

[0021] The storage device 102 stores programs for implementing the functions of an automatic empirical rule extraction means 111, a learning data supplement means 112, an empirical rule model construction means 113, an inference execution means 114, and an erroneous inference cause analysis means 115.

[0022] The storage device 102 also stores various data that is input and output when the program is executed by the calculation device 101. Specifically, the storage device 102 stores target data 121, correct answer labels 122, empirical rules 123, pseudo data 124, empirical rule models 125, and erroneous inference factors 126.

[0023] (Target data) 2 is a diagram showing an example of target data 121. The target data 121 has items of data ID 211 in which a data ID that uniquely identifies each target data 121 is set, and explanatory variables 214 in which one or more explanatory variables are set (in the illustrated example, explanatory variable 1, explanatory variable 2, . . . , explanatory variable m). For example, information that identifies a company is set in the data ID 211, and activity data of the company (for example, sales, net profit, or total assets, etc.) is set in the explanatory variables 124. The target data 121 is learning data to which no correct answer label has been assigned.

[0024] <Processing Overview> FIG. 3 is a diagram for explaining an outline of the processing performed by the classification class reasoning device 1. As shown in FIG. The classification class inference device 1 accepts an input of a correct answer label 122 (classification class) for target data 121 from a user, and writes the input correct answer label 122 into the storage device 102 (S1).

[0025] (Correct label) 4 is a diagram showing an example of the correct label 122. The correct label 122 has items of a data ID 211 in which a data ID is set, and a label 215. In the label 215, a correct label (classification class) input by the user is set.

[0026] Furthermore, the classification class inference device 1 accepts input of noticeable feature amount information 302 of a feature amount to be focused on when inferring a classification class from a user, and writes the input noticeable feature amount information 302 into the storage device 102 (S2).

[0027] (Feature information of interest) 5 is a diagram showing an example of the noticeable feature amount information 302. The noticeable feature amount information 302 has items of a feature amount name 312 in which the name of the feature amount is set, and a feature amount calculation function 313 in which a function for calculating the feature amount is set.

[0028] Furthermore, the classification class reasoning device 1 executes an automatic empirical rule extraction process (S3) for generating empirical rules 123. The automatic empirical rule extraction process S3 will be described in detail later.

[0029] Furthermore, the classification class reasoning device 1 accepts corrections (modifications or deletions) of the generated empirical rules 123 from the user, and updates the corrected empirical rules 123 (S4).

[0030] Furthermore, the classification class reasoning device 1 executes a learning data complementation process (S5) for complementing the learning data for constructing the empirical rule model 125. The learning data complementation process S5 will be described later in detail.

[0031] Furthermore, the classification class reasoning device 1 executes an empirical rule model construction process (S6) for constructing the empirical rule model 125. The details of the empirical rule model construction process S6 will be described later.

[0032] Furthermore, the classification class reasoning device 1 executes an erroneous inference cause analysis process (S7) for analyzing the empirical rules 123 that are factors that cause a decrease in inference accuracy in the constructed empirical rule model 125. The erroneous inference cause analysis process S7 will be described in detail later.

[0033] Furthermore, the classification class reasoning device 1 executes an empirical rule correction process for correcting the empirical rule 123 based on the analysis results from the erroneous inference cause analysis process (S8). The empirical rule correction process S8 will be described in detail later. Next, each of the above-described processes will be described in detail.

[0034] <Automatic extraction of experience rules> 6 is a diagram showing various data input and output to and from the empirical rule automatic extraction means 111 that executes the empirical rule automatic extraction process. The empirical rule automatic extraction means 111 receives the target data 121, the correct answer label 122, and the notable feature amount information 302 as input, and executes the empirical rule extraction process while updating the segment management table 301, thereby outputting the empirical rule 123.

[0035] 7 is a process flow diagram for explaining details of the empirical rule automatic extraction process. The empirical rule model automatic extraction process is executed, for example, when a predetermined input is made to the classification class reasoning device 1 by a user, or at a predetermined timing (for example, at a predetermined time, at a predetermined time interval).

[0036] First, the empirical rule automatic extraction means 111 calculates a feature of interest for each data ID 211 based on the target data 121, the correct label 122, and the feature of interest information 302 (S401). For example, the empirical rule automatic extraction means 111 calculates a feature of interest by substituting the explanatory variable 214 into the feature calculation function 313.

[0037] Next, the empirical rule automatic extraction means 111 sorts the feature amounts of interest calculated in S401 in ascending order, and divides the data ID 211 (target data 121) into a plurality of segments.

[0038] 8 is a diagram for explaining an outline of a process of dividing the target data 121 into segments based on the feature of interest. First, the empirical rule automatic extraction means 111 generates a predetermined number of segments (five segments s1 to s5 in the illustrated example) so that the number of data IDs 211 in each segment is approximately uniform (see reference symbol ST1). In the illustrated example, the segment s1 has data IDs "x2" and "x3" whose feature of interest is less than 0, the segment s2 has data ID "x1" whose feature of interest is 0 or more and less than 0.02, the segment s3 has data ID "x4" whose feature of interest is 0.02 or more and less than 0.04, the segment s4 has data IDs "x5" and "x6" whose feature of interest is 0.04 or more and less than 0.06, and the segment s5 has data IDs "x7" and "x8" whose feature of interest is 0.06 or more.

[0039] Then, the empirical rule automatic extraction means 111 calculates the upper limit, lower limit, correct label distribution, and average label value of the segment management table 301 for each segment (S402).

[0040] (Segment management table) 9 is a diagram showing an example of the segment management table 301. The segment management table 301 has items of segment ID 311 in which a segment ID that uniquely identifies each segment is set, upper limit 314 in which an upper limit value of a feature quantity of interest in the segment is set, lower limit 315 in which a lower limit value of a feature quantity of interest in the segment is set, correct label distribution 316 in which a distribution of correct labels in the segment is set, average label value 317 in which the average value of the correct labels is set, and chi-square value 318 in which the chi-square value of the correct label distribution 316 is set.

[0041] Next, the empirical rule automatic extraction means 111 calculates the similarity of the correct label distribution of the adjacent segments in the segment management table 301 generated in S402 by calculating the chi-square value of the correct label distribution (S403). In the example shown in Fig. 8, the similarity between segment s1 and segment s2 is 0.9, the similarity between segment s2 and segment s3 is 0.2, the similarity between segment s3 and segment s4 is 2.4, and the similarity between segment s4 and segment s5 is 1.3 (see symbol ST2). In this embodiment, since the chi-square value is used as the similarity, the smaller the value, the higher the similarity, and the larger the value, the lower the similarity.

[0042] Next, the empirical rule automatic extraction means 111 merges (integrates) adjacent segments having the smallest chi-squared value (highest similarity) calculated in S403, and reconstructs the segment management table 301 (S404). In the example shown in Fig. 8, the segments s2 and s3 having the highest similarity are merged into one segment (see symbol ST3).

[0043] Then, the automatic empirical rule extraction means 111 repeats the processes of S403 and S404 until the number of segments reaches the number designated in advance by the user (S405).

[0044] Thereafter, the automatic empirical rule extraction means 111 newly generates empirical rules 123, writes the generated empirical rules 123 in the storage device 102 (S406), and ends this empirical rule extraction process.

[0045] (Experience Rules) 10 is a diagram showing an example of the experience rule 123. The experience rule 123 has data items of an experience rule ID 212 in which an experience rule ID for identifying each experience rule 123 is set, and an experience rule function 213 in which an experience rule function is set.

[0046] For example, the automatic empirical rule extraction means 111 sets a unique value to the empirical rule ID 212. In addition, the automatic empirical rule extraction means 111 sets, as the empirical rule function 213, a function that returns the average label value of a segment when a feature of interest is a value between the upper limit and lower limit of the segment.

[0047] <Learning data complementation processing> 11 is a diagram showing various data input and output to the learning data complementing means 112 that executes the learning data complementing process. The learning data complementing means 112 receives target data 121 as input, and executes the learning data complementing process to output pseudo data 124.

[0048] 12 is a process flow diagram illustrating the details of the learning data complementation process S5. The learning data complementation process is executed, for example, when a predetermined input is made by a user to the classification class inference device 1, or at a predetermined timing (for example, at a predetermined time or at a predetermined time interval).

[0049] The learning data complementing means 112 generates pseudo data 124 from the target data 121 using an algorithm such as Tabular GAN (Generative Adversarial Network), and writes the generated pseudo data 124 to the storage device 102 (S601). Note that the learning data complementing means 112 may generate the pseudo data 124 using random numbers. Thereafter, the learning data complementing means 112 ends this learning data complementing process.

[0050] (pseudo data) 13 is a diagram showing an example of pseudo data 124. The pseudo data 124 has items of data ID 211 in which a data ID is set, and explanatory variable 214 in which one or more explanatory variables are set (in the illustrated example, explanatory variable 1, explanatory variable 2, . . . , explanatory variable m). The data configuration of the pseudo data 124 is the same as that of the target data 121.

[0051] <Experience rule model construction process> 14 is a diagram showing various data input and output to the empirical rule model construction means 113 that executes the empirical rule model construction process. The empirical rule model construction means 113 receives the target data 121, the correct answer label 122, the pseudo data 124, and the empirical rules 123 as input, and outputs the empirical rule model 125 by executing the empirical rule model construction process while updating the empirical rule application result 701 and the learning label 702.

[0052] 15 is a process flow diagram illustrating details of the empirical rule model construction process S6. The empirical rule model construction process is executed, for example, when a predetermined input is made to the classification class reasoning device 1 by the user, or at a predetermined timing (for example, at a predetermined time or at a predetermined time interval).

[0053] First, the empirical rule model constructing means 113 applies each empirical rule 123 to the target data 121 and the pseudo data 124, and generates an empirical rule application result 701 (S801).

[0054] (Results of applying experience rules) 16 is a diagram showing an example of an empirical rule application result 701. The empirical rule application result 701 has items of a data ID 211 in which a data ID is set, and an application result 711 in which a result (classification class) of applying each empirical rule 123 to the target data 121 or the pseudo data 124 is set.

[0055] Next, the empirical rule model construction means 113 copies (duplicates) the data of the correct label 122 to generate a learning label 702. The empirical rule model construction means 113 also generates a learning label 702 corresponding to the pseudo data 124 from the empirical rule application result 701 (S802). For example, the empirical rule model construction means 113 sets a statistical value (e.g., an average value or a median value) of the application result 711 as the learning label 702 of the pseudo data 124.

[0056] (Learning Labels) 17 is a diagram showing an example of the learning label 702. The learning label 702 has items of a data ID 211 in which a data ID is set, and a label 215. The data configuration of the learning label 702 is the same as that of the correct answer label 122.

[0057] Next, the empirical rule model construction means 113 constructs an empirical rule model 125 that receives the empirical rule application result 701 as an input and infers (outputs) a learning label 702 (S803). The empirical rule model 125 is constructed based on an algorithm such as a neural network, a decision tree, a random forest, or a support vector machine (SVM). For example, when the empirical rule model construction means 113 constructs the empirical rule model 125 using a neural network, it stores NN (Neural Network) model configuration information and weight and bias information in the empirical rule model 125.

[0058] (Experience rule model) 18 is a diagram showing an example of parameters in the empirical rule model 125. The empirical rule model 125 has items of NN model configuration information 712 in which NN model configuration information is set, and weight and bias information 713 in which parameters of connections of each neural network, their weight values, and bias information are set.

[0059] <Analysis of the cause of incorrect inference> 19 is a diagram showing various data input / output to the inference execution means 114 and the incorrect inference factor analysis means 115 which execute the incorrect inference factor analysis process. The inference execution means 114 receives the target data 121, the correct label 122, and the empirical rule model 125 as inputs, and outputs a SHAP value management table 901, which will be described later. The incorrect inference factor analysis means 115 receives the target data 121, the correct label 122, and the empirical rule model 125 as inputs, and executes the incorrect inference factor analysis process while updating the SHAP value management table 901, thereby outputting an incorrect inference factor 126, which will be described later.

[0060] 20 is a process flow diagram for explaining details of the erroneous inference cause analysis process S7. The erroneous inference cause analysis process is executed, for example, when a predetermined input is made to the classification class inference device 1 by the user, or at a predetermined timing (for example, at a predetermined time or at a predetermined time interval).

[0061] First, the inference execution means 114 copies (duplicates) the correct label 122 to the SHAP value management table 901 (S1001).

[0062] (SHAP value management table) 21 is a diagram showing an example of a SHAP value management table 901. The SHAP value management table 901 has items of data ID 211 in which a data ID is set, explanatory variables 214 in which one or more explanatory variables are set (in the illustrated example, explanatory variable 1, explanatory variable 2, . . ., explanatory variable m), inference value 913 in which a result of inference by the empirical rule model 125 is set, correct answer label 914 in which a correct answer label is set, and SHAP value 915 in which a SHAP (SHapley Additive exPlanations) value of each empirical rule 123 is set. The SHAP value will be described later.

[0063] Next, the inference execution means 114 inputs the empirical rule application results obtained by applying each empirical rule 123 to the target data 121 into the empirical rule model 125, thereby calculating an inference value and storing it in the SHAP value management table 901 (S1002).

[0064] Next, the erroneous inference factor analysis means 115 calculates the contribution (degree of contribution) of each empirical rule 123 to the inference value for each data ID 211. Since the empirical rule model 125 receives the empirical rule application result and outputs the classification class, it is possible to calculate the contribution of each empirical rule 123 to the output value by a method for determining the contribution of each variable (feature amount) to the prediction result of the model, such as SHAP. Therefore, in this embodiment, the erroneous inference factor analysis means 115 uses a value calculated by SHAP (hereinafter, referred to as "SHAP value") as the contribution to the inference value. Then, the erroneous inference factor analysis means 115 stores the calculated SHAP value in the SHAP value management table 901 (S1003).

[0065] Next, the erroneous inference factor analysis means 115 calculates the contribution (hereinafter, referred to as "error contribution") to the error (the difference between the inference value and the correct label) for each empirical rule 123. Specifically, first, the erroneous inference factor analysis means 115 calculates a value {(inference value)-(correct label)}*(SHAP value) obtained by multiplying the value obtained by subtracting the correct label from the inference value by the SHAP value of the empirical rule 123 for each data ID 211. The SHAP value is a positive value when it has a positive effect on the inference value, and a negative value when it has a negative effect. Therefore, the error contribution of each empirical rule 123 for each data ID 211 can be calculated by multiplying the value obtained by subtracting the correct label from the inference value by the SHAP value of the empirical rule 123. Next, the incorrect inference factor analysis means 115 calculates the total value (total value of the error contribution degree for each data ID 211) of the calculated values ​​{(inference value)-(correct label)}*(SHAP value) for each empirical rule 123. Then, the incorrect inference factor analysis means 115 stores the calculated total value for each empirical rule 123 in the incorrect inference factor 126 as the error contribution degree for the entire empirical rule model 125 (S1004). After that, the incorrect inference factor analysis means 115 ends this incorrect inference factor analysis process.

[0066] (Factors of misconception) 22 is a diagram showing an example of the erroneous inference factor 126. The erroneous inference factor 126 has items of an empirical rule ID 916 in which an empirical rule ID is set, and an error contribution degree 917.

[0067] <Experience rule correction process> 23 is a process flow diagram illustrating details of the empirical rule correction process S8. The empirical rule correction process is executed, for example, when a predetermined input is made to the classification class inference device 1 by the user, or at a predetermined timing (for example, at a predetermined time or at a predetermined time interval).

[0068] First, the classification class inference device 1 refers to the erroneous inference factors 126 and corrects (e.g., deletes or modifies) the empirical rules 123 whose error contribution 917 is greater than a predetermined threshold (S1101). The threshold to be compared with the error contribution 917 is, for example, set in advance by a user. Alternatively, the classification class inference device 1 may display the erroneous inference factors 126 on the output device 104 and accept a correction (deletion or modification) of each empirical rule 123 from the user via the input device 103. The user can tell that the greater the value of the erroneous inference contribution is, the more the empirical rule 123 has contributed to the error, so the user refers to the erroneous inference factors 126 and deletes or modifies the empirical rule 123.

[0069] Next, the classification class reasoning device 1 re-executes the above-mentioned processes from S801 to S803 and from S1001 to S1004 (S1102).

[0070] Thereafter, the classification class reasoning device 1 determines whether the inference accuracy of the empirical rule model 125 has reached a predetermined target accuracy (S1103). The target accuracy is, for example, set in advance by a user. For example, the classification class reasoning device 1 calculates the inference accuracy based on the degree of agreement between the inference result obtained by inputting the target data 121 into the reconstructed empirical rule model 125 and the correct label 122. If the inference accuracy of the empirical rule model 125 has not reached the target accuracy (S1103: NO), the classification class reasoning device 1 returns to the process of S1101. On the other hand, if the inference accuracy of the empirical rule model 125 has reached the target accuracy (S1103: YES), the classification class reasoning device 1 ends this empirical rule correction process.

[0071] As described above, the classification class inference device 1 of this embodiment is a classification class inference device 1 that constructs an empirical rule model 125 that infers a classification class from target data 121, and includes a storage device 102 that stores a plurality of empirical rules 123 that determine a classification class corresponding to a condition based on a business-related condition, a plurality of target data 121, and a correct label 122 of the target data 121, and a calculation device 101 that executes an empirical rule model construction process that generates pseudo data 124 that corresponds to the business-related conditions and constructs the empirical rule model 125 based on the pseudo data 124 and the empirical rules 123, an empirical rule model inference process that infers a classification class corresponding to the target data 121 by inputting the target data 121 to the empirical rule model 125, and an erroneous inference factor analysis process that generates and outputs erroneous inference factors 126 for identifying an empirical rule 123 that is a cause of a decrease in inference accuracy of the empirical rule model 125 by comparing the inferred classification class with the correct label 122 of the target data 121.

[0072] That is, the classification class inference device 1 of this embodiment stores in advance empirical rules 123 that determine classification classes corresponding to business conditions from the conditions, and therefore can construct an empirical rule model 125 that infers a classification class of target data 121 even when the amount of learning data is insufficient. Furthermore, the classification class inference device 1 outputs the contribution (error contribution) of each empirical rule 123 to a decrease in the inference accuracy of the empirical rule model 125, and therefore the user can easily grasp the empirical rule 123 that should be corrected to improve the inference accuracy of the empirical rule model 125. This makes it possible to construct an empirical rule model 125 that classifies target data 121 into an appropriate class with fewer man-hours.

[0073] In addition, in the erroneous inference cause analysis process, the classification class inference device 1 of this embodiment calculates the contribution of each empirical rule 123 to the decline in inference accuracy of the empirical rule model 125, corrects the empirical rules 123 whose calculated contribution is greater than a predetermined threshold, and executes an empirical rule correction process to reconstruct the empirical rule model 125.

[0074] With this configuration, the classification class inference device 1 automatically corrects the empirical rules 123 that are a factor in reducing inference accuracy, and reconstructs the empirical rule model 125. This saves the user the effort of manually correcting the empirical rules 123, and makes it possible to automatically generate the empirical rule model 125 that further improves inference accuracy.

[0075] Moreover, the classification class inference device 1 of this embodiment stores prominent feature amount information 302 for defining a new empirical rule 123, and executes an automatic empirical rule extraction process for generating a new empirical rule 123 based on a plurality of target data 121, a correct answer label 122 of the target data 121, and the prominent feature amount information 302. Specifically, in the automatic empirical rule extraction process, the classification class inference device 1 calculates a value of a feature amount for each of the plurality of target data 121, divides the plurality of target data 121 into a plurality of segments based on the feature amount values, and generates an empirical rule 123 based on the feature amount values ​​for each segment and the correct answer label 122 of the target data 121.

[0076] With this configuration, the classification class inference device 1 can automatically generate new empirical rules 123 based on the important feature information 302. Therefore, it is possible to build an empirical rule model 125 with high inference accuracy while reducing the man-hours required for a user to manually create the empirical rules 123.

[0077] Furthermore, in the automatic empirical rule extraction process, the classification class reasoning device 1 of this embodiment integrates segments based on the similarity in distribution of the correct answer labels 122 of the target data 121 between adjacent segments until the number of segments reaches a predetermined number.

[0078] With this configuration, the classification class reasoning device 1 can generate more accurate empirical rules 123. This makes it possible to build an empirical rule model 125 with more accurate inference.

[0079] [Second embodiment] Next, a classification class inference device, a classification class inference method, and a classification class inference program according to a second embodiment of the present invention will be described.

[0080] 24 is a diagram showing an example of the configuration of a classification class inference device 1A in this embodiment. In this diagram, the same components as those in the classification class inference device 1 of the first embodiment are given the same reference numerals, and the description thereof will be omitted.

[0081] Even if there is insufficient learning data for inferring the classification class of the target data 121, there may be sufficient learning data used for purposes other than the target data 121. For example, when a service (inference of classification class) for a specific industry (e.g., agriculture and fisheries) has already been provided, and a new service (inference of classification class) for the industry of the target data 121 (e.g., mining) is to be started in order to expand the service, there may be sufficient accumulated learning data (business data) for the industry (agriculture and fisheries) for which the service is already provided, even if there is insufficient learning data for the industry (mining) of the new service.

[0082] Therefore, the classification class inference device 1 in this embodiment differs from the first embodiment in that it improves the inference accuracy of the empirical rule model 125 by using other-use data 1311, which is learning data having a different use from that of the target data 121.

[0083] The calculation device 101A executes a program stored in the memory device 102A to realize each function of an empirical rule quality evaluation means 1301 and an additional noteworthy feature extraction means 1302 in addition to each function realized by the calculation device 101 in the first embodiment.

[0084] The empirical rule quality evaluation means 1301 executes a rule quality evaluation process to extract empirical rules (reusable empirical rules 1314) to be used in constructing the empirical rule model 125 from empirical rules for other uses (other-use empirical rules 1313 (other-use rules)) generated by the empirical rule automatic extraction means 111 based on the other-use data 1311 and the other-use correct labels 1312 indicating the correct labels of the other-use data 1311.

[0085] The additional noteworthy feature extraction means 1302 executes an additional noteworthy feature extraction process to extract an additional noteworthy feature (additional noteworthy feature 1317) for generating a new empirical rule 123 to be used for constructing the empirical rule model 125.

[0086] The storage device 102A stores programs for realizing each of the functions of an empirical rule quality evaluation means 1301 and an additional noteworthy feature extraction means 1302 in addition to each of the functions realized by the arithmetic device 101 in the first embodiment.

[0087] In addition to the various data stored in the storage device 102 in the first embodiment, the storage device 102A also stores other-purpose data 1311, other-purpose correct labels 1312, other-purpose empirical rules 1313 described later, divertable empirical rules 1314 described later, incorrect inference data 1315 described later, incorrect inference data inference model 1316 described later, and additional attention features 1317 described later.

[0088] The other-purpose data 1311 is learning data used for a purpose different from that of the target data 121. The other-purpose data 1311 has the same data configuration as the target data 121.

[0089] The other-purpose correct label 1312 is a correct label of the other-purpose data 1311. The data configuration of the other-purpose correct label 1312 is the same as that of the correct label 122.

[0090] <Processing Overview> Fig. 25 is a diagram showing various data input and output to each function of the classification class inference device 1A. Fig. 26 is a diagram for explaining an overview of the processing performed by the classification class inference device 1A. The processing shown in this diagram is executed, for example, when a predetermined input is made to the classification class inference device 1A by a user, or at a predetermined timing (for example, at a predetermined time or at a predetermined time interval).

[0091] First, the experiential rule automatic extraction means 111 receives the multi-purpose data 1311 and the multi-purpose correct label 1312 as input, and executes the above-mentioned processes from S401 to S406 to extract the multi-purpose experiential rule 1313 (S1401). The multi-purpose experiential rule 1313 is an experiential rule for a multi-purpose use, and its data structure is the same as that of the experiential rule 123.

[0092] Next, the experience rule quality evaluation means 1301 applies the multi-purpose experience rule 1313 to the target data 121, and compares the empirical rule application result, which is the application result, with the correct label 122 for each multi-purpose experience rule 1313. Then, the experience rule automatic extraction means 111 registers the multi-purpose experience rule 1313 in which the degree of agreement (matching degree) between the experience rule application result and the correct label 122 is equal to or greater than a preset threshold value, as a divertible experience rule 1314 (S1402). The divertible experience rule 1314 is a multi-purpose experience rule 1313 that can be diverted to the experience rule model 125, and its data structure is similar to that of the experience rule 123.

[0093] Next, the empirical rule automatic extraction means 111 receives the target data 121 and the correct label 122 as input, and executes the above-mentioned S401 to S406 to extract the empirical rule 123 (S1403).

[0094] Next, the additional noteworthy feature extraction means 1302 adds the divertable empirical rule 1314 extracted in S1402 to the empirical rule 123. Then, the additional noteworthy feature extraction means 1302 executes the above-mentioned processes from S801 to S803 to construct an empirical rule model 125 from the target data 121, the correct answer label 122, the pseudo data 124, and the empirical rule 123 (S1404).

[0095] Next, the additional noteworthy feature extraction means 1302 calculates an inference value for each of the target data 121 by inputting the target data 121 to the empirical rule model 125. Then, the additional noteworthy feature extraction means 1302 collects the target data 121 whose inference value does not match the correct label 122 as incorrect inference data 1315 (S1405). The data structure of the incorrect inference data 1315 is the same as that of the target data 121.

[0096] Next, the additional noteworthy feature extraction means 1302 constructs a machine learning model that infers the correct label 122 corresponding to the incorrect inference data from the incorrect inference data 1315 (inputs the incorrect inference data 1315 and outputs the correct label 122 corresponding to the incorrect inference data). The machine learning model is constructed based on an algorithm such as a neural network, a decision tree, a random forest, or a support vector machine (SVM). Then, the additional noteworthy feature extraction means 1302 calculates a SHAP value for each explanatory variable of the machine learning model, and stores explanatory variables whose absolute value of the SHAP value is equal to or greater than a predetermined threshold in the additional noteworthy feature 1317 (S1406).

[0097] (Additional feature of interest) 27 is a diagram showing an example of the additional noteworthy feature amount 1317. The additional noteworthy feature amount 1317 has the items of the explanatory variable 1011 in which an explanatory variable is set, and the attention level 1012 in which a SHAP value is set.

[0098] Next, the additional noteworthy feature extraction means 1302 registers the explanatory variable 1011 of the additional noteworthy feature 1317 in the noteworthy feature information 302. Then, the additional noteworthy feature extraction means 1302 executes the above-mentioned processes from S401 to S406 to additionally extract (newly generate) the empirical rule 123 (S1407).

[0099] Next, the empirical rule model construction means 113 executes the above-mentioned processes from S801 to S803 to construct the empirical rule model 125 from the target data 121, the correct label 122, the pseudo data 124, and the empirical rule 123 (S1404). Then, the process ends.

[0100] As described above, the classification class inference device 1A of this embodiment stores the other-purpose data 1311 used for purposes other than the target data 121 and the other-purpose correct labels 1312, generates the other-purpose empirical rules 1313 based on the other-purpose data 1311 and the other-purpose correct labels 1312, and applies the other-purpose empirical rules 1313 to the target data 121 to execute a rule quality evaluation process for extracting, from the other-purpose empirical rules 1313, divertable empirical rules 1314 to be used in constructing the empirical rule model 125.

[0101] That is, the classification class reasoning device 1A of this embodiment automatically extracts the applicable empirical rules 1314 from the other-use empirical rules 1313 for other uses (e.g., other industries) for which there is sufficient learning data. This makes it possible to apply reliable empirical rules used for other uses, thereby saving the user the trouble of manually creating the empirical rules 123 and enabling a more accurate empirical rule model 125 to be constructed with fewer man-hours.

[0102] Moreover, the classification class inference device 1A of this embodiment constructs an empirical rule model 125 based on the empirical rules 123 and the divergent empirical rules 1314, extracts variables of the target data 121 that are factors that cause the inference result of inputting the target data 121 into the constructed empirical rule model 125 to not match the correct label as new features, generates new empirical rules 123 based on the extracted features, and executes an additional noteworthy feature extraction process to reconstruct the empirical rule model 125 based on the newly generated empirical rules 123, the empirical rules 123 stored in the storage device 102A, and the divergent empirical rules 1314.

[0103] That is, the classification class reasoning device 1A focuses on explanatory variables of the target data 121 that are the cause of erroneous inference in which the inference result does not match the correct label when only the current empirical rules 123 are used. Then, the classification class reasoning device 1A can generate new empirical rules 123 that can correct erroneous inference (which is currently lacking) by using the explanatory variables as features. Therefore, it is possible to automatically generate new empirical rules 123 that further improve the inference accuracy of the empirical rule model 125. This saves the user the trouble of manually creating the empirical rules 123, and makes it possible to build a more accurate empirical rule model 125 with fewer man-hours.

[0104] The present invention is not limited to the above-described embodiment, and can be implemented using any components without departing from the scope of the present invention. The above-described embodiments and modifications are merely examples, and the present invention is not limited to these contents as long as the features of the invention are not impaired. In addition, although various embodiments and modifications have been described above, the present invention is not limited to these contents. Other aspects conceivable within the scope of the technical idea of ​​the present invention are also included in the scope of the present invention.

[0105] For example, part of the hardware included in each device of this embodiment may be provided in another device.

[0106] Furthermore, each program of the classification class inference devices 1 and 1A may be provided in another device, a program may be made up of a plurality of programs, or a plurality of programs may be integrated into a single program. [Explanation of symbols]

[0107] 1,1A Classification class inference device, 111 Empirical rule automatic extraction means, 112 Learning data supplement means, 113 Empirical rule model construction means, 114 Inference execution means, 115 Means for analyzing erroneous inference factors, 121 Target data, 122 Correct answer label, 123 Empirical rule, 124 Pseudo data, 125 Empirical rule model, 126 erroneous inference factors, 1301 Empirical rule quality evaluation means, 1302 Additional attention feature extraction means, 1311 Data for other uses 1311, 1312 Correct answer label for other uses, 1313 Empirical rule for other uses 1313, 1314 Reusable empirical rule, 1315 erroneous inference data, 1316 erroneous inference data inference model, 1317 Additional attention feature

Claims

1. A classification class inference device that constructs a model for inferring a classification class from target data, comprising: a storage device that stores a plurality of rules for determining a classification class corresponding to a condition related to a business, a plurality of target data, and a correct answer label for the target data; a model construction process for generating learning data corresponding to the conditions related to the business and constructing a model based on the learning data and the rules; A model inference process for inferring a classification class corresponding to the target data by inputting the target data into the model; a calculation device that executes an erroneous inference cause analysis process that generates and outputs information for identifying the rule that is a cause of a decrease in the inference accuracy of the model by comparing the inferred classification class with a correct label of the target data; A classification class inference device comprising:

2. The computing device includes: In the erroneous inference factor analysis process, a contribution degree of each of the rules to a decrease in the inference accuracy of the model is calculated; A rule correction process for correcting the rules whose contribution ratio is greater than a predetermined threshold value and reconstructing the model. The classification class inference device of claim 1 , further comprising:

3. the storage device stores features for defining the new rule; The computing device includes: An automatic rule extraction process for newly generating the rule based on a plurality of the target data, the correct answer labels of the target data, and the feature amount. The classification class inference device of claim 1 , further comprising:

4. The computing device includes: In the automatic rule extraction process, Calculating the value of the feature amount for each of the plurality of target data; Dividing the plurality of target data into a plurality of segments based on the values ​​of the feature quantities; The rule is generated based on the value of the feature amount in each segment and the correct label of the target data. The classification class inference apparatus according to claim 3.

5. The computing device includes: In the automatic rule extraction process, The segments are integrated based on the similarity of the distribution of the correct answer labels of the target data between the segments until the number of the segments reaches a predetermined number.

5. The classification class inference apparatus according to claim 4.

6. the storage device stores other-purpose data used for a purpose other than the target data, and a correct answer label of the other-purpose data; The computing device includes: generating a rule for another purpose, the rule being for the other purpose, based on the data for another purpose and a correct answer label of the data for another purpose; A rule quality evaluation process for extracting the rule to be used in constructing the model from the multi-purpose rules by applying the multi-purpose rules to the target data.

4. The classification class inference apparatus of claim 3, further comprising:

7. The computing device includes: constructing the model based on the rules stored in the storage device and the rules extracted in the rule quality evaluation process; extracting, as new features, variables of the target data that cause an inference result obtained by inputting the target data into the constructed model to not match a correct label; generating new rules based on the extracted features; An additional feature extraction process for reconstructing the model based on the newly generated rule, the rule stored in the storage device, and the rule extracted in the rule quality evaluation process.

7. The classification class inference apparatus of claim 6, further comprising:

8. An information processing device that constructs a model that infers a classification class from target data, A model construction process that generates learning data corresponding to business conditions, and constructs a model based on the learning data and a plurality of rules that determine a classification class corresponding to the business conditions from the business conditions; A model inference process for inferring a classification class corresponding to the target data by inputting the target data into the model; and performing an erroneous inference factor analysis process to generate and output information for identifying the rule that is a cause of a decrease in the inference accuracy of the model by comparing the inferred classification class with a correct label of the target data. Classification class inference methods.

9. An information processing device that constructs a model that infers a classification class from target data, A model construction process that generates learning data corresponding to business conditions, and constructs a model based on the learning data and a plurality of rules that determine a classification class corresponding to the business conditions from the business conditions; A model inference process for inferring a classification class corresponding to the target data by inputting the target data into the model; and executing an erroneous inference cause analysis process for generating and outputting information for identifying the rule that is a cause of a decrease in the inference accuracy of the model by comparing the inferred classification class with a correct label of the target data. A classification class inference program.

Citation Information

Cited By

  • Classification model training method, information classification method and device and electronic equipment

    CN121388783A

  • Support device, support method, and program

    JP2026163114A

  • Analysis system, analysis program, and analysis method

    JP7869608B1