Information processing device, information processing method, and program
Patent Information
- Application Number
- JP2021064001
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- Filing Date
- 2021-04-05
- Publication Date
- 2024-11-14
- Estimated Expiration
- 2041-04-05
AI Technical Summary
【0010】 1つの側面では、判断処理に影響のあるパラメータに関する条件を判断フローに反映できる。
Smart Images

Figure 0007583276000001 
Figure 0007583276000002 
Figure 0007583276000003
Abstract
Description
[Technical field]
[0001] The present invention relates to an information processing device, an information processing method, and a program. [Background technology]
[0002] The task of obtaining a judgment result according to a judgment procedure described in a document such as a guideline may be performed manually. The judgment procedure can be represented by a workflow in which a series of judgment steps included in the judgment procedure are connected in order. The workflow can be digitized, and a judgment process according to the workflow can be executed by an information processing device.
[0003] For example, there has been a proposal for a workflow generation server that generates a workflow, which generates a workflow consisting of a plurality of data operations using a value that quantifies the relationship between the histories of data operations on a database.
[0004] There has also been a proposal for a device that can identify correlations between a series of tasks performed by a given user for a single business case from log data that records the work of each user, and identify a group of tasks based on the correlations, thereby supporting the playback and visualization of business flows.
[0005] Furthermore, a task performance model optimization device has been proposed that optimizes a task performance model by repeatedly and flexibly modifying the task performance model based on the actual work results of a worker. In addition, there has been a proposal for an information processing device that controls a user interface that allows a user to decide whether or not to use collected sensor data as learning data for machine learning.In addition, there has been a proposal for a method of constructing a case database that, in classifying case data into classes, presents only case data that a computer cannot automatically classify to a human and accepts input of the correct class value by the human. [Prior art documents] [Patent documents]
[0006] [Patent Document 1] JP 2014-48673 A [Patent Document 2] JP 2017-45080 A [Patent Document 3] JP 2011-81620 A [Patent Document 4] International Publication No. 2020 / 170986 [Patent Document 5] JP 2005-352997 A Summary of the Invention [Problem to be solved by the invention]
[0007] As described above, a decision flow, which is a workflow for obtaining a certain decision result, can be created based on the contents of a guideline, etc. However, if the decision conditions included in the created decision flow are not appropriate, there is a problem that the accuracy of the decision result by the decision flow will be low.
[0008] According to one aspect, the present invention has an object to provide an information processing device, an information processing method, and a program capable of reflecting conditions related to parameters that affect a decision process in a decision flow. [Means for solving the problem]
[0009] In one embodiment, an information processing device is provided. The information processing device includes a storage unit and a processing unit. The storage unit stores information on a judgment flow in which a judgment result is obtained by a plurality of first judgment steps indicating judgment conditions for parameters. The processing unit acquires a plurality of second judgment steps associated with a judgment model generated by machine learning based on machine learning using teacher data representing judgment results for the plurality of parameters, identifies parameters associated with each of the plurality of second judgment steps, extracts a set of parameters from the plurality of parameters according to the influence of the parameters on the judgment model, and modifies a part of a plurality of first judgment steps included in the judgment flow based on the extracted set of parameters. Effect of the Invention
[0010] In one aspect, conditions regarding parameters that affect the decision process can be reflected in the decision flow. [Brief description of the drawings]
[0011] [Figure 1] FIG. 1 is a diagram illustrating an information processing apparatus according to a first embodiment. [Diagram 2] FIG. 11 illustrates an example of hardware of an information processing apparatus according to a second embodiment. [Diagram 3] FIG. 1 is a diagram illustrating a functional example (part 1) of an information processing device. [Figure 4] FIG. 13 is a diagram illustrating a functional example (part 2) of the information processing device. [Diagram 5] FIG. 13 is a diagram illustrating a functional example (part 3) of the information processing device. [Figure 6] FIG. 13 is a diagram illustrating an example of information stored in an original rule storage unit. [Figure 7] FIG. 13 is a diagram showing an example (part 1) of information stored in an extraction rule storage unit. [Figure 8] FIG. 13 is a diagram showing an example (part 2) of information stored in an extraction rule storage unit. [Figure 9] FIG. 11 is a diagram illustrating a first example of rule synthesis. [Figure 10] FIG. 13 is a diagram showing a first modified example of the workflow according to rule combination. [Figure 11] FIG. 13 is a diagram illustrating a second example of rule composition. [Figure 12] FIG. 13 is a diagram showing a second modification example of the workflow according to rule combination. [Figure 13] 13 is a flowchart illustrating an example of an overall process of the information processing device. [Figure 14] 13 is a flowchart showing an example of check item deletion. [Figure 15] 13 is a flowchart showing an example of acquiring a second extraction rule group. [Figure 16] 13 is a flowchart illustrating an example of rule synthesis. [Figure 17] 13 is a flowchart showing an example of similar data registration. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0012] The present embodiment will be described below with reference to the drawings. [First embodiment] A first embodiment will be described.
[0013] 1 is a diagram for explaining an information processing device according to a first embodiment. The information processing device 10 has a storage unit 11 and a processing unit 12. The storage unit 11 may be a volatile storage device such as a random access memory (RAM), or a non-volatile storage device such as a hard disk drive (HDD) or a flash memory. The processing unit 12 may include a central processing unit (CPU), a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), or the like. The processing unit 12 may be a processor that executes a program. The "processor" may include a set of multiple processors (multiprocessor).
[0014] The storage unit 11 stores information on the decision flow 20. The decision flow 20 is a workflow that obtains a decision result by a plurality of first decision steps each indicating a decision condition for a parameter. For example, the decision flow 20 has decision steps 21, 22, .... The decision condition in the decision step 21 is a condition for a parameter X, for example, a condition that the value of the parameter X is greater than or smaller than a condition value. The decision condition in the decision step 22 is a condition for a parameter A, for example, a condition that the value of the parameter A is greater than or smaller than a condition value. Each of the decision steps 21, 22, ... includes information indicating a next decision step depending on whether the decision condition is true or false. For example, the decision step 21 includes information indicating that the next decision step is the decision step 22 when the condition for the parameter X is true.
[0015] The processing unit 12 acquires a plurality of second judgment steps related to a judgment model generated by machine learning based on machine learning using teacher data representing judgment results for a plurality of parameters. The plurality of parameters include parameters used in judgment conditions of the judgment flow 20 and parameters not used in judgment conditions of the judgment flow 20.
[0016] The judgment model is a learned model by machine learning, and is a model that generalizes the relationship between a factor parameter (also called an explanatory variable or an independent variable) and a judgment result (also called a target variable or a dependent variable). The processing unit 12 can obtain an unknown judgment result for a new input parameter by an inference process based on an artificial intelligence (AI) technology using the judgment model. An example of a model used in machine learning is a deep neural network (DNN).
[0017] In one example, the processing unit 12 obtains, as a machine learning result, judgment step information 30 indicating a plurality of second judgment steps. The processing unit 12 can obtain the judgment step information 30 using existing technology. The judgment step information 30 includes, for each of a plurality of second judgment steps, information indicating a parameter used in the second judgment step, a judgment condition for the parameter, and other parameters with which the parameter is linked. In the drawing, an item indicating other parameters with which the parameter is linked is abbreviated as "linked".
[0018] For example, the decision step information 30 includes a "decision condition R1" for a parameter "A" used in one second decision step, and information that there is no other parameter ("-") that is linked to the parameter. The decision step information 30 also includes a "decision condition R2" for a parameter "B" used in another second decision step, and information that there is another parameter "A" that is linked to the parameter. Here, the parameter B is a parameter that is not used in the decision flow 20.
[0019] The judgment step information 30 includes information on other second judgment steps, such as information on the second judgment step related to the parameter X, but is not shown in the figure. Furthermore, the judgment step information 30 does not need to include information indicating the order of the multiple second judgment steps. This is because the multiple second judgment steps are information related to the judgment model, and the order of the multiple second judgment steps cannot be obtained.
[0020] The processing unit 12 identifies parameters associated with each of the second judgment steps. For example, the processing unit 12 identifies parameters associated with each of the second judgment steps based on the judgment step information 30 described above.
[0021] The processing unit 12 extracts a set of parameters from the multiple parameters according to the influence of the parameters on the judgment model. For example, the processing unit 12 compares a judgment result by a judgment model using parameter B and a judgment result by a judgment model not using parameter B for multiple inputs whose judgment results are known, with respect to parameter B not included in the judgment flow 20. Then, the processing unit 12 determines that the parameter B has an influence on the judgment model when the accuracy of the judgment result by the judgment model using parameter B is improved more than the accuracy of the judgment result by the judgment model not using parameter B. For example, the processing unit 12 determines that the accuracy of the judgment result is improved by using parameter B when the number of correct answers for known judgment results is higher when using parameter B than when not using parameter B.
[0022] According to the decision step information 30, the parameter B is related to the parameter A. Therefore, when it is determined that the parameter B has an influence on the decision model, the processing unit 12 extracts a pair of parameters A and B based on the decision step information 30.
[0023] The processing unit 12 changes a part of the first judgment steps included in the judgment flow 20 based on the extracted parameter set. For example, the judgment step 22 in the judgment flow 20 is a condition judgment regarding the parameter A. Therefore, the processing unit 12 identifies the judgment step 22 for the extracted parameter set A and B. The processing unit 12 acquires the judgment condition R1 for the parameter A and the judgment condition R2 for the parameter B from the judgment step information 30, and changes the judgment condition of the judgment step 22 to a judgment condition obtained by combining the judgment conditions R1 and R2. The judgment condition obtained by combining the judgment conditions R1 and R2 is, for example, a condition for determining whether or not both the judgment conditions R1 and R2 are true. The judgment condition R1 may be the same as or different from the judgment condition of the judgment step 22 for the parameter A.
[0024] In this way, the processing unit 12 generates a changed decision flow 40 by changing some of the multiple first decision steps included in the decision flow 20. The changed decision flow 40 has decision steps 41, 42, .... For example, the decision step 41 corresponds to the decision step 21. In the decision step 42, the decision condition of the decision step 22 for the parameter A is changed to the decision conditions R1 and R2 for the parameters A and B. More specifically, the decision of "decision step 22 is true" in the decision flow 20 is replaced with the decision of "decision step 42 is true" in the changed decision flow 40. Also, the decision of "decision step 22 is false" in the decision flow 20 is replaced with the decision of "decision step 42 is false" in the changed decision flow 40. In the above example, when both the decision conditions R1 and R2 are true, the decision step 42 is true, and when at least one of the decision conditions R1 and R2 is false, the decision step 42 is false.
[0025] In the example of the changed judgment flow 40, the judgment step 22 is changed to the judgment step 42, but the processing unit 12 may generate the changed judgment flow by treating the judgment conditions R1 and R2 of the parameters A and B in the judgment step 42 as consecutive individual judgment steps. For example, the judgment step 42 may be expressed by dividing it into a first judgment step for determining whether the judgment condition R1 is true or false, and a second judgment step for determining whether the judgment condition R2 is true or false, which is executed when the judgment condition R1 is true. In this way, the processing unit 12 may treat the judgment conditions R1 and R2 as one judgment step 42 or as individual judgment steps. In either case, substantially the same judgment flow is expressed.
[0026] According to the information processing device 10, information on a judgment flow 20 in which a judgment result is obtained by a plurality of first judgment steps indicating judgment conditions for parameters is acquired. Based on machine learning using teacher data representing the judgment results for the plurality of parameters, a plurality of second judgment steps associated with a judgment model generated by machine learning are acquired. Parameters associated with each of the plurality of second judgment steps are identified. A set of parameters is extracted from the plurality of parameters according to the influence of the parameters on the judgment model. A part of the plurality of first judgment steps included in the judgment flow 20 is changed based on the extracted set of parameters.
[0027] This allows the information processing device 10 to reflect conditions related to parameters that affect the judgment process in the judgment flow 20. Furthermore, the information processing device 10 can appropriately improve the judgment flow. As a result, a modified judgment flow 40 that refines the existing judgment flow 20 is obtained. By using the modified judgment flow 40 for the judgment process, the processing unit 12 can improve the accuracy of the judgment result obtained for a new input compared to using the judgment flow 20.
[0028] In addition, the changed decision flow 40 is digitized based on rule-based information that allows the decision content of each decision step to be easily understood by humans. The processing unit 12 executes the data represented by the rule base using a predetermined rule engine, thereby obtaining a decision result based on the changed decision flow 40.
[0029] Here, when a judgment result is obtained by model-based inference processing using AI technology, the judgment process becomes a black box, making it difficult to explain why the judgment result was reached. On the other hand, according to the information processing device 10, since the judgment result is obtained based on the changed judgment flow 40 expressed in a rule base, it is possible to explain why the judgment result was reached, and there is an advantage that the user can easily understand the reason.
[0030] The functions of the information processing device 10 will be described in more detail below. [Second embodiment] Next, a second embodiment will be described.
[0031] FIG. 2 is a diagram illustrating an example of hardware of an information processing device according to a second embodiment. The information processing device 100 supports acquisition of a judgment result based on certain information. The information processing device 100 can be realized by a computer such as a server. In one example, the information processing device 100 supports judgment of the good or bad management status of each of a plurality of companies. In this case, various indexes representing the management status of the company to be evaluated are input as parameters to the information processing device 100. The information processing device 100 outputs a result of judgment of the good or bad management status of the company based on the input parameters. The information processing device 100 provides a function of improving a judgment procedure (basic workflow) that is the basis for obtaining a certain judgment result.
[0032] The information processing device 100 includes a CPU 101, a RAM 102, a HDD 103, a GPU (Graphics Processing Unit) 104, an input interface 105, a medium reader 106, and a NIC (Network Interface Card) 107. The CPU 101 is an example of the processing unit 12 of the first embodiment. The RAM 102 or the HDD 103 is an example of the storage unit 11 of the first embodiment.
[0033] The CPU 101 is a processor that executes program instructions. The CPU 101 loads at least a part of the programs and data stored in the HDD 103 into the RAM 102 and executes the programs. The CPU 101 may include multiple processor cores. The information processing device 100 may have multiple processors. The processes described below may be executed in parallel using multiple processors or processor cores. A set of multiple processors may be called a "multiprocessor" or simply a "processor."
[0034] The RAM 102 is a volatile semiconductor memory that temporarily stores programs executed by the CPU 101 and data used in calculations by the CPU 101. Note that the information processing device 100 may include a type of memory other than a RAM, or may include multiple memories.
[0035] The HDD 103 is a non-volatile storage device that stores software programs such as an OS (Operating System), middleware, and application software, as well as data. The information processing device 100 may include other types of storage devices, such as a flash memory or an SSD (Solid State Drive), or may include multiple non-volatile storage devices.
[0036] The GPU 104 outputs an image to a display 51 connected to the information processing device 100 in accordance with an instruction from the CPU 101. As the display 51, any type of display may be used, such as a CRT (Cathode Ray Tube) display, a liquid crystal display (LCD: Liquid Crystal Display), a plasma display, or an organic EL (OEL: Organic Electro-Luminescence) display.
[0037] The input interface 105 acquires an input signal from an input device 52 connected to the information processing device 100, and outputs the signal to the CPU 101. As the input device 52, a pointing device such as a mouse, a touch panel, a touch pad, or a trackball, a keyboard, a remote controller, a button switch, etc. may be used. Furthermore, a plurality of types of input devices may be connected to the information processing device 100.
[0038] The medium reader 106 is a reading device that reads programs and data recorded on the recording medium 53. For example, a magnetic disk, an optical disk, a magneto-optical disk (MO: Magneto-Optical disk), a semiconductor memory, etc. are used as the recording medium 53. The magnetic disk includes a flexible disk (FD: Flexible Disk) and a HDD. The optical disk includes a compact disc (CD) and a digital versatile disc (DVD).
[0039] The medium reader 106 copies, for example, the program or data read from the recording medium 53 to another recording medium such as the RAM 102 or the HDD 103. The read program is executed, for example, by the CPU 101. Note that the recording medium 53 may be a portable recording medium, and may be used for distributing the program or data. Also, the recording medium 53 and the HDD 103 may be called computer-readable recording media.
[0040] The NIC 107 is an interface that is connected to the network 50 and communicates with other computers via the network 50. The NIC 107 is connected to a communication device such as a switch or a router via a cable.
[0041] 3 is a diagram showing a functional example (part 1) of an information processing device. The information processing device 100 has a machine learning mechanism 110, an original rule storage unit 120, an extracted rule storage unit 130, a revised rule storage unit 140, a new data storage unit 150, a judgment result storage unit 160, a rule creation unit 170, a synthesis unit 180, and a rule execution unit 190. The machine learning mechanism 110 has a past data storage unit 111, a judgment model storage unit 112, a judgment basis storage unit 113, an AI judgment result storage unit 114, a learning unit 115, and an AI judgment unit 116.
[0042] The memory areas of RAM 102 and HDD 103 are used for past data storage unit 111, judgment model storage unit 112, judgment basis storage unit 113, AI judgment result storage unit 114, original rule storage unit 120, extracted rule storage unit 130, revised rule storage unit 140, new data storage unit 150, and judgment result storage unit 160. The learning unit 115, AI judgment unit 116, rule creation unit 170, synthesis unit 180, and rule execution unit 190 are realized by CPU 101 executing programs stored in RAM 102. However, some of these functions may be realized by electronic circuits such as GPU and FPGA.
[0043] The machine learning mechanism 110 performs machine learning based on AI technology and inference processing based on a trained model. Existing technology is used for the machine learning mechanism 110. For example, DNN is used as the model used in the machine learning mechanism 110.
[0044] The past data storage unit 111 stores past data processed by the learning unit 115 and the AI judgment unit 116. The past data stored in the past data storage unit 111 is data related to parameters of judgment objects and many judgment results for the parameters, which were made manually or otherwise in the past. The past data is used as teacher data in the machine learning of the learning unit 115. The past data is also used to evaluate the judgment accuracy of the AI judgment result by the AI judgment unit 116. Hereinafter, the parameters of the judgment object for obtaining the judgment result are referred to as judgment items.
[0045] The judgment model storage unit 112 stores the trained model output by the learning unit 115. The trained model is hereinafter referred to as a judgment model. For example, a judgment model may be a model for judging whether a certain company is in poor financial condition.
[0046] The judgment basis storage unit 113 stores information on the judgment basis output by the learning unit 115. The judgment basis indicates the judgment conditions for each judgment item used by the learning unit 115 to generate a judgment model and the importance of the judgment item. For example, in a company that is judged to be in poor financial condition, the more frequently the value of a certain judgment item falls within a specific range, the higher the importance of the judgment condition that the value of the judgment item falls within the specific range for the judgment result that the financial condition is in poor condition.
[0047] The AI judgment result storage unit 114 stores the output result by the AI judgment unit 116. The output result by the AI judgment unit 116 is the result of the AI judgment made by the AI judgment unit 116. The learning unit 115 performs machine learning based on the past data stored in the past data storage unit 111, generates a judgment model and information on judgment grounds, and stores them in the judgment model storage unit 112 and the judgment grounds storage unit 113, respectively. For example, the judgment grounds may include information such as a judgment condition "profit margin > 10%" for the judgment item "profit margin".
[0048] The AI judgment unit 116 performs inference processing, i.e., AI judgment, based on the past data stored in the past data storage unit 111 and the judgment model stored in the judgment model storage unit 112. The AI judgment unit 116 stores the AI judgment result in the AI judgment result storage unit 114.
[0049] The original rule storage unit 120 stores original rules. An original rule is a judgment step for making a certain judgment, which is created based on a document such as a guideline (e.g., a guideline showing a guideline for a company's management decisions). For example, an original rule is extracted from the document by a user, and information indicating the extracted content is registered in advance in the original rule storage unit 120. A series of original rules created based on the document provides a judgment procedure for making a certain judgment. The judgment procedure is represented by a workflow that is the target of improvement by the information processing device 100, i.e., a basic workflow. The basic workflow is an example of the judgment flow 20 of the first embodiment.
[0050] The extraction rule storage unit 130 stores extraction rules. The extraction rules are generated by the rule creation unit 170 based on the information on the judgment basis stored in the judgment basis storage unit 113. It can be said that the extraction rules are rules for obtaining judgment results, extracted based on the results of machine learning.
[0051] The amended rule storage unit 140 stores amended rules. The amended rules are generated by the synthesis unit 180 synthesizing the original rules and the extracted rules. A set of amended rules (which may include the original rules) results in an improved judgment procedure for making a certain judgment. The judgment procedure is represented by a workflow that improves the basic workflow, i.e., a modified workflow. The modified workflow is an example of the modified judgment flow 40 of the first embodiment.
[0052] The new data storage unit 150 stores new data. The new data is information that represents various parameters related to a new judgment target. The parameters included in the new data include judgment items according to revised rules. The new data is used in the judgment process by the rule execution unit 190.
[0053] The judgment result storage unit 160 stores information on the judgment results by the rule execution unit 190. The judgment results stored in the judgment result storage unit 160 are rule-based judgment results by the rule execution unit 190, unlike the model-based judgment results by the AI judgment unit 116.
[0054] The rule creation unit 170 generates extraction rule information based on the judgment basis information stored in the judgment basis storage unit 113, and stores the extraction rule information in the extraction rule storage unit 130. The extraction rule includes information such as judgment items related to the judgment model obtained from the results of machine learning by the learning unit 115, judgment conditions for the judgment items, and the importance of the judgment items to the AI judgment results.
[0055] The judgment items in one extraction rule may be independent or may be associated with judgment items in other extraction rules. Information indicating the association between the judgment items is included in the judgment basis information output by the machine learning mechanism 110. For example, an extraction rule with a judgment condition of "profit margin > 10%" may be obtained for an independent judgment item "profit margin". Alternatively, an extraction rule with a judgment condition of "profit margin > 10%" for the judgment item "profit margin" and an extraction rule with a judgment condition of "net income > xxx" for another judgment item "net income" related to the judgment item "profit margin" may be obtained.
[0056] As described later, the extraction rules are gradually improved by repeatedly using the machine learning mechanism 110 and evaluating the AI judgment results. The rule creation unit 170 stores the extraction rules finally obtained through the improvement in the extraction rule storage unit 130.
[0057] The synthesis unit 180 generates a revised rule by synthesizing the original rule stored in the original rule storage unit 120 and the extraction rule stored in the extraction rule storage unit 130, and stores the revised rule in the revised rule storage unit 140. In the synthesis by the synthesis unit 180, the extraction rule is added to the original rule. For example, the synthesis unit 180 adds another judgment condition related to another judgment item in the extraction rule to a judgment condition related to a specific judgment item in the original rule. Alternatively, the synthesis unit 180 may replace a judgment condition related to a specific judgment item in the original rule with another judgment condition related to the same judgment item in the extraction rule.
[0058] The rule execution unit 190 is a rule engine that executes rule-based processing. Based on the information of the revised rule stored in the revised rule storage unit 140, the rule execution unit 190 executes the procedure of the post-change workflow for the new data stored in the new data storage unit 150, and obtains a judgment result based on the procedure. The rule execution unit 190 stores the obtained judgment result in the judgment result storage unit 160.
[0059] Furthermore, the information processing device 100 has the following extraction rule improvement function. 4 is a diagram showing a functional example (part 2) of the information processing device. First, the learning unit 115 performs machine learning using all judgment items that can be used for a target judgment process in the past data 111a stored in the past data storage unit 111 as input, and outputs a judgment model 112a and a judgment basis 113a as the results of the machine learning.
[0060] The rule creating unit 170 generates a first extraction rule group 131 based on the determination basis 113a. The first extraction rule group 131 includes all extraction rules that are candidates for addition to the basic workflow. The first extraction rule group 131 is stored in the extraction rule storage unit 130.
[0061] 3, information processing device 100 has narrowing down unit 171, item adding unit 172, accuracy determining unit 173, added item selecting unit 174, and information determining unit 175. Narrowing down unit 171, item adding unit 172, accuracy determining unit 173, added item selecting unit 174, and information determining unit 175 are realized by CPU 101 executing a program stored in RAM 102.
[0062] The narrowing down unit 171 narrows down the judgment items for the past data 111a stored in the past data storage unit 111, and generates past data 111b after narrowing down the judgment items. In the first stage after acquiring the first extraction rule group 131, the narrowing down unit 171 narrows down the judgment items to the minimum necessary based on the original rule group 121 stored in the original rule storage unit 120. The necessary optimum limit of judgment items may be all judgment items used in the judgment process of the original rule group 121. For example, when making a management decision of a company, the minimum necessary judgment items may include "sales", "profit", "operating profit", and "expenses".
[0063] The item adding unit 172 adds judgment items to the past data 111b, thereby generating past data 111c after the judgment items have been added. The judgment items that are candidates for addition are judgment items that are not included in the original rule group 121. The past data 111c will have a larger amount of information than the past data 111b by the amount of the added judgment items. The judgment items to be added are selected by the added item selecting unit 174. For example, when making a management decision about a company, possible judgment items to be added as candidates include "net income for the current period," "number of issued shares," "cash flow per share," "total assets," and "current liabilities."
[0064] The learning unit 115 generates a judgment model 112b and information on judgment grounds (not shown) through machine learning using the past data 111b. The learning unit 115 also generates a judgment model 112c and information on judgment grounds 113b through machine learning using the past data 111c.
[0065] The AI judgment unit 116 performs an AI judgment using the past data 111a as an input based on the judgment model 112b, and generates an AI judgment result 114a. Also, the AI judgment unit 116 performs an AI judgment using the past data 111a as an input based on the judgment model 112c, and generates an AI judgment result 114b.
[0066] The AI judgment results 114a and 114b each include judgment results by the judgment models 112b and 112c for multiple samples (e.g., companies) of judgment targets included in the past data 111a. The judgment results of the samples of judgment targets included in the past data 111a are already known.
[0067] The accuracy determination unit 173 compares the AI judgment results 114a and 114b and determines whether the AI judgment result 114b has a higher accuracy of the judgment result. Specifically, if the AI judgment result 114b has a larger number of samples that match known judgment results than the AI judgment result 114a, the accuracy determination unit 173 determines that the AI judgment result 114b has a higher accuracy of the judgment result. If the AI judgment result 114b has a smaller number of samples that match known judgment results than the AI judgment result 114a, the accuracy determination unit 173 does not determine that the AI judgment result 114b has a higher accuracy of the judgment result.
[0068] When the accuracy of the AI judgment result 114b is higher, the accuracy determination unit 173 determines the judgment item added in the generation of the past data 111c as a target for addition to the basic workflow, and notifies the rule creation unit 170 of the judgment item. The accuracy determination unit 173 also instructs the addition item selection unit 174 to select the next item to be added.
[0069] In the next stage, the current past data 111c is used as the next past data 111b, and the result of adding judgment items to the current past data 111c by the item adding unit 172 is used as the next past data 111c. In this manner, the item adding unit 172 adds items until it is determined that the accuracy of the judgment results is equivalent for the AI judgment results 114a, 114b.
[0070] The additional item selection unit 174 selects judgement items to be added based on the first extraction rule group 131, and notifies the item adding unit 172 of the selected judgement items. The additional item selection unit 174 selects judgement items with high importance from the first extraction rule group 131. Here, some of the judgement items included in the first extraction rule group may be deleted by the information judgment unit 175.
[0071] The information judgment unit 175 deletes judgment items that match similar data designated in advance by the user from among the judgment items included in the first extraction rule group 131. For example, unnecessary judgment items according to the target judgment content, such as judgment items or sets of judgment items that are unnecessary for the management judgment of the company, are registered in advance as similar data.
[0072] The rule creating unit 170 generates the second extraction rule group 132 for the above-mentioned minimum necessary judgment items and judgment items to be added notified by the accuracy determining unit 173 based on the judgment grounds 113b for the finally created judgment model 112c. Note that, each time the rule creating unit 170 is notified of a judgment item to be added by the accuracy determining unit 173 at a stage where each judgment item has been added, the rule creating unit 170 may generate an extraction rule for the judgment item to be added based on the judgment grounds 113b at that stage and include the generated extraction rule in the second extraction rule group 132. The second extraction rule group 132 is stored in the extraction rule storage unit 130. Note that the second extraction rule group 132 also includes information on the minimum necessary judgment items initially extracted from the original rule group 121.
[0073] The synthesis unit 180 synthesizes the extraction rules included in the second extraction rule group 132 thus generated and the original rules included in the original rule group 121 to generate an amended rule group 141. The amended rule group 141 is stored in the amended rule storage unit 140.
[0074] The rule execution unit 190 executes the procedure of the changed workflow for the new data 151 based on the revised rule group 141, and obtains a judgment result 161 based on the procedure. The rule execution unit 190 outputs the judgment result 161.
[0075] Here, the information determining unit 175 has the following functions. 5 is a diagram showing a functional example (part 3) of the information processing device. The information determining unit 175 has a similar data storage unit 175a, a registration unit 175b, a comparison unit 175c, and a deletion unit 175d. The similar data storage unit 175a uses a storage area of the RAM 102 or the HDD 103. The registration unit 175b, the comparison unit 175c, and the deletion unit 175d are realized by the CPU 101 executing a program stored in the RAM 102.
[0076] The similar data storage unit 175a stores similar data. The similar data is a judgment item that is judged by the user to be unnecessary for a target judgment content (for example, judgment of a company's management status).
[0077] The registration unit 175b accepts input of similar data by the user, and registers the accepted similar data in the similar data storage unit 175a. The comparison unit 175c compares the first extraction rule group 131 stored in the extraction rule storage unit 130 with the similar data stored in the similar data storage unit 175a, identifies judgment items from the first extraction rule group 131 that match the similar data, and notifies the deletion unit 175d.
[0078] The deletion unit 175d deletes the check items notified by the comparison unit 175c from the first extraction rule group 131, and generates a first extraction rule group 131a after the check items have been deleted. The first extraction rule group 131a is also stored in the extraction rule storage unit 130.
[0079] The information processing device 100 can perform control by the function of the information judgment unit 175 so as not to use judgment items registered as similar data in learning, thereby shortening the time required to obtain the second extraction rules.
[0080] FIG. 6 is a diagram showing an example of information stored in the original rule storage unit. The original rule storage unit 120 stores an original rule group 121. The original rule group 121 represents a basic workflow 200. The basic workflow 200 includes a plurality of steps for obtaining a certain judgment result. The plurality of steps are represented as steps 1, 2, .... The steps correspond to the "judgment step" in the first embodiment. The basic workflow 200 may obtain a plurality of judgment results. For example, the basic workflow 200 obtains judgment results α and β. In the above-mentioned example of judging the business condition of a company, an example can be considered in which the judgment result α is "business condition is poor" and the judgment result β is "business condition is good."
[0081] The original rule group 121 includes original rule information that represents each of the multiple steps of the basic workflow. One piece of original rule information corresponds to one step. For example, the original rule group 121 includes original rule information 121e1 that represents step 3 of the basic workflow 200. In the original rule group 121, the other steps are also represented by the same data structure as the original rule information 121e1.
[0082] The original rule information 121e1 is information indicating the original rule, and includes information on the previous step, judgment item, condition value, and next step. The previous step is the step immediately before the step in question, for example, step 2 for step 3. The judgment item is the name of the judgment item used in the judgment process. The condition value is a condition value to be compared with the value of the judgment item. The next step is the next step after the step in question. If there is a branch according to a condition judgment for the judgment item, the branch destination according to the result of the condition judgment is registered in the next step. The original rule information 121e1 also includes other information, such as a condition formula indicating the content of the condition judgment using the condition value.
[0083] The revised rule group 141 is also represented by information on revised rules (revised rule information) having the same data structure as the original rule group 121. FIG. 7 is a diagram showing an example (part 1) of information stored in the extraction rule storage unit. The extraction rule storage unit 130 stores a first extraction rule group 131. The first extraction rule group 131 includes extraction rule information representing each of a plurality of steps based on the learning result of the learning unit 115. The extraction rule information is information indicating an extraction rule. The plurality of steps represented by the first extraction rule group 131 are represented as steps A, B, .... The plurality of steps includes steps (represented as steps 1, 2, ...) for the same judgment items as steps 1, 2, ... of the basic workflow 200. For example, the first extraction rule group 131 includes extraction rule information 131e1 representing step B. In the first extraction rule group 131, the other steps are also represented by the same data structure as the extraction rule information 131e1.
[0084] The extraction rule information 131e1 includes information on a judgment item, a condition value, linkage information, and importance. The judgment item is the name of a judgment item used in the judgment process. The condition value is a condition value to be compared with the value of the judgment item. The linkage information is the name of another judgment item related to the judgment item in question. The other judgment items related to the judgment item in question are called related judgment items. The importance is the importance of the judgment item in question. The extraction rule information 131e1 also includes other information, such as a condition expression that indicates the content of a condition judgment using the condition value. Also, unlike the original rule information 121e1, the extraction rule information 131e1 does not need to include information on the previous step or the next step.
[0085] The second extraction rule group 132 is also realized by the same data structure as the first extraction rule group 131. 8 is a diagram showing an example (part 2) of information stored in the extraction rule storage unit. Importance information 133 is stored in the extraction rule storage unit 130. The importance information 133 is generated by the rule creation unit 170 based on the information on judgment grounds stored in the judgment grounds storage unit 113, and stored in the extraction rule storage unit 130. The importance information 133 holds the normalized importance for each judgment item in the first extraction rule set 131. The importance information 133 includes judgment items and importance items.
[0086] In the judgment item field, the name of the judgment item is registered. A set of multiple judgment item names may be registered as a judgment item. In the importance field, a normalized importance for the relevant judgment item is registered. Through normalization, the importance value included in the first extraction rule group 131 is adjusted according to the judgment content. For example, in an example of judging the management status of a company, through normalization, the importance is expressed as a value greater than -1 and less than +1. The rule creation unit 170 may adjust the importance so that the greater the importance, the greater the weight given to an evaluation that the management judgment is poor, and the smaller the importance, the less the weight given to an evaluation that the management judgment is poor.
[0087] For example, the rule creating unit 170 may obtain, as a normalized importance, an average value of the importance levels corresponding to a set of multiple linked judgment items in the first extraction rule group 131. This makes it possible to raise the importance levels of other judgment items related to a judgment item having a relatively high importance level, making it easier for the set of judgment items to be reflected in the judgment model.
[0088] For example, a record of importance "0.7" for the check item "item A" is registered in the importance information 133. This record indicates that the importance of the check item "item A" is "0.7." Note that the check item "item A" is a check item in step A.
[0089] Furthermore, in the importance information 133, a record with an importance of "0.5" is registered for the pair of check items "item C, item 3." This record indicates that the importance of the pair of check items "item C, item 3" is "0.5." Note that check item "item C" is a check item in step C. Check item "item 3" is a check item in step 3.
[0090] Next, a description will be given of an example of rule merging by the merging unit 180. In rule merging, a merging process is performed based on original rule information included in the original rule group 121 and extracted rule information included in the second extracted rule group 132. This merging process corresponds to a process of changing the basic workflow 200 into a changed workflow.
[0091] Fig. 9 is a diagram showing a first example of rule synthesis. In the example of Fig. 9, the original rule group 121 includes original rule information 121e1 related to the judgement item "A1", and the second extraction rule group 132 includes extracted rule information 132e1 related to the judgement item "A1". The original rule information 121e1 corresponds to step 3 of the basic workflow 200. The extracted rule information 132e1 corresponds to step B obtained by the machine learning result.
[0092] In this case, the synthesis unit 180 replaces step 3 with step B. That is, the synthesis unit 180 acquires the condition value "X2" for the decision item "A1" from the extraction rule information 132e1. Then, the synthesis unit 180 changes the condition value included in the condition formula for the decision item "A1" in step 3 in the original rule group 121 from "X1" to "X2." Note that if the condition formula itself in step B is different from that in step 3, the synthesis unit 180 may change the condition formula in step 3 to the condition formula in step B.
[0093] In this way, for example, the synthesis unit 180 may generate a revised rule set 141 that indicates a modified workflow in which step 3 of the basic workflow 200 is replaced with step B. FIG. 10 is a diagram showing a first example of a workflow change according to rule synthesis. The changed workflow 300 is a workflow represented by the modified rule group 141. For example, the basic workflow 200 includes step 3. Step 3 is a judgment condition expressed by a conditional expression of "A1>X1" using a condition value "X1" for the judgment item "A1". In contrast, in the changed workflow 300, step 3 is changed to step B. Step B is a judgment condition expressed by a conditional expression of "A1>X2" using a condition value "X2" for the judgment item "A1". The next step according to the judgment result of step B is the same as the next step according to the judgment result of step 3 in the basic workflow 200. In the modified rule information of the modified rule group 141, the relationship between the previous step and the next step of the corresponding step in the changed workflow 300 is registered.
[0094] Fig. 11 is a diagram showing a second example of rule synthesis. In the example of Fig. 11, the original rule group 121 includes original rule information 121e1 related to the judgement item "A1", and the second extraction rule group 132 includes extraction rule information 132e2 related to the judgement item "K1". The extraction rule information 132e2 includes "A1" as linkage information (related judgement item) for the judgement item "K1". The original rule information 121e1 corresponds to step 3 of the basic workflow 200. The extraction rule information 132e2 corresponds to step B obtained by the machine learning result.
[0095] In this case, the combination unit 180 adds step B to step 3. That is, the combination unit 180 obtains the condition value "P1" for the decision item "K1" from the extraction rule information 132e2, and obtains the decision item "A1" associated with the decision item "K1". Then, the combination unit 180 obtains original rule information for step 3 included in the original rule group 121 based on the associated decision item "A1", and adds the decision condition for decision item "K1" of step B to the original rule information.
[0096] Thus, for example, the synthesizing unit 180 may generate a corrected rule group 142 indicating a modified workflow in which step B is added to step 3 of the basic workflow 200. As a modification example, it is conceivable that the determination step is modified such that the step of the determination condition regarding "profit" in the original rule becomes the step of the determination condition regarding "profit" and "net profit for the current period" in the corrected rule.
[0097] FIG. 12 is a diagram showing a second modification example of the workflow according to rule synthesis. The modified workflow 300a is a workflow represented by the corrected rule group 142. For example, the basic workflow 200 includes step 3. Step 3 is a determination condition represented by a conditional expression using the condition value "X1" for the determination item "A1". On the other hand, in the modified workflow 300a, step B is added to step 3. Step B is a determination condition regarding the determination item "A1" and represented by a conditional expression of "K1 < P1" using the condition value "P1".
[0098] Thereby, in the basic workflow 200, the procedure of proceeding to the next first candidate step when step 3 is true is changed to the procedure of proceeding to the first candidate step when both step 3 and step B are true in the modified workflow 300a. Also, in the basic workflow 200, the procedure of proceeding to the next second candidate step when step 3 is false is changed to the procedure of proceeding to the second candidate step when at least one of step 3 and step B is false in the modified workflow 300a. In the rule information of the corrected rule group 142, the relationship between the previous step and the next step of the corresponding step in the modified workflow 300a is registered.
[0099] Note that, as also exemplified in the first embodiment, the determination conditions for both step 3 and step B may be represented as one step or two consecutive steps in the modified workflow 300a.
[0100] The information processing apparatus 100 executes the following processing procedure. FIG. 13 is a flowchart showing an example of the overall process of the information processing device. (S1) The rule creating unit 170 acquires the first extraction rule group 131. As described above, the rule creating unit 170 generates the first extraction rule group 131 based on the decision grounds 113a output as a result of machine learning by the learning unit 115 using all decision items included in the past data 111a, for example, and stores the first extraction rule group 131 in the extraction rule storage unit 130. In addition, the rule creating unit 170 generates importance information 133 together with the first extraction rule group 131, and stores the importance information 133 in the extraction rule storage unit 130.
[0101] (S2) The information judgment unit 175 executes check item deletion for the first extraction rule set 131. The check item deletion process will be described in detail later. (S3) The rule creating section 170 acquires the second extraction rule group 132. The process of acquiring the second extraction rule group 132 will be described in detail later.
[0102] (S4) The merging unit 180 performs rule merging to merge the second extracted rule group 132 with the original rule group 121. Details of the rule merging process will be described later. FIG. 14 is a flow chart showing an example of check item deletion.
[0103] Deleting a check item corresponds to step S2. (S10) The comparison unit 175c acquires one of all the check items in the first extraction rule group 131.
[0104] (S11) The comparison unit 175c judges whether or not the judgment item acquired in step S10 matches the judgment item of the similar data stored in the similar data storage unit 175a. If they match, the process proceeds to step S12. If they do not match, the process proceeds to step S13.
[0105] (S12) The deletion unit 175d deletes the judgment items that match the judgment items of the similar data from the first extraction rule group 131. Deleting a judgment item corresponds to deleting the extraction rule information of the corresponding judgment item from the first extraction rule group 131. Then, the process proceeds to step S14.
[0106] (S13) The deletion unit 175d leaves the check items that do not match the check items of the similar data without deleting them from the first extraction rule set. Then, the process proceeds to step S14. (S14) The comparison unit 175c determines whether or not all the decision items included in the first extraction rule group 131 have been processed. If all the decision items included in the first extraction rule group 131 have been processed, the decision item deletion ends. If all the decision items included in the first extraction rule group 131 have not been processed, the process proceeds to step S10.
[0107] In this way, unnecessary check items are deleted from the first extraction rule group 131. In the following, it is assumed that the first extraction rule group 131 is a result of check items being deleted by the check item deletion process.
[0108] FIG. 15 is a flowchart showing an example of obtaining the second extraction rule group. Obtaining the second extraction rule group corresponds to step S3. (S20) The addition item selection unit 174 selects judgment items to be added for machine learning from the first extraction rule group 131 according to the importance of each judgment item included in the importance information 133. For example, the addition item selection unit 174 preferentially selects judgment items with high importance for the judgment that the company's financial condition is poor. The addition item selection unit 174 notifies the item addition unit 172 of the selected judgment items.
[0109] (S21) The AI judgment unit 116 acquires the AI judgment result 114a based on the current judgment model (judgment model 112b) obtained by learning using the current judgment item group. In the initial stage, the current judgment item group used to create the current judgment model in step S21 is only the minimum necessary judgment items in the past data 111a. The minimum necessary judgment items may be, for example, all judgment items included in the original rule group 121.
[0110] (S22) The AI judgment unit 116 acquires the AI judgment result 114b based on the new judgment model (judgment model 112c) obtained by learning using a group of judgment items obtained by adding the judgment item of step S20 to the current group of judgment items. The addition of a judgment item to the current group of judgment items is performed by the item adding unit 172.
[0111] (S23) The accuracy determination unit 173 compares the AI judgment results 114a, 114b and determines whether the judgment accuracy has improved by adding the judgment item. If the judgment accuracy has improved, the process proceeds to step S24. If the judgment accuracy has not improved, the process proceeds to step S25. As described above, the accuracy determination unit 173 can evaluate the judgment accuracy of the AI judgment results 114a, 114b based on the number of correct answers for past data 111a including multiple samples whose judgment results are known. If the number of correct answers is greater for the AI judgment result 114b than for the AI judgment result 114a, the judgment accuracy has improved by adding the corresponding judgment item, and in other cases, the judgment accuracy has not improved even if the corresponding judgment item is added.
[0112] (S24) The accuracy determining unit 173 determines the check item added for machine learning as a target to be added to the second extraction rule set 132, and notifies the rule creating unit 170 of the check item. Then, the process proceeds to step S26.
[0113] (S25) The accuracy determination unit 173 determines whether the accuracy of both AI judgment results, i.e., the AI judgment results 114a and 114b, is equivalent. If they are equivalent, the process proceeds to step S27. If they are not equivalent, the process proceeds to step S26. Being equivalent means that the number of correct answers of the judgment results described in step S23 is the same.
[0114] (S26) The addition item selection unit 174 judges whether or not there are any candidate judgment items to be added remaining in the first extraction rule group 131. If there are any candidate judgment items to be added remaining, the process proceeds to step S20. If there are no candidate judgment items to be added remaining, the process proceeds to step S27. Note that the next time step S21 is executed, the current judgment model 112c will be treated as the next judgment model 112b. Also, the next time step S22 is executed, the result of adding a new judgment item to the current judgment model 112c will be treated as the next judgment model 112c.
[0115] (S27) The rule creation unit 170 acquires the second extraction rule set 132 from the final learning result by the learning unit 115. That is, the rule creation unit 170 generates the second extraction rule set 132 based on the information of the judgment grounds 113b output together with the judgment model 112c finally created by the learning unit 115. At this time, the rule creation unit 170 generates extraction rule information on the additional judgment items notified by the accuracy determination unit 173, in addition to the extraction rule information on the judgment items included in the original rule set 121. Then, the process of acquiring the second extraction rule set ends.
[0116] In the above procedure, the rule creation unit 170 generates extraction rule information in the second extraction rule set 132 from the final learning result, but it may also generate extraction rule information for the corresponding judgment item each time a notification is received in step S24.
[0117] FIG. 16 is a flowchart showing an example of rule synthesis. The rule synthesis corresponds to step S4. (S30) The synthesis unit 180 selects a check item in the second extracted rule group 132, and searches for the corresponding check item in the original rule group 121 (basic workflow 200).
[0118] (S31) The synthesis unit 180 determines whether or not the original rule group 121 contains a judgment item that is the same as the judgment item selected from the second extraction rule group 132 as a result of the search in step S30. If the original rule group 121 contains a judgment item that is the same as the judgment item selected from the second extraction rule group 132, the process proceeds to step S32. If the original rule group 121 does not contain a judgment item that is the same as the judgment item selected from the second extraction rule group 132, the process proceeds to step S33.
[0119] (S32) The synthesis unit 180 replaces the conditional judgment of the corresponding step in the original rule group 121 with the conditional judgment of the judgment item selected from the second extracted rule group 132. As a result, as illustrated in Figures 9 and 10, the conditional judgment of the corresponding step in the original rule group 121 is changed to the conditional judgment extracted based on machine learning, and revised rule information reflecting the change is generated. Then, the process proceeds to step S36.
[0120] (S33) Synthesis unit 180 determines whether or not the related judgment item registered in the extraction rule information of the judgment item selected in step S30 is included in original rule group 121. If the related judgment item is included in original rule group 121, processing proceeds to step S34. If the related judgment item is not included in original rule group 121, processing proceeds to step S35.
[0121] (S34) The synthesis unit 180 adds a conditional decision for the decision item selected in step S30 to the step of the relevant related decision item in the original rule group 121. As a result, as illustrated in Figures 11 and 12, a new conditional decision extracted based on machine learning is added to the conditional decision of the relevant step in the original rule group 121, and revised rule information reflecting the addition is generated. Then, the process proceeds to step S36.
[0122] (S35) The synthesis unit 180 adds a conditional judgment for the judgment item selected in step S30 to a predetermined position of the original rule group 121. Here, the predetermined position may be immediately before the first step or immediately after the last step of the basic workflow 200 represented by the original rule group 121. As a result, revised rule information reflecting the addition of the conditional judgment is generated. Then, the process proceeds to step S36.
[0123] (S36) The synthesis unit 180 determines whether or not all the decision items in the second extraction rule group 132 have been processed. If all the decision items in the second extraction rule group 132 have been processed, the rule synthesis process ends. If all the decision items in the second extraction rule group 132 have not been processed, the process proceeds to step S30.
[0124] In this way, the information processing device 100 generates, for example, the amended rule set 141 or the amended rule set 142 from the original rule set 121. In other words, the information processing device 100 generates, for example, the changed workflow 300 or the changed workflow 300a from the basic workflow 200. This allows the information processing device 100 to refine a series of procedures included in the basic workflow 200. The information processing device 100 can improve the accuracy of the judgment result by performing a judgment process using the refined changed workflow 300 or the changed workflow 300a. Furthermore, the information processing device 100 can support the correction and improvement of the basic workflow 200.
[0125] The revised rule groups 141, 142 are rule-based information representing the changed workflows 300, 300a, similar to the original rule information in the original rule group 121. The rule execution unit 190 executes a series of procedures represented by the rule base based on the revised rule groups 141, 142, thereby obtaining a judgment result based on the procedures represented by the changed workflows 300, 300a. On the other hand, in a model-based judgment using a judgment model, the judgment process becomes a black box, making it difficult to explain the reason why an AI judgment result was obtained. In contrast, according to the information processing device 100, a judgment result can be obtained by a procedure represented by the rule base, so that the user can more easily understand the reason for the judgment result than when an AI judgment is made based on a model base.
[0126] Here, when registering similar data, the information processing device 100 executes the following procedure. For example, the similarity data may be registered in advance in the similar data storage unit 175a by the registration unit 175b of the information determination unit 175 as follows.
[0127] FIG. 17 is a flowchart showing an example of similar data registration. (S40) The registration unit 175b analyzes past extraction results of the judgment items. The past extraction results are, for example, information representing past judgment results, such as judgments of the business status of many companies, and the values of many judgment items when the past judgment results were obtained. For example, the registration unit 175b may analyze whether or not each of the many judgment items has an influence on the past judgment results, based on big data that has a larger amount of information than the past data 111a. The analysis in step S40 may be performed by a user.
[0128] (S41) The registration unit 175b determines whether or not each check item can explain the reason for the past check result based on the analysis result of step S40. If the check item can explain the reason for the past check result, the process for the check item is terminated without registering the check item as similar data. If the check item cannot explain the reason for the past check result, the process proceeds to step S42.
[0129] For example, when it is estimated that the corresponding check item has no effect on past judgment results, the registration unit 175b determines that the check item cannot explain the reason for the past judgment results. Also, when it is estimated that the corresponding check item has some effect on past judgment results, the registration unit 175b determines that the check item can explain the reason for the past judgment results. Note that step S41 may be executed by the registration unit 175b receiving the result of the judgment by the user.
[0130] (S42) The registration unit 175b stores the check item that is determined to be able to explain the reason for the past check result as similar data in the similar data storage unit 175a. Then, the process for that check item ends.
[0131] In this way, the information judgment unit 175 acquires judgment items that are clearly unrelated to the target judgment content, such as judging the business condition of a company, in advance as similar data, and removes judgment items included in the similar data from the first extraction rule set 131. This prevents the additional item selection unit 174 from selecting unnecessary judgment items, reduces the amount of processing for creating the second extraction rule set 132, and further improves the accuracy of creating the second extraction rule set 132.
[0132] An example of the judgment process handled by the information processing device 100 is the audit of the business status of a corporation such as a company, as described above. In such an audit, even if there are audit guidelines describing the judgment steps, they may not describe all items that serve as the criteria for judgment, or the reference values that serve as the criteria for judgment may be industry-standard values. For this reason, the audit guidelines alone may not be sufficient to properly judge whether the business status is good or bad. Therefore, the information processing device 100 can refine the judgment steps in the audit and improve the accuracy of judgment.
[0133] Another example is judging the business status of an investment target or a business partner. In such a judgment, documents such as financial statements may be checked, and even if the documents include general judgment items, the values for the judgment items may be general numbers, and no specific values may be presented. Therefore, the information processing device 100 can refine the judgment steps in the audit and improve the judgment accuracy.
[0134] In this way, the tasks and decisions to be made in a certain decision process are created in a guideline (document), and when the contents of the document are written down in the form of a workflow, the workflow can be modified by the information processing device 100. As described above, the workflow is described in rule-based information, and the contents of the decisions are easy for users to understand, so it is also possible to refine the guidelines from the workflow.
[0135] In addition, after the guidelines are put into the form of a workflow, the workflow is made more precise, thereby improving the accuracy of the judgment results of the information processing device 100 based on the workflow. As a result, the accuracy of the judgment results of an audit or the like and the accuracy of extracting judgment factors can be improved.
[0136] Furthermore, when basic rules or regulations need to be changed, it may be difficult to know which of the many types of parameters contained in the past data has the greatest impact on the decision-making process. Also, it is necessary for someone who is familiar with the parameters of the decision-making process to consider the degree of impact. For this reason, when changing the decision-making flow based on past data, it may be difficult to appropriately select the parameters that will have the greatest impact on improvement.
[0137] In response to this, the information processing device 100 can support the user in making changes when basic rules or regulations need to be changed. The information processing device 100 can support appropriate changes to basic rules by using machine learning to discover and extract new rules related to parameters that have a large impact on the decision process.
[0138] As described above, the information processing device 100 executes, for example, the following process. The original rule storage unit 120 stores information on a judgment flow in which a judgment result is obtained by a plurality of first judgment steps, each of which indicates a judgment condition for a parameter. The rule creation unit 170 acquires a plurality of second judgment steps associated with a judgment model generated by machine learning based on machine learning using teacher data representing judgment results for a plurality of parameters. The rule creation unit 170 identifies parameters associated with each of the plurality of second judgment steps, and extracts a set of parameters from the plurality of parameters according to the influence of the parameters on the judgment model. The synthesis unit 180 changes a part of the plurality of first judgment steps included in the judgment flow based on the extracted set of parameters.
[0139] This allows the information processing device 100 to reflect conditions related to parameters that affect the judgment process in the judgment flow. Furthermore, the information processing device 100 can appropriately improve the judgment flow. The judgment items are an example of parameters. The basic workflow 200 is an example of a judgment flow. The past data 111a, 111b, 111c are an example of teacher data. Steps 1, 2, ... represented by the original rule group 121 are an example of a plurality of first judgment steps. Steps A, B, ... in the first extraction rule group 131 and the second extraction rule group 132 are an example of a plurality of second judgment steps.
[0140] The influence of a parameter on a judgment model is judged, for example, by whether the accuracy of the judgment result in the judgment model with the parameter is improved compared to the judgment result in the judgment model without the parameter when the judgment result is obtained using a known input. If the accuracy of the judgment result in the judgment model with the parameter is improved compared to the case without the parameter, the parameter has an influence on the judgment model. On the other hand, if the accuracy of the judgment result in the judgment model with the parameter is not improved compared to the case without the parameter, the parameter has no influence on the judgment model.
[0141] Furthermore, the similar data storage unit 175a stores information indicating a specific parameter. The information determination unit 175 determines not to use the specific parameter from among the parameter group included in the teacher data for machine learning.
[0142] This allows the information processing device 100 to reduce the amount of processing required to create the second extraction rule set 132. Furthermore, by registering parameters that are obviously unnecessary for the judgment process as specific parameters, it is possible to improve the accuracy of creating the second extraction rule set 132. As a result, the information processing device 100 can increase the accuracy of improving the judgment flow.
[0143] The set of parameters extracted by rule creating unit 170 includes a first parameter included in the decision flow and a second parameter not included in the decision flow. Combining unit 180 identifies a first decision step corresponding to the first parameter from the multiple first decision steps, and adds a decision condition related to the second parameter to the identified first decision step.
[0144] This allows the information processing device 100 to refine the decision conditions included in the decision flow based on the machine learning result, thereby improving the accuracy of the decision result based on the decision flow.
[0145] Furthermore, the synthesis unit 180 extracts a new judgment condition for the third parameter included in the judgment flow from the second judgment steps. The synthesis unit 180 replaces the judgment condition for the third parameter included in the judgment flow with the extracted new judgment condition.
[0146] This allows the information processing device 100 to refine the decision conditions included in the decision flow based on the machine learning result, thereby improving the accuracy of the decision result based on the decision flow.
[0147] Furthermore, the synthesis unit 180 extracts a decision condition for a fourth parameter that is not included in the decision flow from the second decision steps. The synthesis unit 180 adds a decision step indicating a decision condition for the fourth parameter to the decision flow.
[0148] This allows the information processing device 100 to refine the decision conditions included in the decision flow based on the machine learning result, thereby improving the accuracy of the decision result based on the decision flow.
[0149] Furthermore, the synthesis unit 180 generates rule information indicating each judgment step included in a judgment flow after a change in part of the first judgment steps, the rule information describing judgment conditions for parameters used in the judgment flow after the change. The rule execution unit 190 executes the judgment flow after the change based on the rule information, thereby obtaining a new judgment result for new input data.
[0150] In this way, by executing a changed decision flow based on rule-based information in which decision conditions for parameters are described and obtaining a decision result, the user can more easily understand why the decision result is obtained than by obtaining an AI decision result based on a model. Here, the changed workflows 300 and 300a are an example of a changed decision flow. The amended rule groups 141 and 142 or the amended rule information contained in the amended rule groups 141 and 142 are an example of rule information indicating a changed decision flow. The new data 151 is an example of new input data. The decision result 161 is an example of a new decision result.
[0151] The information processing in the first embodiment can be realized by causing the processing unit 12 to execute a program. The information processing in the second embodiment can be realized by causing the CPU 101 to execute a program. The program is recorded in a computer-readable recording medium 53.
[0152] For example, the program is distributed by distributing recording medium 53 on which the program is recorded. The program may also be stored in another computer and distributed via a network. The computer may store (install) the program recorded in recording medium 53 or a program received from another computer in a storage device such as RAM 102 or HDD 103, and read and execute the program from the storage device.
[0153] Note that parameters are not limited to words that appear in the workflow or those included in the original rule group. For example, the data to be learned may contain data other than character information, such as images. If the image contains a face image facing right and the workflow contains the phrase "facing right", the image may be preprocessed to extract the information "facing right" and then used to handle the situation. There may also be cases where the workflow does not contain information about "facing right". In that case, the relevant part is identified by searching for content that matches the word "right". If there is no corresponding parameter, a new parameter may be added to any position in the workflow, or the parameter may be presented to the user as a candidate for a new parameter to be added to the workflow, and the user may select and add the parameter. [Explanation of symbols]
[0154] 10. Information processing device 11 Storage section 12 Processing section 20 Decision Flow 30 Decision Step Information 40 Post-change decision flow 21, 22, 41, 42 Decision Steps
Claims
1. a storage unit that stores information on a decision flow for obtaining a decision result by a plurality of first decision steps indicating decision conditions for parameters; a processing unit that acquires, based on machine learning using teacher data representing the judgment results for a plurality of parameters, a plurality of second judgment steps associated with a judgment model generated by the machine learning, identifies the parameters associated with each of the plurality of second judgment steps, extracts a set of the parameters from the plurality of parameters according to the influence of the parameters on the judgment model, and modifies a portion of the plurality of first judgment steps included in the judgment flow based on the extracted set of parameters; An information processing device having the above configuration.
2. The storage unit stores information indicating a specific parameter, The processing unit determines not to use the specific parameter from among a group of parameters included in the training data in the machine learning.
2. The information processing device according to claim 1.
3. the set of parameters includes a first parameter that is included in the decision flow and a second parameter that is not included in the decision flow; The processing unit includes: identifying a first determination step corresponding to the first parameter from among the plurality of first determination steps; adding a judgment condition regarding the second parameter to the identified first judgment step; 2. The information processing device according to claim 1.
4. The processing unit includes: extracting a new judgment condition for a third parameter included in the judgment flow from the plurality of second judgment steps; replacing a judgment condition for the third parameter included in the judgment flow with the extracted new judgment condition; 2. The information processing device according to claim 1.
5. The processing unit includes: extracting a judgment condition for a fourth parameter not included in the judgment flow from the plurality of second judgment steps; adding a judgment step indicating a judgment condition for the fourth parameter to the judgment flow; 2. The information processing device according to claim 1.
6. The processing unit includes: generating rule information indicating each judgment step included in the judgment flow after a part of the first judgment steps is changed, the rule information describing judgment conditions for the parameters used in the judgment flow after the change; and executing the changed judgment flow based on the rule information to obtain a new judgment result for new input data.
2. The information processing device according to claim 1.
7. The computer Acquire information on a decision flow for obtaining a decision result by a plurality of first decision steps indicating decision conditions for parameters; obtaining a plurality of second judgment steps associated with a judgment model generated by machine learning based on machine learning using teacher data representing the judgment results for a plurality of parameters; identifying the parameters associated with each of the plurality of second decision steps; extracting the set of parameters from the plurality of parameters according to an influence of the parameters on the judgment model; changing a part of the plurality of first judgment steps included in the judgment flow based on the extracted set of parameters; Information processing methods.
8. On the computer, Acquire information on a decision flow for obtaining a decision result by a plurality of first decision steps indicating decision conditions for parameters; obtaining a plurality of second judgment steps associated with a judgment model generated by machine learning based on machine learning using teacher data representing the judgment results for a plurality of parameters; identifying the parameters associated with each of the plurality of second decision steps; extracting the set of parameters from the plurality of parameters according to an influence of the parameters on the judgment model; changing a part of the plurality of first judgment steps included in the judgment flow based on the extracted set of parameters; A program that executes a process.
Citation Information
Patent Citations
Construction method of case database, learning method of distinguishing device, data distinguishing support device, data distinguishing support program
JP2005352997A
Business execution model optimization device, business execution model optimization method and business execution model optimization program
JP2011081620A
Workflow generation server and method
JP2014048673A
Machine learning device, machine learning method, and machine learning program
JP2016192007A
Business flow specification regeneration method
JP2017045080A
