Plan evaluation device and plan evaluation method

Abstract

To extract, from a schedule, a component having a large direct or indirect influence degree to a KPI to which partial evaluation is unable to be applied, in an evaluation of the schedule.SOLUTION: A plan evaluation apparatus 1000 which evaluates a schedule 1014 planned by combining a plurality of plans, includes: a feature amount conversion unit 1041 that divides the schedule 1014 into plan components based on a prescribed conversion rule, and converts each of the divided plan components into a feature amount; a model learning unit 1042 that uses the feature amount as an input and creates a machine learning model 1021 having a KPI (Key Performance Indicator) of the schedule 1014 as an objective variable; a contribution rate calculation unit 1043 that calculates a contribution rate of each feature amount with respect to the machine learning model 1021; and an influence degree calculation unit 1044 that calculates an influence degree of the plan component representing a conversion source of the feature amount to influence the KPI of the schedule 1014, based on the contribution rate of the feature amount.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a plan evaluation device and a plan evaluation method, and is suitable for application to a plan evaluation device and a plan evaluation method that support planning, evaluate, and extract information about a plan. [Background technology]

[0002] When creating business plans such as product production plans and employee shift schedules, it is necessary to create plans that comply with various constraints on resources such as time, space, equipment, and people, while also taking into account key performance indicators (KPIs) for the plan, such as maximizing production volume, maximizing equipment operating rate, and minimizing the number of workers. Traditionally, employees (craftsmen) with advanced know-how created these plans, but due to a lack of successors and the need to improve business efficiency, there are an increasing number of cases where computer-based plan creation devices are created to automatically create plans while taking into account the above conditions.

[0003] On the other hand, there are a great number of constraints and KPIs in planning, which affect each other in a complex way. As a result, the development of a planning device can require a lot of man-hours, and there is a need to reduce this man-hours. One of the reasons for the increase in man-hours is the time required to evaluate the planned plan, find problems, and consider corrections. There are two evaluation methods: a quantitative method based on the numerical value of KPIs and the degree of violation of constraints, and a method of visually checking the plan to find unintended bugs and conditions that are not fully formulated compared to a craftsman's plan. However, in some cases, the plan can be very large, with more than several thousand items, making it difficult to check it visually.

[0004] With regard to the evaluation support technology for planning tables such as those described above, Patent Document 1 describes a method in which the overall evaluation value of a planning table is calculated by superimposing multiple partial evaluation components, and the partial evaluation component that contributes most to worsening the overall evaluation value is extracted. If this partial evaluation component is calculated by superimposing even smaller partial evaluation components, the same operation is repeated to extract the partial evaluation component that contributes most in a similar manner, thereby extracting the component of the planning table that contributes most to the overall evaluation value in the most unfavorable direction.

[0005] Furthermore, Non-Patent Document 1 describes a method of expressing combinations of decision variables in each plan, such as whether or not to produce a certain product, using dummy variables of (0,1) as a basic technology for numerically evaluating planning tables.

[0006] Furthermore, Non-Patent Document 2 describes a method in the field of AI models such as machine learning, for calculating the contribution rate of each feature, which is a simple numeric sequence, to an AI predicted value, using a set of multiple perturbation data generated by varying data to be evaluated and a predicted value obtained by inputting each perturbation data into an AI. [Prior art documents] [Patent documents]

[0007] [Patent Document 1] JP 2019-209796 A [Non-patent literature]

[0008] [Non-Patent Document 1] Christian D. Hubbs, Can Li, Nikolaos V. Sahinidis, Ignacio E.Grossmann, John M. Wassick, “A deep reinforcement learning approach for chemical production scheduling,” Computers and Chemical Engineering, 106982,vol.141, (2020). [Non-Patent Document 2] Lundberg, Scott M., and Su-In Lee, "A unified approach to interpreting model predictions," Advances in Neural Information Processing Systems, pp. 4765-4774, (2017). Summary of the Invention [Problem to be solved by the invention]

[0009] However, the technology described in Patent Document 1 has a problem that only KPIs that can be partially evaluated can be analyzed. Partial evaluation means that the planning table is divided and then evaluated individually. In addition, the evaluator needs to specify the planning table division rule for partial evaluation, which requires additional labor. Furthermore, the final planning component is a component that directly worsens the value of the KPI, but since the components of the planning table have a complex influence on each other, there are also components that indirectly affect the KPI by affecting other planning components even if they do not directly change the KPI. However, the technology in Patent Document 1 has a problem that it is not possible to extract components that do not directly change the KPI.

[0010] Furthermore, when describing a schedule using the technology described in Non-Patent Document 1, dummy variables must be created for all possible combinations, which results in the schedule becoming a high-dimensional, sparse data structure, making it very difficult to handle.

[0011] In addition, the technology described in Non-Patent Document 2 can calculate the contribution rate to prediction of a feature value that is a simple numeric string, but has the problem that it cannot be applied to evaluation of a schedule as it is.

[0012] The present invention has been made in consideration of the above points, and aims to propose a plan evaluation device and plan evaluation method that can extract components from a plan table that have a large direct or indirect influence even on KPIs that cannot be partially evaluated, by extending the method for calculating the influence of features in a machine learning model to the evaluation of a plan table. [Means for solving the problem]

[0013] In order to solve such problems, the present invention provides a plan evaluation device that evaluates a planning table that is formulated by combining a plurality of plans, the plan evaluation device including: a feature conversion unit that divides the planning table into planning components based on a predetermined conversion rule and converts each divided planning component into a feature; a model learning unit that uses the feature as an input and creates a machine learning model with a KPI (Key Performance Indicator) of the planning table as an objective variable; a contribution rate calculation unit that calculates a contribution rate of each of the feature amounts to the machine learning model; and an influence calculation unit that calculates, based on the contribution rates of the feature amounts, the influence of the planning components from which the feature amounts were converted on the KPI of the planning table.

[0014] In addition, in order to solve such problems, the present invention provides a plan evaluation method by a plan evaluation device that evaluates a plan table that is formulated by combining a plurality of plans, the plan evaluation method comprising: a feature conversion step in which the plan evaluation device divides the plan table into plan components based on a predetermined conversion rule and converts each divided plan component into a feature; a model learning step in which the plan evaluation device uses the feature converted in the feature conversion step as an input and creates a machine learning model with a KPI (Key Performance Indicator) of the plan table as an objective variable; a contribution rate calculation step in which the plan evaluation device calculates a contribution rate of each of the feature amounts to the machine learning model created in the model learning step; and an influence calculation step in which the plan evaluation device calculates a degree of influence of the plan components from which the feature amounts were converted on the KPI of the plan table based on the contribution rates of the feature amounts calculated in the contribution rate calculation step. Effect of the Invention

[0015] According to the present invention, in evaluating a planning table, it is possible to extract from the planning table components that have a large direct or indirect influence on KPIs that cannot be partially evaluated. [Brief description of the drawings]

[0016] [Figure 1] 1 is a block diagram showing an example of the configuration of a plan evaluation device 1000 according to a first embodiment of the present invention. [Diagram 2] FIG. 13 is a diagram showing an example of an evaluation index list 1013. [Diagram 3] FIG. 10 is a diagram showing an example of a schedule table 1014. [Figure 4] FIG. 10 is a diagram showing an example of KPI data 1015. [Diagram 5] 13 is a flowchart illustrating an example of a processing procedure for an influence degree calculation process. [Figure 6] FIG. 10 is a diagram showing an example of a feature conversion rule 1022. [Figure 7] 13 is a flowchart illustrating an example of a processing procedure for feature amount conversion processing. [Figure 8] FIG. 13 is a diagram for explaining an image of compression conversion in feature conversion processing. [Figure 9] FIG. 10 is a diagram showing an example of a planning feature table 1023. [Figure 10] FIG. 10 is a diagram showing an example of a contribution rate calculation result 1024. [Figure 11] 13 is a flowchart illustrating an example of a processing procedure for impact level information aggregation processing. [Figure 12] FIG. 10 is a diagram showing an example of an influence degree calculation result 1025. [Figure 13] FIG. 13 is a diagram for explaining an image of a process of distributing and integrating contribution rates. [Figure 14] FIG. 10 is a diagram showing an example of an output screen based on an influence degree calculation result 1025. [Figure 15] 13 is a flowchart illustrating an example of a processing procedure for an influence degree calculation process using an integrated model. [Figure 16] 11 is a diagram for explaining an image of a process for calculating an influence degree in the influence degree calculation process using an integrated model. FIG. [Figure 17] FIG. 17 is a block diagram showing an example of the configuration of a plan evaluation device 1700 according to a second embodiment of the present invention. [Figure 18] 13 is a flowchart showing an example of a processing procedure of a first indirect influence relationship extraction process. [Figure 19] FIG. 13 is a diagram for explaining an image of a process for searching for an indirect influence relationship. [Figure 20] FIG. 10 is a diagram showing an example of an influence degree calculation result 1025 including an indirect influence relationship. [Figure 21] 13 is a flowchart showing an example of a processing procedure for a second indirect influence relationship extraction process. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0017] Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings.

[0018] Note that the following mainly describes the evaluation of staffing plans that assign requests to appropriate employees, but the plan evaluation method of the present invention can be widely applied to systems that require the creation / modification of plans by combining a variety of evaluation perspectives in a complex manner, such as operation plans for transportation such as airplanes, buses, and trains, and product manufacturing plans in factories.

[0019] In addition, the embodiments described below are merely examples for realizing the present invention, and do not limit the technical scope of the present invention. It will be easily understood by those skilled in the art that the specific configurations can be changed without departing from the concept or spirit of the present invention.

[0020] In the configurations of the invention described below, the same or similar configurations or functions are given the same reference numerals, and duplicated descriptions are omitted. In addition, the configurations, functions, processing units, processing means, etc. described in each embodiment may be realized in hardware, partly or in whole, by designing them as integrated circuits, for example. In addition, the above configurations or functions, etc. may be realized in software by a processor interpreting and executing a program that realizes each function. Information such as programs, tables, files, etc. that realize each function can be stored in a memory, a recording device such as a hard disk or SSD (Solid State Drive), or a recording medium such as an IC card, SD card, or DVD.

[0021] In addition, the position, size, shape, range, etc. of each component shown in the drawings may not represent the actual position, size, shape, range, etc., in order to facilitate understanding of the invention. In addition, the control lines and information lines in each drawing are those that are considered necessary for explanation, and do not necessarily show all control lines and information lines in the product. In reality, it may be considered that almost all components are connected to each other. Therefore, the present invention is not limited to the position, size, shape, range, etc. disclosed in the drawings.

[0022] (1) First embodiment 1 is a block diagram showing an example of the configuration of a plan evaluation device 1000 according to a first embodiment of the present invention. As shown in FIG. 1, the plan evaluation device 1000 includes a storage device 1001, a processing device 1002, an input device 1003, and an output device 1004.

[0023] The storage device 1001 is a general-purpose device that permanently stores data, such as an HDD (Hard Disk Drive) or an SSD (Solid State Drive), and has plan information 1010 and impact-related information 1020. The storage device 1001 may be configured on the same terminal as the other devices that configure the plan evaluation device 1000, or may exist not on the same terminal as the other devices but on a cloud or an external server, and data can be referenced via a network.

[0024] The plan information 1010 includes business-related data 1011, a constraint list 1012, and an evaluation index list 1013 (see FIG. 2 for details), which are inputs for generating a schedule table that is drawn up by combining a plurality of plans, a schedule table 1014 that is the evaluation target (see FIG. 3 for details), and KPI data 1015 that is the evaluation result (see FIG. 5 for details). Note that in this text, the evaluation index and KPI (Key Performance Indicator) are synonymous.

[0025] The business related data 1011 is, for example, employee skill information, request assignment available time information, assignment request date and start / end time information, skill condition information, and other master information.

[0026] The constraint list 1012 is data that stores constraint conditions that the schedule 1014 must adhere to, such as whether a request is assigned to time that an employee cannot assign, whether overtime hours exceed a specified value, etc., and is used for generating the schedule.

[0027] The impact-related information 1020 includes a machine learning model 1021 that learns the impact relationship between the planning table 1014 and KPIs, feature conversion rules 1022 (see Figure 6 for details) that hold rules for converting the components of the planning table 1014 in order to input the planning table 1014 into a model (machine learning model 1021), a planning feature table 1023 (see Figure 9 for details) that holds features obtained by converting the planning table 1014, a contribution rate calculation result 1024 (see Figure 10 for details) that holds the calculation result of the contribution rate of the feature to the machine learning model 1021, and an impact calculation result 1025 (see Figure 12 for details) that holds the calculation result of the impact.

[0028] The processing device 1002 is, for example, a general-purpose computer including a CPU (Central Processing Unit), a memory, etc. The processing device 1002 includes therein a planning related processing unit 1030, an impact evaluation processing unit 1040, a screen output unit 1050, and a data input unit 1060, which are stored in the memory as software programs.

[0029] The planning related processing unit 1030 has a plan generation unit 1031 which inputs data from the storage device 1001 and outputs a plan table 1014, and an evaluation index calculation unit 1032 which evaluates the obtained plan table 1014 and outputs a KPI value.

[0030] The impact evaluation processing unit 1040 includes a feature conversion unit 1041 that obtains a planning feature table 1023 from the planning table 1014 automatically or based on a feature conversion rule 1022, a model learning unit 1042 that generates a machine learning model 1021, a contribution rate calculation unit 1043 that calculates the contribution rate of the feature in the model (machine learning model 1021), and an impact calculation unit 1044 that extracts the influenced part of the planning table 1014 on the KPI.

[0031] The screen output unit 1050 has a function of outputting information for displaying a predetermined screen on the output device 1004. Specifically, for example, the screen output unit 1050 generates information for screen display based on the impact calculation result 1025 calculated by the impact calculation unit 1044, and transmits the information to the output device 1004. As a result, the output device 1004 displays an output screen based on the impact calculation result 1025 (see FIG. 14).

[0032] The data input unit 1060 has a function of inputting processing execution instructions and data in response to input operations by a developer (hereinafter, user) on the input device 1003, and is used when the user makes changes to the schedule or when setting parameters. Specifically, for example, when a user-specific feature transformation rule is specified, the data input unit 1060 receives the feature transformation rule from the input device 1003 operated by the user and stores it in the storage device 1001 as the feature transformation rule 1022.

[0033] The input device 1003 is a general-purpose input device for a computer, such as a mouse, a keyboard, or a touch panel.

[0034] The output device 1004 is a device such as a display, and displays an output screen showing the evaluation result (e.g., the impact degree calculation result 1025) by the processing device 1002 via the screen output unit 1050. Note that if there is no need for a human to check the evaluation result by the processing device 1002 (for example, if the evaluation result is directly passed to a system that automatically creates a plan), the output device 1004 does not need to be provided in the plan evaluation device 1000.

[0035] 2 is a diagram showing an example of an evaluation index list 1013. The evaluation index list 1013 is a database having index numbers 21 in which numbers assigned to each index are stored, index names 22 in which the names of the evaluation indexes related to the index numbers are stored, index definitions 23 in which the definitions of the evaluation indexes related to the index numbers are stored, and calculation processes 24 for the values ​​of the evaluation indexes related to the index numbers. The evaluation index list 1013 is, for example, information input by a user or acquired from a specified information processing device.

[0036] Fig. 3 is a diagram showing an example of a schedule 1014. The schedule 1014 shown in Fig. 3 is a schedule to be evaluated (target schedule), and more specifically, is data to be used for calculating the impact in the impact calculation process described later with reference to Fig. 5.

[0037] The planning table 1014 is configured as matrix data (table data) and is divided into fixed planning master information 31 common to all planning tables and target items 32 specific to each planning table. The types of planning tables 1014 will be described later.

[0038] The plan master information 31 stores basic information of the plan table such as the date, the start time and end time of the request, and the request number. Data of each item of the plan master information 31 is output from the business related data 1011 and stored in the plan table 1014.

[0039] The target items 32 store information specific to the plan table. The combination of the target items 32 is determined by the plan generating unit 1031 so as to optimize the value of the evaluation index list 1013 while satisfying the conditions of the constraint list 1012. For example, when the plan evaluation device 1000 evaluates a personnel allocation plan, as shown in the example of Fig. 3, the target items 32 are employee numbers, and the evaluation of the personnel allocation plan becomes a combination problem of determining the allocation of employees to predetermined requests.

[0040] As described above, any problem can be evaluated by the plan evaluation device 1000 as long as it is a plan table 1014 that requires a combination of certain items. Furthermore, a plan component as described in this invention refers to one row in the plan table 1014. Furthermore, "calculating the influence of a plan table on a KPI" refers to "extracting a plan component that has made a large contribution to a change in a KPI in a certain plan table."

[0041] In this embodiment, three types of plan tables 1014 are used: a target plan table, a historical plan table, and a master plan table. All plan tables share the same plan master information 31, and include different or the same target items 32.

[0042] The target schedule is a schedule that is the target for calculating the degree of influence and performing an analysis, and is generated in S501 in FIG. 5, which will be described later.

[0043] A historical plan table is a plan table that has been generated in advance (generated in the past) in a process different from that of the target plan table. The generation process of the historical plan table may be manual work by a craftsman, a plan generation algorithm different from that of the target plan table, or a process utilizing the randomness of an algorithm similar to that of the target plan table, and the generated historical plan table may have the same plan master information 31 as that of the target plan table and target items 32 in the same format as that of the target plan table. A historical plan table is a plan table that is useful for improving the accuracy of calculation of the degree of influence on the target plan table.

[0044] A baseline schedule can be considered a type of historical schedule, and is a schedule that is clearly compared to a target schedule. A baseline schedule can be, for example, a sample schedule created by a craftsman, or a schedule created by a plan generation algorithm before a certain correction work was performed.

[0045] 4 is a diagram showing an example of KPI data 1015. The KPI data 1015 is data that stores the results (KPI values) of evaluating a plan table 1014 based on an evaluation index list 1013. The KPI data 1015 exists for each type of plan table, including a target plan table, a historical plan table, and a master plan table.

[0046] KPI data 1015 is a database having index numbers 41 in which numbers assigned to each index are stored, index names 42 in which the names of evaluation indexes related to index numbers 41 are stored, and evaluation result values ​​43. The numbers stored in index numbers 41 correspond to index numbers 21 in evaluation index list 1013 of FIG. 2. Values ​​43 store KPI values ​​such as numerical values, binary values, and category values. Furthermore, KPI data 1015 corresponds one-to-one to schedules 1014 from which evaluation results are calculated, and the correspondence between schedules 1014 and KPI data 1015 can be distinguished by file names, etc.

[0047] The impact calculation process for calculating the impact of the target schedule will be described below with reference to the drawings as appropriate.

[0048] 5 is a flowchart showing an example of the processing procedure of the influence degree calculation process. In this embodiment, the plan evaluation device 1000 executes the influence degree calculation process to compress and convert the components of the plan table 1014 into the feature amount format, which is a simple numeric string, and creates a machine learning model 1021 that learns the relationship between the obtained feature amount and the KPI value. Furthermore, the plan evaluation device 1000 calculates the contribution rate of each feature amount in the obtained machine learning model 1021 to the output, and distributes and integrates the contribution rate to the components of the target plan table to calculate the influence of the components of the target plan table on the KPI value.

[0049] 5, first, the plan generation unit 1031 of the processing device 1002 reads the business related data 1011 and the constraint list 1012 from the storage device 1001, and generates a target plan table (step S501). The target plan table generated in step S501 is, for example, the plan table 1014 shown in FIG. 3, and is stored in the storage device 1001. Note that a specific method by which the plan generation unit 1031 generates the target plan table in step S501 may be any method that finds a combination solution for the target items 32 while taking into account constraint conditions, such as a combinatorial optimization solver or a machine learning model.

[0050] Next, the evaluation index calculation unit 1032 inputs the evaluation index list 1013 and the target plan table (plan table 1014) output in step S501, calculates the KPI value of the target plan table, and saves the KPI data 1015 indicating the calculation result in the storage device 1001 (step S502). Note that when the plan generation unit 1031 and the evaluation index calculation unit 1032 are configured as the same program or the same architecture, the processes of steps S501 and S502 are executed simultaneously.

[0051] Next, the user selects whether or not to introduce the user's own feature conversion rule 1022 when performing feature conversion to input each planning table to the model learning unit 1042 (step S503). If the introduction of the user's own feature conversion rule 1022 is not selected in step S503, the automatic rule that automatically converts the planning table into features is used in the feature conversion, and the process proceeds to step S505.

[0052] On the other hand, if introduction of the user's own feature transformation rule 1022 is selected in step S503 (YES in step S503), the user operates the input device 1003 to write the feature transformation rule 1022 to be introduced (step S504). Details of the method of writing the feature transformation rule 1022 will be described later with reference to Fig. 6. Then, the data input unit 1060 stores the written feature transformation rule 1022 in the storage device 1001, and the process proceeds to step S505.

[0053] In step S505, the feature conversion unit 1041 inputs the target plan table generated in step S501, the historical plan table obtained in the past, and the KPI values ​​(KPI data 1015) of the target plan table and the historical plan table, and executes feature conversion processing to convert the target plan table into features multiple times using the feature conversion rules 1022 or automatic rules, or a combination of the feature conversion rules 1022 and the automatic rules, selected in steps S503 to S504. A detailed processing procedure of the feature conversion processing will be described later with reference to Fig. 7, but in the feature conversion processing, the feature conversion unit 1041 stores the features obtained by converting the plan table in the storage device 1001 as the plan feature table 1023.

[0054] Next, in step S506, the impact evaluation processing unit 1040 selects one from the multiple planned feature quantity tables 1023 obtained in the feature quantity conversion processing in step S505, and starts a machine learning loop processing.

[0055] In the machine learning loop process, first, in step S507, the model learning unit 1042 inputs the plan feature table 1023 selected in step S506, inputs the KPI data 1015 (which may be read as KPI 93 of the plan feature table 1023) corresponding to the plan feature table 1023 as an output (objective variable), and performs machine learning to create a machine learning model 1021 that has learned the relationship between each plan table and the KPI data 1015. The machine learning model 1021 created in step S507 is stored in the storage device 1001. Note that the machine learning model 1021 created by the model learning unit 1042 is assumed to be based on a decision tree or a neural network, which is supervised learning that executes processing on input data and outputs a predicted value.

[0056] In the next step S508, the contribution rate calculation unit 1043 calculates the contribution rate of each feature based on the machine learning model 1021 created in step S507, the planning feature table 1023 selected in step S506, and the KPI data 1015 corresponding to the planning feature table 1023, and stores information indicating the calculation results (contribution rate calculation result 1024) in the storage device 1001.

[0057] The method of calculating the contribution rate by the contribution rate calculation unit 1043 may be, for example, the method described in Non-Patent Document 2. However, the method is not limited to this method, and any method may be used as long as it can calculate the contribution rate to the prediction of each feature amount in the machine learning model. Here, it is preferable that the initial value used for calculating the contribution rate is a master schedule.

[0058] In addition, if no contradictions arise in the planning tables even if the features are swapped for the planning feature tables 1023 obtained from the target planning table and the base planning table, the contribution rate calculation unit 1043 may directly calculate the contribution rate to the KPI (i.e., the degree of influence) in step S508 using the Shapley value calculation method described in Non-Patent Document 2, without creating the machine learning model 1021 in step S507.

[0059] The above-mentioned machine learning loop process (steps S506 to S508) is repeatedly executed for all of the multiple planning feature tables 1023 obtained in the feature conversion process in step S505 (step S509), and when the process is completed for all planning feature tables 1023, the process proceeds to step S510.

[0060] In step S510, the influence calculation unit 1044 executes an influence information aggregation process that distributes and integrates the contribution rates obtained in step S508 in the above loop process and calculates the influence of the target schedule table components on the KPI value. The detailed processing procedure of the influence information aggregation process will be described later with reference to FIG.

[0061] Next, the impact calculation unit 1044 creates an impact calculation result 1025 by aggregating each impact calculated in the processing of step S510 and the KPI value and target plan table used to calculate each impact, and stores the created impact calculation result 1025 in the storage device 1001 (step S511).

[0062] Finally, the screen output unit 1050 generates information for screen display based on the impact degree calculation result 1025 created in step S511, and transmits it to the output device 1004. As a result, an output screen based on the impact degree calculation result 1025 is displayed on the output device 1004 (step S512). A specific example of the output screen is shown in FIG. 14, which will be described later.

[0063] Fig. 6 is a diagram showing an example of the feature transformation rule 1022. The feature transformation rule 1022 shown in Fig. 6 is a specific example of the feature transformation rule 1022 described in step S504 in Fig. 5.

[0064] When directly converting a planning table into features that can be machine-learned, a method of converting into dummy variables as described in Non-Patent Document 1 is common, but converting into such dummy variables results in a high-dimensional, sparse data structure that is difficult to handle. Therefore, in this embodiment, the planning table is compressed and converted using one or more methods to obtain features. The compression method may utilize some column information of the planning table, or may be determined randomly.

[0065] In the feature conversion rule 1022 illustrated in Fig. 6, a rule number 61 for feature conversion, a planning table column 62 which is column information of the planning table 1014 to be used for compression, and its compression condition 63 are described. In Fig. 6, as an example of a personnel allocation plan, specifically, rule number "1" describes that the column information of "date" in the planning table is compressed "by date", and rule number "2" describes that the column information of "employee number" in the planning table is compressed "by number". Note that in this explanation, the compression condition 63 is described based on natural language for readability, but ultimately, the compression condition 63 is converted into a program or formula format suitable for the feature conversion unit 1041.

[0066] Fig. 7 is a flowchart showing an example of a processing procedure for the feature conversion process. The feature conversion process shown in Fig. 7 corresponds to the process in step S505 in Fig. 5, and is executed by the feature conversion unit 1041.

[0067] According to FIG. 7, the feature conversion unit 1041 inputs the target plan table generated in step S501, the historical plan table obtained in the past, and the KPI values ​​(KPI data 1015) of the target plan table and the historical plan table, and starts the following feature conversion loop processing (step S701).

[0068] In the feature conversion loop process, the feature conversion unit 1041 first determines whether or not to apply the feature conversion rule 1022 to the conversion of the feature based on the user's designation in step S503 in Fig. 5 (step S702). If the feature conversion rule 1022 is to be applied (YES in step S702), the feature conversion unit 1041 reads the feature conversion rule 1022 created in step S504 in Fig. 5, and then proceeds to step S703, selects one rule from the read feature conversion rules 1022, and proceeds to step S705. Note that the rules not read in step S703 are selected sequentially in the feature conversion loop process from the next time onwards.

[0069] On the other hand, if it is not specified that the feature conversion rule 1022 is applied to the feature conversion (NO in step S702), the feature conversion unit 1041 specifies a random division or an arbitrary column of the plan master information 31 in the target plan table, automatically creates a feature conversion rule (automatic rule) for the information in the column (step S704), and proceeds to step S705. In detail, in step S704, the feature conversion unit 1041 automatically specifies a feature conversion rule for the information in the specified column based on each category if the information in the column is categorical, or based on the range of the maximum and minimum values ​​if the information is numerical. When specifying this conversion rule, a basic clustering algorithm or conditional branching can be used.

[0070] As is clear from the evaluation index list 1013 shown in FIG. 2, since the calculation process 24 is usually different for each KPI (index), it is difficult to automatically divide the schedule into a format that can be calculated for each KPI. However, the plan evaluation device 1000 according to the present embodiment does not divide the schedule to calculate the KPI value by numerical calculation, but converts the schedule to input to the machine learning model 1021, so it is sufficient if the schedule is compressed into a simple numeric string as a feature, even if it is not in a format that can be calculated for each KPI. Therefore, there is no problem in dividing the schedule by any method, according to a rule specified by the user, an automatically specified rule, or randomly, and it is also possible to calculate the impact on KPIs that cannot be partially evaluated.

[0071] In step S705, the feature conversion unit 1041 determines whether or not to extract differences in the planning table. The differences in the planning table are the differences between the target planning table and the standard planning table, and by limiting the differences between the target items 32 of the target planning table and the standard planning table and evaluating these differences, comparison and model learning may become easier. For this reason, in this embodiment, the user can specify whether or not to extract differences in the planning table. If differences in the planning table are to be extracted (YES in step S705), the process proceeds to step S707 via step S706, and if differences in the planning table are not to be extracted (NO in step S705), the process proceeds directly to step S707.

[0072] In step S706, the feature conversion unit 1041 extracts only components (in the example of Figure 3, one row corresponds to "component") whose target items 32 differ from those of the basic planning table for historical planning tables other than the target planning table and the base planning table, and saves these as new target planning tables and historical planning tables in the planning table 1014 of the storage device 1001.

[0073] In step S707, the feature conversion unit 1041 divides the plan table (the target plan table and the history plan table) in accordance with the rule obtained in step S703 or step S704.

[0074] Next, the feature conversion unit 1041 performs compression conversion for each of the planning tables divided in step S707, converting the values ​​of the items (target items 32) for each plan into "feature amounts" based on categories or numerical values ​​(step S708). The compression conversion may be performed by converting into a feature amount (category or numerical value) unique to each combination of the planning tables, or by converting into numerical values ​​based on an existing distance measure such as cosine distance. A specific example of the compression conversion will be described later with reference to FIG. 8.

[0075] Next, the feature conversion unit 1041 creates a plan feature table 1023 by combining the feature for each plan compressed and converted in step S708 with the corresponding KPI value (the value 43 of the KPI data 1015), and stores it in the storage device 1001 (step S709). Note that, although a specific example will be described later with reference to Fig. 9, in the plan feature table 1023, linking is performed so that the correspondence between the components of each feature and the target items 32 before conversion in step S708 can be confirmed later.

[0076] Then, the feature transformation unit 1041 repeats the processes of steps S702 to S709 until a predetermined repetition upper limit is reached (step S710). The repetition upper limit may be specified based on the number of feature transformation rules 1022 specified by the user, or the time or number of transformations (loop count) may be explicitly specified in advance. However, when a feature transformation rule 1022 is newly written by the user (see step S504 in FIG. 5), it is preferable to give priority to the execution of feature transformation according to this feature transformation rule 1022.

[0077] As described above, in the feature conversion process, a loop of feature conversion is performed to perform multiple compression conversions, enabling the plan evaluation device 1000 to analyze the plan table from various perspectives and increasing the reliability of the calculated impact degree.

[0078] Fig. 8 is a diagram for explaining an image of compression conversion in the feature conversion process. The image of compression conversion shown in Fig. 8 is a visualization of the process of step S708 in Fig. 7 using a specific example. Specifically, for plan A, which is a base plan table, a base plan table 81 before compression conversion to feature values ​​and base plan tables 84 and 85 after compression conversion to feature values ​​are shown, for plan B, which is a target plan table, a target plan table 82 before compression conversion to feature values ​​and target plan tables 86 and 87 after compression conversion to feature values ​​are shown, and for plan C, which is one of a plurality of existing history plan tables, a history plan table 83 before compression conversion to feature values ​​and history plan tables 88 and 89 after compression conversion to feature values ​​are shown.

[0079] In the example of Figure 8, "by date" and "random" are used as the division rules for the schedule. The compression conversion according to each division rule will be specifically explained below.

[0080] In the compression conversion using the "by date" division rule, after compression conversion into feature quantities, numerical values ​​are assigned to the columns for each day in the baseline plan table 84, the target plan table 86, and the history plan table 88. These numerical values ​​are obtained by compressing and converting the employee numbers in the original plan tables, and specifically, for example, a compression rule is adopted in which combinations that appear in the original baseline plan table 81 for plan A are set to "0," and different combinations that do not appear in the baseline plan table 81 are set to a different numerical value such as "1."

[0081] By adopting the above compression rules, the values ​​of each date in the master schedule table 84 after the compression conversion of plan A are naturally all converted to "0".

[0082] In addition, in the target planning table 86 after the compression conversion of plan B, the employee number for "8 / 1" in the target planning table 82 is a different combination from that in the base planning table 81, so the value of that date is converted to "1," and the employee numbers for "8 / 2" and "8 / 3" in the target planning table 82 are the same combination as in the base planning table 81, so the value of that date is converted to "0."

[0083] In the case of plan C, the employee number of "8 / 2" in the historical plan table 83 is the same as that in the basic plan table 81. J group This is a combination in which the employee numbers of "8 / 1" and "8 / 3" in historical planning table 83 are a different combination from those in base planning table 81. If compression conversion is performed in the same way as for plan B described above, then in historical planning table 88 after compression conversion for plan C, the value of "8 / 2" will be converted to "0" and the values ​​of "8 / 1" and "8 / 3" will be converted to "1".

[0084] However, in the case of Figure 8, as an additional compression rule, in the compression conversion of the history plan table (plan C), if the combination that appears in the original history plan table is different from both the base plan table (plan A) and the target plan table (plan B), it is converted to "2". In this case, since the employee number combination ("0002" and "0004") for "8 / 1" in the history plan table 83 is different from the employee number combination ("0001" and "0002") for "8 / 1" in the base plan table 81 and the employee number combination ("0001" and "0004") for "8 / 1" in the target plan table 82, in the history plan table 88 after the compression conversion of plan C, 1 " has been converted to "2".

[0085] Next, in the compression conversion using the "random" splitting rule, each planning table is split into rows by random combinations of columns, rather than by clear rules such as the values ​​for each column (e.g. date), and a compression rule similar to the "by date" splitting rule is applied to the split items, thereby converting combinations of employee numbers into numerical features.

[0086] Specifically, for example, when "row 1 and row 3" are selected as a column combination in the "random" division rule, the combination of employee numbers in "row 1 and row 3" in the history plan table 83 (plan C) before compression conversion is "0001" and "0003." Therefore, the feature conversion unit 1041 can determine the feature (value) of the relevant item in the history plan table 89 by comparing this combination of employee numbers with similar combinations of employee numbers in the base plan table 81 (plan A) and the target plan table 82 (plan B).

[0087] The above-mentioned compression conversion method is merely an example, and the compression conversion of the feature amount in this embodiment is not limited to this. For example, the above-mentioned compression rule uses the combination in the master schedule 81 (plan A) as the baseline, but the baseline is not limited to the master schedule.

[0088] 9 is a diagram showing an example of the planned feature table 1023. The planned feature table 1023 shown in Fig. 9 is a specific example of the planned feature table 1023 created by the feature conversion unit 1041 in step S709 in Fig. 7 and stored in the storage device 1001, and is configured to include plan numbers 91 on the basis of each feature (information for linking the components of the feature with the target items 32 before compression conversion, described in step S709), features 92 serving as input variables of the machine learning model 1021, and KPIs 93 serving as objective variables of the machine learning model 1021.

[0089] In step S507 in FIG. 5 described above, the planned feature table 1023 configured as above is imported into the machine learning model 1021, and in the next step S508, the contribution rate of the feature to the model (contribution rate 103 in the contribution rate calculation result 1024 in FIG. 10) is calculated.

[0090] Fig. 10 is a diagram showing an example of a contribution rate calculation result 1024. The contribution rate calculation result 1024 shown in Fig. 10 is information indicating a calculation result of a contribution rate of each feature amount to the machine learning model 1021, which is calculated by the contribution rate calculation unit 1043 in step S508 in Fig. 5.

[0091] 10, the contribution rate calculation result 1024 includes a feature value name 101, which is the column name of each feature value, a feature value value 102, and a contribution rate 103 of the feature value to a machine learning model 1021. The contribution rate calculation result 1024 is generated for each of the one or more machine learning models 1021, and after the contribution rate for each planned feature value is obtained in the one or more machine learning models 1021, the influence information aggregation process shown in FIG. 11 (step S510 in FIG. 5) is executed.

[0092] Fig. 11 is a flowchart showing an example of the processing procedure of the impact information aggregation process. The impact information aggregation process shown in Fig. 11 corresponds to the process of step S510 in Fig. 5, and is a process in which the impact calculation unit 1044 distributes and integrates the contribution rate calculated in step S509 to the components of the target planning table to calculate the impact of the components of the target planning table on the KPI value.

[0093] 11, the impact calculation unit 1044 starts the following loop process of integrating contribution rates (step S1101). At this time, since the machine learning loop of the impact calculation process (steps S506 to S509 in FIG. 5) has ended, the contribution rate calculation result 1024 has been calculated for each target schedule.

[0094] In the loop process of contribution rate integration, the influence calculation unit 1044 first extracts one set of the contribution rate calculation result 1024 and the corresponding target schedule table (schedule table 1014) (step S1102).

[0095] Next, the influence calculation unit 1044 starts a loop process of contribution rate distribution to distribute the contribution rate to the planning components for each feature whose contribution rate is indicated in the contribution rate calculation result 1024 extracted in step S1102 (step S1103). The contribution rate is the influence on the prediction for each feature of a simple numeric sequence. However, since the feature is a numerical value obtained by converting each planning component, the influence of the planning component on the KPI cannot be obtained from the contribution rate alone. Therefore, in the present invention, the contribution rate in the machine learning model 1021 is applied to the planning table by executing a loop process of contribution rate distribution to distribute the contribution rate to the planning components.

[0096] In the loop process of contribution rate distribution, the influence calculation unit 1044 selects one feature amount from the contribution rate calculation result 1024 extracted in step S1102 (step S1104). Taking the contribution rate calculation result 1024 in Fig. 10 as an example, the influence calculation unit 1044 extracts one row in step S1104.

[0097] Next, the influence calculation unit 1044 extracts a planning component that is the pre-conversion component of the feature quantity selected in step S1104 from the corresponding planning table extracted in step S1102 (step S1105).

[0098] Next, the influence calculation unit 1044 distributes the row contribution rate 103 extracted from the contribution rate calculation result 1024 in step S1104 to the planning components extracted in step S1105 (step S1106). This distribution method is not particularly limited, but may be, for example, a method of equally allocating the contribution rate to each planning component, or a method of allocating the value (contribution rate) as it is. Also, for example, the distribution of the contribution rate may be determined using an algorithm such as a machine learning model.

[0099] Furthermore, in step S1106, if a contribution rate in another machine learning model 1021 already exists in the planned component, the influence calculation unit 1044 integrates the existing contribution rate with the newly distributed contribution rate. This integration method is not particularly limited, and any method such as simple addition, weighting, or multiplication may be adopted.

[0100] Then, the influence calculation unit 1044 calculates the contribution rate distribution by the number of types of feature amounts shown in the contribution rate calculation result 1024 extracted in step S1102. Distribution The above loop process (steps S1104 to S1106) is repeated (step S1107).

[0101] And the contribution rate Distribution After the loop process is completed, the impact calculation unit 1044 repeats the loop process of integrating the contribution rates (steps S1102 to S1107) for all the contribution rate calculation results 1024 (step S1108), and then ends the impact information counting process.

[0102] Fig. 12 is a diagram showing an example of the impact calculation result 1025. The impact calculation result 1025 shown in Fig. 12 is a specific example of the impact calculation result 1025 created by the impact calculation unit 1044 and stored in the storage device 1001 in step S511 in Fig. 5.

[0103] The impact calculation result 1025 is structured as matrix data (table data), similar to the planning table 1014 shown in FIG. 3, and has fixed planning master information 1201 common to all planning tables, target items 1202 unique to each planning table, and an item for calculated impact 1203.

[0104] By creating the impact calculation result 1025 in the format shown in FIG. 12, the user (developer) can instantly find components that have a large impact on the KPI from among the vast number of components of the planning table by referring to the impact calculation result 1025, and can quickly start considering revision proposals for improving the KPI.

[0105] Furthermore, the influence 1203 shown in the influence calculation result 1025 is an influence obtained as a result of learning the relationship between the plan table 1014 and the KPI value (KPI data 1015) by the machine learning model 1021. Therefore, unlike the technology of Patent Document 1 which can only extract components that directly change the KPI value, the plan evaluation device 1000 of this embodiment can extract not only components that directly affect the KPI value, but also components that may indirectly affect the KPI value.

[0106] Fig. 13 is a diagram for explaining an image of the process of distributing and integrating the contribution rates. The image shown in Fig. 8 is a visualization of the process of step S1106 in Fig. 11 using a specific example.

[0107] FIG. 13 shows an image of creating an impact calculation result 1303 by distributing and integrating contribution rates to each planning component based on contribution rate calculation result 1301 for features generated by compression conversion using the division rule "by date" and contribution rate calculation result 1302 for features generated by compression conversion using the division rule "by number" of employee numbers.

[0108] In the case of FIG. 13, the contribution rate is distributed by "average allocation" and the contribution rate is integrated by "simple addition". For example, when looking at the first row of the planned component of the impact calculation result 1303, the date is "8 / 1", ​​so the contribution rate of the feature value "8 / 1" of the contribution rate calculation result 1301 is "+0.3", and the distribution of this contribution rate, "+0.1", is entered in the contribution rate of "distribution by date". Here, the distributed value of the contribution rate of "+0.3" is "+0.1" because there are three components (first to third rows from the top) with the date "8 / 1" in the impact calculation result 1303, and they are divided into three according to the distribution method of average allocation. Similarly, in the first row of the planned component of the impact calculation result 1303, the contribution rate of employee number "0001" is also distributed, and "+0.1" is entered as the contribution rate of "distribution by employee number". Then, in the first row of the planning component of the impact calculation result 1303, the contribution rate of the date "8 / 1" "+0.1" and the contribution rate of the employee number "0001" "+0.1" are finally integrated (here, simply added together) to calculate the impact rate of the integrated result as "+0.2".

[0109] Fig. 14 is a diagram showing an example of an output screen based on the impact calculation result 1025. An output screen 1400 shown in Fig. 14 is a specific example of an output screen displayed on the output device 1004 in step S512 in Fig. 5, and displays the contents of the impact calculation result 1025 created in step S511 in Fig. 5 in a format that is easy for the user to understand.

[0110] 14 is output in the form of a Gantt chart for personnel allocation planning, and is composed of employee number 1401, date of each request 1402, plan component with small impact 1403, and plan component with large impact 1404. In this example, the values ​​of the plan components entered in plan components 1403 and 1404 refer to the request numbers in personnel allocation planning (for example, the "request number" in plan master information 1201 in FIG. 12).

[0111] In addition, on the output screen 1400, whether each plan component (in this example, the request number) is classified into either the plan component with low influence 1403 or the plan component with high influence 1404 may be determined in any manner based on the influence value in the impact calculation result 1025.For example, request numbers showing an influence level from the top to a predetermined number may be classified into the plan component with high influence 1404, or request numbers showing an influence level exceeding a predetermined threshold may be classified into the plan component with high influence 1404.

[0112] 14, request number "105" and request number "112" are highlighted as plan components 1404 with high influence, allowing the user to immediately identify important parts from output screen 1400. Note that the display method is not limited to this, and any method may be used, such as a method of omitting display of plan components 1403 with low influence, or a method of displaying the magnitude of influence with a color gradation based on the value of the influence in influence calculation result 1025.

[0113] 5 described above, the contribution rate is calculated for each machine learning model 1021, and then the contribution rates are distributed and integrated to calculate the influence for each component (planning component) of the plan table, but the plan evaluation device 1000 according to this embodiment can also calculate the influence for each plan component by first integrating the machine learning models 1021 as a separate processing procedure of the influence calculation process. The influence calculation process using such a processing procedure is referred to as an influence calculation process using an integrated model, and an example of the processing procedure is shown in the following FIG. 15.

[0114] Fig. 15 is a flowchart showing an example of a processing procedure of an influence degree calculation process using an integrated model. Among the processes shown in Fig. 15, detailed explanations of processes similar to the influence degree calculation process shown in Fig. 5 may be omitted.

[0115] In FIG. 15, first, the same processes as steps S501 to S505 in FIG. 5 are performed, and finally, the feature conversion unit 1041 generates the planned feature table 1023 (step S1501).

[0116] Next, the impact evaluation processing unit 1040 selects one of the multiple plan feature tables 1023 obtained in step S1501, and starts a machine learning loop process (step S1502).

[0117] In the machine learning loop processing, first, in a manner similar to step S507 in Figure 5, the model learning unit 1042 inputs the plan feature table 1023 and inputs the KPI data 1015 (which may be read as KPI93 of the plan feature table 1023) corresponding to the plan feature table 1023 as output, and performs machine learning to create a machine learning model 1021 that has learned the relationship between each plan table and the KPI data 1015 (step S1503).

[0118] The machine learning loop processing in FIG. 15 differs from the machine learning loop processing in FIG. 5 in that, instead of performing processing equivalent to step S508 in FIG. 5, the processing in step S1503 is repeatedly performed for all of the multiple planning feature tables 1023 (step S1504). When machine learning models 1021 have been created for all planning feature tables 1023, the process proceeds to step S1505.

[0119] In step S1505, the model learning unit 1042 integrates multiple machine learning models 1021 created in the machine learning loop process. The integrated model is called an integrated model. The integration method of the integrated model in step S1505 may adopt an existing integration method, such as a weighted average of outputs or a stacking method that further stacks machine learning models 1021.

[0120] Next, the contribution rate calculation unit 1043 calculates the contribution rate of each feature amount in the integrated model integrated in step S1505, and stores information indicating the calculation result (contribution rate calculation result 1024) in the storage device 1001 (step S1506). The contribution rate may be calculated using existing technology, as described in step S508 of FIG.

[0121] Next, the influence calculation unit 1044 starts a loop process of the influence calculation for each planning component of the target planning table (step S1507). When focusing on a certain planning component of the target planning table, in the influence calculation process of Fig. 5, one machine learning model 1021 is compressed and converted into one feature amount, but in the influence calculation process of Fig. 15, since multiple machine learning models 1021 are integrated into an integrated model, the one integrated model is converted into multiple feature amounts.

[0122] In the loop process of the influence degree calculation, first, the influence degree calculation unit 1044 selects one plan component to be the target of the influence degree calculation from the target plan table (step S1508).

[0123] Next, the influence calculation unit 1044 selects, from the integrated model, feature amounts whose source contains the planned component selected in step S1508 (step S1509).

[0124] Next, the influence calculation unit 1044 calculates the sum of the contribution rates of the feature quantities selected in step S1509 (step S1510). The sum of the contribution rates calculated in step S1510 is adopted as an approximation of the influence rate of the target planning component. Therefore, the more machine learning models 1021 are generated and integrated from more planning feature tables 1023, the more feature quantities can be compressed and converted for the target planning component, which leads to the calculation of the influence rate of the feature quantities. In other words, the more machine learning models 1021 are generated and integrated from more planning feature tables 1023, the more information that is the basis of the influence rate is generated, which leads to an improvement in the reliability of the influence rate (sum of the contribution rates) calculated in step S1510.

[0125] The above-mentioned loop process of the influence degree calculation (steps S1507 to S1510) is repeatedly executed for all the plan components included in the target plan table (step S1511), and when the influence degree is calculated for all the plan components, the process proceeds to step S1512.

[0126] In step S1512, the impact calculation unit 1044 creates an impact calculation result 1025 by aggregating the impact of each plan component calculated in the impact calculation loop processing and the KPI value and target plan table used in calculating each impact, and stores the created impact calculation result 1025 in the memory device 1001.

[0127] Finally, the screen output unit 1050 generates information for screen display based on the impact degree calculation result 1025 created in step S1512, and transmits it to the output device 1004. As a result, an output screen based on the impact degree calculation result 1025 is displayed on the output device 1004 (step S1513).

[0128] Fig. 16 is a diagram for explaining an image of the process of calculating the influence degree in the influence degree calculation process by the integrated model. The image shown in Fig. 16 is a visualization of the process of step S1510 in Fig. 15 using a concrete example.

[0129] Figure 16 shows a target planning table 1601 as an example of a target planning table for which impact is calculated in the impact calculation process of Figure 15, a contribution rate calculation result 1602 as an example of the contribution rate calculation result 1024 of the integrated model calculated in step S1506 of the same process, and an impact calculation result 1603 as an example of the impact calculation result 1025 created in step S1512 of the same process.

[0130] An example of creating the influence calculation result 1603 in Fig. 16 will be described with reference to specific values. Focusing on the planning component in the first row of the target planning table 1601, it can be seen that the date is "8 / 1" and the employee number is "0001", and therefore, in the contribution rate calculation result 1602, these are converted into feature quantities of "8 / 1" and "0001". The contribution rates of these feature quantities are both "+0.3", and in this example, the result of simply adding up these contribution rates is output as the influence of the focused planning component in the influence calculation result 1603. That is, in the influence calculation result 1603, the influence of the feature quantity of employee number "0001" on date "8 / 1" is calculated as "+0.6".

[0131] As described above, according to the plan evaluation device 1000 of the first embodiment, the method of calculating the influence of features in the machine learning model 1021 is extended to the evaluation of the plan table (evaluation of each plan component), and the evaluation results are created and output, making it possible to extract components from the plan table that have a large direct or indirect influence not only on KPIs that can be partially evaluated, but also on KPIs that cannot be partially evaluated.

[0132] In the above-described first embodiment, a method for interpreting and comparing the evaluation results by the plan evaluation device 1000 has been described in the case where there is one target plan table for which the degree of influence is calculated. However, for example, when the plan generation unit 1031 is an algorithm that performs repeated calculations, there are cases where multiple plans (target plan tables) are created at one time.

[0133] The plan evaluation device 1000 according to this embodiment can be applied even when there are multiple target plan tables as described above. Specifically, the plan evaluation device 1000 determines one common reference plan table and calculates the influence of each target plan table, thereby evaluating and visualizing changes in plan components with high influence. As a result, the user (developer) can obtain insights such as, for example, why a plan part with high influence was obtained in the middle of repeated calculations, but was replaced with another item in the final output.

[0134] In the above description of the present embodiment, the degree of influence on a single KPI was calculated, but the plan evaluation device 1000 can calculate the degree of influence of each plan component on multiple KPIs by repeating this process.Then, by adding up the KPIs using the weights of each KPI determined by any method, the plan evaluation device 1000 can calculate the degree of influence of each plan component on the overall KPI.

[0135] Furthermore, in the plan evaluation device 1000 according to this embodiment, it is also possible to execute an operation in which the plan components of the target plan table are partially replaced and the degree of influence in the plan table after the replacement is calculated by utilizing the data input unit 1060. By carrying out such an operation, it becomes possible to interactively or automatically evaluate the case where the plan table is modified, which can contribute to reducing the number of development steps.

[0136] (2) Second embodiment The second embodiment of the present invention is an extension of the first embodiment. In the above-mentioned first embodiment, the influence of the planning components was calculated based on the machine learning model 1021, and therefore the planning components included both planning components that directly change the KPI and planning components that indirectly change the KPI. That is, in the first embodiment, it was not possible to extract which parts of the planning components are in an indirect influence relationship. Therefore, in the second embodiment, an indirect influence relationship (hereinafter referred to as an indirect influence relationship) between the planning components is further extracted from the influence obtained in the first embodiment.

[0137] Fig. 17 is a block diagram showing an example of the configuration of a plan evaluation device 1700 according to the second embodiment of the present invention. As is clear from a comparison with Fig. 1, the plan evaluation device 1700 according to the second embodiment has a configuration that is an extension of the plan evaluation device 1000 according to the first embodiment. Specifically, as additions to the plan evaluation device 1000, the plan evaluation device 1700 has a plan generation control unit 1701 in the planning relationship processing unit 1030 and an indirect influence relationship calculation unit 1702 in the influence evaluation processing unit 1040. Specific methods of utilizing these added configurations will be described later.

[0138] In the second embodiment, in order to extract indirect influence relationships, a first approach that is effective when there is randomness in plan generation and a second approach that is effective in cases where it is difficult to generate plans multiple times can be adopted.

[0139] First, the process of extracting an indirect influence relationship according to the first approach will be described.

[0140] Fig. 18 is a flowchart showing an example of the processing procedure of the first indirect influence relationship extraction processing. The first indirect influence relationship extraction processing is processing for extracting an indirect influence relationship by using an already-planned history plan table based on the first approach, and is executed by the indirect influence relationship calculation unit 1702 after the end of the influence degree calculation processing of Fig. 5 or Fig. 15 described in the first embodiment.

[0141] According to FIG. 18, first, the indirect influence relationship calculation unit 1702 reads the influence calculation result 1025 of the target plan table calculated in the influence calculation process and the historical plan table obtained by the process of the plan generation unit 1031 (step S1801).

[0142] Next, the indirect influence relation calculation unit 1702 determines one planning component to be focused on from the influence degree calculation result 1025 read in step S1801 (step S1802). The following process is to extract the indirect influence relation to this planning component of interest. Therefore, it is preferable that the planning component selected in step S1802 is a planning component for which detailed analysis is desired, such as one with a large influence degree.

[0143] Next, the indirect influence relationship calculation unit 1702 extracts a history plan table in which the target plan component determined in step S1802 appears from the history plan tables read in step S1801 (step S1803).

[0144] Next, the indirect influence relationship calculation unit 1702 starts a loop process for searching for an indirect influence relationship (step S1804).

[0145] In the loop process of the indirect influence search, the indirect influence relation calculation unit 1702 first sets one "indirect influence candidate" that is a candidate for a planning component that has an indirect influence relationship with the target planning component from the target planning table (step S1805). The method of setting the indirect influence candidate may be any method, but since there is a high possibility that a planning component that is important in the sense of an indirect influence is included in the part with a large influence degree by the machine learning model 1021, it is common to set the planning components other than the target planning component with the largest influence degree as indirect influence candidates and perform the following search.

[0146] Next, the indirect influence relationship calculation unit 1702 extracts a history schedule table in which the indirect influence candidate set in step S1805 appears from the history schedule tables read in step S1801, and judges whether or not there is a relationship between the history schedule table and the history schedule table including the target schedule component extracted in step S1803 (step S1806). The most common method for judging the relationship is to compare the numbers of the respective history schedule tables, but the relationship may be judged using some quantitative index such as correlation.

[0147] In step S1806, if there is a relationship between the two plan tables, such as the number of history plan tables including indirect influence candidates being equal to the number of history plan tables including the focus plan component (YES in step S1806), the indirect influence relationship calculation unit 1702 determines that there is an indirect influence relationship and records the indirect influence candidates set in step S1805 in a predetermined recording destination (step S1807). That is, the indirect influence candidates recorded in step S1807 correspond to plan components (indirect influence components) that indirectly affect the change in KPI on the focus plan component.

[0148] On the other hand, in step S1806, if the number of history schedule tables including the indirect influence candidate is equal to the number of history schedule tables including the target schedule component, Kuna If there is no relationship between the two schedules (NO in step S1806), the indirect influence relationship calculation unit 1702 determines that there is no indirect influence relationship and excludes the schedule component set in step S1805 from the indirect influence candidates (step S1808).

[0149] Then, after step S1807 or step S1808, the indirect influence relationship calculation unit 1702 repeatedly executes the loop processing of steps S1804 to S1807 (or S1808) until a predetermined search upper limit for searching for indirect influence candidates is reached (step S1809), and proceeds to step S1810 after the loop processing ends. Note that the predetermined search upper limit for searching for indirect influence candidates may be determined, for example, by the calculation time or a threshold value of the influence degree of the candidate component.

[0150] In step S1810, the search results for indirect influence relationships by the above process (specifically, the indirect influence candidates recorded in step S1807) are added to and updated with the influence degree calculation result 1025 read in step S1801, and the indirect influence relationship extraction process is terminated.

[0151] Fig. 19 is a diagram for explaining an image of the process of searching for an indirect influence relationship. The image shown in Fig. 19 is a visualization of the outline of the process of steps S1801 to S1808 in Fig. 18 using a concrete example.

[0152] FIG. 19 shows a target schedule 1901 and other historical schedules 1902 and 1903 as examples of historical schedules for which the presence or absence of an indirect influence relationship with respect to a target schedule component of the target schedule 1901 is determined.

[0153] In the target schedule table 1901, focusing on the schedule component in the first row, the rows with dates "8 / 2" and "8 / 3" are set as indirect influence candidates. When the other history schedule tables 1902 and 1903 are checked, indirect influence candidate 1 shown in FIG. 19 appears in both history schedule tables 1902 and 1903, so indirect influence candidate 1 is determined to be in an indirect influence relationship and is recorded. On the other hand, indirect influence candidate 2 shown in FIG. 19 appears in history schedule table 1902 but does not appear in history schedule table 1903, so indirect influence candidate 2 is determined not to be in an indirect influence relationship and is excluded from the indirect influence candidates.

[0154] Fig. 20 is a diagram showing an example of an influence degree calculation result 1025 including an indirect influence relationship. The influence degree calculation result 2000 shown in Fig. 20 is a specific example of the influence degree calculation result 1025 after the determination result of the indirect influence relationship is updated by the indirect influence relationship extraction process shown in Fig. 18.

[0155] The impact calculation result 2000 is configured as matrix data (table data) like the planning table 1014 shown in Fig. 3 and the impact calculation result 1025 shown in Fig. 12, and has fixed planning master information 1201 common to all planning tables, target items 1202 specific to each planning table, and items of calculated impact 1203. Furthermore, the impact calculation result 2000 has an indirect impact relationship 2004 indicating planning components in an indirect impact relationship as a new item added from the impact calculation result 1025 in Fig. 12.

[0156] In the example of Fig. 20, a request number is entered in the indirect influence relationship 2004. Specifically, for example, the plan component in the second row has a relationship of indirect influence with the plan component in the third row (request number "102") by entering a request number of "102" in the indirect influence relationship 2004. Note that the description format of the indirect influence relationship 2004 is not limited to the above-mentioned format that references a request number, and may be, for example, a different column or row number, as long as the format can identify the plan component.

[0157] Next, the process of extracting an indirect influence relationship using the second approach will be described.

[0158] Fig. 21 is a flowchart showing an example of the processing procedure of the second indirect influence relationship extraction processing. The second indirect influence relationship extraction processing is processing for extracting an indirect influence relationship after planning a schedule based on the second approach, and is executed by a plan generation control unit 1701 and an indirect influence relationship calculation unit 1702 after the end of the influence degree calculation processing of Fig. 5 or Fig. 15 described in the first embodiment. In Fig. 21, the same processes as those in the indirect influence relationship extraction processing of Fig. 18 are given the same step numbers, and detailed descriptions thereof will be omitted.

[0159] According to FIG. 21, first, the indirect influence relationship calculation section 1702 executes the processes of steps S1801, S1802, S1804, and S1805 in the same manner as in FIG.

[0160] Next, in the loop process of the indirect influence search, the indirect influence relation calculation unit 1702 replaces the indirect influence candidates in the subject schedule with the target items existing in the same component of the base schedule (step S2101).

[0161] Next, the indirect influence relationship calculation unit 1702 issues a command to the plan generation control unit 1701 so that the target plan component and the indirect influence candidate replaced in step S2101 (i.e., the target item in the standard schedule) appear in the final output of the plan generation unit 1031 (i.e., the schedule drawn up by the plan generation unit 1031), and the plan generation control unit 1701 causes the plan generation unit 1031 to draw up a simulated plan in accordance with the command (step S2102). Note that the control method related to the command and drawing up by the plan generation control unit 1701 depends on the algorithm of the plan generation unit 1031.

[0162] Next, the indirect influence relationship calculation unit 1702 checks the schedule table output in step S2102 and judges whether the schedule has been created without any problems (step S2103). Specifically, the judgment in step S2103 checks whether any abnormality has occurred, such as a program error occurring without the schedule table being output, a specified component not being output, or a schedule not satisfying a constraint condition that would have been observed in the normal output being output.

[0163] If the above abnormality is not found in step S2103 and a normal schedule is generated (YES in step S2103), this indicates that there is no problem (no influence) even if the indirect influence candidates replaced in step S2101 are not present, and therefore the indirect influence relationship calculation unit 1702 excludes the schedule components set in step S1805 from the indirect influence candidates (step S2104).

[0164] On the other hand, if the above abnormality is confirmed in step S2103 (NO in step S2103), if the indirect influence candidate replaced in step S2101 cannot be output, the focused plan component does not appear either, so the indirect influence relationship calculation unit 1702 determines that there is an indirect influence relationship and records the indirect influence candidate set in step S1805 in a predetermined recording destination (step S2105). That is, the indirect influence candidate recorded in step S2105 corresponds to the plan component (indirect influence component) that indirectly affects the change in KPI with respect to the focused plan component.

[0165] Then, after step S2104 or step S2105, the indirect influence relationship calculation unit 1702 repeatedly executes the loop processing of steps S1804 to S2104 (or S2105) until a predetermined search upper limit for searching for indirect influence candidates is reached (step S2106), and proceeds to step S1810 after the loop processing ends. Note that the predetermined search upper limit for searching for indirect influence candidates may be determined, for example, by the calculation time or a threshold value of the influence degree of the candidate component.

[0166] In step S1810, similar to step S1810 in FIG. 18, the search results for indirect influence relationships (specifically, the indirect influence candidates recorded in step S2105) are added to and updated by the influence calculation result 1025 read in step S1801, and the indirect influence relationship extraction process is terminated.

[0167] The above describes the indirect influence relationship extraction process using the first approach and the second approach. In the second embodiment, however, by using these approaches appropriately depending on the characteristics of the plan, it is possible to assist in appropriate planning.

[0168] In other words, when there are many combinations of plan components and a different pattern of plan table is created each time the user (developer) presses the plan creation button, it can be said that there is randomness in the plan generation. In such a case, the first approach can be adopted, and by using plans that have already been created (history plan tables) to explore the indirect influence relationships (indirect influence relationships) between the plan components, it is possible to obtain information for creating an appropriate plan even in the presence of randomness.

[0169] Furthermore, when the configuration is complex and it takes a long time to create one schedule, it is difficult to generate multiple schedules, and in such cases, the second approach is adopted, and by using a schedule created based on explicit instructions, indirect influence relationships (indirect influence relationships) between schedule components are searched for, making it possible to extract components that are essential for generating a target schedule component (components that must not change randomly), and the analysis of these influence relationships can contribute to the creation of an appropriate schedule. Note that the above explicit instructions may be given by the user (developer). [Explanation of symbols]

[0170] 1000, 1700...Plan evaluation device, 1001...Storage device, 1002...Processing device, 1003...Input device, 1004...Output device, 1010...Plan information, 1011...Business related data, 1012...Constraint list, 1013...Evaluation index list, 1014...Plan table, 1015...KPI data, 1020...Influence related information, 1021...Machine learning model, 1022...Feature conversion rule, 1023...Plan feature table , 1024...Contribution rate calculation result, 1025...Influence calculation result, 1030...Planning relation processing unit, 1031...Plan generation unit, 1032...Evaluation index calculation unit, 1040...Influence evaluation processing unit, 1041...Feature conversion unit, 1042...Model learning unit, 1043...Contribution rate calculation unit, 1044...Influence calculation unit, 1050...Screen output unit, 1060...Data input unit, 1701...Plan generation control unit, 1702...Indirect influence relation calculation unit

Claims

1. A plan evaluation device for evaluating a plan table that is created by combining a plurality of plans, a feature conversion unit that divides the planning table into planning components based on a predetermined conversion rule and converts each divided planning component into a feature; A model learning unit that creates a machine learning model using the feature amount as an input and a key performance indicator (KPI) of the planning table as a target variable; a contribution rate calculation unit that calculates a contribution rate of each of the feature amounts to the machine learning model; an influence calculation unit that calculates an influence of the planning component, which is a conversion source of the feature amount, on a KPI of the planning table based on a contribution rate of the feature amount; A plan evaluation device comprising:

2. The feature conversion unit converts each of the divided planning components into the feature based on a plurality of the conversion rules; the model learning unit creates a plurality of the machine learning models for each of the feature amounts converted based on each of the conversion rules; the contribution rate calculation unit calculates a contribution rate of the feature amount in the machine learning model, The influence degree calculation unit calculates the influence degree by superimposing the feature amounts for which the contribution rates are calculated to be high in each of the machine learning models, and extracts the planning component from which the superimposed feature amounts are converted.

2. The plan evaluation apparatus according to claim 1,

3. The feature conversion unit divides the plan table into plan components based on the conversion rule by an arbitrary method, and obtains the feature by compressing and converting the item value of the plan component into a single numerical value or a category value for each divided plan component.

3. The plan evaluation apparatus according to claim 2.

4. The influence degree calculation unit distributes the contribution rates of the feature amounts calculated by the contribution rate calculation unit to the planning components from which each feature amount is converted, and calculates the influence degree by superimposing the feature amounts whose contribution rates are calculated to be high in each of the machine learning models using the contribution rates after distribution.

2. The plan evaluation apparatus according to claim 1,

5. The model learning unit integrates the created multiple machine learning models, The contribution rate calculation unit calculates a contribution rate of the feature amount from the integrated model integrated by the model learning unit, The influence degree calculation unit distributes the contribution rates of the feature amounts in the integrated model calculated by the contribution rate calculation unit to the planning components from which the feature amounts are converted, and calculates the influence degree by superimposing the distributed contribution rates for each planning component.

3. The plan evaluation apparatus according to claim 2.

6. The schedule includes a target schedule to be used for calculating the degree of influence and a base schedule to be used for comparing the target schedule, a difference between the target schedule and the base schedule is evaluated, and no inconsistency occurs between the two schedules even if a first feature in the target schedule converted by the feature conversion unit and a second feature in the base schedule converted by the feature conversion unit are exchanged for a schedule component of the difference; The influence calculation unit calculates, for each planning component of the difference, a combination of the contribution rates in both planning tables when the planning components are exchanged, thereby calculating an influence due to the planning components.

2. The plan evaluation apparatus according to claim 1,

7. When the plan has multiple KPIs, The influence degree calculation unit calculates the influence degree for each KPI, and calculates the influence degree for the plurality of KPIs as a whole by adding up the influence degrees based on the weighting of each KPI.

2. The plan evaluation apparatus according to claim 1,

8. The feature quantity conversion unit divides the schedule into schedule components based on the conversion rule that is automatically created based on values ​​or ranges of items included in the schedule, and converts each divided schedule component into the feature quantity.

2. The plan evaluation apparatus according to claim 1,

9. The feature quantity conversion unit divides the plan table into plan components based on the unique conversion rule designated by a user, and converts each divided plan component into the feature quantity.

2. The plan evaluation apparatus according to claim 1,

10. The system further includes a plan generating unit that creates the plan table by a predetermined algorithm so as to satisfy given conditions.

2. The plan evaluation apparatus according to claim 1,

11. An indirect influence relationship calculation unit that extracts, from the planning table in which the influence on the KPI has been calculated by the influence calculation unit, an indirect influence component that means a planning component that indirectly contributes to a change in the KPI, based on a dependency relationship between planning components in the planning table.

2. The plan evaluation apparatus according to claim 1,

12. The schedule includes a target schedule that is a target for calculating the degree of influence, and a history schedule that has been planned and is separate from the target schedule, The indirect influence relationship calculation unit determines one target planning component from the planning components of the target planning table for which the influence on the KPI has been calculated by the influence calculation unit, searches for planning components that appear simultaneously with the target planning component in the history planning table having the target planning component, and determines and extracts the planning component found in the search as the indirect influence component on the target planning component.

12. The plan evaluation apparatus according to claim 11 .

13. a schedule generating unit that creates a schedule that satisfies given conditions using a predetermined algorithm; a plan generation control unit that instructs the plan generation unit to create a plan table; Further equipped with The schedule includes a target schedule to be used for calculating the degree of influence and a base schedule to be used for comparing the target schedule, The indirect influence relationship calculation unit determines one focus plan component from the plan components of the target plan table for which the influence on the KPI has been calculated by the influence calculation unit, sets a plan component other than the focus plan component as an indirect influence candidate, and further replaces the indirect influence candidate with a plan component of the base plan table corresponding to the indirect influence candidate; the plan generation control unit instructs the plan generation unit to create a simulated plan table having the plan component of interest and the indirect influence candidate after the replacement; The indirect influence relationship calculation unit, when a problem occurs in the planning of the simulated schedule by the schedule generation unit, determines and extracts the indirect influence candidate before the replacement as the indirect influence component for the target schedule component.

12. The plan evaluation apparatus according to claim 11 .

14. a screen output unit that displays a result of the impact degree calculation by the impact degree calculation unit on an output device, The screen output unit, in displaying the calculation result, highlights a planning component for which a high influence exceeding a predetermined standard is calculated among the plurality of planning components for which the influence is calculated by the influence calculation unit.

14. The plan evaluation device according to claim 1, wherein the plan evaluation device is a device for evaluating a plan of a target object.

15. A plan evaluation method using a plan evaluation device that evaluates a plan table that is created by combining a plurality of plans, comprising the steps of: a feature conversion step in which the plan evaluation device divides the plan table into plan components based on a predetermined conversion rule and converts each divided plan component into a feature; a model learning step in which the plan evaluation device creates a machine learning model using the feature converted in the feature conversion step as an input and a key performance indicator (KPI) of the plan table as a response variable; a contribution rate calculation step in which the plan evaluation device calculates a contribution rate of each of the feature amounts to the machine learning model created in the model learning step; an influence calculation step in which the plan evaluation device calculates an influence degree of the plan component from which the feature amount is converted on a KPI of the plan table based on the contribution rate of the feature amount calculated in the contribution rate calculation step; A plan evaluation method comprising:

Images