Computer system, method for evaluating policy definition, and program

Abstract

To evaluate bias of elements included in a comparison group with respect to a population of elements.SOLUTION: A computer system is configured to: hold measure definition management information that stores measure definition data including allocation conditions for generating a comparison group which is a set of elements of measures, the comparison group being used for evaluating the effect of the measures; specify a comparison group generated based on the measure definition data and a population of elements that meets the allocation conditions included in the measure definition data; and determine, on the basis of a distribution of attributes of the elements included in the comparison group and a distribution of attributes of the elements included in the population, whether a bias of the elements in the comparison group has occurred or not.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a technique for assessing bias in elements included in a comparison group used in evaluating a policy. [Background technology]

[0002] Various measures are implemented to improve indicators such as business efficiency, productivity, and sales. In this specification, a "measure" is defined as an action aimed at achieving a certain effect by using resources on the target of the measure.

[0003] A / B testing is known as a method for evaluating policies. In A / B testing, people are divided into two comparison groups, A and B, and different actions are performed on A and B. The results of the actions in A and B are compared to evaluate the effectiveness and causal relationships of the policies.

[0004] In AB testing, the allocation of elements to comparison groups is important. In the measures of this specification, it is necessary to appropriately allocate targets and resources. In response to this, a method of allocating comparison groups based on the attributes of the target to be divided is known (for example, paragraphs

[0046] and

[0047] of Patent Document 1). [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Patent Publication No. 2021-89485 Summary of the Invention [Problem to be solved by the invention]

[0006] Conventional techniques focus on bias in the attributes of elements between comparison groups, but do not address bias in attributes between the set of elements to be selected (population) and the group of elements included in the comparison group. It is desirable that the distribution of attributes of elements included in the comparison group be similar to the distribution of attributes of elements included in the population. If the two distributions differ, it indicates that bias has occurred.

[0007] The present invention provides a system and method for assessing the bias of elements included in a comparison group relative to a population of elements. [Means for solving the problem]

[0008] A representative example of the invention disclosed in the present application is as follows: That is, a computer system including at least one computer having a processor and a storage device connected to the processor, holds policy definition management information storing policy definition data including allocation conditions for generating a comparison group, which is a set of elements of a policy used to evaluate the effectiveness of the policy, and the at least one computer identifies the comparison group generated based on the policy definition data and a population of the elements that meets the allocation conditions included in the policy definition data, and calculates a distribution of attributes of the elements included in the comparison group and a distribution of attributes of the elements included in the population. and calculating a similarity between the two images based on the similarity. It is determined whether bias of the elements in the comparison group occurs. [Effects of the Invention]

[0009] According to the present invention, a computer system can evaluate and present the bias of elements included in a comparison group relative to the population of elements. Objects, configurations, and effects other than those described above will become apparent from the following description of the embodiments. [Brief explanation of the drawings]

[0010] [Figure 1] FIG. 1 is a diagram illustrating an example of the configuration of a computer system according to a first embodiment. [Figure 2]FIG. 2 is a diagram illustrating an example of a hardware configuration of a computer included in the policy evaluation system of the first embodiment. [Figure 3] FIG. 10 is a diagram illustrating an example of policy definition management information according to the first embodiment. [Figure 4] FIG. 10 is a diagram illustrating an example of resource management information according to the first embodiment. [Figure 5] FIG. 10 is a diagram illustrating an example of policy target management information according to the first embodiment. [Figure 6] FIG. 10 is a diagram illustrating an example of policy implementation plan management information according to the first embodiment. [Figure 7A] FIG. 10 is a diagram illustrating an example of bias analysis result management information according to the first embodiment. [Figure 7B] FIG. 10 is a diagram illustrating an example of bias analysis result management information according to the first embodiment. [Figure 8] FIG. 10 is a diagram illustrating an example of a screen displayed by the terminal according to the first embodiment. [Figure 9] 10 is a flowchart illustrating an example of a policy implementation plan generation process executed by the policy evaluation system of the first embodiment. [Figure 10A] 10 is a flowchart illustrating an example of a bias evaluation index calculation process executed by the policy evaluation system of the first embodiment. [Figure 10B] 10 is a flowchart illustrating an example of a bias evaluation index calculation process executed by the policy evaluation system of the first embodiment. [Figure 11] FIG. 10 is a diagram illustrating an example of second bias analysis information according to the second embodiment. [Figure 12] FIG. 10 is a diagram illustrating an example of a screen displayed by the terminal according to the first embodiment. [Figure 13] 10 is a flowchart illustrating an example of a factor analysis process executed by the policy evaluation system according to the second embodiment. [Figure 14] FIG. 10 is a diagram illustrating an example of a screen displayed by a terminal according to a second embodiment. [Figure 15] FIG. 11 is a diagram illustrating an example of a screen displayed by a terminal according to a third embodiment. [Figure 16] 11 is a flowchart illustrating an example of a policy definition evaluation process executed by the policy evaluation system according to the third embodiment. [Figure 17]FIG. 1 is a diagram illustrating a conventional problem. DETAILED DESCRIPTION OF THE INVENTION

[0011] Hereinafter, embodiments of the present invention will be described with reference to the drawings. However, the present invention should not be construed as being limited to the description of the embodiments shown below. Those skilled in the art will readily understand that the specific configuration can be changed without departing from the spirit or intent of the present invention.

[0012] In the configuration of the invention described below, the same or similar configurations or functions are denoted by the same reference numerals, and redundant explanations will be omitted.

[0013] In this specification, the terms "first," "second," "third," etc. are used to identify components and do not necessarily limit the number or order.

[0014] In this specification, the comparison groups for evaluating measures are defined as Group A and Group B. Furthermore, when there is no distinction between targets and resources, they are referred to as elements. [Example]

[0015] Fig. 1 is a diagram illustrating an example of the configuration of a computer system according to Example 1. Fig. 2 is a diagram illustrating an example of the hardware configuration of a computer included in the policy evaluation system according to Example 1.

[0016] The computer system comprises a policy evaluation system 100 and a terminal 101. The policy evaluation system 100 and the terminal 101 are connected to each other via a network 102 such as a WAN (Wide Area Network) or a LAN (Local Area Network). The network 102 may be connected via either a wired or wireless method.

[0017] The policy evaluation system 100 is a system that generates an implementation plan for a policy and evaluates the bias of a comparison group used in evaluating the policy, and is composed of a computer 200 as shown in FIG.

[0018] The computer 200 includes a processor 201, a main memory device 202, a secondary memory device 203, and a network interface 204. The hardware elements are connected to each other via a bus. The computer 200 may include input devices such as a keyboard, a mouse, and a touch panel, and may also include output devices such as a display and a printer.

[0019] The processor 201 executes a program stored in the main memory device 202. The processor 201 executes processing in accordance with the program, thereby operating as a functional unit (module) that realizes a specific function. In the following description, when a processing is described using a functional unit as the subject, it indicates that the processor 201 is executing a program that realizes the functional unit.

[0020] The main memory device 202 is a storage device such as a DRAM (Dynamic Random Access Memory) and stores programs executed by the processor 201 and data used by the programs. The main memory device 202 is also used as a work area. The secondary memory device 203 is a storage device such as an HDD (Hard Disk Drive) or SSD (Solid State Drive) and permanently stores data.

[0021] The programs and data stored in the main memory device 202 may be stored in the secondary memory device 203. In this case, the processor 201 reads the programs and data from the secondary memory device 203 and loads them into the main memory device 202.

[0022] The policy evaluation system 100 includes a policy planning unit 110, a bias analysis unit 111, and a policy definition evaluation unit 112. The policy evaluation system 100 also holds policy definition management information 120, resource management information 121, policy target management information 122, policy implementation plan management information 123, and bias analysis result management information 124.

[0023] The policy definition management information 120 is information for managing data including the definition of a policy (policy definition data). Here, the definition of a policy is a concept including the specific content of the policy and the allocation conditions of a comparison group for evaluating the policy. The resource management information 121 is information for managing the resources used in the policy. The policy target management information 122 is information for managing the target of the policy. The policy implementation plan management information 123 is information for managing the implementation plan of the policy. The bias analysis result management information 124 is information for managing the analysis results of the bias of the comparison group.

[0024] The policy planning unit 110 generates an implementation plan based on the policy definition data and registers the generated implementation plan in the policy implementation plan management information 123. When generating implementation plans for multiple policies, the policy planning unit 110 of this embodiment allocates elements to comparison groups so as to prevent bias of elements in groups A and B and bias of elements included in the comparison group relative to a set (population) of elements that meet the allocation conditions of the comparison group. Details will be described later.

[0025] In the following description, the bias of elements in groups A and B will be referred to as a first bias, and the bias of elements included in the comparison group relative to the set of elements that meet the allocation conditions of the comparison group will be referred to as a second bias.

[0026] The bias analysis unit 111 analyzes various biases of the comparison group. The bias analysis unit 111 includes an evaluation index calculation unit 130 and a factor analysis unit 131. The evaluation index calculation unit 130 calculates an index representing the intensity of occurrence of each of the first bias and the second bias. The factor analysis unit 131 analyzes the factor causing the second bias. Details of the factor analysis unit 131 will be described in Example 2.

[0027] The policy definition evaluation unit 112 evaluates the likelihood of the new policy causing the second bias based on the policy definition data of the new policy. Details of the policy definition evaluation unit 112 will be described in a third embodiment.

[0028] It should be noted that with regard to each functional unit of the policy evaluation system 100, multiple functional units may be combined into one functional unit, or one functional unit may be divided into multiple functional units for each function.

[0029] The terminal 101 is a terminal operated by a user who uses the policy evaluation system 100. The terminal 101 has a processor, a main memory device, a network interface, an input device, and an output device, all of which are not shown. The terminal 101 has a data receiving unit 140 and a display unit 141. The data receiving unit 140 receives data input from the user. The display unit 141 displays various information.

[0030] The second bias occurs most noticeably when multiple measures are implemented simultaneously. Figure 17 is a diagram illustrating the conventional problem. Here, we will explain the example of grouping employees (resources) in call center operations. In Figure 17, circles represent employees. Consider the following grouping of resources for Measure 1 and Measure 2. Measure 1 Group A: Women Measure 1 Group B: Male Measure 2 Group A: 30s Measure 2 Group B: 20s

[0031] As shown in Figure 17, when grouping is performed for each measure in the order of Measure 1 and Measure 2, Group B for Measure 2 contains only female employees. However, the set of elements (population) that meets the allocation conditions for Group B for Measure 2 includes men. Therefore, it can be determined that the second bias has occurred in Group B for Measure 2.

[0032] 3 is a diagram showing an example of the policy definition management information 120 according to the first embodiment. The policy definition management information 120 stores entries including a policy ID 301, an action 302, a customer allocation condition 303, and a resource allocation condition 304. One entry exists for one policy.

[0033] The policy ID 301 is a field that stores identification information for uniquely identifying a policy. The action 302 is a field that stores the action of the policy. The customer allocation conditions 303 are a group of fields that store the allocation conditions of a comparison group related to customers who are the target of the policy. The customer allocation conditions 303 store the allocation conditions of each of group A and group B. The resource allocation conditions 304 are a group of fields that store the allocation conditions of a comparison group related to resources. The resource allocation conditions 304 store the allocation conditions of each of group A and group B.

[0034] 4 is a diagram showing an example of the resource management information 121 according to the first embodiment. The resource management information 121 stores entries each including a resource ID 401 and an attribute 402. One entry exists for one resource.

[0035] The resource ID 401 is a field that stores identification information for uniquely identifying a resource. The attributes 402 are a group of fields that store the attributes of the resource. If the resource is an employee, the attributes 402 store fields such as staff ID, age, gender, and working hours. Note that the present invention is not limited to the fields included in the attributes 402.

[0036] 5 is a diagram showing an example of the policy target management information 122 according to the embodiment 1. The policy target management information 122 stores entries each including a target ID 501 and an attribute 502. One entry exists for one target.

[0037] The target ID 501 is a field that stores identification information for uniquely identifying a target. The attributes 502 are a group of fields that store the attributes of the target. If the target is a customer, the attributes 502 store fields such as age, gender, and annual income. Note that the present invention is not limited to the fields included in the attributes 502.

[0038] 6 is a diagram showing an example of the policy implementation plan management information 123 according to the first embodiment. The policy implementation plan management information 123 stores entries including a policy ID 601, an AB type 602, a target list 603, and a resource list 604. One entry exists for each combination of a policy and a comparison group type.

[0039] The policy ID 601 is the same field as the policy ID 301. The AB type 602 is a field that stores the type of comparison group. The AB type 602 stores either "A" representing group A or "B" representing group B. The target list 603 is a field that stores a list of targets assigned to the comparison group of the policy. The resource list 604 is a field that stores a list of resources assigned to the comparison group of the policy.

[0040] 7A and 7B are diagrams illustrating an example of the bias analysis result management information 124 according to the first embodiment. The bias analysis result management information 124 includes first bias analysis information 700 and second bias analysis information 710.

[0041] The first bias analysis information 700 stores an entry including a measure ID 701 and an AB group bias 702. One entry exists for one measure.

[0042] The policy ID 701 is the same field as the policy ID 301. The group AB bias 702 is a field that stores an index (first bias index) that indicates the intensity of occurrence of the first bias.

[0043] The second bias analysis information 710 stores entries including a measure ID 711, an AB type 712, and a distribution bias 713. There is one entry for each combination of a measure and a type of comparison group.

[0044] Policy ID 711 is the same field as policy ID 301. AB type 712 is the same field as AB type 602. Distribution bias 713 is a group of fields that store an index (second bias index) that indicates the intensity of occurrence of the second bias. Distribution bias 713 includes a group of fields that store the intensity of occurrence of the second bias for each attribute of the target, and a group of fields that store the intensity of occurrence of the second bias for each attribute of the resource.

[0045] 8 is a diagram showing an example of a screen displayed by the terminal 101 of the embodiment 1. The display unit 141 of the terminal 101 displays a screen 800 for registering policy definition data.

[0046] The screen 800 includes a group A setting area 810, a group B setting area 820, a manual condition setting area 830, a measure ID setting field 840, and a registration button 850.

[0047] The group A setting area 810 includes fields for setting the allocation conditions for resources and targets for group A and the actions in group A. The group B setting area 820 includes fields for setting the allocation conditions for resources and targets for group B and the actions in group B. The manual condition setting area 830 includes fields for setting other conditions. Fields are added by pressing the add button. The policy ID setting field 840 is a field for setting identification information for a policy. The register button 850 is an operation button for registering policy definition data.

[0048] The user makes various settings on the screen 800 and presses the register button 850. The data receiving unit 140 of the terminal 101 transmits the policy definition data including the various settings on the screen 800 to the policy evaluation system 100. The policy evaluation system 100 registers the policy definition data in the policy definition management information 120.

[0049] FIG. 9 is a flowchart illustrating an example of a policy implementation plan generation process executed by the policy evaluation system 100 according to the first embodiment.

[0050] When the policy evaluation system 100 receives an instruction to generate a policy implementation plan from the terminal 101, it starts the process described below.

[0051] The policy planning unit 110 identifies policies to be implemented by referring to the policy definition management information 120 (step S101). Specifically, the policy planning unit 110 acquires each entry in the policy definition management information 120 and generates a list of policies. In addition, the policy planning unit 110 adds entries for group A and group B for each policy to the policy implementation plan management information 123.

[0052] The policy planning unit 110 extracts, for each policy, a set of targets (target population) that meets the allocation conditions for targets in group A and group B, and extracts a set of resources (resource population) that meets the allocation conditions for resources in group A and group B (step S102). Here, for each policy, a target population for group A, a target population for group B, a resource population for group A, and a resource population for group B are extracted.

[0053] The policy planning unit 110 starts a loop process of policies (step S103). Specifically, the policy planning unit 110 selects one policy from the list of policies.

[0054] The policy planning unit 110 selects targets to be allocated to group A and targets to be allocated to group B from the target population of the selected policy, and selects resources to be allocated to group A and resources to be allocated to group B from the resource population of the selected policy (step S104). The targets and resources are selected randomly from each population. The policy planning unit 110 sets identification information of the targets and resources in the target list 603 and resource list 604 of each entry in the policy implementation plan management information 123.

[0055] The policy planning unit 110 determines whether or not the processing has been completed for all policies (step S105). If the processing has not been completed for all policies, the policy planning unit 110 selects a new policy and returns to step S104.

[0056] In this way, by selecting one element from the population for each measure, it is possible to suppress the second bias caused by selecting elements at once, and by selecting elements randomly, it is possible to suppress the first bias.

[0057] The policy planning unit 110 determines whether or not there is a policy for which allocation of elements to the comparison group has been completed (step S106). For example, if the number of elements in each of group A and group B is greater than a threshold, or if the population is depleted, it is determined that there is a policy for which allocation of elements to the comparison group has been completed.

[0058] If there is no measure for which allocation of elements to comparison groups has been completed, the measure planning unit 110 executes the loop processing of measures again.

[0059] If there is a measure for which allocation of elements to comparison groups has been completed, the measure planning unit 110 excludes the measure from the selection targets (step S107). Specifically, the measure planning unit 110 deletes the entry for the measure from the list of measures.

[0060] The policy planning unit 110 determines whether or not the allocation of all policy elements has been completed (step S108).

[0061] If allocation of all the elements of the measures has not been completed, the measure planning unit 110 returns to step S103 and executes the loop process of the measures again.

[0062] When the allocation of all the policy elements is completed, the policy planning unit 110 ends the policy implementation plan generation process.

[0063] 10A and 10B are flowcharts illustrating an example of the bias evaluation index calculation process executed by the policy evaluation system 100 according to the first embodiment.

[0064] After the policy implementation plan generation process is completed, the evaluation index calculation unit 130 of the bias analysis unit 111 starts the process described below. Note that the trigger for executing the process is not limited to this. For example, the policy implementation plan generation process may be executed when an execution instruction is received from the terminal 101.

[0065] The evaluation index calculation unit 130 refers to the policy implementation plan management information 123 to identify a policy and starts loop processing of the policy (step S201). Specifically, the evaluation index calculation unit 130 refers to the policy implementation plan management information 123 to generate a list of policies, and selects one policy from the list of policies.

[0066] The evaluation index calculation unit 130 calculates a first bias index for the selected measure (step S202).

[0067] For example, the evaluation index calculation unit 130 calculates the first bias index of the policy using an index indicating the variation in the attributes of the target between group A and group B, and an index indicating the variation in the attributes of the target resources between group A and group B.

[0068] The evaluation index calculation unit 130 adds an entry to the first bias analysis information 700 , sets the identification information of the selected measure in the measure ID 701 , and sets the first bias index in the group AB bias 702 .

[0069] The evaluation index calculation unit 130 starts loop processing of the comparison groups (step S203). Here, it is assumed that the comparison groups are selected in the order of group A and group B. At this time, the evaluation index calculation unit 130 adds an entry to the second bias analysis information 710, sets identification information of the selected measure to the measure ID 711 of the entry, and sets the type of the selected comparison group to the AB type 712.

[0070] The evaluation index calculation unit 130 extracts a set of targets (target population) that meet the allocation conditions of the targets of the comparison group for the selected policy, and extracts a set of resources (resource population) that meet the allocation conditions of the resources of the comparison group (step S204).

[0071] The evaluation index calculation unit 130 starts loop processing of the target attribute (step S205). Specifically, the evaluation index calculation unit 130 selects one attribute from the target attribute group.

[0072] The evaluation index calculation unit 130 calculates a second bias index for the target attribute (step S206). Specifically, the evaluation index calculation unit 130 calculates the similarity between the attribute distributions of the comparison group and the population as the second bias index. For example, the Jaccard coefficient is calculated as the similarity. At this time, the evaluation index calculation unit 130 sets the similarity in the field corresponding to the selected target attribute of the distribution bias 713 of the entry added in step S203.

[0073] The evaluation index calculation unit 130 determines whether the second bias index is equal to or less than a threshold value (step S207). In this embodiment, if the distribution of the attributes of the comparison group is not similar to the distribution of the attributes of the population, that is, if the second bias index is equal to or less than a threshold value, it is determined that there is a possibility that the second bias has occurred.

[0074] If the second bias index is greater than the threshold, the evaluation index calculation unit 130 proceeds to step S209.

[0075] If the second bias index is equal to or less than the threshold, the evaluation index calculation unit 130 sets an alert (step S208), and then proceeds to step S209.

[0076] In step S209, the evaluation index calculation unit 130 determines whether or not the processing has been completed for all the target attributes (step S209).

[0077] If the processing has not been completed for all of the target attributes, the evaluation index calculation unit 130 selects a new target attribute and returns to S206.

[0078] When the processing for all target attributes is completed, the evaluation index calculation unit 130 starts a loop processing of the resource attributes (step S210). Specifically, the evaluation index calculation unit 130 selects one attribute from the group of resource attributes.

[0079] The evaluation index calculation unit 130 calculates a second bias index for the attribute of the resource (step S211). Specifically, the evaluation index calculation unit 130 calculates the similarity between the attribute distributions of the comparison group and the population as the second bias index. For example, the Jaccard coefficient is calculated as the similarity. At this time, the evaluation index calculation unit 130 sets the similarity in the field corresponding to the attribute of the selected resource in the distribution bias 713 of the entry added in step S203.

[0080] The evaluation index calculation unit 130 determines whether the second bias index is equal to or less than a threshold value (step S212).

[0081] If the second bias index is greater than the threshold, the evaluation index calculation unit 130 proceeds to step S214.

[0082] If the second bias index is equal to or less than the threshold, the evaluation index calculation unit 130 sets an alert (step S213), and then proceeds to step S214.

[0083] In step S214, the evaluation index calculation unit 130 determines whether or not the processing has been completed for all attributes of the resource (step S214).

[0084] If the processing has not been completed for all the attributes of the resource, the evaluation index calculation unit 130 selects a new attribute of the resource and returns to S211.

[0085] When the processing has been completed for all attributes of the resource, the evaluation index calculation unit 130 determines whether the processing for each comparison group has been completed (step S215).

[0086] If the processing of each comparison group has not been completed, the evaluation index calculation section 130 selects a new comparison group and returns to step S204.

[0087] When the processing of each comparison group is completed, the evaluation index calculation unit 130 determines whether the processing of all measures is completed (step S216).

[0088] If the processing has not been completed for all measures, the evaluation index calculation unit 130 selects a new measure and returns to step S202.

[0089] When the processing for all measures is completed, the evaluation index calculation unit 130 ends the bias evaluation index calculation processing. The bias analysis unit 111 displays on the terminal 101 the entry for which the alert is set.

[0090] In step S204, the evaluation index calculation unit 130 may randomly select a predetermined number of targets from the target population to generate a target set for processing, and may also randomly select a predetermined number of targets from the resource population to generate a resource set for processing. In this case, in step S206, the similarity between the distribution of attributes of the comparison group and the set for processing is calculated.

[0091] According to the first embodiment, the policy evaluation system 100 can quantitatively evaluate the intensity of the occurrence of the second bias and present the evaluation result, which allows the user to review policy definitions and change settings, etc. [Example]

[0092] The policy evaluation system 100 of the second embodiment analyzes the factors that cause the second bias. The second embodiment will be described below, focusing on the differences from the first embodiment.

[0093] The configuration of the computer system of the second embodiment is the same as that of the first embodiment. The policy definition management information 120, resource management information 121, policy target management information 122, and policy implementation plan management information 123 of the second embodiment are the same as those of the first embodiment. The first bias analysis information 700 of the bias analysis result management information 124 of the second embodiment is the same as that of the first embodiment.

[0094] The second embodiment differs from the first embodiment in the second bias analysis information 710. Fig. 11 is a diagram showing an example of the second bias analysis information 710 of the second embodiment.

[0095] An entry of the second bias analysis information 710 includes a bias factor score 714. The bias factor score 714 is a group of fields that store scores indicating the degree of influence of the combination of the allocation conditions of targets and resources of the measure corresponding to the entry (target measure) and the allocation conditions of targets and resources of other measures (comparison measures) on the occurrence of the second bias. In this embodiment, the bias factor score 714 stores, as a score, the degree of overlap between the population of elements that match the allocation conditions of the target measure and the population of elements that match the allocation conditions of the comparison measure, as shown in formula (1).

[0096]

number

[0097] FIG. 12 is a diagram illustrating an example of a screen displayed on the terminal 101 according to the first embodiment.

[0098] The display unit 141 of the terminal 101 displays a screen 900 for inputting a policy definition.

[0099] The screen 1200 includes a group A setting area 1210, a group B setting area 1220, a policy ID setting field 1230, and a registration button 1240.

[0100] The group A setting area 1210 includes fields for setting allocation conditions for resources and targets for group A and actions in group A. The group B setting area 1220 includes fields for setting allocation conditions for resources and targets for group B and actions in group B. The policy ID setting field 1230 is a field for setting identification information for a policy. The register button 1240 is an operation button for registering a policy definition.

[0101] In the second embodiment, before registering the policy definition data, the display unit 141 inquires about the contents of the group A setting area 1210 and the group B setting area 1220 and displays the number of resources and targets occupied. The denominator of the number of occupancies is the number of elements that match the allocation conditions, and the numerator is the number of elements to be allocated to the comparison group.

[0102] Fig. 13 is a flowchart illustrating an example of the factor analysis process executed by the policy evaluation system 100 of the second embodiment. Fig. 14 is a diagram illustrating an example of a screen displayed on the terminal 101 of the second embodiment.

[0103] After the policy implementation plan generation process is completed, the factor analysis unit 131 of the bias analysis unit 111 starts the process described below. Note that the trigger for executing the process is not limited to this. For example, the factor analysis process may be executed when an execution instruction is received from the terminal 101.

[0104] The factor analysis unit 131 starts a loop process (1) of measures (step S301). Specifically, the factor analysis unit 131 selects one measure by referring to the measure implementation plan management information 123. In the following description, the selected measure will be referred to as a target measure.

[0105] The factor analysis unit 131 starts a loop process (1) of the comparison groups of the target measures (step S302). Here, it is assumed that the comparison groups are selected in the order of group A and group B.

[0106] The factor analysis unit 131 extracts a set of targets (target population) that match the allocation conditions of targets of the comparison group for the target measure, and extracts a set of resources (resource population) that match the allocation conditions of resources of the comparison group (step S303).

[0107] The factor analysis unit 131 starts the policy loop process (2) (step S304). Specifically, the factor analysis unit 131 selects one policy by referring to the policy implementation plan management information 123. Note that the target policy is excluded from the selection targets. In the following explanation, the selected policy will be referred to as a comparison policy.

[0108] The factor analysis unit 131 starts a loop process (2) of the comparison groups of the comparison measures (step S305). Here, it is assumed that the comparison groups are selected in the order of group A and group B.

[0109] The factor analysis unit 131 extracts a set of targets (target population) that match the allocation conditions of targets of the comparison group for the comparison measure, and extracts a set of resources (resource population) based on the resource allocation conditions of the comparison group (step S306).

[0110] The factor analysis unit 131 calculates the score (target) and the score (resource) (step S307). Specifically, the score (target) and the score (resource) are calculated using the formula (1).

[0111] The factor analysis unit 131 refers to the second bias analysis information 710 and searches for an entry in which the identification information of the target measure is stored in the measure ID 711 and the type of comparison group selected in step S302 is stored in the AB type 712. The factor analysis unit 131 refers to the bias factor score 714 of the searched entry and sets the calculated score (target) and score (resource) in fields that match the combination of the comparison measure and the type of comparison group selected in step S305.

[0112] The factor analysis unit 131 may calculate the number of occupancies and set it in the field.

[0113] The factor analysis unit 131 determines whether at least one of the determination formula (1) and the determination formula (2) is satisfied (step S308). Judgment formula (1): Score (target) is above threshold Judgment formula (2): Score (resource) is above threshold

[0114] When at least one of the judgment formula (1) and the judgment formula (2) is satisfied, it is determined that the combination of the allocation condition of the target measure and the allocation condition of the comparison measure is a factor causing the second bias in the comparison group of the target measure.

[0115] If neither the determination formula (1) nor the determination formula (2) is satisfied, the factor analysis unit 131 proceeds to step S310.

[0116] If at least one of the determination formula (1) and the determination formula (2) is satisfied, the factor analysis unit 131 sets an alert (step S309), and the process proceeds to step S310.

[0117] In step S310, the factor analysis unit 131 determines whether or not the processing of each comparison group of the comparison measures has been completed (step S310).

[0118] If the processing of each comparison group of the comparison measures has not been completed, the factor analysis unit 131 selects a new comparison group and returns to step S306.

[0119] When the processing of each comparison group of the comparison measures is completed, the factor analysis unit 131 determines whether the processing of all measures except the target measure is completed (step S311).

[0120] If the processing has not been completed for all measures other than the target measure, the factor analysis unit 131 selects a new comparison measure and returns to step S305.

[0121] When the processing is completed for all measures except the target measure, the factor analysis unit 131 determines whether the processing for each comparison group of the target measure is completed (step S312).

[0122] If the processing of each comparison group for the target measure has not been completed, the factor analysis unit 131 selects a new comparison group and returns to step S303.

[0123] When the processing of each comparison group of the target measures is completed, the factor analysis unit 131 determines whether the processing is completed for all measures (step S313).

[0124] If the processing has not been completed for all measures, the factor analysis unit 131 selects a new target measure and returns to step S302.

[0125] When processing has been completed for all measures, the factor analysis unit 131 ends the factor analysis processing. The bias analysis unit 111 displays the entry for which the alert has been set on the terminal 101. For example, a screen 1400 as shown in Fig. 14 is displayed on the terminal 101. The screen 1400 includes a measure selection field 1410 and alert information 1420 and 1430.

[0126] The measure selection field 1410 is a field for selecting a measure for which an alert has been set, and measures for which an alert has been set are displayed in a pull-down format.

[0127] The alert information 1420 is information indicating a combination of a comparison measure and a comparison group for which an alert has been set, and a score (target). The alert information 1430 is information indicating a combination of a comparison measure and a comparison group for which an alert has been set, and a score (resource).

[0128] According to the second embodiment, the policy evaluation system 100 can present to the user a combination of allocation conditions for elements that are factors causing the second bias, thereby enabling the user to examine policy definitions and change settings in more detail. [Example]

[0129] When new policy definition data is registered, the policy evaluation system 100 of the third embodiment determines whether a second bias occurs in the relationship between the existing policy definition data and the policy definition data, and outputs the determination result. The following describes the third embodiment, focusing on the differences from the first embodiment.

[0130] The configuration of the computer system of the third embodiment is the same as that of the first embodiment. The policy definition management information 120, resource management information 121, policy target management information 122, and policy implementation plan management information 123 of the third embodiment are the same as those of the first embodiment. The first bias analysis information 700 of the bias analysis result management information 124 of the third embodiment is the same as that of the first embodiment. The second bias analysis information 710 of the third embodiment is the same as that of the second embodiment.

[0131] Fig. 15 is a diagram showing an example of a screen displayed on the terminal 101 of the embodiment 3. Fig. 16 is a flowchart illustrating an example of a policy definition evaluation process executed by the policy evaluation system 100 of the embodiment 3.

[0132] The display unit 141 of the terminal 101 displays a screen 800 for inputting a policy definition. The group A setting area 810, group B setting area 820, manual condition setting area 830, policy ID setting field 840, and registration button 850 are the same as those in the first embodiment. Specific items in the group A setting area 810, group B setting area 820, and manual condition setting area 830 are omitted. In the third embodiment, the screen 800 newly includes a bias evaluation result display area 860 and an evaluation button 870.

[0133] After making various inputs in the group A setting area 810, group B setting area 820, manual condition setting area 830, and policy ID setting field 840, the user presses the evaluation button 870. The terminal 101 transmits a policy definition evaluation instruction to the policy evaluation system 100. When the policy definition evaluation unit 112 of the policy evaluation system 100 receives the policy definition evaluation instruction, it starts the policy definition evaluation process.

[0134] The policy definition evaluation unit 112 instructs the factor analysis unit 131 to execute a factor analysis process (step S401). At this time, the policy definition evaluation unit 112 inputs policy definition data of a new policy to the factor analysis unit 131.

[0135] The factor analysis unit 131 executes factor analysis processing with the target measure fixed. In other words, the measure loop processing (1) is not executed. The factor analysis unit 131 does not register data in the second bias analysis information 710, but stores the processing results in the work area. When the factor analysis processing is completed, the factor analysis unit 131 outputs the processing results stored in the work area to the measure definition evaluation unit 112.

[0136] The policy definition evaluation unit 112 generates evaluation display information based on the processing result acquired from the factor analysis unit 131, and transmits it to the terminal 101 (step S402). After that, the policy definition evaluation unit 112 ends the processing.

[0137] The terminal 101 performs display in the bias evaluation result display area 860 based on the evaluation display information.

[0138] According to the third embodiment, the policy evaluation system 100 can support the setting of a policy definition that prevents the occurrence of the second bias when setting the policy definition. This allows the user to perform settings that prevent the occurrence of the second bias when registering policy definition data.

[0139] The present invention is not limited to the above-described embodiments, but includes various modifications. For example, the above-described embodiments are provided to explain the present invention in detail, and the present invention is not necessarily limited to those including all of the described configurations. Furthermore, some of the configurations of each embodiment can be added to, deleted from, or replaced with other configurations.

[0140] Furthermore, the above-described configurations, functions, processing units, processing means, etc. may be partially or entirely implemented in hardware, for example, by designing them as integrated circuits. The present invention can also be realized by software program code that implements the functions of the embodiments. In this case, a storage medium on which the program code is recorded is provided to a computer, and a processor included in the computer reads the program code stored in the storage medium. In this case, the program code itself read from the storage medium implements the functions of the above-described embodiments, and the program code itself and the storage medium on which it is stored constitute the present invention. Examples of storage media for providing such program code include flexible disks, CD-ROMs, DVD-ROMs, hard disks, solid-state drives (SSDs), optical disks, magneto-optical disks, CD-Rs, magnetic tapes, non-volatile memory cards, and ROMs.

[0141] Furthermore, the program code that realizes the functions described in this embodiment can be implemented in a wide range of program or script languages, such as assembler, C / C++, perl, Shell, PHP, Python, and Java (registered trademark).

[0142] Furthermore, the program code of the software that realizes the functions of the embodiments may be distributed via a network and stored in a storage means such as a computer's hard disk or memory, or in a storage medium such as a CD-RW or CD-R, and the processor of the computer may read and execute the program code stored in the storage means or storage medium.

[0143] In the above-described embodiment, the control lines and information lines are those that are considered necessary for the explanation, and not all control lines and information lines are necessarily shown in the product. All components may be interconnected. [Explanation of symbols]

[0144] 100 Policy Evaluation System 101 terminals 102 Network 110 Policy Planning Department 111 Bias Analysis Department 112 Policy Definition and Evaluation Department 120 Measure definition management information 121 Resource Management Information 122 Policy target management information 123 Policy Implementation Plan Management Information 124 Bias Analysis Results Management Information 130 Evaluation index calculation unit 131 Factor Analysis Department 140 Data Reception Department 141 Display section 200 calculator 201 processor 202 Main storage 203 Secondary storage device 204 Network Interface 710 Secondary Bias Analysis Information 800, 900, 1200, 1400 screen

Claims

1. A computer system including at least one computer having a processor and a storage device connected to the processor, retaining policy definition management information that stores policy definition data including allocation conditions for generating a comparison group, which is a set of elements of the policy, used to evaluate the effectiveness of the policy; The at least one computer Identifying the comparison group generated based on the policy definition data and a population of the elements that meet the allocation conditions included in the policy definition data; Calculating the similarity between the distribution of the attributes of the elements included in the comparison group and the distribution of the attributes of the elements included in the population; A computer system that determines whether or not bias of the elements in the comparison group occurs based on the similarity.

2. A computer system according to claim 1, the policy definition management information stores first policy definition data of a first policy including a first allocation condition and second policy definition data of a second policy including a second allocation condition; The at least one computer calculating a score indicating the degree of influence of a combination of the first allocation condition and the second allocation condition on the occurrence of the bias in the comparison group generated based on the first policy definition data, based on a set of elements that match the first allocation condition and a set of elements that match the first allocation condition and the second allocation condition; A computer system characterized by determining, based on the score, whether the combination of the first allocation condition and the second allocation condition is a factor that causes the bias in the comparison group generated based on the first policy definition data.

3. A computer system according to claim 1, The at least one computer When a registration request for third policy definition data of a third policy including a third allocation condition is received, the policy definition data stored in the policy definition management information is acquired; calculating a score indicating the degree of influence of a combination of the third allocation condition and the allocation condition included in the acquired policy definition management information on the occurrence of the bias in the comparison group generated based on the third policy definition data; A computer system characterized by determining, based on the score, whether the combination of the third allocation condition and the allocation condition contained in the acquired policy definition management information is a factor that causes the bias in the comparison group generated based on the third policy definition data.

4. A computer system according to claim 1, A computer system characterized in that the at least one computer acquires one policy definition data from the policy definition management information and repeatedly executes a process of selecting one element based on the allocation condition contained in the acquired policy definition data, thereby generating the comparison group of policies corresponding to the multiple policy definition data stored in the policy definition management information.

5. A method for evaluating a definition of a measure executed by a computer system, comprising: The computer system at least one computer having a processor and a storage device coupled to the processor; retaining policy definition management information that stores policy definition data including allocation conditions for generating a comparison group, which is a set of elements of the policy, used to evaluate the effectiveness of the policy; The evaluation method for the definition of the measures is as follows: a first step in which the at least one computer identifies the comparison group generated based on the policy definition data and a population of the elements that meet the allocation condition included in the policy definition data; a second step in which the at least one computer determines whether bias of the elements in the comparison group occurs based on a distribution of attributes of the elements included in the comparison group and a distribution of attributes of the elements included in the population, The second step includes: a step of calculating a similarity between a distribution of attributes of the elements included in the comparison group and a distribution of attributes of the elements included in the population by the at least one computer; and a step of determining, by the at least one computer, whether or not the bias has occurred based on the similarity.

6. A method for evaluating the definition of a measure according to claim 5, comprising: the policy definition management information stores first policy definition data of a first policy including a first allocation condition and second policy definition data of a second policy including a second allocation condition; The evaluation method for the definition of the measures is as follows: a step in which the at least one computer calculates a score indicating the degree of influence of a combination of the first allocation condition and the second allocation condition on the occurrence of the bias in the comparison group generated based on the first policy definition data, based on a set of elements that match the first allocation condition and a set of elements that match the first allocation condition and the second allocation condition; A method for evaluating a policy definition, characterized by including a step in which the at least one computer determines, based on the score, whether the combination of the first allocation condition and the second allocation condition is a factor that causes the bias in the comparison group generated based on the first policy definition data.

7. A method for evaluating the definition of a measure according to claim 5, comprising: When the at least one computer receives a request to register third policy definition data of a third policy including a third allocation condition, the at least one computer acquires the policy definition data stored in the policy definition management information; a step in which the at least one computer calculates a score indicating the degree of influence of a combination of the third allocation condition and the allocation condition included in the acquired policy definition management information on the occurrence of the bias in the comparison group generated based on the third policy definition data; A method for evaluating a policy definition, characterized by including a step in which the at least one computer determines, based on the score, whether the combination of the third allocation condition and the allocation condition included in the acquired policy definition management information is a factor that causes the bias in the comparison group generated based on the third policy definition data.

8. A method for evaluating the definition of a measure according to claim 5, comprising: A method for evaluating a policy definition, characterized in that it includes a step in which at least one computer acquires one policy definition data from the policy definition management information and repeatedly executes a process of selecting one element based on the allocation condition contained in the acquired policy definition data, thereby generating a comparison group of policies corresponding to multiple policy definition data stored in the policy definition management information.

9. A program for causing a computer having a processor and a storage device connected to the processor to execute a procedure, the program having policy definition management information storing policy definition data including allocation conditions for generating a comparison group, which is a set of elements of a policy, used to evaluate the effectiveness of the policy, the program comprising: a first step of identifying the comparison group generated based on the policy definition data and a population of the elements that meet the allocation conditions included in the policy definition data; a second step of determining whether or not bias of the elements in the comparison group has occurred based on a distribution of attributes of the elements in the comparison group and a distribution of attributes of the elements in the population, The second step comprises: a step of calculating a similarity between a distribution of attributes of the elements included in the comparison group and a distribution of attributes of the elements included in the population; and a procedure for determining whether or not the bias occurs based on the degree of similarity.

10. The program according to claim 9, the policy definition management information stores first policy definition data of a first policy including a first allocation condition and second policy definition data of a second policy including a second allocation condition; The program a step of calculating a score indicating the degree of influence of a combination of the first allocation condition and the second allocation condition on the occurrence of the bias in the comparison group generated based on the first policy definition data, based on a set of elements that match the first allocation condition and a set of elements that match the first allocation condition and the second allocation condition; and a procedure for determining, based on the score, whether the combination of the first allocation condition and the second allocation condition is a factor causing the bias in the comparison group generated based on the first policy definition data.

11. The program according to claim 9, a step of acquiring the policy definition data stored in the policy definition management information when a registration request for third policy definition data of a third policy including a third allocation condition is received; a step of calculating a score indicating the degree of influence of a combination of the third allocation condition and the allocation condition included in the acquired policy definition management information on the occurrence of the bias in the comparison group generated based on the third policy definition data; A program characterized by causing the computer to execute the following steps: based on the score, determine whether the combination of the third allocation condition and the allocation condition included in the acquired policy definition management information is a factor that causes the bias in the comparison group generated based on the third policy definition data.

12. The program according to claim 9, A program characterized by causing a computer to execute a procedure for generating a comparison group of measures corresponding to multiple policy definition data stored in the policy definition management information by repeatedly executing a process of obtaining one policy definition data from the policy definition management information and selecting one element based on the allocation condition contained in the obtained policy definition data.

Images