Information processing device, information processing method and information processing program

Abstract

To provide an information processing device, an information processing method, and an information processing program that can reduce a bias related to evaluation and improve the fairness of evaluation.SOLUTION: An information processing device according to the present application comprises an acquisition unit, a determination unit, and a calculation unit. The acquisition unit acquires evaluation information including a first evaluation value indicating the evaluator's evaluation of each of a plurality of evaluated subjects. The determination unit determines a comparative assessment between evaluated subjects for each pair of evaluated subjects based on the first evaluation values of the same evaluator for the evaluated subjects in the pair. The calculation unit calculates a second evaluation value for each of the multiple evaluated subjects based on the comparative assessments determined for each pair of evaluated subjects by the determination unit.SELECTED DRAWING: Figure 3

Description

Technical Field

[0001] The present invention relates to an information processing apparatus, an information processing method, and an information processing program.

Background Art

[0002] Bias may occur because there are variations in the evaluation content among evaluators. For example, in personnel evaluations, biases may be included in the evaluations due to variations in the number of evaluations and experience of the evaluators, the relationship between the evaluator and the evaluated person (e.g., closeness), etc., or due to variations in evaluations among departments.

[0003] Conventionally, as an example of a technology for strategic decision-making and business efficiency improvement by utilizing various information accumulated in organizations such as companies, a technology called people analytics is known. For example, technologies for supporting personnel evaluations are known.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0005] However, in the above prior art, there is room for further improvement to reduce the bias related to evaluation and improve the fairness of evaluation.

[0006] The present application has been made in view of the above, and an object thereof is to provide an information processing apparatus, an information processing method, and an information processing program capable of reducing the bias related to evaluation and improving the fairness of evaluation.

Means for Solving the Problems

[0007] The information processing apparatus according to the present application includes an acquisition unit, a determination unit, and a calculation unit. The acquisition unit acquires evaluation information including a first evaluation value indicating the evaluation of an evaluator for each of a plurality of evaluatees. The determination unit determines a relative evaluation between evaluatees for each pair of evaluatees based on the first evaluation values given by the same evaluator for pairs of evaluatees among the plurality of evaluatees. The calculation unit calculates a second evaluation value for each of the plurality of evaluatees based on the relative evaluation determined for each pair of evaluatees by the determination unit.

Effect of the Invention

[0008] According to one aspect of the embodiment, there is an effect that it is possible to reduce the bias related to evaluation and improve the fairness of evaluation.

Brief Description of the Drawings

[0009]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

MODE FOR CARRYING OUT THE INVENTION

[0010] Hereinafter, embodiments for implementing an information processing apparatus, an information processing method, and an information processing program according to the present application (hereinafter referred to as "embodiments") will be described in detail with reference to the drawings. Note that the information processing apparatus, the information processing method, and the information processing program according to the present application are not limited by this embodiment. Also, each embodiment can be appropriately combined within a range that does not conflict with the processing content. In addition, the same reference numerals are given to the same parts in the following embodiments, and duplicate explanations are omitted.

[0011] 〔1. An example of information processing〕 First, with reference to FIG. 1, an example of information processing according to the embodiment will be described. FIG. 1 is a diagram for explaining the information processing according to the embodiment.

[0012] The information processing apparatus 1 shown in FIG. 1 is an information processing apparatus for realizing various data collections and analyses, and is realized by, for example, one or more servers or a cloud system. For example, the information processing apparatus 1 is used to manage data within an organization such as a company.

[0013] As shown in FIG. 1, evaluation information is acquired from the terminal device 2 of each user U (step S1). The evaluation information includes information indicating an evaluation made by the user U as an evaluator to another user U who is the evaluated person. The user U can be either the evaluator or the evaluated person.

[0014] The information indicating the evaluation included in the evaluation information is, for example, information indicating a multi-faceted evaluation by the evaluator of the evaluated person, but is not limited to such an example. Multi-faceted evaluation is a method of evaluating the evaluated person from multiple aspects by a plurality of evaluators with different positions and relationships with the evaluated person, and is also called 360-degree evaluation or 360-degree feedback. The evaluator is, for example, the supervisor, colleague, subordinate, etc. of the evaluated person, but is not limited to such an example.

[0015] The information indicating the evaluation is the first evaluation value which is an evaluation value (score) indicating the evaluation of the evaluated person determined by the evaluator. The first evaluation value is, for example, the comprehensive evaluation value or the evaluation value for each evaluation item, but is not limited to such examples. When the information indicating the evaluation is the evaluation value for each evaluation item, the information processing apparatus 1 can also calculate the average value or the weighted average value of the evaluation values for each evaluation item as the first evaluation value.

[0016] Note that the information processing apparatus 1 can also acquire the information indicating the response regarding the evaluation of the evaluated person by the evaluator, and based on such information, calculate the first evaluation value to acquire the first evaluation value.

[0017] Subsequently, the information processing apparatus 1 specifies, for each evaluator, the first evaluation values for a plurality of evaluated persons evaluated by the same evaluator based on the evaluation information acquired from each terminal device 2 in step S1 (step S2).

[0018] For example, in the example shown in FIG. 1(a), for the sake of convenience of explanation, six users U among the plurality of users U are regarded as users UA, UB, UC, UD, UE, UF, and the first evaluation values of these users UA, UB, UC, UD, UE, UF are shown numerically. In FIG. 1(a), the user U on the starting point side of the arrow indicates that the user is an evaluator, the user U on the end point side of the arrow indicates that the user is an evaluated person, and the first evaluation value is an evaluation value with a full score of 4 points.

[0019] In the example shown in FIG. 1(a), user UA evaluates users UB, UC, UD, UE as an evaluator, user UB evaluates user UA as an evaluator, and user UC evaluates users UA, UB as an evaluator. Also, in the example shown in FIG. 1(a), user UD evaluates users UA, UE, UF as an evaluator, user UE evaluates user UF as an evaluator, and user UF evaluates users UD, UE as an evaluator.

[0020] In this case, in the process of step S2, the information processing apparatus 1 identifies the first evaluation values for a plurality of users UB, UC, UD, UE evaluated by the user UA, and identifies the first evaluation values for a plurality of users UA, UB evaluated by the user UC. Further, the information processing apparatus 1 identifies the first evaluation values for a plurality of users UA, UE, UF evaluated by the user UD, and identifies the first evaluation values for a plurality of users UD, UE evaluated by the user UF.

[0021] Subsequently, the information processing apparatus 1 performs, for each evaluator, a process of comparing the first evaluation values of two evaluated users for each pair of evaluated users with different combinations among the plurality of evaluated users evaluated by the same evaluator (step S3).

[0022] For example, in the example shown in FIG. 1(a), the pairs of evaluated users with different combinations among the plurality of users UB, UC, UD, UE evaluated by the user UA are the pair of users UB, UC, the pair of users UB, UD, the pair of users UB, UE, the pair of users UC, UD, the pair of users UC, UE, and the pair of users UD, UE, a total of six pairs.

[0023] In this case, as shown in FIG. 1(b), the information processing apparatus 1 compares the first evaluation values of the two evaluated users included in each of the six pairs. The information processing apparatus 1 compares the first evaluation values of the two evaluated users, and for each pair, performs a process of determining which of the first evaluation values of the two evaluated users is higher or the same.

[0024] Subsequently, the information processing apparatus 1 performs, for each evaluator, a process of determining the relative evaluation between evaluated users for each pair of evaluated users based on the comparison results in step S3 (step S4). In step S4, for example, when the first evaluation value of either of the two evaluated users constituting the pair is higher, the relative evaluation of the evaluated user with the higher first evaluation value is set to 1 win, the relative evaluation of the evaluated user with the lower first evaluation value is set to 0 win, and when the first evaluation values of the two evaluated users constituting the pair are the same, the relative evaluations of the two evaluated users are set to 0.5 win.

[0025] In the example shown in FIG. 1(b), the first evaluation value of user UB is lower than the first evaluation values of users UC, UD, and UE. The first evaluation value of user UC is higher than the first evaluation values of users UD and UE. The first evaluation value of user UD is the same as the first evaluation value of user UE.

[0026] Therefore, as shown in FIG. 1(c), information processing apparatus 1 sets the relative evaluations of users UC, UD, and UE to 1 win in relation to user UB, sets the relative evaluation of user UC to 1 win in relation to users UD and UE, and sets the relative evaluations of users UD and UE to 0.5 win in relation to each other.

[0027] Information processing apparatus 1 performs such processing for each evaluator who has evaluated a plurality of evaluatees. Specifically, in the example shown in FIG. 1(a), information processing apparatus 1 determines the relative evaluations between users UA and UB when user UC is the evaluator, determines the relative evaluations of pairs of users U with different combinations among users UA, UE, and UF when user UD is the evaluator, and determines the relative evaluations between users UD and UE when user UF is the evaluator.

[0028] Subsequently, information processing apparatus 1 aggregates the relative evaluations for each pair of evaluatees (step S5). In step S5, information processing apparatus 1 aggregates, for example, the number of wins, which is the total relative evaluation of each evaluatee for each pair of evaluatees.

[0029] In the example shown in FIG. 1(d), for example, in the pair of user UB and user UC, the total relative evaluation of user UB is 2 wins and the total relative evaluation of user UC is 4 wins. In the pair of user UB and user UD, the total relative evaluation of user UB is 1 win and the total relative evaluation of user UD is 5 wins.

[0030] Also, in the pair of user UB and user UE, the total relative evaluation of user UB is 3 wins and the total relative evaluation of user UE is 2 wins. In the pair of user UC and user UD, the total relative evaluation of user UC is 2 wins and the total relative evaluation of user UD is 0 wins.

[0031] Also, in the pair of user UC and user UE, the total relative evaluation of user UC is 5 wins and the total relative evaluation of user UE is 3 wins. In the pair of user UD and user UE, the total relative evaluation of user UD is 2 wins and the total relative evaluation of user UE is 4 wins. Also, in the pair of user UA and user UE, the total relative evaluation of user UA is 1 win and the total relative evaluation of user UE is 3 wins.

[0032] Subsequently, the information processing apparatus 1 calculates a second evaluation value, which is the evaluation score of each evaluated person, using a static rating algorithm from the aggregation result of step S5 (step S6). The static rating algorithm is also called the Bradley-Terry model.

[0033] In the Bradley-Terry model, in the pair of evaluated person i and evaluated person j, the probability p ij = p(i>j) that evaluated person i has a better first evaluation than evaluated person j is represented by the following formula (1). In the following formula (1), θ i is a numerical value representing the "height (strength) of evaluation" of evaluated person i. Note that i ≠ j, i = 1, 2, ···, n, and j = 1, 2, ···, n. n is the total number of evaluated persons.

[0034]

Equation

[0035] The maximum likelihood solution θ i ^ can be represented by the following formula (2). Note that θ i ^ has meaning only in terms of the ratio to each other, so it is defined by the following formula (3) to eliminate the indeterminacy.

[0036] [Number]

[0037] The information processing apparatus 1 calculates the maximum likelihood solution θ i ^, for example, using an iterative method. For example, the information processing apparatus 1 sets an initial value of θ i for the evaluated person i, and then executes the process of updating θ i for each evaluated person i until the process converges, thereby obtaining the maximum likelihood solution θ i ^(i = 1, 2, ···, n). Then, the information processing apparatus 1 calculates the second evaluation value of each evaluated person by scaling the maximum likelihood solution θ i ^(i = 1, 2, ···, n).

[0038] In step S1, the information processing apparatus 1 can also acquire the evaluation information for a plurality of periods T1, ···, Tm. m is an integer of 2 or more. The period T1 is the newest period, the period Tm is the oldest period, and the periods become older from the period T1 to the period Tm. Hereinafter, when each of the plurality of periods T1, ···, Tm is not individually distinguished and shown, it may be described as the period T.

[0039] In this case, the information processing apparatus 1 determines the relative evaluation between the evaluated persons for each pair of evaluated persons for each of the periods T1, ···, Tm, and calculates the second evaluation value of each of the plurality of evaluated persons based on the result obtained by weighted addition of the relative evaluations for each of the periods T1, ···, Tm with weights such that the relative evaluation in the newer period has a larger weight. Thereby, the information processing apparatus 1 can appropriately reflect the relative evaluation in the latest period in the second evaluation value while utilizing the relative evaluations in the past periods.

[0040] For example, when m = 4, assume that the weight of the period T1 is represented by the weight ω1, the weight of the period T2 is represented by the weight ω2, the weight of the period T3 is represented by the weight ω3, and the weight of the period T4 is represented by the weight ω4. In this case, the weights ω1, ω2, ω3, ω4 are, for example, ω1 = 1, ω2 = 0.8, ω3 = 0.64 (= 0.8 2 ), ω4 = 0.512 (= 0.83 ) is an exponential weight, but is not limited to such an example.

[0041] For example, in the pair of the evaluated person i and the evaluated person j, the ratio that the evaluated person i is superior to the evaluated person j is R ij (T1) at the time of the period T1, and R ij (T2) at the time of the period T2, and R ij (T3) at the time of the period T3, and R ij (T4) at the time of the period T4.

[0042] In this case, the information processing apparatus 1 calculates, by weighted addition represented by the arithmetic expression of R ij which is the ratio that the evaluated person i is superior to the evaluated person j, as ij R = ω1 × R ij (T1) + ω2 × R ij (T2) + ω3 × R ij (T3) + ω4 × R ij (T4).

[0043] Also, the information processing apparatus 1 can calculate the second evaluation value of each evaluated person for each period T based on the relative evaluation determined for each pair of evaluated persons, and calculate, as the second evaluation value of each evaluated person, the result obtained by weighted addition of the second evaluation value for each period T with a weight such that the second evaluation value for a new period T is larger. Thereby, the information processing apparatus 1 can perform the evaluation for each period T by calculating the second evaluation value for each period T, and can appropriately reflect the relative evaluation in the latest period on the second evaluation value while utilizing the relative evaluation in the past periods.

[0044] For example, when m = 4, it is assumed that the weight of the period T1 is represented by the weight ω1, the weight of the period T2 is represented by the weight ω2, the weight of the period T3 is represented by the weight ω3, and the weight of the period T4 is represented by the weight ω4. The weights ω1, ω2, ω3, ω4 in this case are also exponential weights as described above, but are not limited to such an example. Also, the second evaluation value of the evaluated person i is v2 i (T1) at the time of the period T1, and v2 iIt is (T2), and it is v2 at time period T3 i It is (T3), and it is v2 at time period T4 i Suppose it is (T4).

[0045] In this case, the information processing apparatus 1 calculates the second evaluation value v2 of the evaluated person i i as v2 i = ω1 × v2 i (T1) + ω2 × v2 i (T2) + ω3 × v2 i (T3) + ω4 × v2 i by weighted addition represented by the arithmetic expression of (T4).

[0046] Also, when determining the relative evaluation among the evaluated persons in step S4, the information processing apparatus 1 can also determine the relative evaluation according to the difference in the first evaluation values between two evaluated persons who are a pair as the relative evaluation among the evaluated persons. Thereby, the information processing apparatus 1 can calculate, for example, the second evaluation value of the evaluated person more appropriately.

[0047] For example, in a pair of the evaluated person i and the evaluated person j, let the first evaluation value of the evaluated person i be v1 i and the first evaluation value of the evaluated person j be v1 j (<v1 i ). When this is the case, the difference between the first evaluation value v1 i of the evaluated person i and the first evaluation value v1 j of the evaluated person j is Δv1 ij (= v1 i - v1 j ). Suppose it is so.

[0048] In this case, the information processing apparatus 1 determines the relative evaluation according to the difference Δv1 ij as the relative evaluation between the evaluated person i and the evaluated person j. For example, when the difference Δv1 ij in the first evaluation values between the evaluated person i and the evaluated person j is equal to or greater than the threshold value, the information processing apparatus 1 determines the relative evaluation according to the difference Δv1 ij in the first evaluation values between the evaluated person i and the evaluated person j as the relative evaluation between the evaluated person i and the evaluated person j.

[0049] For example, when the difference Δv1 in the first evaluation value between the evaluated person i and the evaluated person j ij is less than the threshold value, the relative evaluation of the evaluated person i is set to 1win and the relative evaluation of the evaluated person j is set to 0win, and the difference Δv1 ij is greater than or equal to the threshold value, the relative evaluation of the evaluated person i is set to k1×1win and the relative evaluation of the evaluated person j is set to 0win. k1 is a value greater than 1.

[0050] Also, the information processing apparatus 1 can also set the relative evaluation of the evaluated person i to k2×1win and the relative evaluation of the evaluated person j to 0win. The coefficient k2 is represented by, for example, k2 = Δv1 ij ×k3, etc., but is not limited to such an example. The coefficient k3 is set so that, for example, k2 becomes a value of 1 or more.

[0051] Also, the information processing apparatus 1 can determine a dummy relative evaluation that is the relative evaluation between the evaluated person and a virtual evaluated person, and calculate the second evaluation value of each of the plurality of evaluated persons based on the relative evaluation including the dummy relative evaluation. Thereby, the information processing apparatus 1 can, for example, calculate the second evaluation value of the evaluated person more appropriately.

[0052] For example, the information processing apparatus 1 sets the relative evaluation between the virtual evaluated person and each evaluated person to an average relative evaluation. For example, the relative evaluation between the virtual evaluated person and each evaluated person is the average value or the median value of the relative evaluations of the evaluated persons, and the ratio of the virtual evaluated person being superior in evaluation compared to the evaluated persons is the average value of the ratios of each evaluated person being superior in evaluation compared to other evaluated persons.

[0053] For example, when the information processing apparatus 1 sets the case where the first evaluation is excellent as a win and the case where the second evaluation is excellent as a loss, as the relative evaluation between the virtual evaluated person and each evaluated person, for example, the win-loss can be set to p wins and p losses. p is a natural number.

[0054] In this way, the information processing apparatus 1 acquires evaluation information including first evaluation values indicating the evaluations of evaluators for each of a plurality of evaluatees, and determines the relative evaluations between evaluatees for each pair of evaluatees based on the first evaluation values indicating the evaluations of the same evaluator for pairs of evaluatees among the plurality of evaluatees. Then, the information processing apparatus 1 calculates second evaluation values for each of the plurality of evaluatees based on the relative evaluations determined for each pair of evaluatees. Thereby, the information processing apparatus 1 can reduce the bias involved in the evaluation and improve the fairness of the evaluation.

[0055] Hereinafter, the configuration of the information processing system including the information processing apparatus 1 that performs such processing, a plurality of terminal devices 2, and the terminal device 3 will be described in detail.

[0056] [2. Configuration of Information Processing System] FIG. 2 is a diagram showing an example of the configuration of the information processing system according to the embodiment. As shown in FIG. 2, the information processing system 100 according to the embodiment includes an information processing apparatus 1, a plurality of terminal devices 2, and a terminal device 3.

[0057] The plurality of terminal devices 2 are used by different users U. The terminal device 3 is, for example, the terminal device of a personnel officer O in the personnel department of a company or organization to which a plurality of users U belong. The terminal devices 2 and 3 are, for example, notebook PCs (Personal Computers), desktop PCs, smartphones, tablet PCs, or wearable devices. The wearable device is, for example, smart glasses or a smartwatch, but is not limited to such examples.

[0058] Each of the information processing apparatus 1, the terminal device 2, and the terminal device 3 is communicably connected to each other by wire or wirelessly via the network N. Note that the information processing system 100 shown in FIG. 2 may include a plurality of information processing apparatuses 1 and the like.

[0059] The network N includes, for example, a WAN (Wide Area Network) such as the Internet, and a mobile communication network such as LTE (Long Term Evolution), 4G (4th Generation), and 5G (5th Generation: the 5th generation mobile communication system).

[0060] The terminal devices 2 and 3 can be connected to the network N via a mobile communication network, short-range wireless communication such as Bluetooth (registered trademark), or a wireless LAN (Local Area Network), and communicate with the information processing device 1.

[0061] [3. Configuration of the information processing device 1] FIG. 3 is a diagram showing an example of the configuration of the information processing device 1 according to the embodiment. As shown in FIG. 3, the information processing device 1 includes a communication unit 10, a storage unit 11, and a processing unit 12.

[0062] [3.1. Communication unit 10] The communication unit 10 is realized, for example, by a communication module or a NIC (Network Interface Card). The communication unit 10 is connected to the network N by wire or wirelessly, and transmits and receives information to and from other various devices. For example, the communication unit 10 transmits and receives information to and from each of the terminal device 2, the terminal device 3, and the information processing device 4 via the network N.

[0063] [3.2. Storage unit 11] The storage unit 11 is realized, for example, by a semiconductor memory element such as a RAM (Random Access Memory) or a flash memory, or a storage device such as a hard disk or an optical disk. The storage unit 11 includes a user information storage unit 20 and an evaluation result storage unit 21.

[0064] [3.2.1. User information storage unit 20] The user information storage unit 20 stores user information including information about the user U. FIG. 4 is a diagram showing an example of a user information table stored in the user information storage unit 20 of the information processing apparatus 1 according to the embodiment. As shown in FIG. 4, the user information table stored in the user information storage unit 20 includes items such as "user ID", "attribute information", "affiliation information", and "evaluation information".

[0065] The "user ID" is identification information for identifying the user U. The "attribute information" is attribute information of the user U corresponding to the "user ID", and includes, for example, information on psychographic attributes and information on demographic attributes. Demographic attributes are, for example, gender, age, place of residence, and occupation, and psychographic attributes are objects of interest such as travel, clothing, cars, religion, lifestyle, thoughts, and trends of thoughts.

[0066] The "affiliation information" is information regarding the department to which the user U corresponding to the "user ID" belongs, and includes, for example, information indicating the department, the user ID of the supervisor, the user ID of colleagues, and the user ID of subordinates.

[0067] The "evaluation information" is information regarding the evaluation by the evaluator of the user U associated with the "user ID", and includes, for example, a second evaluation value, but may also include information indicating the evaluation of another user U as the evaluator and information indicating the evaluation performed by the evaluator as the evaluated person.

[0068] 〔3.2.2. Evaluation result storage unit 21〕 The evaluation result storage unit 21 stores information regarding the evaluation of the evaluated person by the evaluator. FIG. 5 is a diagram showing an example of an evaluation result table stored in the evaluation result storage unit 21 of the information processing apparatus 1 according to the embodiment. In the evaluation result storage unit 21, an evaluation result table for each period T is stored.

[0069] As shown in FIG. 5, the evaluation result table stored in the evaluation result storage unit 21 includes items such as "evaluation result ID", "evaluator ID", "evaluated person ID", and "evaluation result". The "evaluation result ID" is identification information for identifying the evaluation result of the evaluator for the evaluated person.

[0070] The "evaluator ID" is the user ID of the user U (evaluator) who performed the evaluation corresponding to the "evaluation result ID", and the "evaluated person ID" is the user ID of the user U (evaluated person) for whom the evaluation corresponding to the "evaluation result ID" was performed.

[0071] The "evaluation result" is the evaluation result corresponding to the "evaluation result ID", for example, it is the first evaluation value which is the evaluation value (score) for the evaluated person. For example, the first evaluation value is the comprehensive evaluation value or the evaluation value for each evaluation item, but is not limited to such examples.

[0072] [3.3. Processing Unit 12] The processing unit 12 is a controller, and is realized, for example, by various programs (corresponding to an example of an information processing program) stored in the storage device inside the information processing apparatus 1 being executed with a RAM or the like as a work area by a processor such as a CPU (Central Processing Unit) or an MPU (Micro Processing Unit).

[0073] Also, the processing unit 12 is a controller and may be realized by an integrated circuit such as an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or a GPGPU (General Purpose Graphic Processing Unit).

[0074] As shown in FIG. 3, the processing unit 12 includes an acquisition unit 30, a reception unit 31, a determination unit 32, a calculation unit 33, and a provision unit 34, and realizes or executes the information processing functions and operations described below. Note that the internal configuration of the processing unit 12 is not limited to the configuration shown in FIG. 3, and other configurations may be used as long as they can perform the information processing described later.

[0075] [3.3.1. Acquisition Unit 30] The acquisition unit 30 acquires various information. For example, the acquisition unit 30 acquires various information from the terminal devices 2 and 3 via the network N and the communication unit 10.

[0076] For example, the acquisition unit 30 acquires evaluation information from the terminal device 2 of each user U, and adds the acquired evaluation information or information based on the evaluation information to the evaluation result table stored in the evaluation result storage unit 21 as an evaluation result.

[0077] The evaluation information includes information indicating an evaluation performed by the user U as an evaluator on another user U as an evaluatee. The user U can be either an evaluator or an evaluatee. The information indicating the evaluation included in the evaluation information is, for example, information indicating a comprehensive evaluation by the evaluator on the evaluatee, but is not limited to such an example.

[0078] The comprehensive evaluation is a method of comprehensively evaluating an evaluatee by a plurality of evaluators with different positions and relationships with the evaluatee, and is also called 360-degree evaluation or 360-degree feedback. The evaluators are, for example, the supervisor, colleagues, subordinates, etc. of the evaluatee, but are not limited to such examples.

[0079] The information indicating the evaluation is a first evaluation value that is an evaluation value (score) indicating the evaluation of the evaluatee determined by the evaluator. The first evaluation value is, for example, a comprehensive evaluation value by comprehensive evaluation or an evaluation value for each evaluation item, but is not limited to such examples. The calculation unit 33 described later can also calculate the average value or weighted average value of the evaluation values for each evaluation item as the first evaluation value when the information indicating the evaluation is the evaluation value for each evaluation item.

[0080] Note that the acquisition unit 30 can also acquire the first evaluation value by acquiring information indicating an answer regarding the evaluation of the evaluated person by the evaluator from the terminal device 2 and calculating the first evaluation value based on such information. Note that the calculation of the first evaluation value from the information indicating the answer regarding the evaluation of the evaluated person by the evaluator can also be performed by the calculation unit 33.

[0081] For example, when a first evaluation value processing request is received by the reception unit 31, the acquisition unit 30 acquires evaluation information for a plurality of periods T including the first evaluation value indicating the evaluation of the evaluated person by the evaluator for each of the plurality of evaluated persons from the evaluation result table for each period T stored in the evaluation result storage unit 21. Further, the acquisition unit 30 can also acquire the evaluation information for each period T or the latest evaluation information from the evaluation result storage unit 21.

[0082] 〔3.3.2. Reception Unit 31〕 The reception unit 31 receives various requests and information. For example, the reception unit 31 receives a first evaluation value processing request transmitted from the terminal device 3.

[0083] 〔3.3.3. Decision Unit 32〕 The decision unit 32 makes various decisions. For example, the decision unit 32 specifies the first evaluation value indicating the evaluation of the same evaluator for a pair of evaluated persons among the plurality of evaluated persons from the evaluation information acquired by the acquisition unit 30, and determines the relative evaluation between the evaluated persons for each pair of evaluated persons based on the first evaluation value indicating the evaluation of the same evaluator for a pair of evaluated persons among the plurality of evaluated persons. The decision unit 32 includes a specification processing unit 40, a comparison processing unit 41, and a decision processing unit 42.

[0084] For example, based on the evaluation information for each period T acquired by the acquisition unit 30, the specification processing unit 40 performs processing for specifying, for each period T, the first evaluation value for a plurality of evaluated persons evaluated by the same evaluator in units of evaluators.

[0085] FIG. 6 is a diagram showing an example of an evaluation relationship among users U indicated by evaluation information acquired by an acquisition unit 30 in a processing unit 12 of the information processing apparatus 1 according to the embodiment. FIG. 7 is a diagram showing an example of a first evaluation value for each evaluator specified by a specifying processing unit 40 in the processing unit 12 of the information processing apparatus 1 according to the embodiment.

[0086] In the example shown in FIG. 6, for the sake of convenience of explanation, six users U among a plurality of users U are regarded as users UA, UB, UC, UD, UE, and UF, and the first evaluation values of these users UA, UB, UC, UD, UE, and UF are indicated by numbers. In FIG. 6, the user U on the starting point side of the arrow indicates the evaluator, the user U on the end point side of the arrow indicates the evaluated person, and the first evaluation value is an evaluation value with a full score of 4 points.

[0087] In the example shown in FIG. 6, user UA evaluates users UB, UC, UD, and UE as an evaluator, user UB evaluates user UA as an evaluator, and user UC evaluates users UA and UB as an evaluator. Also, in the example shown in FIG. 6, user UD evaluates users UA, UE, and UF as an evaluator, user UE evaluates user UF as an evaluator, and user UF evaluates users UD and UE as an evaluator.

[0088] In this case, the specifying processing unit 40 specifies the first evaluation values for the plurality of users UB, UC, UD, and UE evaluated by user UA, and specifies the first evaluation values for the plurality of users UA and UB evaluated by user UC. Also, the specifying processing unit 40 specifies the first evaluation values for the plurality of users UA, UE, and UF evaluated by user UD, and specifies the first evaluation values for the plurality of users UD and UE evaluated by user UF.

[0089] Based on the specifying result by the specifying processing unit 40, the comparison processing unit 41 performs, for each pair of evaluated persons with different combinations among the plurality of evaluated persons evaluated by the same evaluator, a process of comparing the first evaluation values of the two evaluated persons in units of evaluators.

[0090] For example, in the example shown in FIG. 6, among the plurality of users UB, UC, UD, and UE evaluated by the user UA, the pairs of evaluated persons in different combinations from each other are the pair of users UB and UC, the pair of users UB and UD, the pair of users UB and UE, the pair of users UC and UD, the pair of users UC and UE, and the pair of users UD and UE, a total of six pairs.

[0091] In this case, as shown in FIG. 7, the comparison processing unit 41 compares the first evaluation values of the two evaluated persons included in each of the six pairs. The comparison processing unit 41 compares the first evaluation values of the two evaluated persons, and for each pair, performs a process of determining which of the two evaluated persons has a higher first evaluation value or whether they are the same.

[0092] Based on the comparison results by the comparison processing unit 41, the determination processing unit 42 performs a process of determining the relative evaluation between the evaluated persons for each pair of evaluated persons in units of evaluators. For example, when the first evaluation value of either one of the two evaluated persons constituting the pair is higher, the determination processing unit 42 sets the relative evaluation of the evaluated person with the higher first evaluation value to 1 win and the relative evaluation of the evaluated person with the lower first evaluation value to 0 win. When the first evaluation values of the two evaluated persons constituting the pair are the same, the determination processing unit 42 sets the relative evaluation of the two evaluated persons to 0.5 win.

[0093] FIG. 8 is a diagram showing an example of the relative evaluation between the evaluated persons determined by the determination processing unit 42 in the processing unit 12 of the information processing apparatus 1 according to the embodiment.

[0094] In the example shown in FIG. 6 described above, the first evaluation value of the user UB is lower than the first evaluation values of the users UC, UD, and UE. The first evaluation value of the user UC is higher than the first evaluation values of the users UD and UE. The first evaluation value of the user UD is the same as the first evaluation value of the user UE.

[0095] Therefore, as shown in FIG. 8, the determination processing unit 42 sets the relative evaluations of the users UC, UD, and UE to 1 win in relation to the user UB, sets the relative evaluation of the user UC to 1 win in relation to the users UD and UE, and sets the relative evaluations of the users UD and UE to 0.5 win in the relationship between the users UD and UE.

[0096] The determination processing unit 42 performs such processing in units of evaluators who have evaluated a plurality of evaluatees. Specifically, in the example shown in FIG. 6 described above, when the user UC is the evaluator, the determination processing unit 42 determines the relative evaluation between the users UA and UB, and when the user UD is the evaluator, it determines the relative evaluation of pairs of users U with different combinations among the users UA, UE, and UF, and when the user UF is the evaluator, it determines the relative evaluation between the users UD and UE.

[0097] When the acquisition unit 30 acquires the evaluation information for a plurality of periods T1, ···, Tm, the determination unit 32 can determine the relative evaluation between the evaluatees for each pair of evaluatees based on the evaluation information for the plurality of periods T1, ···, Tm. m is an integer of 2 or more. The period T1 is the newest period, the period Tm is the oldest period, and the periods become older from the period T1 toward the period Tm.

[0098] In this case, the determination processing unit 42 determines the relative evaluation between the evaluatees for each pair of evaluatees for each of the periods T1, ···, Tm, and calculates the second evaluation value for each of the plurality of evaluatees based on the result obtained by weighted addition of the relative evaluations for each of the periods T1, ···, Tm with weights that are larger for the relative evaluations in the newer periods. Thereby, the calculation unit 33 described later can appropriately reflect the relative evaluation in the latest period in the second evaluation value while utilizing the relative evaluations in the past periods.

[0099] For example, when m = 4, assume that the weight of the period T1 is represented by the weight ω1, the weight of the period T2 is represented by the weight ω2, the weight of the period T3 is represented by the weight ω3, and the weight of the period T4 is represented by the weight ω4. In this case, the weights ω1, ω2, ω3, ω4 are, for example, ω1 = 1, ω2 = 0.8, ω3 = 0.64 (= 0.8 2 )、ω4 = 0.512(= 0.8 3 )), which are exponential weights, but are not limited to such examples.

[0100] For example, in the pair of the evaluated person i and the evaluated person j, the ratio by which the evaluated person i is superior to the evaluated person j is R ij (T1) at the time of period T1, and is R ij (T2) at the time of period T2, and is R ij (T3) at the time of period T3, and is R ij (T4) at the time of period T4.

[0101] In this case, the determination processing unit 42 calculates R ij , which is the ratio by which the evaluated person i is superior to the evaluated person j, ij =ω1×R ij (T1)+ω2×R ij (T2)+ω3×R ij (T3)+ω4×R ij (T4) by weighted addition represented by the arithmetic expression.

[0102] Also, when determining the relative evaluation between the evaluated persons, the determination processing unit 42 can determine the relative evaluation according to the difference in the first evaluation values between the two paired evaluated persons as the relative evaluation between the evaluated persons. Thereby, for example, the calculation unit 33 described later can calculate the second evaluation value of the evaluated person more appropriately.

[0103] For example, in the pair of the evaluated person i and the evaluated person j, let the first evaluation value of the evaluated person i be v1 i , and the first evaluation value of the evaluated person j be v1 j (<v1 i ). In this case, the difference between the first evaluation value v1 i of the evaluated person i and the first evaluation value v1 j of the evaluated person j is Δv1 ij (=v1 i -v1 j ).

[0104] In this case, the determination processing unit 42 determines the relative evaluation according to the difference Δv1 ij as the relative evaluation between the evaluated person i and the evaluated person j. For example, the determination processing unit 42 determines the relative evaluation according to the difference Δv1 ijWhen it is equal to or greater than the threshold value, the difference Δv1 in the first evaluation value between the evaluated person i and the evaluated person j ij Determine the relative evaluation according to the relative evaluation between the evaluated person i and the evaluated person j as the relative evaluation between the evaluated person i and the evaluated person j.

[0105] For example, the determination processing unit 42 calculates the difference Δv1 in the first evaluation value between the evaluated person i and the evaluated person j ij When it is less than the threshold value, set the relative evaluation of the evaluated person i to 1win and the relative evaluation of the evaluated person j to 0win, and the difference Δv1 ij When it is equal to or greater than the threshold value, set the relative evaluation of the evaluated person i to k1×1win and the relative evaluation of the evaluated person j to 0win. k1 is a value greater than 1.

[0106] In addition, the determination processing unit 42 can also set the relative evaluation of the evaluated person i to k2×1win and the relative evaluation of the evaluated person j to 0win. The coefficient k2 is represented by, for example, k2 = Δv1 ij ×k3, etc., but is not limited to such examples. The coefficient k3 is set so that, for example, k2 becomes a value of 1 or more.

[0107] In addition, the determination processing unit 42 can also determine the dummy relative evaluation, which is the relative evaluation between the evaluated person and the virtual evaluated person. For example, the determination processing unit 42 sets the relative evaluation between the virtual evaluated person and each evaluated person to the average relative evaluation. For example, the relative evaluation between the virtual evaluated person and each evaluated person is the average value or the median value of the relative evaluations of the evaluated persons, and the ratio of the virtual evaluated person being superior in evaluation compared to the evaluated persons is the average value of the ratios of each evaluated person being superior in evaluation compared to other evaluated persons.

[0108] For example, when the determination processing unit 42 regards the case where the first evaluation is excellent as a win and the case where the second evaluation is excellent as a loss, as the relative evaluation between the virtual evaluated person and each evaluated person, for example, the win-loss can be set to p wins and p losses. p is a natural number.

[0109] 〔3.3.4. Calculation Unit 33〕 The calculation unit 33 calculates a second evaluation value for each of the plurality of evaluatees based on the relative evaluation determined for each pair of evaluatees by the determination unit 32. For example, the calculation unit 33 totals the win counts, which are the total relative evaluations of each evaluatee for each pair of evaluatees, and calculates the second evaluation value for each of the plurality of evaluatees based on the total result.

[0110] FIG. 9 is a diagram showing an example of the total relative evaluation of each evaluatee for each pair of evaluatees aggregated by the calculation unit 33 in the processing unit 12 of the information processing apparatus 1 according to the embodiment. In the example shown in FIG. 9, for example, in the pair of user UB and user UC, the total relative evaluation of user UB is 2 wins and the total relative evaluation of user UC is 4 wins, and in the pair of user UB and user UD, the total relative evaluation of user UB is 1 win and the total relative evaluation of user UD is 5 wins.

[0111] Also, in the pair of user UB and user UE, the total relative evaluation of user UB is 3 wins and the total relative evaluation of user UE is 2 wins, and in the pair of user UC and user UD, the total relative evaluation of user UC is 2 wins and the total relative evaluation of user UD is 0 wins.

[0112] Also, in the pair of user UC and user UE, the total relative evaluation of user UC is 5 wins and the total relative evaluation of user UE is 3 wins, and in the pair of user UD and user UE, the total relative evaluation of user UD is 2 wins and the total relative evaluation of user UE is 4 wins. Also, in the pair of user UA and user UE, the total relative evaluation of user UA is 1 win and the total relative evaluation of user UE is 3 wins.

[0113] The calculation unit 33 calculates a second evaluation value, which is an evaluation score for each evaluatee, using a static rating algorithm from the total result of the total relative evaluation of each evaluatee for each pair of evaluatees. The static rating algorithm is also called the Bradley-Terry model.

[0114] In the Bradley-Terry model, for a pair of an evaluated person i and an evaluated person j, the probability p that the evaluated person i is superior in the first evaluation to the evaluated person j (the probability of winning) ij = p(i > j) is represented by the above formula (1). In the above formula (1), θ i is a numerical value representing the "height (strength) of evaluation" of the evaluated person i. The maximum likelihood solution θ i ^ can be represented by the above formula (2).

[0115] The calculation unit 33 calculates the maximum likelihood solution θ i ^, for example, using an iterative method. For example, the calculation unit 33 sets an initial value of θ i for the evaluated person i, and then executes a process of updating θ i for each evaluated person i until the process converges, thereby obtaining the maximum likelihood solution θ i ^(i = 1, 2, ···, n).

[0116] Then, the calculation unit 33 calculates the second evaluation value of each evaluated person by scaling the maximum likelihood solution θ i ^(i = 1, 2, ···, n). Note that the calculation unit 33 can also calculate the maximum likelihood solution θ i ^(i = 1, 2, ···, n) as the second evaluation value of the evaluated person.

[0117] Also, the calculation unit 33 calculates the second evaluation value of each evaluated person for each period T based on the relative evaluation determined by the determination unit 32 for each pair of evaluated persons, and calculates as the second evaluation value of each evaluated person the result obtained by weighted addition with a weight such that the second evaluation value for each period T is larger as the second evaluation value for the new period T. Thereby, the calculation unit 33 can perform the evaluation for each period T by calculating the second evaluation value for each period T, and can appropriately reflect the relative evaluation in the latest period on the second evaluation value while utilizing the relative evaluation in the past periods.

[0118] For example, when m = 4, assume that the weight of period T1 is represented by weight ω1, the weight of period T2 is represented by weight ω2, the weight of period T3 is represented by weight ω3, and the weight of period T4 is represented by weight ω4. The weights ω1, ω2, ω3, ω4 in this case are also exponential weights as described above, but are not limited to such examples. Also, the second evaluation value of the evaluated person i is v2 i (T1) at time T1, v2 i (T2) at time T2, v2 i (T3) at time T3, and v2 i (T4) at time T4.

[0119] In this case, the calculation unit 33 calculates the second evaluation value v2 i of the evaluated person i by weighted addition represented by the arithmetic expression v2 i = ω1 × v2 i (T1) + ω2 × v2 i (T2) + ω3 × v2 i (T3) + ω4 × v2 i (T4).

[0120] Also, when the dummy relative evaluation is determined by the determination unit 32, the calculation unit 33 can also calculate the second evaluation value of each of the plurality of evaluated persons based on the relative evaluation including the dummy relative evaluation. Thereby, the calculation unit 33 can calculate the second evaluation value of the evaluated person more appropriately.

[0121] Here, the second evaluation value calculated by the calculation unit 33 will be described. FIG. 10 is a diagram showing the relationship between the average value of the first evaluation value for each evaluator shown in the evaluation information acquired by the acquisition unit 30 in the processing unit 12 of the information processing apparatus 1 according to the embodiment and the number of evaluators. FIG. 11 is a diagram showing the distribution of the first evaluation value shown in the evaluation information acquired by the acquisition unit 30 in the processing unit 12 of the information processing apparatus 1 according to the embodiment.

[0122] As shown in Fig. 10, the average first evaluation value, which is the average value of the first evaluation values for each evaluator, varies greatly, indicating that the first evaluation value for the evaluated person by the evaluator's perception of sweetness or spiciness can change. Also, as shown in Fig. 11, when the first evaluation value is represented on a 4-point scale, 4 points are often given as the first evaluation value, making it difficult to distinguish between outstandingly excellent and fairly excellent, and it can be seen that the evaluations given to excellent talents cannot be sufficiently distinguished.

[0123] Fig. 12 is a diagram showing an example for explaining the relationship between the first evaluation value and the second evaluation value calculated by the calculation unit 33 in the processing unit 12 of the information processing apparatus 1 according to the embodiment, and the average first evaluation value. Fig. 13 is a diagram showing another example for explaining the relationship between the second evaluation value calculated by the calculation unit 33 in the processing unit 12 of the information processing apparatus 1 according to the embodiment and the average first evaluation value.

[0124] As shown in Fig. 12(a), a strong correlation is seen in the relationship between the average first evaluation value of the evaluator and the first evaluation value. That is, there is a tendency for a higher first evaluation value to be obtained when evaluated by a lenient evaluator. On the other hand, in the example shown in Fig. 12(b), the correlation in the relationship between the average first evaluation value of the evaluator and the second evaluation value is weak, indicating that the bias due to the evaluator's perception of sweetness or spiciness is reduced.

[0125] Also, as shown in Fig. 13(a), when using the evaluation information for only one period T1, the evaluators with harsher evaluations tend to have higher second evaluation values compared to the evaluators with lenient evaluations. However, as shown in Fig. 13(b), by using the evaluation information for four periods T1 to T4, it can be seen that there is not much difference in the tendency for the second evaluation value to be higher due to the perception of sweetness or spiciness of the evaluation.

[0126] 〔3.3.5. Provision Unit 34〕 The provision unit 34 provides various types of information to the user U and the personnel department staff O. For example, the provision unit 34 provides various types of information to the user U by transmitting various types of information to the terminal device 2. Also, the provision unit 34 provides various types of information to the personnel department staff O by transmitting various types of information to the terminal device 3.

[0127] For example, when the provision unit 34 receives a first evaluation value processing request by the reception unit 31, the provision unit 34 provides the second evaluation value of each of the plurality of evaluatees calculated by the calculation unit 33 to the personnel staff O by transmitting evaluation information including the second evaluation value of each of the plurality of evaluatees to the terminal device 3.

[0128] The terminal device 2 receives the second evaluation value of each of the plurality of evaluatees and displays the received information. Thereby, the personnel staff O can obtain the second evaluation value of each of the plurality of evaluatees.

[0129] 〔4. Processing procedure〕 Next, the information processing procedure by the processing unit 12 of the information processing apparatus 1 according to the embodiment will be described. FIG. 14 is a flowchart showing an example of information processing by the processing unit 12 of the information processing apparatus 1 according to the embodiment.

[0130] As shown in FIG. 14, the processing unit 12 of the information processing apparatus 1 determines whether or not evaluation information has been acquired (step S10). When the processing unit 12 determines that the evaluation information has been acquired (step S10: Yes), the processing unit 12 stores the acquired evaluation information in the storage unit 11 (step S11).

[0131] When the processing of step S11 is completed, or when the processing unit 12 determines that the evaluation information has not been acquired (step S10: No), the processing unit 12 determines whether or not a first evaluation value processing request has been received (step S12). When the processing unit 12 determines that the first evaluation information processing request has been received (step S12: Yes), the processing unit 12 specifies the first evaluation value indicated by the evaluation information stored in the storage unit 11 for each evaluator (step S13).

[0132] Subsequently, the processing unit 12 compares the first evaluation values for each pair of the evaluated persons (step S14), and determines the relative evaluation between the evaluated persons for each pair of the evaluated persons (step S15). Then, the processing unit 12 aggregates the relative evaluations for each pair of the evaluated persons (step S16), and calculates the second evaluation value for each evaluated person from the aggregation result (step S17). Thereafter, the processing unit 12 provides the second evaluation value of each evaluated person to the personnel staff O (step S18).

[0133] When the processing in step S18 is completed, or when it is determined that the first evaluation information processing request has not been received (step S12: No), the processing unit 12 determines whether the operation end timing has arrived (step S19). The processing unit 12 determines that the operation end timing has arrived, for example, when the power of the information processing apparatus 1 is turned off.

[0134] When the processing unit 12 determines that the operation end timing has not arrived (step S19: No), the process proceeds to step S10, and when the processing unit 12 determines that the operation end timing has arrived (step S19: Yes), the process shown in FIG. 14 is terminated.

[0135] [5. Modification Example] The providing unit 34 can also transmit, to the terminal device 3, evaluation information including the first evaluation value indicating the evaluation of the evaluator for each evaluated person, in addition to the second evaluation value of each of the plurality of evaluated persons calculated by the calculating unit 33. Further, the providing unit 34 can also transmit, as evaluation information, information further including information indicating the calculation content of the second evaluation value to the terminal device 3.

[0136] In addition, the providing unit 34 can also transmit information including the graphs shown in FIGS. 12(a) and (b) to the terminal device 3, or transmit information including the graphs shown in FIGS. 13(a) and (b) to the terminal device 3.

[0137] In addition, when there is an evaluated person who is only evaluated by evaluators who have not evaluated a plurality of evaluated persons, the providing unit 34 can also transmit information indicating the evaluated person to the terminal device 3. The evaluators who have not evaluated a plurality of evaluated persons are identified by the identification processing unit 40.

[0138] Further, for example, when the identification processing unit 40 identifies that there is an evaluated person who is only evaluated by evaluators who have not evaluated a plurality of evaluated persons, the determination processing unit 42 can also determine a dummy relative evaluation, which is a relative evaluation between the evaluated person and a virtual evaluated person.

[0139] [[6. Hardware Configuration]] The information processing apparatus 1 according to the above-described embodiment is realized by a computer 80 having a configuration as shown in FIG. 15, for example. FIG. 15 is a hardware configuration diagram showing an example of a computer 80 that realizes the functions of the information processing apparatus 1 according to the embodiment. The computer 80 includes a CPU 81, a RAM 82, a ROM (Read Only Memory) 83, an HDD (Hard Disk Drive) 84, a communication interface (I / F) 85, an input / output interface (I / F) 86, and a media interface (I / F) 87.

[0140] The CPU 81 operates based on programs stored in the ROM 83 or the HDD 84 and controls each unit. The ROM 83 stores a boot program executed by the CPU 81 when the computer 80 is started up, programs dependent on the hardware of the computer 80, and the like.

[0141] The HDD 84 stores programs executed by the CPU 81, data used by such programs, and the like. The communication interface 85 receives data from other devices via the network N (see FIG. 2) and sends it to the CPU 81, and transmits data generated by the CPU 81 to other devices via the network N.

[0142] The CPU 81 controls output devices such as a display and a printer, and input devices such as a keyboard or a mouse via the input / output interface 86. The CPU 81 acquires data from the input device via the input / output interface 86. Also, the CPU 81 outputs the data generated via the input / output interface 86 to the output device.

[0143] The media interface 87 reads a program or data stored in the recording medium 88 and provides it to the CPU 81 via the RAM 82. The CPU 81 loads such a program from the recording medium 88 onto the RAM 82 via the media interface 87 and executes the loaded program. The recording medium 88 is, for example, an optical recording medium such as a DVD (Digital Versatile Disc) or a PD (Phase change rewritable Disk), a magneto-optical recording medium such as an MO (Magneto-Optical disk), a tape medium, a magnetic recording medium, or a semiconductor memory.

[0144] For example, when the computer 80 functions as the information processing apparatus 1 according to the embodiment, the CPU 81 of the computer 80 realizes the functions of the processing unit 12 by executing the program loaded onto the RAM 82. Also, the data in the storage unit 11 is stored in the HDD 84. The CPU 81 of the computer 80 reads and executes these programs from the recording medium 88, but as another example, these programs may be acquired from another device via the network N.

[0145] 〔7. Others〕 In addition, among the processes described in the above embodiments, all or part of the processes described as being automatically performed can be manually performed, or all or part of the processes described as being manually performed can be automatically performed by a known method. In addition, the processing procedures, specific names, and information including various data and parameters shown in the above documents and drawings can be arbitrarily changed unless otherwise specified. For example, the various information shown in each figure is not limited to the illustrated information.

[0146] In addition, each component of each illustrated device is conceptually functional and does not necessarily have to be physically configured as shown. That is, the specific form of the distribution and integration of each device is not limited to that shown, and all or part of it can be functionally or physically distributed and integrated in any unit according to various loads and usage conditions.

[0147] For example, the above-described information processing apparatus 1 may be implemented by a terminal device and a server computer, may be implemented by a plurality of server computers, or may be implemented by calling an external platform or the like by an API or network computing depending on the function, and the configuration can be flexibly changed.

[0148] In addition, the above-described embodiments and modified examples can be appropriately combined as long as the processing contents do not conflict.

[0149] 〔8. Effects〕 As described above, the information processing apparatus 1 according to the embodiment includes an acquisition unit 30, a determination unit 32, and a calculation unit 33. The acquisition unit 30 acquires evaluation information including first evaluation values indicating evaluations by evaluators for each of a plurality of evaluatees. The determination unit 32 determines a relative evaluation between evaluatees for each pair of evaluatees based on the first evaluation values by the same evaluator for the pair of evaluatees among the plurality of evaluatees. The calculation unit 33 calculates a second evaluation value for each of the plurality of evaluatees based on the relative evaluation determined for each pair of evaluatees by the determination unit 32. Thereby, the information processing apparatus 1 can reduce the bias related to the evaluation and improve the fairness of the evaluation.

[0150] Further, the acquisition unit 30 acquires evaluation information for a plurality of periods T, the determination unit 32 determines a relative evaluation between evaluatees for each pair of evaluatees for each period T, and the calculation unit 33 calculates a second evaluation value for each of the plurality of evaluatees based on the result of weighted addition of the relative evaluations for each period T with a weight that is larger for the relative evaluation in a newer period. Thereby, the information processing apparatus 1 can further reduce the bias related to the evaluation and improve the fairness of the evaluation.

[0151] Further, the acquisition unit 30 acquires evaluation information for a plurality of periods T, the calculation unit 33 calculates a second evaluation value for each evaluatee for each period T based on the relative evaluation determined for each pair of evaluatees by the determination unit 32, and calculates, as the second evaluation value for each evaluatee, the result obtained by weighted addition of the second evaluation values for each period T with a weight that is larger for the second evaluation value in a newer period T. Thereby, the information processing apparatus 1 can further reduce the bias related to the evaluation and improve the fairness of the evaluation.

[0152] Further, the determination unit 32 determines a relative evaluation corresponding to the difference in the first evaluation values between the evaluatees as the relative evaluation. Thereby, the information processing apparatus 1 can further reduce the bias related to the evaluation and improve the fairness of the evaluation.

[0153] In addition, when the difference between the first evaluation values of the evaluated persons is equal to or greater than the threshold value, the determination unit 32 determines the relative evaluation corresponding to the difference between the first evaluation values of the evaluated persons as the relative evaluation. Thereby, the information processing apparatus 1 can further reduce the bias related to the evaluation and improve the fairness of the evaluation.

[0154] In addition, the determination unit 32 determines a dummy relative evaluation that is a relative evaluation between an evaluated person and a virtual evaluated person, and the calculation unit 33 calculates the second evaluation value of each of the plurality of evaluated persons based on the relative evaluation including the dummy relative evaluation. Thereby, the information processing apparatus 1 can further reduce the bias related to the evaluation and improve the fairness of the evaluation.

[0155] The first evaluation value is an evaluation value by multi-faceted evaluation. Thereby, the information processing apparatus 1 can reduce the bias related to the multi-faceted evaluation and improve the fairness of the evaluation.

[0156] As described above, the embodiments of the present application have been described in detail based on the drawings. However, this is an example, and the present invention can be implemented in other forms in which various modifications and improvements are made based on the knowledge of those skilled in the art, including the aspects described in the column of the disclosure of the invention.

[0157] In addition, the "section (section, module, unit)" described above can be read as "means" or "circuit". For example, the acquisition unit can be read as an acquisition means or an acquisition circuit.

Explanation of Reference Numerals

[0158] 1 Information processing apparatus 2, 3 Terminal device 10 Communication unit 11 Storage unit 12 Processing unit 20 User information storage unit 21 Evaluation result storage unit 30 Acquisition unit 31 Reception unit 32 Determination unit 33 Calculation unit 34 Provision unit 40 Specific Processing Unit 41 Comparison Processing Unit 42 Decision Processing Unit 100 Information Processing System N Network

Claims

1. An acquisition unit that acquires evaluation information including a first evaluation value indicating an evaluator's evaluation for each of a plurality of evaluatees; A determination unit that determines a relative evaluation between the evaluatees for each pair of evaluatees based on the first evaluation values given by the same evaluator for pairs of evaluatees among the plurality of evaluatees; A calculation unit that calculates a second evaluation value for each of the plurality of evaluatees based on the relative evaluation determined for each pair of evaluatees by the determination unit, comprising An information processing apparatus characterized by the above.

2. The acquisition unit acquires the evaluation information for a plurality of periods, The determination unit determines the relative evaluation between the evaluatees for each pair of evaluatees for each period, The calculation unit calculates the second evaluation value for each of the plurality of evaluatees based on the result of weighted addition of the relative evaluations for each period with a weight that increases as the relative evaluation for a new period increases. The information processing apparatus according to claim 1, characterized by the above.

3. The acquisition unit acquires the evaluation information for a plurality of periods, The calculation unit calculates the second evaluation value for each evaluatee for each period based on the relative evaluation determined for each pair of evaluatees by the determination unit, and calculates, as the second evaluation value for each evaluatee, the result obtained by weighted addition of the second evaluation values for each period with a weight that increases as the second evaluation value for a new period increases. The information processing apparatus according to claim 1, characterized by the above.

4. The determination unit determines the relative evaluation according to the difference in the first evaluation values between the evaluatees as the relative evaluation. The information processing apparatus according to any one of claims 1 to 3, characterized by the above.

5. The determination unit determines, as the relative evaluation, the relative evaluation according to the difference in the first evaluation values between the evaluatees when the difference in the first evaluation values between the evaluatees is equal to or greater than a threshold value. The information processing apparatus according to claim 4, characterized by the above.

6. The determination unit determines a dummy relative evaluation, which is a relative evaluation between the evaluatee and a virtual evaluatee, The calculation unit calculates the second evaluation value for each of the plurality of evaluatees based on the relative evaluation including the dummy relative evaluation. The information processing apparatus according to any one of claims 1 to 3, characterized by the above.

7. The first evaluation value is an evaluation value by multi-faceted evaluation. The information processing apparatus according to any one of claims 1 to 3, characterized by the above.

8. An information processing method executed by a computer, An acquisition step of acquiring evaluation information including a first evaluation value indicating an evaluator's evaluation for each of a plurality of evaluatees; A determination step of determining a relative evaluation between the evaluatees for each pair of the evaluatees based on the first evaluation values by the same evaluator for pairs of the evaluatees among the plurality of evaluatees; A calculation step of calculating a second evaluation value for each of the plurality of evaluatees based on the relative evaluation determined for each pair of the evaluatees by the determination step, comprising An information processing method characterized by the above.

9. An acquisition procedure of acquiring evaluation information including a first evaluation value indicating an evaluator's evaluation for each of a plurality of evaluatees; A determination procedure of determining a relative evaluation between the evaluatees for each pair of the evaluatees based on the first evaluation values by the same evaluator for pairs of the evaluatees among the plurality of evaluatees; Causing a computer to execute a calculation procedure of calculating a second evaluation value for each of the plurality of evaluatees based on the relative evaluation determined for each pair of the evaluatees by the determination procedure, An information processing program characterized by the above.

Images