Information processing method, information processing system, and program

Abstract

The information processing method includes: a step for acquiring evaluation object information (i1) including information relating to an evaluation object; a step for acquiring evaluator information (i2) including information relating to an evaluator; a determination step for determining, on the basis of the evaluation object information (i1) and the evaluator information (i2), at least one of the domain professional degree (S) and the interest relationship degree (I) of the evaluator with respect to the evaluation object; and a reliability derivation step for deriving the reliability (T) of the evaluation on the basis of at least one of the domain professional degree (S) and the interest relationship degree (I) determined in the determination step.

Description

Technical Field

[0001] The present disclosure relates to an information processing method, an information processing system, and a program. Background Art

[0002] A patent document 1 discloses an evaluation assistance device that can exclude unbalanced evaluations and assist in appropriate evaluations of people or things. The evaluation assistance device includes: an evaluation score input unit that inputs an evaluation score of an evaluator for an evaluation object; a deviation value calculation unit that calculates a deviation value of the evaluation score input by the evaluation score input unit; a correlation value calculation unit that calculates a correlation value between the deviation value calculated by the deviation value calculation unit and a specified value; and an evaluator information output unit that outputs evaluator information related to the evaluator corresponding to the smallest correlation value among the correlation values calculated by the correlation value calculation unit.

[0003] Prior Art Documents

[0004] Patent Documents

[0005] Patent Document 1: Japanese Patent Application Laid-Open No. 2003-196432 Summary of the Invention

[0006] Problems to be Solved by the Invention

[0007] However, in conventional devices, the evaluation results sometimes vary depending on the professional knowledge of the evaluators and the like. Therefore, there is a problem of doubting the evaluation results obtained by the evaluators.

[0008] The present disclosure provides an information processing method and the like that can obtain the reliability of the evaluation that an evaluator is about to perform or has performed on an evaluation object.

[0009] Means for Solving the Problems

[0010] An information processing method according to one aspect of the present disclosure is an information processing method for deriving the reliability of an evaluation that an evaluator is about to perform or has performed on an evaluation object, including: a step of obtaining evaluation object information including information related to the evaluation object; a step of obtaining evaluator information including information related to the evaluator; a determination step of determining at least one of the field expertise and the degree of interest of the evaluator with respect to the evaluation object based on the evaluation object information and the evaluator information; and a reliability derivation step of deriving the reliability of the evaluation based on at least one of the field expertise and the degree of interest determined in the determination step.

[0011] An information processing system according to one aspect of the present disclosure is an information processing system that derives the reliability of an evaluation that an evaluator is to perform or has performed on an evaluation object, and includes: an object information acquisition unit that acquires evaluation object information including information related to the evaluation object; an evaluator information acquisition unit that acquires evaluator information including information related to the evaluator; a level determination unit that determines at least one of the field expertise and the interest association degree of the evaluator with respect to the evaluation object based on the evaluation object information and the evaluator information; and a reliability derivation unit that derives the reliability of the evaluation based on at least one of the field expertise and the interest association degree determined by the level determination unit.

[0012] A program according to one aspect of the present disclosure is a program for causing a computer to execute the above information processing method.

[0013] Advantages of the Invention

[0014] According to an information processing method and the like according to one aspect of the present disclosure, it is possible to obtain the reliability of an evaluation that an evaluator is to perform or has performed on an evaluation object. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 It is a schematic diagram showing an information processing system according to an embodiment.

[0016] Figure 2 It is a diagram showing the module configuration of an information processing system according to an embodiment and an information processing device included in the information processing system.

[0017] Figure 3 It is a diagram showing an example of evaluation object information acquired by an information processing device.

[0018] Figure 4 It is a diagram showing an example of evaluator information acquired by an information processing device.

[0019] Figure 5 It is a diagram showing an example of the field expertise determined by an information processing device.

[0020] Figure 6 It is a diagram showing an example of the degree of interest relationship determined by an information processing device.

[0021] Figure 7 It is a diagram showing an example of the reliability derived by an information processing device.

[0022] Figure 8 It is a flowchart showing an information processing method according to an embodiment.

[0023] Figure 9It is a diagram showing the module configuration of an information processing system related to a modified example of an embodiment.

[0024] Figure 10 It is a flowchart showing an information processing method related to a modified example of an embodiment. Detailed implementation manners

[0025] The information processing method of the present disclosure is a method for obtaining the reliability of an evaluation that an evaluator has performed or is to perform on an evaluation object.

[0026] The evaluation object is a matter to be evaluated, for example, a research and development theme for which the company to which the evaluator belongs makes future-oriented research and development investments. The evaluation object may include projects formed by members of multiple organizations, group activities within an organization, personal target themes, etc.

[0027] The evaluator is a person who evaluates the evaluation object. The evaluator evaluates the evaluation object, for example, based on the business contribution degree and technical strength of the evaluation object. For the evaluation object, resources (people, things, property, etc.) are invested according to the evaluation result. Therefore, the reliability of the evaluation that the evaluator has performed or is to perform on the evaluation object is obtained.

[0028] The information processing method of the present disclosure includes the following steps, etc., in order to obtain the reliability of the evaluation that the evaluator has performed or is to perform on the evaluation object.

[0029] The information processing method of Example 1 is an information processing method for deriving the reliability of an evaluation that an evaluator has performed or is to perform on an evaluation object, including: a step of obtaining evaluation object information including information related to the evaluation object; a step of obtaining evaluator information including information related to the evaluator; a determination step of determining at least one of the field expertise degree and the degree of interest of the evaluator with respect to the evaluation object based on the evaluation object information and the evaluator information; and a reliability derivation step of deriving the reliability of the evaluation based on at least one of the field expertise degree and the degree of interest determined in the determination step.

[0030] In this way, it is possible to determine at least one of the field expertise degree and the degree of interest of the evaluator with respect to the evaluation object based on the evaluation object information and the evaluator information, and to obtain the reliability of the above evaluation based on at least one of the determined field expertise degree and the degree of interest.

[0031] In addition, the information processing method of Example 2 may also be that in the information processing method of Example 1, in the determination step, for each of the multiple evaluation objects, at least one of the domain expertise and the degree of interest of the evaluator is determined, and in the reliability derivation step, the reliability is derived for each of the multiple evaluation objects.

[0032] In this way, by determining at least one of the domain expertise and the degree of interest of the evaluator for each of the multiple evaluation objects, the reliability of the above evaluation can be obtained for each of the multiple evaluation objects.

[0033] In addition, the information processing method of Example 3 may also be that in the information processing method of Example 1 or Example 2, in the determination step, based on the evaluation object information and the evaluator information, the domain expertise and the degree of interest of the evaluator are respectively determined, and in the reliability derivation step, the reliability is derived based on the domain expertise and the degree of interest.

[0034] In this way, by deriving the reliability based on both the domain expertise and the degree of interest, the reliability of the above evaluation can be obtained more appropriately.

[0035] In addition, the information processing method of Example 4 may also be that in the information processing method of Example 3, in the reliability derivation step, the reliability is derived by weighting the domain expertise and the degree of interest respectively.

[0036] In this way, by weighting the domain expertise and the degree of interest respectively, the reliability of the above evaluation can be obtained more appropriately.

[0037] In addition, the information processing method of Example 5 may also be that in the information processing method of any one of Examples 1 to 4, in the reliability derivation step, the reliability is derived based on Equation 1,

[0038] T = αS - βI... (Equation 1)

[0039] where T: reliability, S: domain expertise, I: degree of interest, α: a coefficient greater than or equal to 0, β: a coefficient greater than or equal to 0.

[0040] By using the above (Equation 1), the reliability of the above evaluation can be obtained more appropriately.

[0041] In addition, the information processing method of Example 6 may also be that in the information processing method of any one of Examples 1 to 5, the evaluation object is a research and development theme for which the company to which the evaluator belongs makes future-oriented research and development investments.

[0042] Accordingly, it is possible to obtain the reliability of the evaluation that the evaluator has made or will make on the research and development subject.

[0043] In addition, the information processing method of Example 7 may also be that in any one of the information processing methods of Examples 1 to 6, in the determination step, based on the relationship between the search formula for retrieving documents related to the evaluation object and the documents of which the evaluator is the inventor or author, the field expertise is determined.

[0044] Accordingly, it is possible to appropriately determine the field expertise of the evaluator with respect to the evaluation object. Accordingly, it is possible to more appropriately obtain the reliability of the above evaluation.

[0045] In addition, the information processing method of Example 8 may also be that in any one of the information processing methods of Examples 1 to 7, the document is at least one of a patent document and a thesis.

[0046] Accordingly, it is possible to appropriately determine the field expertise based on at least one of a patent document and a thesis. Accordingly, it is possible to more appropriately obtain the reliability of the above evaluation.

[0047] In addition, the information processing method of Example 9 may also be that in any one of the information processing methods of Examples 1 to 8, in the determination step, based on the relationship between the research and development organization that has upgraded the subject of the evaluation object and the past and present affiliated organizations of the evaluator, the degree of interest is determined.

[0048] Accordingly, it is possible to appropriately determine the degree of interest of the evaluator with respect to the evaluation object. Accordingly, it is possible to more appropriately obtain the reliability of the above evaluation.

[0049] In addition, the information processing method of Example 10 may also be that in any one of the information processing methods of Examples 1 to 9, an evaluation adoption determination step is further included, and in this evaluation adoption determination step, it is determined whether to adopt the evaluation that a specified evaluator has made on the evaluation object. In the evaluation adoption determination step, when the reliability is greater than a predetermined threshold, it is determined to adopt the evaluation that the specified evaluator has made on the evaluation object.

[0050] Accordingly, when the above reliability is greater than a predetermined threshold, it is possible to determine to adopt the evaluation that a specified evaluator has made on the evaluation object. Accordingly, it is possible to appropriately evaluate the evaluation object.

[0051] In addition, the information processing method of Example 11 may also be that in any one of the information processing methods of Examples 1 to 9, an evaluation correction step is further included. In this evaluation correction step, the evaluation made by a specified evaluator on the evaluation object is corrected. In the evaluation correction step, the reliability is used to weight the evaluation made by the specified evaluator on the evaluation object, thereby correcting the evaluation.

[0052] Thus, the evaluation of the evaluation object can be corrected using the above-mentioned reliability. Thus, the evaluation object can be appropriately evaluated.

[0053] The information processing system of Example 12 is an information processing system that derives the reliability of the evaluation to be performed or already performed by an evaluator on an evaluation object, and includes: an object information acquisition unit that acquires evaluation object information including information related to the evaluation object; an evaluator information acquisition unit that acquires evaluator information including information related to the evaluator; a level determination unit that determines at least one of the field expertise and the degree of interest of the evaluator with respect to the evaluation object based on the evaluation object information and the evaluator information; and a reliability derivation unit that derives the reliability of the evaluation based on at least one of the field expertise and the degree of interest determined by the level determination unit.

[0054] In this way, by determining at least one of the field expertise and the degree of interest of the evaluator with respect to the evaluation object based on the evaluation object information and the evaluator information, the reliability of the above-mentioned evaluation can be obtained based on at least one of the field expertise and the degree of interest.

[0055] The program of Example 12 is a program for causing a computer to execute the information processing method of any one of Examples 1 to 11.

[0056] Thus, the computer can execute the above-mentioned information processing method according to the program.

[0057] In addition, these general or specific technical solutions can be implemented either by a system, a device, a method, an integrated circuit, a computer program, or a non-transitory recording medium such as a computer-readable CD-ROM, or by any combination of a system, a device, a method, an integrated circuit, a computer program, and a recording medium.

[0058] Hereinafter, the embodiments will be described with reference to the drawings. In addition, the embodiments described below are all general or specific examples. The numerical values, shapes, materials, constituent elements, arrangement positions of the constituent elements, connection methods, steps, order of steps, etc. shown in the following embodiments are examples and are not intended to limit the claims.

[0059] (Embodiment)

[0060] [Configuration of Information Processing System]

[0061] Refer to Figures 1 to 7 The configuration of the information processing system involved in the embodiment will be described.

[0062] Figure 1 It is a schematic diagram showing the information processing system 1 involved in the embodiment. Figure 2 It is a diagram showing the module configuration of the information processing system 1 and the information processing device 10 included in the information processing system 1.

[0063] As Figure 1 and Figure 2 shown, the information processing system 1 includes an information processing device 10 and a database 50.

[0064] The information processing device 10 and the database 50 are connected to each other via a communication network.

[0065] The database 50 is, for example, a computer device that performs input / output, editing, and storage of various information. In addition, the database 50 also includes an external storage device such as a NAS (Network Attached Storage).

[0066] The database 50 includes a first database 51 and a second database 52. The first database 51 and the second database 52 are, for example, set in a facility managed by the company to which the evaluator belongs. The first database 51 is, for example, a technology planning database that stores information related to the evaluation object. The second database 52 is, for example, a personnel database that stores information related to the evaluator.

[0067] In addition, the database 50 includes a third database 53 and a fourth database. The third database 53 and the fourth database 54 are, for example, set in an external server on the Internet. The third database 53 is, for example, a patent literature database that stores information related to patent literature. The fourth database 54 is, for example, a paper database that stores information related to technical papers.

[0068] The information processing device 10 can access the database 50 to obtain the required information.

[0069] As Figure 2 shown, the information processing device 10 includes an object information acquisition unit 11, an evaluator information acquisition unit 12, a grade determination unit 13, a reliability derivation unit 14, and an evaluation adoption determination unit 15. The information processing device 10 is, for example, a computer device that performs input / output and arithmetic processing of various information.

[0070] The object information acquisition unit 11 is a processing circuit that acquires the evaluation object information i1. The object information acquisition unit 11 acquires the evaluation object information i1 based on the information related to the evaluation object output from the database 50.

[0071] Figure 3 It is a diagram showing an example of the evaluation object information i1 acquired by the information processing apparatus 10.

[0072] The evaluation object information i1 includes information related to the name, content, and executing organization of the evaluation object. For example, the name of the evaluation object is the research and development theme name, and the content of the evaluation object is the technical content of the research and development theme. Hereinafter, an example where the evaluation object is a research and development theme will be described. The executing organization is an organization that promotes and executes the evaluation object. Information related to the executing organization is hierarchically stored in a form in which multiple small organizations are associated with a large organization, such as a research and development center, a research and development department, a development section, etc. Information related to the executing organization also includes organization history information indicating the relationship between the current organization and the past organization.

[0073] In addition, the evaluation object information i1 includes an evaluator who evaluates the evaluation object, and an evaluation result that the evaluator has to perform or has performed on the evaluation object. The evaluator evaluates multiple evaluation objects separately. In addition, the evaluation of the evaluation object is performed for each evaluator among multiple evaluators.

[0074] The evaluator scores multiple evaluation items such as customer demand degree, business scale, and competitive differentiation degree for the evaluation object, and appropriately takes a weighted average, etc. to determine the evaluation score of the evaluation object. The evaluation of the evaluation object is performed at the time of application of the research and development theme, but it can also be performed at the end of the research and development theme, or at the time of regularly monitoring the progress of the research and development theme, during budget research, etc.

[0075] In addition, the evaluation object information i1 includes a search formula for retrieving documents related to the evaluation object. The search formula is, for example, a patent search formula or a paper search formula.

[0076] The patent search formula is a classification or keyword used when retrieving patent documents, such as the International Patent Classification (IPC) and the F term (File Forming Term). The paper search formula is a classification or keyword used when retrieving papers. As the paper search formula, the technical classification (ASJC: All Science Journal Classification) given in a paper database such as SCOPUS (registered trademark) can be used.

[0077] In addition, when the above search formula is not included in the information related to the evaluation object stored in the first database 51, the object information acquisition unit 11 can acquire a patent search formula and a paper search formula based on the technical content of the evaluation object. For example, the patent search formula is composed of keywords included in the technical content and patent classifications (such as IPC) including the keywords. For example, the paper search formula is composed of keywords included in the technical content and technical classifications (such as ASJC) including the keywords. These search formulas can also be obtained through the automatic arithmetic processing of the information processing device 10. The evaluation object information i1 acquired by the object information acquisition unit 11 is output to the grade determination unit 13 described later.

[0078] The evaluator information acquisition unit 12 is a processing circuit that acquires evaluator information i2. The evaluator information acquisition unit 12 acquires the evaluator information i2 based on the information related to the evaluator output from the database 50.

[0079] Figure 4 It is a diagram showing an example of the evaluator information i2 acquired by the information processing device 10.

[0080] The evaluator information i2 includes the identification information of the evaluator and the affiliated organization. For example, the identification information of the evaluator is the name and employee number of the evaluator. The affiliated organization of the evaluator is the past and current affiliated organization of the evaluator. The affiliated organization is an organization within the company to which the evaluator belongs, and is stored in a hierarchical state such as a research and development center, a research and development department, a development section, etc. The affiliated organization of the evaluator is used when determining the degree of interest of the evaluator in the evaluation object.

[0081] In addition, the evaluator information i2 includes information related to the documents of which the evaluator is the inventor or author, specifically patent documents and papers (non-patent documents). Hereinafter, the documents of which the evaluator is the inventor or author are sometimes referred to as the evaluator's documents. The evaluator's documents may also include past documents created by the evaluator, that is, documents created when the evaluator did not belong to the company. In addition, the papers may include in-company papers and technical reports that are only disclosed within the company.

[0082] The patent documents of the evaluator are assigned patent classifications (such as IPC), and the papers of the evaluator are assigned technical classifications (such as ASJC). In the evaluator information i2, a list of the patent documents of the evaluator assigned with each classification and a list of the papers of the evaluator are stored. These lists are used when determining the field expertise of the evaluator in the evaluation object.

[0083] In addition, when the above classification is not included in the information related to the reviewer stored in the second database 52, the reviewer information acquisition unit 12 may also obtain the patent classification or technical classification corresponding to the literature based on the literature of the reviewer. For example, for patent classification, refer to the patent classification described in the patent literature, and for technical classification, refer to the technical classification assigned to the paper. These classifications can also be obtained through the automatic arithmetic processing of the information processing device 10. The reviewer information i2 obtained by the reviewer information acquisition unit 12 is output to the grade determination unit 13.

[0084] The grade determination unit 13 is a processing circuit that determines at least one of the field expertise S and the degree of interest I of the reviewer in the evaluation object based on the evaluation object information i1 and the reviewer information i2. The field expertise S and the degree of interest I are determination items required to derive the reliability T of the evaluation. Hereinafter, the field expertise S and the degree of interest I will be described separately.

[0085] Figure 5 It is a diagram showing an example of the field expertise S determined by the information processing device 10.

[0086] The field expertise S is a grade indicating the height of the reviewer's professional knowledge in the technical field of the evaluation object. The field expertise S becomes different grades according to the different professional knowledge of the reviewer a for each evaluation object AA, BB, CC. Hereinafter, taking the evaluation object AA among the evaluation objects AA, BB, CC as a representative example for explanation.

[0087] As Figure 5 shown, the field expertise S also becomes different grades according to each reviewer a, b, c who evaluates the evaluation object AA. The larger the value of the field expertise S, the higher the professional knowledge. For example, it is represented by 11 grades from 0 or more and 1.0 or less (0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0). Figure 5 shows an example where the field expertise S of the reviewer a for the evaluation object AA is 0.9, the field expertise S of the reviewer b for the evaluation object AA is 0.8, and the field expertise S of the reviewer c for the evaluation object AA is 0.4. In addition, the field expertise S is not limited to 11 grades and can also be represented by two or more grades.

[0088] As described above, the evaluation object information i1 includes a search formula for retrieving literature related to the evaluation object, and the reviewer information i2 includes the classification of the literature for which the reviewer is the inventor or author. The grade determination unit 13 determines the field expertise S based on the relationship between the above search formula and the reviewer's literature.

[0089] For example, the level determination unit 13 determines the field expertise S based on the degree of relevance between the above-mentioned search formula and the classification of the reviewer's literature. Specifically, the level determination unit 13 compares the above-mentioned patent search formula with the patent classification in the reviewer's patent literature. The lower the level in each hierarchy (department, class, subclass, main group, subgroup, etc.) that matches, the higher the determined relevance. Also, the level determination unit 13 compares the above-mentioned paper search formula with the technical classification in the reviewer's paper. The lower the level in each hierarchy that matches, the higher the determined relevance. Moreover, when the relevance between the above-mentioned search formula and the classification of the reviewer's literature is high, the level determination unit 13 determines that the field expertise S is high, and when the relevance is low, it determines that the field expertise S is low. Information related to the field expertise S determined by the level determination unit 13 is output to the reliability derivation unit 14.

[0090] Figure 6 FIG. is an example showing the degree of interest I determined by the information processing device 10.

[0091] The degree of interest I is a rank indicating the intensity of the reviewer's interest in the executing organization that has upgraded the subject of the evaluation object. The degree of interest I varies depending on the combination of the executing organization that has upgraded the subject of each evaluation object AA, BB, CC and the past and present affiliated organizations of each reviewer a, b, c. The larger the value of the degree of interest I, the stronger the interest. For example, it is represented by 11 ranks from 0 or more and 1.0 or less (0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0). Figure 6 An example is shown in which the degree of interest I of reviewer a for evaluation object AA is 0.8, the degree of interest I of reviewer b for evaluation object AA is 0.4, and the degree of interest I of reviewer c for evaluation object AA is 0.2. In addition, the degree of interest I is not limited to 11 ranks and can also be represented by two or more ranks.

[0092] As described above, the evaluation object information i1 contains information related to the executing organization that has upgraded the subject of the evaluation object, and the reviewer information i2 contains information related to the present and past affiliated organizations of the reviewer. The level determination unit 13 determines the degree of interest I based on the relationship between the executing organization that has upgraded the subject of the evaluation object and the affiliated organization of the reviewer.

[0093] For example, the level determination unit 13 determines the degree of interest I based on the level of relevance between the executing organization that has upgraded the theme of the evaluation object and the organization to which the evaluator belongs. Specifically, the level determination unit 13 compares the executing organization that has upgraded the theme of the evaluation object with the organization to which the evaluator belongs, and determines that the relevance is higher if the lower levels (such as research and development centers, research and development departments, development sections, etc.) are the same. Moreover, the level determination unit 13 determines that the degree of interest I is high when the relevance between the executing organization that has upgraded the theme of the evaluation object and the organization to which the evaluator belongs is high, and determines that the degree of interest I is low when the relevance is low. Information related to the degree of interest I determined by the level determination unit 13 is output to the reliability derivation unit 14.

[0094] The reliability derivation unit 14 is a processing circuit that derives the reliability T of the evaluation that the evaluator is to perform or has performed on the evaluation object based on at least one of the domain expertise S and the degree of interest I determined by the level determination unit 13.

[0095] Figure 7 It is a diagram showing an example of the reliability T derived by the information processing device 10.

[0096] The reliability T is the reliability of the evaluation that the evaluator is to perform or has performed on the evaluation object. The reliability derivation unit 14 derives the reliability T by weighting the domain expertise S and the degree of interest I respectively. The larger the value of the reliability T, the higher the reliability, which is expressed, for example, on a scale of 0 or more and 1.0 or less. The domain expertise S when deriving the reliability T is a positive factor, and the degree of interest I is a negative factor.

[0097] For example, the reliability derivation unit 14 derives the reliability T based on the following (Equation 1) (where T: reliability, S: domain expertise, I: degree of interest, α: coefficient of 0 or more, β: coefficient of 0 or more).

[0098] T = αS - βI... (Equation 1)

[0099] In Figure 7 the example shown, an example is shown where the reliability T of the evaluation that evaluator a has performed on evaluation object AA is 0.1, the reliability T of the evaluation that evaluator b has performed on evaluation object AA is 0.4, and the reliability T of the evaluation that evaluator c has performed on evaluation object AA is 0.2. Additionally, it is preset that the coefficients α = β = 1.

[0100] The coefficients α and β are preset in the information processing apparatus 10 according to which one of the field expertise S and the degree of interest I is emphasized. For example, when the field expertise S is set to a weight twice that of the degree of interest I, the coefficients are set to α = 1 and β = 0.5. When the field expertise S is set to a weight of 1 / 2 times that of the degree of interest I, the coefficients are set to α = 0.5 and β = 1. When deriving the reliability T based only on the field expertise S, β = 0 is set, and when deriving the reliability T based only on the degree of interest I, α = 0 is set.

[0101] In the above, the evaluation object AA has been taken as an example for explanation, but the reliability T of the evaluation made by the evaluator can also be derived for other evaluation objects BB and CC in the same way. The reliability T derived by the reliability derivation unit 14 is output to the evaluation adoption determination unit 15.

[0102] The evaluation adoption determination unit 15 is a processing circuit that determines whether to adopt the evaluation made by a specified evaluator for the evaluation object. The evaluation adoption determination unit 15 determines to adopt the evaluation made by a specified evaluator for the evaluation object when the reliability T is greater than a predetermined threshold.

[0103] For example, when the above-mentioned threshold is set to 0.3, in Figure 7 the example shown, the evaluation adoption determination unit 15 determines not to adopt the evaluations made by the evaluators a and c for the evaluation object AA. On the other hand, the evaluation adoption determination unit 15 determines to adopt the evaluation made by the evaluator b for the evaluation object AA. Thus, the evaluation of the evaluation object can be appropriately performed.

[0104] The information processing system 1 of the present embodiment includes: an object information acquisition unit 11 that acquires evaluation object information i1; an evaluator information acquisition unit 12 that acquires evaluator information i2; a level determination unit 13 that determines at least one of the field expertise S and the degree of interest I based on the evaluation object information i1 and the evaluator information i2; and a reliability derivation unit 14 that derives the reliability T of the evaluation based on at least one of the field expertise S and the degree of interest I determined by the level determination unit 13. With this configuration, the reliability T of the evaluation that the evaluator is to perform or has performed on the evaluation object can be appropriately obtained.

[0105] In addition, in the above example, an example is shown in which the evaluation performed by a specified evaluator on an evaluation object is adopted when the reliability T is greater than a predetermined threshold, but it is not limited to this. For example, the evaluation adoption determination unit 15 may also determine not to adopt the evaluation performed by a specified evaluator on an evaluation object when the field expertise S is less than a predetermined threshold (for example, 0.5). For example, the evaluation adoption determination unit 15 may also determine not to adopt the evaluation performed by a specified evaluator on an evaluation object when the degree of interest I is equal to or greater than a predetermined threshold (for example, 0.7).

[0106] [Information Processing Method]

[0107] Refer to Figure 8 The information processing method according to the embodiment will be described.

[0108] Figure 8 It is a flowchart showing the information processing method according to the embodiment.

[0109] The information processing apparatus 10 acquires evaluation object information i1 including information related to an evaluation object (step S10). For example, the information processing apparatus 10 acquires the evaluation object information i1 based on the information obtained from the first database 51, the third database 53, and the fourth database 54.

[0110] The evaluation object information i1 includes information related to the name, content, and executing organization of the evaluation object. The evaluation object is, for example, a research and development theme for which a company to which the evaluator belongs makes investment in future research and development. In addition, the evaluation object information i1 includes the evaluator who evaluates the evaluation object and the evaluation result that the evaluator has performed on the evaluation object. In addition, the evaluation object information i1 includes a search formula for searching for literature related to the evaluation object (research and development theme). The search formula is, for example, a patent search formula or a paper search formula.

[0111] Next, the information processing device 10 acquires evaluator information i2 including information related to the evaluator (step S20). For example, the information processing device 10 acquires the evaluator information i2 based on the information obtained from the second database 52, the third database 53, and the fourth database 54.

[0112] The evaluator information i2 includes the identification information and the affiliated organization of the evaluator. In addition, the evaluator information i2 includes information related to the literature in which the evaluator is the inventor or the author, specifically, patent literature and papers (non-patent literature). In addition, the order of steps S10 and S20 may be different, or they may be executed simultaneously.

[0113] Next, the information processing device 10 determines at least one of the field expertise S and the degree of interest I of the evaluator with respect to the evaluation object based on the evaluation object information i1 and the evaluator information i2 (step S30). For example, the information processing device 10 determines the field expertise S based on the relationship between the search formula for retrieving documents related to the evaluation object and the documents of which the evaluator is the inventor or author. In addition, the information processing device 10 determines the degree of interest I based on the relationship between the research and development organization that has upgraded the theme of the evaluation object and the past and current affiliated organizations of the evaluator.

[0114] In addition, the information processing device 10 determines the field expertise S and the degree of interest I of the evaluator for each of the multiple evaluation objects. In the case of multiple evaluators, the field expertise S and the degree of interest I of each of the multiple evaluators who have evaluated one evaluation object are determined. The information processing device 10 can determine both the field expertise S and the degree of interest I of the evaluator based on the evaluation object information i1 and the evaluator information i2, or can determine only one of them.

[0115] Next, the information processing device 10 derives the reliability T of the evaluation that the evaluator has performed on the evaluation object based on at least one of the field expertise S and the degree of interest I determined in step S30 (step S40). For example, the information processing device 10 derives the reliability T by weighting the field expertise S and the degree of interest I respectively.

[0116] Next, it is determined whether to adopt the evaluation that the specified evaluator has performed on the evaluation object. Specifically, the information processing device 10 determines whether the reliability T is greater than a predetermined threshold value (step S50). When the reliability T is equal to or less than the predetermined threshold value (in S50, "no"), the information processing device 10 determines not to adopt the evaluation that the specified evaluator has performed on the evaluation object (step S60). On the other hand, when the reliability T is greater than the predetermined threshold value (in S50, "yes"), the information processing device 10 determines to adopt the evaluation that the specified evaluator has performed on the evaluation object (step S70).

[0117] By executing these steps S10 to S40, the reliability T of the evaluation that the evaluator has performed on the evaluation object can be obtained. In addition, by further executing steps S50 to S70, the evaluation of the evaluation object can be appropriately performed.

[0118] [Modification Example of the Embodiment]

[0119] The information processing system 1A according to the modification example of the embodiment will be described. In this modification example, an example of correcting the evaluation performed by the evaluator using the reliability T derived by the above method will be described.

[0120] Figure 9 FIG. is a diagram showing a module configuration of an information processing system 1A according to a modified example of an embodiment.

[0121] As Figure 9 shown, the information processing system 1A includes an information processing device 10A and a database 50. The information processing device 10A includes an object information acquisition unit 11, an evaluator information acquisition unit 12, a rank determination unit 13, a reliability derivation unit 14, and an evaluation correction unit 16.

[0122] The evaluation correction unit 16 is a processing circuit that corrects the evaluation made by a specified evaluator on an evaluation object. The evaluation correction unit 16 uses the reliability T derived by the reliability derivation unit 14 to weight the evaluation of the evaluation object by the specified evaluator, thereby correcting the evaluation. According to this configuration, the evaluation of the evaluation object can be appropriately performed.

[0123] In addition, the evaluation correction unit 16 may also determine the comprehensive evaluation score for the evaluation object by averaging the evaluation scores of multiple evaluators. Alternatively, the evaluation correction unit 16 may also determine the comprehensive evaluation score for the evaluation object by removing the highest and lowest evaluation scores among the multiple evaluators and then averaging the remaining evaluation scores.

[0124] An information processing method according to a modified example of the embodiment will be described.

[0125] Figure 10 FIG. is a flowchart showing an information processing method according to a modified example of the embodiment.

[0126] Steps S10 to S40 are the same as those in the embodiment.

[0127] In the modified example, the evaluation made by a specified evaluator on an evaluation object is corrected (step S80). Specifically, the information processing device 10 uses the reliability T derived in step S40 to weight the evaluation of the evaluation object by the specified evaluator, thereby correcting the evaluation. By executing step S80, the evaluation of the evaluation object can be appropriately performed.

[0128] (Other Embodiments)

[0129] As described above, the embodiments of the information processing system and the like have been described, but the embodiments of the information processing system and the like are not limited to the embodiments. Modifications that can be conceived by those skilled in the art may be made to the embodiments, and multiple components in the embodiments may be arbitrarily combined.

[0130] For example, the information processing apparatus 10 may include an evaluator determination unit that determines whether a specified person should be an evaluator to be evaluated. When the field expertise S of the specified person is less than a specified threshold, the evaluator determination unit determines that the specified person is not to be an evaluator. In addition, the evaluator determination unit may also determine that the specified person is not to be an evaluator when the degree of interest I of the specified person is equal to or greater than the specified threshold.

[0131] For example, instead of specific components, other components may perform the processing performed by the specific components in the embodiments. In addition, the order of a plurality of processes may be changed, or a plurality of processes may be performed in parallel. In addition, ordinal numbers such as first and second used in the description may be appropriately changed, removed, or newly assigned. These ordinal numbers do not necessarily correspond to a meaningful order and may also be used for element identification.

[0132] The information processing method may be executed by any system or apparatus. That is, the information processing method may be executed by the above-described information processing system or the like, or may be executed by other systems or apparatuses.

[0133] For example, part or all of the information processing method may also be executed by a computer including a processor, a memory, an input / output circuit, and the like. In this case, the information processing method may also be executed by executing a program for causing the computer to execute the information processing method.

[0134] In addition, the above program may be recorded on a non-transitory computer-readable recording medium such as a CD-ROM.

[0135] In addition, each component of the information processing system or the like may be configured by dedicated hardware, may be configured by general hardware that executes the above program or the like, or may be configured by a combination thereof. In addition, the general hardware may also be composed of a memory recording a program and a general processor that reads and executes the program from the memory. Here, the memory may be a semiconductor memory or a hard disk, and the general processor may be a CPU or the like.

[0136] In addition, the dedicated hardware may also be composed of a memory and a dedicated processor or the like. For example, the dedicated processor may also execute the above information processing method with reference to the memory.

[0137] In addition, each component of the information processing system may also be an electric circuit. These electric circuits may be configured as a single electric circuit as a whole, or may be different electric circuits respectively. In addition, these electric circuits may correspond to dedicated hardware or may correspond to general hardware that executes the above program or the like.

[0138] Industrial Applicability

[0139] The present disclosure can be applied to a reliability derivation method and the like for obtaining the reliability of an evaluation made by an evaluator on a research and development subject to be carried out or already carried out.

[0140] Explanation of Reference Numerals

[0141] 1, 1A Information Processing System

[0142] 10, 10A Information Processing Device

[0143] 11 Object Information Acquisition Unit

[0144] 12 Evaluator Information Acquisition Unit

[0145] 13 Grade Determination Unit

[0146] 14 Reliability Derivation Unit

[0147] 15 Evaluation Adoption Determination Unit

[0148] 16 Evaluation Correction Unit

[0149] 50 Database

[0150] 51 First Database

[0151] 52 Second Database

[0152] 53 Third Database

[0153] 54 Fourth Database

[0154] i1 Evaluation Object Information

[0155] i2 Evaluator Information

[0156] I Degree of Interest

[0157] S Field Expertise

[0158] T Reliability

Claims

1. An information processing method for deriving the reliability of an evaluation that an evaluator is to conduct or has conducted on an evaluation object, where Comprising: A step of obtaining evaluation object information including information related to the evaluation object; A step of obtaining evaluator information including information related to the evaluator; A determination step of determining at least one of the field expertise and the degree of interest of the evaluator with respect to the evaluation object based on the evaluation object information and the evaluator information; And A reliability derivation step of deriving the reliability of the evaluation based on at least one of the field expertise and the degree of interest determined in the determination step.

2. The information processing method according to claim 1, wherein In the determination step, at least one of the field expertise and the degree of interest of the evaluator is determined for each evaluation object among a plurality of the evaluation objects, In the reliability derivation step, the reliability is derived for each evaluation object among a plurality of the evaluation objects.

3. The information processing method according to claim 1, wherein In the determination step, the field expertise and the degree of interest of the evaluator are respectively determined based on the evaluation object information and the evaluator information, In the reliability derivation step, the reliability is derived based on the field expertise and the degree of interest.

4. The information processing method according to claim 3, wherein In the reliability derivation step, the reliability is derived by respectively weighting the field expertise and the degree of interest.

5. The information processing method according to claim 1, wherein In the reliability derivation step, the reliability is derived based on (Equation 1), T = αS - βI…(Equation 1) wherein, T: reliability, S: field expertise, I: degree of interest, α: a coefficient of 0 or more, β: a coefficient of 0 or more.

6. The information processing method according to any one of claims 1 to 5, wherein The evaluation object is a research and development theme for which the company to which the evaluator belongs makes investment in future research and development.

7. The information processing method according to claim 1, wherein In the determination step, the field expertise is determined based on the relationship between the search formula for retrieving literature related to the evaluation object and the literature for which the evaluator is the inventor or the author.

8. The information processing method according to claim 7, wherein The literature is at least one of patent literature and papers.

9. The information processing method according to claim 6, wherein In the determination step, the degree of interest is determined based on the relationship between the research and development organization that has upgraded the theme of the evaluation object and the past and current affiliated organizations of the evaluator.

10. The information processing method according to any one of claims 1 to 5, wherein It further includes an evaluation adoption determination step of determining whether to adopt the evaluation of the evaluation object performed by a specified evaluator, In the evaluation adoption determination step, when the reliability is greater than a predetermined threshold, it is determined to adopt the evaluation of the evaluation object performed by the specified evaluator.

11. The information processing method according to any one of claims 1 to 5, wherein: it further includes an evaluation correction step in which the evaluation made by a specified evaluator on the evaluation object is corrected; in the evaluation correction step, the reliability is used to weight the evaluation made by the specified evaluator on the evaluation object, thereby correcting the evaluation.

12. An information processing system that derives the reliability of an evaluation that an evaluator is to perform or has performed on an evaluation object, where, comprising: an object information acquisition unit that acquires evaluation object information including information related to the evaluation object; an evaluator information acquisition unit that acquires evaluator information including information related to the evaluator; a level determination unit that determines at least one of the field expertise and the degree of interest of the evaluator with respect to the evaluation object based on the evaluation object information and the evaluator information; and a reliability derivation unit that derives the reliability of the evaluation based on at least one of the field expertise and the degree of interest determined by the level determination unit.

13. A program for causing a computer to execute the information processing method according to any one of claims 1 to 5.

Images