Information processing device, information processing method, and program

Abstract

[Problem] To effectively reduce the cost of an evaluation with respect to a prescribed goal while securing the quality of the evaluation. [Solution] The present invention provides an information processing device provided with an evaluation unit that, on the basis of text in which characteristics of an evaluation target are described, performs an evaluation of the evaluation target with respect to a prescribed goal including a plurality of sub-goals. When a feature value extracted from the text in which the characteristics of the evaluation target are described is input to a first classifier generated by means of supervised learning using a feature value extracted from text in which characteristics of a comparison target are described, the comparison target having been evaluated as satisfying one of prescribed evaluation criteria defined with respect to the prescribed goal by the combinations of the plurality of sub-goals, the evaluation unit acquires a similarity between the evaluation target and the comparison target for each of the sub-goals, and on the basis of an ideal pattern for each of the prescribed evaluation criteria defined by the combinations of the plurality of sub-goals, evaluates the similarity between the ideal pattern and the evaluation target for each of the prescribed evaluation criteria.

Description

[Technical Field]

[0001] The present disclosure relates to an information processing device, an information processing method, and a program. [Background technology]

[0002] In recent years, techniques for evaluating an evaluation target with respect to some goal have been developed. For example, Patent Document 1 discloses a technique for evaluating services with respect to the SDGs (Sustainable Development Goals). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2020-135726 Summary of the Invention [Problem to be solved by the invention]

[0004] However, the evaluation method disclosed in Patent Document 1 is limited to a relative evaluation based on existing services that have been shown to contribute to the goal. [Means for solving the problem]

[0005] According to one aspect of the present disclosure, there is provided an information processing device including an evaluation unit that evaluates the evaluation target against a predetermined goal including a plurality of sub-goals based on a text describing the characteristics of the evaluation target, wherein the evaluation unit obtains a similarity between the evaluation target and the comparison target for each of the sub-goals by inputting features extracted from the text describing the characteristics of the evaluation target into a first classifier generated by supervised learning using features extracted from the text describing the characteristics of the evaluation target, the text describing the characteristics of a comparison target that has been evaluated as meeting any of predetermined evaluation criteria defined by a combination of a plurality of the sub-goals for the predetermined goal, and evaluates a similarity between the evaluation target and an ideal pattern for each of the predetermined evaluation criteria based on the similarity between the evaluation target and the comparison target for each of the sub-goals and an ideal pattern for each of the predetermined evaluation criteria defined by a combination of the plurality of the sub-goals.

[0006] According to another aspect of the present disclosure, there is provided an information processing method including: a processor evaluating the evaluation target against a predetermined goal including a plurality of sub-goals based on a text describing the characteristics of the evaluation target; wherein the evaluation further includes: acquiring a similarity between the evaluation target and the comparison target for each of the sub-goals by inputting features extracted from the text describing the characteristics of the evaluation target into a first classifier generated by supervised learning using features extracted from the text describing the characteristics of the evaluation target, the text describing the characteristics of a comparison target that has been evaluated as satisfying any of predetermined evaluation criteria defined by a combination of a plurality of the sub-goals for the predetermined goal; and evaluating a similarity between the evaluation target and an ideal pattern for each of the predetermined evaluation criteria based on the similarity between the evaluation target and the comparison target for each of the sub-goals and an ideal pattern for each of the predetermined evaluation criteria defined by a combination of the plurality of sub-goals.

[0007] According to another aspect of the present disclosure, there is provided a program for causing a computer to function as an information processing device, comprising: an evaluation unit that evaluates the evaluation target against a predetermined goal including a plurality of sub-goals based on a text describing the characteristics of the evaluation target; the evaluation unit inputs features extracted from the text describing the characteristics of the evaluation target into a first classifier generated by supervised learning using features extracted from the text describing the characteristics of the evaluation target, the first classifier being generated by supervised learning using features extracted from the text describing the characteristics of a comparison target that has been evaluated as meeting one of predetermined evaluation criteria defined by a combination of a plurality of the sub-goals for the predetermined goal, thereby obtaining a similarity between the evaluation target and the comparison target for each of the sub-goals; and evaluating a similarity between the evaluation target and an ideal pattern for each of the predetermined evaluation criteria, based on the similarity between the evaluation target and the comparison target for each of the sub-goals and an ideal pattern for each of the predetermined evaluation criteria defined by a combination of the plurality of sub-goals. [Brief explanation of the drawings]

[0008] [Figure 1] 1 is a flowchart illustrating an example of the flow of an evaluation method according to an embodiment of the present disclosure. [Figure 2] 2 is a block diagram showing an example of a functional configuration of an evaluation device 10 according to the embodiment. FIG. [Figure 3] FIG. 10 is a diagram for explaining screening according to the embodiment. [Figure 4] FIG. 10 is a diagram showing an example of a target score TS according to the embodiment. [Figure 5] FIG. 2 is a diagram showing an example of a two-dimensional map TM according to the embodiment. [Figure 6] FIG. 10 is a diagram for explaining in detail the rating of an evaluation target according to the embodiment. [Figure 7] FIG. 10 is a diagram showing an example of a sub-goal score SS according to the embodiment. [Figure 8] FIG. 10 is a diagram showing an example of evaluation criterion definition information SI according to the embodiment. [Figure 9]10 is a diagram for explaining calculation of the similarity between evaluation criterion definition information SI and a target-specific score SS according to the embodiment. FIG. [Figure 10] FIG. 10 is a diagram illustrating an example of a rating according to the embodiment. [Figure 11] 10A and 10B are diagrams illustrating examples of output of rating results and the like according to the embodiment. [Figure 12] 10 is a diagram illustrating highlighting of sentences that contributed to improving the similarity between an evaluation target and a comparison target for each sub-goal according to the embodiment. FIG. [Figure 13] 10 is a diagram illustrating highlighting of sentences that contributed to improving the similarity between an evaluation target and a comparison target for each sub-goal according to the embodiment. FIG. [Figure 14] 10A and 10B are diagrams for explaining anomaly detection based on time-series evaluation according to the embodiment. [Figure 15] FIG. 10 is a diagram showing an example of an estimation result ER of a time-series evaluation according to the embodiment. [Figure 16] FIG. 2 is a block diagram showing an example of the hardware configuration of an information processing device 90 according to the embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0009] Preferred embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings. In this specification and drawings, components having substantially the same functional configurations are designated by the same reference numerals, and redundant description will be omitted.

[0010] The explanation will be given in the following order. 1. Embodiment Overview 1.2. Example of Functional Configuration of Evaluation Device 10 1.3. Screening Details 1.4. Rating Details 1.5. Details of Anomaly Detection 1.6.Effects 2. Hardware configuration example 3. Summary

[0011] <1. Embodiment> <<1.1. Overview>> As mentioned above, in recent years, techniques have been developed for evaluating an evaluation target with respect to a certain goal.

[0012] An example of the above goals is the SDGs, which are international goals set by the United Nations for sustainable development.

[0013] In recent years, an increasing number of companies are trying to improve their image by highlighting their SDG-related initiatives to stakeholders.

[0014] SDG bonds, in which the proceeds are allocated to projects that contribute to the SDGs, are also attracting attention.

[0015] SDG bonds include green bonds, social bonds, and sustainability bonds.

[0016] Green bonds are debt instruments whose proceeds are used to finance projects with clear environmental benefits (green projects).

[0017] Social bonds are debt securities used to finance social projects, including projects that directly address or mitigate specified social challenges or that aim to achieve positive social outcomes.

[0018] Sustainability bonds are debt securities used to finance a combination of green and social projects.

[0019] However, at present, the certification (rating) of SDG bonds as described above is done manually by rating agencies, which is costly. As a result, the number of bonds recognized as SDG bonds is still small, and it is believed that many bonds that should be classified as SDG bonds are buried among non-SDG bonds.

[0020] In order to expand the market for SDG bonds, it is important to reduce the costs required for evaluating SDG bonds and ensure the quality of the evaluations.

[0021] The technical idea of ​​the present disclosure was conceived with the above points in mind, and effectively reduces the cost of the evaluation while ensuring the quality of the evaluation for a specified goal.

[0022] To achieve the above, one of the features of the evaluation device 10 according to one embodiment of the present disclosure is that it evaluates the evaluation target based on the similarity between a text describing the characteristics of a comparison target that has been determined to satisfy a certain criterion with respect to a predetermined goal and a text describing the characteristics of the evaluation target.

[0023] According to the above-mentioned features, it is possible to eliminate much of the manual work involved in evaluating an evaluation target with respect to a predetermined goal, and to significantly reduce the cost associated with the evaluation.

[0024] Furthermore, one of the features of the evaluation device 10 according to an embodiment of the present disclosure is that, in addition to relative evaluation based on comparison with the above-mentioned comparison object, it also performs absolute evaluation based on standards set for a predetermined target, thereby comprehensively evaluating the comparison object.

[0025] The above features enable more detailed and higher quality evaluation than when only relative evaluation is performed. Furthermore, the above features enable more flexible evaluation by changing the criteria for the predetermined goal depending on the situation, etc.

[0026] Here, an overview of an evaluation method according to an embodiment of the present disclosure will be described.

[0027] FIG. 1 is a flowchart showing an example of the flow of the evaluation method according to this embodiment.

[0028] As shown in FIG. 1, the evaluation method according to this embodiment may include three stages: screening (S102), grading (S104), and anomaly detection (S106).

[0029] In the screening in step S102, a process is performed to select evaluation targets that are appropriate as targets for rating from among the screening targets.

[0030] This makes it possible to effectively reduce the cost required for rating by narrowing down the evaluation targets from a huge number of candidates.

[0031] In the rating in step S104, the evaluation targets selected in step S102 are rated based on a comprehensive evaluation including a relative evaluation with a comparison target and an absolute evaluation based on standards set for a predetermined target.

[0032] As a result, as described above, it is possible to achieve more detailed, higher quality, and more flexible evaluation compared to when only relative evaluation is performed.

[0033] Furthermore, in the abnormality detection in step S106, a predetermined abnormality pattern is detected based on the time-series evaluation of the evaluation target.

[0034] The predetermined abnormal pattern may be, for example, a large fluctuation in the evaluation.

[0035] This allows the evaluation target to be continuously evaluated, and when a change in the evaluation occurs, it is detected and notified to the user.

[0036] The evaluation method according to this embodiment has been outlined above. An example of the configuration of the evaluation device 10 that realizes this evaluation method will now be described.

[0037] In the following, a case will be mainly described as an example in which the predetermined goals in this embodiment are the SDGs and the evaluation target is an SDG bond.

[0038] <<1.2. Example of Functional Configuration of Evaluation Device 10>> The evaluation device 10 according to this embodiment is an example of an information processing device that evaluates an evaluation target with respect to a predetermined goal including a plurality of sub-goals, based on a text describing the characteristics of the evaluation target.

[0039] FIG. 2 is a block diagram showing an example of the functional configuration of the evaluation device 10 according to this embodiment.

[0040] As shown in FIG. 2, the evaluation device 10 according to this embodiment may include an input unit 110, an evaluation unit 120, and an output unit 160.

[0041] The evaluation unit 120 may also include a screening unit 130 that performs the screening in step S102, a rating unit 140 that performs the rating in step S104, and an anomaly detection unit 150 in step S106.

[0042] (input unit 110) The input unit 110 according to this embodiment inputs information to the evaluation unit 120 based on an operation by a user.

[0043] For this purpose, the input unit 110 according to this embodiment includes input devices such as a mouse and a keyboard.

[0044] The information includes, for example, text describing the characteristics of the screening target, text describing the characteristics of the evaluation target, comments by third parties on the evaluation target, and text describing the characteristics of the comparison target.

[0045] (Evaluation unit 120) The evaluation unit 120 according to this embodiment evaluates an evaluation target with respect to a predetermined goal including a plurality of sub-goals, based on a text describing the characteristics of the evaluation target.

[0046] The predetermined goal may be, for example, the SDGs, and in this case, the sub-goals may be goals for each of the 17 fields defined in the SDGs.

[0047] In addition, the evaluation unit 120 according to this embodiment may obtain the similarity between the evaluation target and the comparison target for each sub-goal by inputting features extracted from text describing the characteristics of the evaluation target into the first classifier.

[0048] The comparison object according to this embodiment may be, for example, a financial product.

[0049] As an example, if the predetermined goal is the SDGs, the evaluation target according to this embodiment may be a bond that has the potential to be an SDG bond.

[0050] In this case, the comparison may be with SDG bonds that have been assessed by an assessment agency as meeting any of the predetermined assessment criteria defined by a combination of multiple sub-goals.

[0051] Furthermore, the first classifier may be a classification model generated by supervised learning using features extracted from text describing the characteristics of the comparison target.

[0052] Furthermore, one of the features of the evaluation unit 120 according to this embodiment is that it evaluates the similarity between the evaluation target and the comparison target for each sub-goal obtained using the first classifier, and the similarity between the ideal pattern for each predetermined evaluation criterion and the evaluation target based on the similarity between the evaluation target and the comparison target for each sub-goal obtained using the first classifier, and the ideal pattern for each predetermined evaluation criterion defined by a combination of multiple sub-goals.

[0053] For example, if the predetermined goals are the SDGs, the predetermined evaluation criteria may include green criteria, social criteria, and sustainability criteria.

[0054] For example, the green standard is defined by a combination of several sub-goals related to green projects out of 17 sub-goals.

[0055] For example, the social standard is defined by combining multiple sub-goals related to social projects among the 17 sub-goals.

[0056] For example, sustainability standards are defined by combining multiple sub-goals that are commonly related to green projects and social projects out of the 17 sub-goals.

[0057] The evaluation section 120 according to this embodiment may rank the evaluation target with respect to a predetermined goal based on the degree of similarity between the evaluation target and the ideal pattern for each of the above-mentioned predetermined evaluation criteria.

[0058] According to the evaluation unit 120 of this embodiment, it is possible to realize a more detailed and high-quality evaluation by combining a relative evaluation based on a comparison with a comparison target and an absolute evaluation based on a predetermined evaluation standard.

[0059] Furthermore, the evaluation unit 120 according to this embodiment can change the ideal pattern of the predetermined evaluation criterion depending on the situation, etc., thereby enabling more flexible evaluation.

[0060] The functions of the evaluation unit 120 according to this embodiment are realized by various processors. Details of the functions of the evaluation unit 120 according to this embodiment will be described separately.

[0061] (output unit 160) The output section 160 according to this embodiment outputs the result of the evaluation by the evaluation section 120 .

[0062] For this reason, the output unit 160 according to this embodiment includes various displays, printers, and the like.

[0063] An example of output by the output unit 160 according to this embodiment will be described separately.

[0064] The above has described an example of the functional configuration of the evaluation device 10 according to this embodiment. Note that the functional configuration described above using Fig. 2 is merely an example, and the functional configuration of the evaluation device 10 according to this embodiment is not limited to this example.

[0065] The functional configuration of the evaluation device 10 according to this embodiment can be flexibly modified according to specifications, operation, and the like.

[0066] <<1.3. Screening Details>> Next, the screening of the evaluation target by the screening unit 130 according to this embodiment will be described in detail.

[0067] The screening unit 130 according to this embodiment selects evaluation targets from among screening targets. By narrowing down evaluation targets from a vast number of candidates, the screening unit 130 can effectively reduce the cost required for rating.

[0068] At this time, the screening unit 130 according to this embodiment may determine whether the screening target is appropriate as an evaluation target based on text describing the characteristics of the screening target.

[0069] FIG. 3 is a diagram for explaining screening according to this embodiment.

[0070] As shown in FIG. 3, the screening unit 130 according to this embodiment may include a sentence feature extraction unit 310, a target score calculation unit 320, and a map generation unit 330.

[0071] In the screening according to this embodiment, first, an input sentence IS1 is input to the sentence feature extraction unit 310.

[0072] Here, the input text IS1 is a free-form text including non-financial information describing the characteristics of the screening target, and the data format and the written language are not limited.

[0073] When the screening target is a financial product such as a bond, the input text IS1 may be any type of text containing information about the financial product or the issuer of the financial product.

[0074] The input document IS1 may be, for example, a shelf registration supplement, a securities report, an integrated report, a sustainability report, or the like.

[0075] The text feature extraction unit 310 according to this embodiment extracts feature vectors from the input text IS1 through natural language processing using a neural network. Note that the diamonds in the figure represent neural networks.

[0076] The text feature extraction unit 310 according to this embodiment may extract a feature vector from the input text IS1 using, for example, Bidirectional Encoder Representations from Transformers (BERT).

[0077] The feature vector extracted from the input sentence IS1 is input to the target score calculation unit 320.

[0078] The target score calculation unit 320 in this embodiment inputs the feature vector extracted from the input sentence IS1 into a second classifier to obtain a target score TS indicating the similarity between the comparison target and the screening target for each specified evaluation criterion.

[0079] The second classifier is generated by supervised learning using features extracted from text describing characteristics of the comparison target that have been evaluated by an evaluation organization as meeting one of the predetermined evaluation criteria.

[0080] The second classifier may be generated using a model such as BiLSTM (Bidirectional Long Short Term Memory).

[0081] In addition, if the predetermined goal is the SDGs, the comparison may be with a green bond, social bond, or sustainability bond certified by an evaluation agency.

[0082] In this case, the target score TS may include the probability that each screening target is a green bond, the probability that it is a social bond, and the probability that it is a sustainability bond, as shown in Figure 4.

[0083] 4 is a diagram showing an example of a target score TS according to this embodiment. In the example shown in FIG. 4, the screening target "Bond X" has a 61% probability of being a social bond and a 35% probability of being a sustainability bond.

[0084] Furthermore, the screening target "Bond Y" has a 92% probability of being a social bond and a 5% probability of being a sustainability bond.

[0085] Furthermore, the screening target "Bond Z" has a 97% probability of being a green bond.

[0086] The screening section 130 according to this embodiment may determine whether or not the screening target is appropriate as an evaluation target based on the target score TS as described above, and may perform selection based on the result of this determination.

[0087] As an example, the screening unit 130 may select as evaluation targets those bonds whose probability of being a green bond, probability of being a social bond, or probability of being a sustainability bond exceeds a predetermined standard.

[0088] Furthermore, the map generating section 330 according to this embodiment may generate a two-dimensional map that represents the similarity between the comparison target and the screening target for each predetermined evaluation criterion as a distance in a two-dimensional space.

[0089] The map generation unit 330 according to this embodiment may generate the two-dimensional map TM by inputting the feature vector obtained in the intermediate layer of the above-described second classifier to an encoder 332 generated by learning a variational auto-encoder (VAE).

[0090] Note that in generating the two-dimensional map TM, the output by the decoder 334 may be discarded.

[0091] Figure 5 is a diagram showing an example of a two-dimensional map TM according to this embodiment. In the example shown in Figure 5, the green bonds, social bonds, sustainability bonds, and non-SDG bonds that are the targets of comparison are represented by filled circles, squares, triangles, and stars, respectively.

[0092] In the example shown in FIG. 5, the bonds to be screened are represented by hollow pentagonal markers.

[0093] The output unit 160 according to this embodiment may output a two-dimensional map TM as shown in FIG. 5 to the display in addition to the target score TS as shown in FIG.

[0094] This allows the user to grasp detailed values ​​relating to the similarity between the comparison target and the screening target, and also allows the user to intuitively visually recognize the similarity.

[0095] <<1.4. Rating Details>> Next, the rating of the evaluation object by the rating unit 140 according to this embodiment will be described in detail.

[0096] FIG. 6 is a diagram for explaining in detail the rating of the evaluation object according to this embodiment.

[0097] As shown in FIG. 6, the rating unit 140 according to this embodiment may include a sentence feature extraction unit 410, a sub-goal score calculation unit 420, and a similarity calculation unit 430.

[0098] In the screening according to this embodiment, first, the input sentence IS2 is input to the sentence feature extraction unit 410.

[0099] Here, the input text IS2 is a free-form text including non-financial information describing the characteristics of the object to be evaluated, and the data format and the written language are not limited.

[0100] When the evaluation target is a financial product such as a bond, the input text IS2 may be any type of text containing information about the financial product or the issuer of the financial product.

[0101] The input document IS2 may be, for example, a shelf registration supplement, a securities report, an integrated report, a sustainability report, or the like.

[0102] The text feature extraction unit 310 according to this embodiment extracts feature vectors from the input text IS2 by natural language processing using a neural network. Note that the diamonds in the figure represent neural networks.

[0103] The text feature extraction unit 410 according to this embodiment may extract a feature vector from the input text IS2 using, for example, BERT.

[0104] The feature vector extracted from the input sentence IS2 is input to the sub-goal score calculation unit 420.

[0105] The sub-goal score calculation unit 420 in this embodiment inputs the feature vector extracted from the input sentence IS2 into the first classifier, and obtains the sub-goal score SS, which indicates the similarity between the evaluation target and the comparison target for each sub-goal.

[0106] The first classifier, like the second classifier, may be a classification model generated by supervised learning using features extracted from text describing characteristics of a comparison target that have been evaluated by an evaluation organization as meeting one of predetermined evaluation criteria.

[0107] However, unlike the second classifier, the first classifier outputs a value relating to the similarity between the evaluation target and the comparison target for each sub-goal.

[0108] The first classifier may be generated using a model such as BiLSTM, for example.

[0109] Fig. 7 is a diagram showing an example of a sub-goal score SS according to this embodiment. Fig. 7 illustrates an example of a sub-goal score SS when the predetermined goal is the SDGs and the sub-goals are goals for each of the 17 fields defined in the SDGs.

[0110] In addition, in the example shown in FIG. 7, the similarity between each sub-goal and the comparison target is rounded to either "0" or "1."

[0111] For example, the evaluation target "Bond X" has sub-goals "03" to "07" and "09" to "13" that are similar to the SDG bonds being compared.

[0112] In addition, the evaluation target "Bond Y" has sub-goals "03" and "05" to "13" that are similar to the SDG bonds that are being compared.

[0113] In addition, the evaluated bond "Bond Z" has sub-goals "09" and "11" to "13" that are similar to the SDG bonds it is being compared to.

[0114] According to such a sub-goal score SS, it is possible to evaluate the evaluation target with respect to a predetermined goal based on a relative comparison with the comparison target.

[0115] Furthermore, the similarity calculation section 430 according to this embodiment may further perform absolute evaluation based on evaluation criterion definition information SI that defines an ideal pattern for each of the predetermined evaluation criteria.

[0116] Fig. 8 is a diagram showing an example of evaluation criteria definition information SI according to this embodiment. Note that Fig. 8 illustrates an example of evaluation criteria definition information SI in the case where the predetermined goal is the SDGs and the sub-goals are the 17 field-specific goals defined in the SDGs.

[0117] For example, the evaluation criterion "Green" is defined by a combination of sub-goals "06," "07," "09" to "14," and "17."

[0118] In addition, the evaluation criterion "social" is defined by a combination of sub-goals "03," "04," and "09" to "13."

[0119] In addition, the evaluation criterion "Sustainability" is defined by a combination of sub-goals "03" to "09" and "11" to "14."

[0120] The similarity calculation section 430 according to this embodiment may calculate the similarity between the evaluation criterion definition information SI and the sub-goal score SS as described above.

[0121] FIG. 9 is a diagram for explaining calculation of the similarity between the evaluation criterion definition information SI and the sub-goal score SS according to this embodiment.

[0122] The similarity calculation unit 430 according to this embodiment may, for example, compare the values ​​("0" or "1") of the evaluation criterion definition information SI and the sub-goal score SS for each sub-goal, and calculate a similarity that summarizes the results of the comparison for each evaluation criterion.

[0123] In the example shown in Figure 9, the evaluation target "Bond X" has a similarity of 74% to the evaluation criterion "Green", a similarity of 84% to the evaluation criterion "Social", and a similarity of 86% to the evaluation criterion "Sustainability".

[0124] Furthermore, the evaluation target "Bond Y" has a similarity of 74% to the evaluation criterion "green", a similarity of 72% to the evaluation criterion "social", and a similarity of 86% to the evaluation criterion "sustainability".

[0125] Furthermore, the evaluation target "Bond Z" has a similarity of 67% to the evaluation criterion "green", a similarity of 76% to the evaluation criterion "social", and a similarity of 60% to the evaluation criterion "sustainability".

[0126] Furthermore, the similarity calculation section 430 according to this embodiment may output rating information RI based on the similarity as described above.

[0127] FIG. 10 is a diagram for explaining an example of the rating of the evaluation object according to this embodiment.

[0128] The rating of the evaluation target according to this embodiment may be determined based on the evaluation criterion that has the highest similarity between the evaluation criterion definition information SI and the sub-goal score SS described above, for example.

[0129] For example, if the similarity for the evaluation criterion "green" is the highest, the rating of the corresponding evaluation target may be determined as either "GA," "GB," or "GC" depending on the value of the similarity, as shown in Figure 10.

[0130] Furthermore, for example, if the similarity regarding the evaluation criterion "social" is the highest, the rating of the corresponding evaluation target may be determined as either "So-A," "So-B," or "So-C" depending on the value of the similarity, as shown in Figure 10.

[0131] Furthermore, for example, if the similarity regarding the evaluation criterion "sustainability" is the highest, the rating of the corresponding evaluation target may be determined as either "Su-A," "Su-B," or "Su-C" depending on the value of the similarity, as shown in Figure 10.

[0132] The rating of the evaluation object by the rating unit 140 according to this embodiment has been described above.

[0133] Next, an example of output of the rating results etc. according to this embodiment will be described.

[0134] The output section 160 according to this embodiment may output the rating results by the rating section 140 to a display or the like.

[0135] FIG. 11 is a diagram showing an example of an output of a rating result etc. according to this embodiment.

[0136] As shown in the output example RO in Figure 11, the output unit 160 may output the name of the evaluation target (here, the name of the bond), the name of the document used in the evaluation, the rating result, and the score used in the rating (for example, the similarity between the evaluation criteria definition information SI and the sub-goal score SS).

[0137] Furthermore, the rating unit 140 according to this embodiment may extract sentences that contributed to improving the similarity between the evaluation target and the comparison target for each sub-goal from the text describing the characteristics of the evaluation target.

[0138] In this case, the output unit 160 according to this embodiment may display a list of sentences (or words) that contributed to improving the similarity between the evaluation target and the comparison target for each sub-goal, as shown in FIG.

[0139] In the example shown in Figure 11, the output unit 160 displays a list of sentences such as "social Welfare" and "medical facilities" extracted by the rating unit 140 as sentences that contributed to improving the above-mentioned similarity in sub-goals "01" and "02."

[0140] The output unit 160 also displays a list of sentences such as "medical Welfare" and "earthquake disaster" extracted by the rating unit 140 as sentences that contributed to improving the similarity for the sub-goal "032."

[0141] This type of list display allows users to clearly understand the sentences that contributed to improving the similarity.

[0142] Furthermore, the output unit 160 may highlight sentences that contribute to improving the similarity in the text (input sentence IS2) in which the characteristic to be evaluated is described.

[0143] 12 and 13 are diagrams illustrating highlighting of sentences that contributed to improving the similarity between the evaluation target and the comparison target for each sub-goal according to this embodiment.

[0144] For example, when the user selects sub-goal "01" in the list display in FIG. 11, the output section 160 may output detailed information DI1 as shown in FIG.

[0145] The detailed information DI1 may include the selected sub-objective "01", the name of the text (document name) containing the relevant sentence, an excerpt ED1 of the portion of the text where the relevant sentence is written, and the like.

[0146] The output unit 160 may highlight the relevant sentence in the excerpt ED1 by, for example, changing the background color or adding decoration such as underlining.

[0147] On the other hand, when the user selects sub-goal "03" in the list display in FIG. 11, the output section 160 may output detailed information DI2 as shown in FIG.

[0148] The detailed information DI2 may include the selected sub-objective "03", the name of the text (document name) containing the relevant sentence, an excerpt ED2 of the portion of the text where the relevant sentence is written, and the like.

[0149] As shown in excerpt ED2, the text that contributed to improving the similarity between the evaluation target and the comparison target for each sub-objective may be included in a table or graph.

[0150] By inputting the sentences contained in the tables and graphs into the sentence feature extraction unit 410, the rating unit 140 can calculate the degree of contribution of the sentences to the similarity between the evaluation target and the comparison target for each sub-goal.

[0151] Furthermore, the contribution of each sentence to the similarity between the evaluation target and the comparison target may be used to suggest sentence corrections to the user.

[0152] For example, the output unit 160 can output a rating result for a certain sentence when the sentence's contribution to the similarity between the evaluation target and the comparison target for each sub-goal is improved.

[0153] At this time, the output unit 160 may also output sentences relating to other bonds that are evaluated as having a high degree of contribution to the similarity with the comparison target in the target sub-goal.

[0154] The output described above allows the user to refer to specific examples and revise the text to improve the rating results.

[0155] <<1.5. Details of Anomaly Detection>> Next, abnormality detection by the abnormality detection unit 150 according to this embodiment will be described in detail.

[0156] The rating unit 140 according to this embodiment may repeatedly output the sub-goal score SS for the same evaluation target on a regular or irregular basis.

[0157] In this case, the anomaly detection unit 150 according to this embodiment may perform a time-series evaluation of the evaluation object with respect to a predetermined target.

[0158] Furthermore, the anomaly detection unit 150 according to this embodiment can detect a predetermined anomaly pattern related to the evaluation of the evaluation target based on the above-mentioned time-series evaluation.

[0159] FIG. 14 is a diagram for explaining anomaly detection based on time-series evaluation according to this embodiment.

[0160] The rating unit 140 according to this embodiment receives an input sentence IS3 as an input and repeatedly outputs a sub-goal score SS for a certain evaluation target, either periodically or irregularly.

[0161] Here, the input text IS2 is a free-form text including non-financial information describing the characteristics of the object to be evaluated, and the data format and the written language are not limited.

[0162] When the object of evaluation is a financial product such as a bond, the input text IS3 may include various texts including information about the financial product or the issuer of the financial product.

[0163] Input documents IS3 may include, for example, shelf registration supplements, securities reports, integrated reports, sustainability reports, etc.

[0164] The input text IS3 also includes text in which a third party's comments on the evaluation target are written.

[0165] The text containing third-party comments on the subject of evaluation may be, for example, news reported by a third party (e.g., a news organization) regarding a financial product or the issuer of the financial product, or a report issued by a third party (e.g., an NGO / NPO).

[0166] Since the input sentence IS3 includes text containing comments from third parties as described above, the rating unit 140 can recalculate the sub-goal score SS each time, reflecting the comments from third parties.

[0167] The time series estimation unit 510 provided in the anomaly detection unit 150 in this embodiment performs time series evaluation (time series estimation) based on multiple sub-target scores SS for the same evaluation target output by the rating unit 140, and outputs the estimation result ER.

[0168] The time series estimation unit 510 may perform the above time series evaluation using, for example, a Long Short Term Memory (LSTM) model.

[0169] FIG. 15 is a diagram showing an example of the estimation result ER of the time-series evaluation according to this embodiment.

[0170] The estimation results ER shown in Figure 15 show the time series trends of the score indicating the probability that the evaluation target is likely to be an SDG bond and the score indicating the probability that the evaluation target is likely to be a non-SDG bond.

[0171] The abnormality detection unit 150 according to this embodiment detects a predetermined abnormal pattern related to the evaluation of the evaluation target based on such an estimation result ER.

[0172] The predetermined abnormal pattern may be, for example, a large fluctuation in the evaluation (score).

[0173] As an example, the abnormal pattern according to this embodiment may include greenwashing.

[0174] Here, greenwashing refers to the practice of appearing to be environmentally conscious when in reality it is not, misleading environmentally conscious consumers.

[0175] For example, when referring to texts published by the issuer at the time of issuance of the bond being evaluated, even if it is determined that the bond or issuer is highly conscious of the SDGs, there may be a discrepancy with the actual situation, or there may be a discrepancy with the actual situation in the future.

[0176] In the anomaly detection according to this embodiment, it is possible to detect the above-mentioned deviation by performing a time-series evaluation of the evaluation target based on comments by a third party.

[0177] In the example shown in Figure 15, the anomaly detection unit 150 may detect greenwashing when the score indicating the probability that the evaluation target is likely to be a non-SDG bond exceeds the score indicating the probability that the evaluation target is likely to be an SDG bond.

[0178] In this way, the anomaly detection unit 150 according to this embodiment can detect an abnormal pattern such as greenwashing by performing a time-series evaluation of the same evaluation target, and notify the user of information related to the abnormal pattern. <<1.6.Effects>>

[0179] The evaluation method according to this embodiment has been described in detail above.

[0180] According to the evaluation method of this embodiment, it is possible to automate evaluation work that has been performed manually up until now, thereby achieving scale-out of evaluation efficiency.

[0181] According to the evaluation method of this embodiment, by making highly descriptive evaluation algorithms and architectures public, it is possible to improve the transparency of the evaluation logic, which was previously a gray box, and ensure the objectivity of the evaluation.

[0182] According to the evaluation method of this embodiment, it is possible to formalize the intention of rating by using an ideal pattern for each predetermined evaluation criterion.

[0183] According to the evaluation method of this embodiment, it is expected that the explanatory power of the rating will be improved by extracting the sentences that are the basis for the rating.

[0184] According to the evaluation method of this embodiment, it is possible to recommend to the user sentences that improve the rating.

[0185] Furthermore, according to the evaluation method of this embodiment, the rating can be revised each time by time-series evaluation, making it possible to guarantee the quality of credit information.

[0186] Although the evaluation of SDG bonds has been mainly described above as an example, the scope of application of the evaluation method according to this embodiment is not limited to this example.

[0187] For example, it can be used to rate ordinary credits based on financial information.

[0188] For example, the evaluation method according to the present embodiment can be applied to the evaluation of the credit information of a borrower in a personal loan. In this case, the evaluation method according to the present embodiment can be used to assign credit information using non-financial information of the borrower, and follow-up investigations can be conducted even after the loan is taken out to evaluate the bankruptcy risk in advance.

[0189] Furthermore, for example, the evaluation method according to this embodiment can also be applied to personnel evaluation. In this case, evaluation is based on documents containing information about the recruit, and a chronological evaluation is performed based on the output of the recruit after the recruitment, thereby ensuring the quality of personnel evaluation.

[0190] <2. Hardware configuration example> Next, a hardware configuration example of the evaluation device 10 according to an embodiment of the present disclosure will be described. Fig. 16 is a block diagram showing a hardware configuration example of an information processing device 90 according to an embodiment of the present disclosure. The information processing device 90 may be a device having a hardware configuration equivalent to that of the evaluation device 10.

[0191] 16, the information processing device 90 includes, for example, a processor 871, a ROM 872, a RAM 873, a host bus 874, a bridge 875, an external bus 876, an interface 877, an input device 878, an output device 879, a storage 880, a drive 881, a connection port 882, and a communication device 883. Note that the hardware configuration shown here is an example, and some of the components may be omitted. Furthermore, the information processing device 90 may further include components other than those shown here.

[0192] (Processor 871) The processor 871 functions, for example, as an arithmetic processing device or control device, and controls the overall operation of each component or part of it based on various programs recorded in the ROM 872, RAM 873, storage 880, or removable storage medium 901.

[0193] (ROM872, RAM873) The ROM 872 is a means for storing programs to be read into the processor 871, data to be used for calculations, etc. The RAM 873 temporarily or permanently stores, for example, programs to be read into the processor 871, and various parameters that change as appropriate when the programs are executed.

[0194] (Host bus 874, bridge 875, external bus 876, interface 877) The processor 871, ROM 872, and RAM 873 are connected to one another via, for example, a host bus 874 that is capable of high-speed data transmission. On the other hand, the host bus 874 is connected to, for example, an external bus 876 that has a relatively low data transmission speed via a bridge 875. In addition, the external bus 876 is connected to various components via an interface 877.

[0195] (Input Device 878) The input device 878 may be, for example, a mouse, keyboard, touch panel, button, switch, lever, etc. Furthermore, a remote controller (hereinafter referred to as a remote control) capable of transmitting control signals using infrared rays or other radio waves may also be used as the input device 878. The input device 878 may also include an audio input device such as a microphone.

[0196] (Output Device 879) The output device 879 is a device capable of visually or audibly notifying the user of acquired information, such as a display device such as a CRT (Cathode Ray Tube), LCD, or organic EL, an audio output device such as a speaker or headphones, a printer, a mobile phone, a facsimile, etc. The output device 879 according to the present disclosure also includes various vibration devices capable of outputting tactile stimuli.

[0197] (Storage 880) The storage 880 is a device for storing various types of data. For example, a magnetic storage device such as a hard disk drive (HDD), a semiconductor storage device, an optical storage device, or a magneto-optical storage device may be used as the storage 880.

[0198] (Drive 881) The drive 881 is a device that reads information recorded on a removable storage medium 901 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory, or writes information to the removable storage medium 901 .

[0199] (Removable storage medium 901) The removable storage medium 901 is, for example, a DVD medium, a Blu-ray (registered trademark) medium, an HD DVD medium, various semiconductor storage media, etc. Of course, the removable storage medium 901 may also be, for example, an IC card equipped with a contactless IC chip, an electronic device, etc.

[0200] (Connection port 882) The connection port 882 is a port for connecting an external device 902, such as a USB (Universal Serial Bus) port, an IEEE1394 port, a SCSI (Small Computer System Interface), an RS-232C port, or an optical audio terminal.

[0201] (External connection device 902) The externally connected device 902 is, for example, a printer, a portable music player, a digital camera, a digital video camera, or an IC recorder.

[0202] (Communication Device 883) The communication device 883 is a communication device for connecting to a network, such as a communication card for wired or wireless LAN, Bluetooth (registered trademark), or WUSB (Wireless USB), a router for optical communication, a router for ADSL (Asymmetric Digital Subscriber Line), or a modem for various types of communication.

[0203] <3. Summary> As described above, the evaluation device 10 according to one embodiment of the present disclosure includes an evaluation unit 120 that evaluates an evaluation target with respect to a predetermined goal including multiple sub-goals, based on text describing the characteristics of the evaluation target.

[0204] Furthermore, one of the features of the evaluation unit 120 according to an embodiment of the present disclosure is that it obtains the similarity between the evaluation target and the comparison target for each sub-goal by inputting features extracted from text describing the characteristics of the evaluation target into a first classifier generated by supervised learning using features extracted from text describing the characteristics of the comparison target that has been evaluated as satisfying any of the predetermined evaluation criteria defined by a combination of multiple sub-goals for the predetermined goal, and evaluates the similarity between the evaluation target and the ideal pattern for each predetermined evaluation criterion based on the similarity between the evaluation target and the comparison target for each sub-goal and the ideal pattern for each predetermined evaluation criterion defined by a combination of multiple sub-goals.

[0205] According to the above configuration, it is possible to effectively reduce the cost of the evaluation while ensuring the quality of the evaluation against a predetermined target.

[0206] Although the preferred embodiments of the present disclosure have been described in detail above with reference to the accompanying drawings, the technical scope of the present disclosure is not limited to such examples. It is clear that a person skilled in the art of the present disclosure can conceive of various modified or altered examples within the scope of the technical idea described in the claims, and it is understood that these also naturally fall within the technical scope of the present disclosure.

[0207] Furthermore, the steps of the processes described in this specification do not necessarily have to be processed in chronological order according to the order shown in the flowcharts or sequence diagrams. For example, the steps of the processes of each device may be processed in an order different from the order shown, or may be processed in parallel.

[0208] Furthermore, the series of processes performed by each device described in this specification may be realized using software, hardware, or a combination of software and hardware. The programs constituting the software may be provided, for example, inside or outside each device and stored in advance in a non-transitory computer-readable medium. Each program is then loaded into RAM when executed by a computer and executed by various processors. Examples of the storage medium include a magnetic disk, an optical disk, a magneto-optical disk, and a flash memory. Furthermore, the computer programs may be distributed, for example, via a network, without using a storage medium.

[0209] Furthermore, the effects described herein are merely descriptive or exemplary and are not limiting. In other words, the technology according to the present disclosure may achieve other effects that are apparent to those skilled in the art from the description of this specification, in addition to or in place of the above-described effects.

[0210] The following configurations also fall within the technical scope of the present disclosure. (1) an evaluation unit that evaluates the evaluation object with respect to a predetermined goal including a plurality of subgoals based on a text describing the characteristics of the evaluation object; Equipped with the evaluation unit acquires a similarity between the evaluation target and the comparison target for each sub-goal by inputting feature amounts extracted from text describing characteristics of the evaluation target into a first classifier generated by supervised learning using feature amounts extracted from text describing characteristics of the comparison target that have been evaluated as meeting any of predetermined evaluation criteria defined by a combination of a plurality of sub-goals with respect to the predetermined goal; evaluating the similarity between the evaluation target and the comparison target for each of the sub-goals, and the ideal pattern for each of the predetermined evaluation criteria defined by a combination of a plurality of the sub-goals, and Information processing device. (2) the evaluation unit ranks the evaluation target with respect to the predetermined goal based on the similarity between the ideal pattern for each of the predetermined evaluation criteria and the evaluation target; The information processing device according to (1) above. (3) The evaluation unit extracts sentences that contributed to improving the similarity between the evaluation target and the comparison target for each sub-goal from a text describing the characteristics of the evaluation target. The information processing device according to (1) or (2). (4) an output unit that outputs a result of the evaluation by the evaluation unit; Further provided with The information processing device according to (3) above. (5) the output unit displays a list of sentences that contributed to improving the similarity between the evaluation target and the comparison target for each of the sub-goals. The information processing device according to (4) above. (6) the output unit highlights and displays sentences that contributed to improving the similarity between the evaluation target and the comparison target for each sub-goal in the text describing the characteristics of the evaluation target. The information processing device according to (4) or (5). (7) The evaluation unit performs a time-series evaluation of the evaluation object with respect to the predetermined goal. 7. The information processing device according to any one of (1) to 6. (8) the evaluation unit detects a predetermined abnormal pattern related to the evaluation of the evaluation target based on the time-series evaluation. The information processing device according to (7) above. (9) The predetermined abnormal pattern includes greenwashing. The information processing device according to (8). (10) the evaluation unit performs the time-series evaluation based on a text describing the characteristics of the evaluation target and a text describing a third party's comment on the evaluation target; The information processing device according to any one of (7) to (9). (11) the evaluation unit determines whether the screening target is appropriate as the evaluation target based on a text describing the characteristics of the screening target; The information processing device according to any one of (1) to (10) above. (12) the evaluation unit inputs features extracted from text describing the characteristics of the screening target into a second classifier generated by supervised learning using features extracted from text describing the characteristics of the comparison target evaluated as satisfying any of the predetermined evaluation criteria, thereby obtaining a similarity between the comparison target and the screening target for each of the predetermined evaluation criteria, and determines whether the screening target is appropriate as the evaluation target based on the similarity. The information processing device according to (11) above. (13) The evaluation object includes a financial product. The information processing device according to any one of (1) to (12). (14) The text describing the characteristics of the object to be evaluated includes information about the financial instrument or the issuer of the financial instrument; The information processing device according to (13) above. (15) The predetermined goals include the SDGs. The information processing device according to any one of (1) to (14). (16) The predetermined evaluation criteria include green criteria, social criteria, and sustainability criteria; The information processing device according to (15) above. (17) a processor evaluating the evaluation target against a predetermined goal including a plurality of subgoals based on a text describing the characteristics of the evaluation target; Including, The evaluation includes inputting features extracted from text describing characteristics of the evaluation target into a first classifier generated by supervised learning using features extracted from text describing characteristics of a comparison target evaluated as satisfying any of predetermined evaluation criteria defined by a combination of a plurality of sub-goals with respect to the predetermined goal, thereby obtaining a similarity between the evaluation target and the comparison target for each of the sub-goals; Evaluating the similarity between the evaluation target and the comparison target for each of the sub-goals and the ideal pattern for each of the predetermined evaluation criteria defined by a combination of a plurality of the sub-goals, based on the similarity between the evaluation target and the comparison target for each of the sub-goals and the ideal pattern for each of the predetermined evaluation criteria; further comprising: Information processing methods. (18) Computer, an evaluation unit that evaluates the evaluation object with respect to a predetermined goal including a plurality of subgoals based on a text describing the characteristics of the evaluation object; Equipped with the evaluation unit acquires a similarity between the evaluation target and the comparison target for each sub-goal by inputting feature amounts extracted from text describing characteristics of the evaluation target into a first classifier generated by supervised learning using feature amounts extracted from text describing characteristics of the comparison target that have been evaluated as meeting any of predetermined evaluation criteria defined by a combination of a plurality of sub-goals with respect to the predetermined goal; evaluating the similarity between the evaluation target and the comparison target for each of the sub-goals, and the ideal pattern for each of the predetermined evaluation criteria defined by a combination of a plurality of the sub-goals, and information processing device, A program to function as a [Explanation of symbols]

[0211] 10 Evaluation equipment 110 Input section 120 Evaluation Department 130 Screening Department 140 Rating Department 150 Abnormality detection unit 160 Output section 310 Text feature extraction unit 320 Target Score Calculation Section 330 Map Generation Unit 410 Text feature extraction unit 420 Sub-goal score calculation section 430 Similarity calculation unit 510 Time Series Estimation Unit

Claims

1. an evaluation unit that evaluates the evaluation object with respect to a predetermined goal including a plurality of subgoals based on a text describing the characteristics of the evaluation object; Equipped with the evaluation unit acquires a similarity between the evaluation target and the comparison target for each sub-goal by inputting feature amounts extracted from text describing characteristics of the evaluation target into a first classifier generated by supervised learning using feature amounts extracted from text describing characteristics of the comparison target that has been evaluated as satisfying any of predetermined evaluation criteria defined by a combination of a plurality of sub-goals with respect to the predetermined goal; evaluating the similarity between the evaluation target and the comparison target for each of the sub-goals, and the ideal pattern for each of the predetermined evaluation criteria defined by a combination of a plurality of the sub-goals, and Information processing device.

2. the evaluation unit ranks the evaluation target with respect to the predetermined goal based on the similarity between the ideal pattern for each of the predetermined evaluation criteria and the evaluation target; The information processing device according to claim 1 .

3. The evaluation unit extracts sentences that contributed to improving the similarity between the evaluation target and the comparison target for each sub-goal from a text describing the characteristics of the evaluation target. The information processing device according to claim 1 .

4. an output unit that outputs a result of the evaluation by the evaluation unit; Further provided with The information processing device according to claim 3 .

5. the output unit displays a list of sentences that contributed to improving the similarity between the evaluation target and the comparison target for each of the sub-goals. The information processing device according to claim 4 .

6. the output unit highlights and displays sentences that contributed to improving the similarity between the evaluation target and the comparison target for each sub-goal in the text describing the characteristics of the evaluation target. The information processing device according to claim 4 .

7. The evaluation unit performs a time-series evaluation of the evaluation object with respect to the predetermined goal. The information processing device according to claim 1 .

8. the evaluation unit detects a predetermined abnormal pattern related to the evaluation of the evaluation target based on the time-series evaluation. The information processing device according to claim 7 .

9. The predetermined abnormal pattern includes greenwashing. The information processing device according to claim 8 .

10. the evaluation unit performs the time-series evaluation based on a text describing the characteristics of the evaluation target and a text describing a third party's comment on the evaluation target; The information processing device according to claim 7 .

11. the evaluation unit determines whether the screening target is appropriate as the evaluation target based on a text describing the characteristics of the screening target; The information processing device according to claim 1 .

12. the evaluation unit inputs features extracted from text describing the characteristics of the screening target into a second classifier generated by supervised learning using features extracted from text describing the characteristics of the comparison target evaluated as satisfying any of the predetermined evaluation criteria, thereby obtaining a similarity between the comparison target and the screening target for each of the predetermined evaluation criteria, and determines whether the screening target is appropriate as the evaluation target based on the similarity; The information processing device according to claim 11.

13. The evaluation object includes a financial product. The information processing device according to claim 1 .

14. The text describing the characteristics of the object to be evaluated includes information about the financial instrument or the issuer of the financial instrument; The information processing device according to claim 13.

15. The predetermined goals include the SDGs. The information processing device according to claim 1 .

16. The predetermined evaluation criteria include green criteria, social criteria, and sustainability criteria; The information processing device according to claim 15.

17. a processor evaluating the evaluation target against a predetermined goal including a plurality of subgoals based on a text describing the characteristics of the evaluation target; Including, The evaluation includes inputting features extracted from text describing characteristics of the evaluation target into a first classifier generated by supervised learning using features extracted from text describing characteristics of a comparison target evaluated as satisfying any of predetermined evaluation criteria defined by a combination of a plurality of sub-goals with respect to the predetermined goal, thereby obtaining a similarity between the evaluation target and the comparison target for each of the sub-goals; Evaluating the similarity between the evaluation target and the comparison target for each of the sub-goals and the ideal pattern for each of the predetermined evaluation criteria defined by a combination of a plurality of the sub-goals, based on the similarity between the evaluation target and the comparison target for each of the sub-goals and the ideal pattern for each of the predetermined evaluation criteria; further comprising: Information processing methods.

18. Computer, an evaluation unit that evaluates the evaluation object with respect to a predetermined goal including a plurality of subgoals based on a text describing the characteristics of the evaluation object; Equipped with the evaluation unit acquires a similarity between the evaluation target and the comparison target for each sub-goal by inputting feature amounts extracted from text describing characteristics of the evaluation target into a first classifier generated by supervised learning using feature amounts extracted from text describing characteristics of the comparison target that has been evaluated as satisfying any of predetermined evaluation criteria defined by a combination of a plurality of sub-goals with respect to the predetermined goal; evaluating the similarity between the evaluation target and the comparison target for each of the sub-goals, and the ideal pattern for each of the predetermined evaluation criteria defined by a combination of a plurality of the sub-goals, and information processing device, A program to function as a

Images