Information processing program, information processing device, and information processing method

Abstract

Provided is an information processing program that can assist in creating answers to questions used by ESG evaluation agencies in evaluating specified organizations. [Solution] The information processing program relating to the disclosed technology causes a computer to execute the following steps: a reception procedure (ST110) for receiving from a user terminal question designation information that can identify a question designated by a user as a question to be answered, which is a question used by an ESG evaluation agency in the evaluation of a specified organization; an information acquisition procedure (ST120) for acquiring from a database, based on the question designation information, disclosure information to be disclosed by the specified organization, information regarding the question specified by the question designation information, and information regarding the answer format to the question; an information input procedure (ST130) for inputting each of the acquired information into a machine learning model; and an information output procedure (ST140) for outputting information regarding the answer to the question output from the machine learning model to the user terminal.

Description

[Technical field]

[0001] The present disclosure relates to an information processing program, an information processing device, and an information processing method. [Background technology]

[0002] In recent years, ESG management has been attracting worldwide attention as a management policy essential for the sustainable growth of companies and other organizations, and an increasing number of organizations are working to introduce it. In addition, various disclosure systems have been established in countries and regions around the world that require organizations involved in activities within the region to disclose information in accordance with disclosure standards from the perspective of ESG (an acronym for Environment, Social, and Governance). There are also ESG evaluation organizations that evaluate whether organizations are conducting activities that take ESG into consideration. In this way, ESG is gaining attention in the same way as the Sustainable Development Goals (SDGs). In addition to the above-mentioned "ESG management," the term has become widely used in general, including phrases that include ESG, such as "ESG investment," "ESG report," and "ESG analytics."

[0003] ESG rating agencies collect information on ESG from companies and other organizations, and based on the collected information, evaluate whether companies and other organizations are conducting ESG-conscious activities. ESG rating agencies may send questions to these organizations in order to collect information on ESG from these organizations. Generally, the questions sent to organizations by ESG rating agencies vary in format and content depending on the rating agency. Therefore, in order for each organization to answer these questions, the person in charge at each organization needs to understand the format and content of each question, visually search for the answers to the questions, and then create a response.

[0004] Incidentally, for example, Patent Document 1 does not disclose any application to ESG, but it does disclose a technology that accepts questions about public gambling, acquires data related to the questions using a RAG (Retrieval Augmented Generation) mechanism, and generates answers to the questions using the accepted questions, the acquired data, and a trained model. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Patent No. 7550329 Summary of the Invention [Problem to be solved by the invention]

[0006] There have been conventional technologies that use a trained model to automatically generate answers to questions and provide the generated answers to users, such as the technology disclosed in Patent Document 1. However, with conventional technologies, it has been difficult to support the generation of answers to questions written in various formats, such as questions used by multiple ESG rating agencies to evaluate a given organization.

[0007] The present disclosed technology is intended to solve the above-mentioned problem, and provides an information processing program, an information processing device, and an information processing method that can assist ESG evaluation agencies in creating answers to questions used in evaluating a specified organization. [Means for solving the problem]

[0008] The information processing program relating to the disclosed technology is configured to cause a computer to execute the following steps: a reception procedure for receiving from a user terminal question designation information capable of identifying a question designated by a user as a question to be answered, which is a question used by an ESG evaluation agency in the evaluation of a specified organization; an information acquisition procedure for acquiring from a database disclosure information to be disclosed by the specified organization, information regarding the question identified by the question designation information, and information regarding the answer format to the question, based on the question designation information received in the reception procedure; an information input procedure for inputting the disclosure information acquired in the information acquisition procedure, information regarding the question, and information regarding the answer format to the question into a machine learning model; and an information output procedure for acquiring information regarding the answer to the question output from the machine learning model, and outputting the acquired information regarding the answer to the user terminal. Effect of the Invention

[0009] The information processing program according to the disclosed technology has the above-mentioned technical features and has the effect of being able to assist in creating answers to questions used by ESG rating agencies in evaluating specified organizations. [Brief description of the drawings]

[0010] [Figure 1] 1 is a block diagram showing a configuration example of an information processing system according to the present disclosure. [Diagram 2] 1 is a diagram showing an example of a question including a question body and an answer auxiliary text in the technology disclosed herein; [Diagram 3] FIG. 13 is a diagram showing an example of a question in a format that prompts a user to select an appropriate option from a plurality of options in the technology disclosed herein. [Figure 4] FIG. 13 is a diagram showing an example of question information in the present disclosure. [Diagram 5] FIG. 13 is a diagram showing an example of question format information in the present disclosure. [Figure 6] 1 is a diagram for explaining an example of operation of an information processing device according to the present disclosure and a flow of each piece of information. [Figure 7]FIG. 13 is a diagram showing an example of questions used in an experiment in the present disclosure, and options shown as answers. [Figure 8] 13 is a diagram showing an example of a prompt generated by an information acquisition unit in an experiment according to the disclosed technology. [Figure 9] FIG. 13 is a diagram showing an example of a response output from a large-scale language model in an experiment according to the present disclosure. [Figure 10] FIG. 1 is a diagram illustrating a hardware configuration of an information processing device according to the present disclosure. [Figure 11] 1 is a flowchart showing a processing procedure of an information processing method according to the present disclosed technique. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] Next, some important terms used in the present disclosure are defined as follows:

[0012] ESG Evaluation Items ESG evaluation items are items related to ESG that are defined as one and the same item, even if the evaluation items are described differently by each ESG evaluation agency, if there is a common concept or idea that is recognized. ESG evaluation items include, for example, four items: ESG classification, code (number), major classification, and minor classification. The ESG classification in ESG evaluation items indicates whether each data is classified as Environment, Social, or Governance, and letters such as "E," "S," and "G" are used to represent the ESG classification. The codes (numbers) in the ESG evaluation items are used to distinguish each subcategory, and a different code is assigned to each subcategory. These codes that distinguish the subcategories are also called "classification codes." The major and minor categories in the ESG evaluation items are expressed in letters so that people can easily understand what the data showing information on ESG is, and the names of these major and minor categories are generalized or standardized expressions of the ESG-related evaluation items used by each ESG evaluation organization. Examples of the major categories include "general environment," "climate change," "water," "resource circulation," "biodiversity," "supplier (environment)," "environmental opportunities and impacts," "human rights," "labor practices," "diversity," "human resource development," "employee safety and health," "corporate citizenship," "product quality and product safety," "corporate governance," "risk management," and "information security and privacy." Examples of minor categories include "environmental management policy" as belonging to "general environment," and "climate change" as belonging to "policy to address climate change," "policy to reduce energy consumption," and "carbon pricing."

[0013] "Specified organizations" A specified organization is an organization that provides ESG-related information to an ESG rating organization and is evaluated by the ESG rating organization. Specified organizations include, for example, companies, government agencies, non-profit organizations, industry associations, academic institutions, etc. The ESG-related information provided by these organizations serves as a reference for the ESG evaluation of, for example, companies, and is also an important element for understanding the environmental and social efforts of the company, etc., in society as a whole.

[0014] Disclosure documents and information Disclosure documents are documents in which each organization discloses its own ESG-related status. Disclosure documents include, for example, integrated reports and corporate governance reports. In addition, the disclosure information refers to information disclosed by each organization as information corresponding to each indicator stipulated by various disclosure systems that require disclosure in accordance with disclosure standards from an ESG perspective. In other words, the disclosure information can be said to be information described in the section of the disclosure document that corresponds to one of the ESG evaluation items (for example, its subcategory).

[0015] Embodiment 1 The first embodiment shows a case in which the disclosed technique is realized as an information processing device 100. 1 is a block diagram showing an example of the configuration of an information processing system according to the disclosed technique, in which an information processing device 100 and a user terminal UT are connected via the Internet, as shown in FIG. Although three user terminals UT are shown in Fig. 1, the number of user terminals UT may be any number equal to or greater than 1. Also, in Fig. 1, the information processing device 100 and the user terminal UT are connected via the Internet, but this connection is not limited to the Internet and any electric communication line may be used.

[0016] Information processing device 100 The information processing device 100 uses the RAG mechanism to support a specific organization to which a user belongs in creating an answer to a question from an ESG rating agency, and includes a reception unit 110, an information acquisition unit 120, an information input unit 130, and an information output unit 140. In the following description, for the sake of specificity, it is assumed that the specific organization to which the user belongs is a company. The information processing device 100 also includes a disclosure information database 150 and a question information database 160. Note that the information processing device 100 is not limited to this configuration, and the disclosure information database 150 and the question information database 160 may be provided in an external device that can be connected to the information processing device 100 for communication. The information processing device 100 provides a user with a service related to the present disclosed technology, namely, a service that uses the RAG mechanism to assist a company to which the user belongs in creating answers to questions posed by an ESG rating agency (hereinafter referred to as the "answer creation support service"). The information processing device 100 is, for example, any computer such as a server or a terminal used by a user. Here, the answer creation support service may be provided in the form of, for example, SaaS (Software as a Service). In this case, the information processing device 100 is owned or managed by a provider of the answer creation support service (hereinafter referred to as a "service provider"). In this specification, details will be given of a case where the information processing device 100 according to the disclosed technique is, for example, a server owned or managed by a service provider.

[0017] <Disclosure Information Database> The disclosure information database 150 holds disclosure information of the company to which the user belongs. The disclosure information held in the disclosure information database 150 is information related to the ESG of the company in relation to the ESG evaluation items. The disclosure information held in the disclosure information database 150 is disclosure information of one or more companies. The disclosure information of each company is acquired, for example, by crawling. Crawling is a well-known technique in which a program called a crawler periodically visits multiple websites to acquire and store information.

[0018] Here, for example, first, the information processing device 100 periodically visits websites of multiple companies designated in advance by a service provider to acquire the disclosed information of the multiple companies. The information processing device 100 converts the acquired disclosed information of the companies into text data by optical character recognition (OCR), divides the converted text data into predetermined ranges, and converts the text data into vectors for each divided range. The information processing device 100 then associates the vectors obtained by the conversion with information identifying the companies disclosing the disclosed information (hereinafter referred to as "company identifying information"), and stores the vectors in the disclosed information database 150. The company specific information is, for example, the official name of the company, the abbreviation of the company, or an ID or code specific to the company, etc. Hereinafter, the company specific information is assumed to be the company name such as the official name or abbreviation of the company. When the disclosure information is created in PDF format, text data may already exist in the disclosure information. In that case, the information processing device 100 may not need to convert the disclosure information into text data by the optical character recognition (OCR) described above. The disclosure information of multiple companies acquired by the information processing device 100 may be created in HTML format. When dividing the above-mentioned text data into predetermined ranges, the information processing device 100 can divide the data by page, paragraph, or number of characters. When dividing the text data by number of characters, the information processing device 100 can set an arbitrary number of characters, such as a maximum of 400 characters, as a division unit. In this case, the information processing device 100 may divide the text data such that an arbitrary number of characters, such as 100 characters, overlap before and after the divided range.

[0019] Question information database The question information database 160 holds information on questions used by ESG rating agencies when evaluating companies (hereinafter referred to as "question information"). In the following explanation, for the sake of concreteness, it is assumed that the questions used by ESG rating agencies when evaluating companies are, for example, questions sent from the ESG rating agencies to companies. The question information database 160 is created and managed by the service provider for each company to which a user belongs. For example, the service provider requests the company to respond in advance with questions sent by the ESG rating agency using a file in a specific data format, such as Excel. The service provider reads the file sent back from the company in response to this request using the information processing device 100, and stores the question information included in the read file in the question information database 160. Note that the question information may be stored in question information database 160 after the description of the question has been generalized by, for example, the service provider. Also, the question information may be stored in question information database 160 after the question style has been unified among a plurality of questions by, for example, the service provider.

[0020] Furthermore, the question indicated by the question information may be stored in the question information database 160 after being divided in advance into a question body and an answer auxiliary text for assisting in answering the question. Figure 2 shows an example of a question, including the question body and answer support text. Typically, questions sent to companies by ESG rating agencies consist of multiple sentences per question. Among them, for example, a sentence at the beginning of a question that roughly explains the question can be regarded as the main text of the question, and a sentence following this main text to assist in answering the question can be regarded as the auxiliary answer text. The auxiliary answer text explains, for example, what the main text of the question is specifically asking about, or what the answer should be specifically. In the example of FIG. 2, "Material Issues for..." marked with the reference number 201 is the main text of the question, and "Does your company..." marked with the reference number 202 is the auxiliary answer text. In addition, when the question indicated by the question information is a question in a format that prompts the user to select an appropriate option from a plurality of options, the question information may include information regarding the option indicated as the answer.

[0021] FIG. 3 shows an example of a question in a format that prompts a person to select an appropriate option from a number of options. In FIG. 3, "Climate Governance" marked with reference number 301 is the main text of the question, and "Is your..." marked with reference number 302 is the auxiliary answer text. Furthermore, each sentence marked with reference number 303 and accompanied by a check box is an option to be selected by the company. For example, a person in charge of a company reads the main text of the question and the auxiliary answer text, and selects an option that is appropriate as an answer to the question by checking the check box. Here, the options 303 are displayed with an indentation for each level. A person in charge of a company can select one or more appropriate options for each level from among the options at the same level. 3 shows an example in which the question includes only the options 303, but some questions may include, in addition to the options 303, a free-text field corresponding to the options 303. In this case, the person in charge at the company can, for example, select a predetermined option and then write any answer in the free-text field corresponding to the option.

[0022] Fig. 4 shows an example of question information stored in the question information database 160. In Fig. 4, the question indicated by the question information includes a question body, an auxiliary answer sentence, and a plurality of options, similar to the question exemplified in Fig. 3. However, the content of the question in Fig. 4 is different from the content of the question in Fig. 3. 4, "Material Issues..." labeled with reference number 401 is the main text of the question, and "Does your company..." labeled with reference number 402 is the auxiliary answer text. The table labeled with reference number 403 is a table showing the options that the company can select. In this way, the question information can be divided into the main text of the question 401, the auxiliary answer text 402, and the table 403 related to the options for each question, and then stored in the question information database 160.

[0023] Furthermore, the question information database 160 holds information regarding the answer format of the question (hereinafter referred to as "question format information"). The question format information includes information regarding an identifier for identifying a question (hereinafter referred to as "question identification information"). The identifier for identifying a question is an identifier capable of uniquely identifying a question, and is composed of a combination of the name of an ESG rating organization and a question number, such as "DJSI (S&P Dow Jones Indices); Q1" or "DJSI; Q2". Furthermore, the question identification information includes, for each question, tag information related to one or more answer formats, dependencies between other questions, and an answer input format. FIG. 5 shows an example of question format information stored in the question information database 160. As shown in FIG. 5, the column labeled with reference numeral 501 indicates the above-mentioned question identification information, and the column labeled with reference numeral 502 indicates the above-mentioned tag information. The "answer format" included in the tag information 502 indicates the format of the answer to the question. The answer formats are, for example, "free description," "YES / NO," "multiple choice," and "table format." Furthermore, "answer format 2" included in tag information 502 indicates a second answer format for the question. For example, Q3 and Q4 sent by DJSI indicate that they should be answered first using "multiple choice" and then using "free writing after selection." In this way, some questions require answers to be answered using a number of different answer formats. Here, examples are given up to the second answer format, but depending on the question, "answer format 3 (third answer format)" and onwards may be provided.

[0024] The "dependency" included in the tag information 502 indicates the dependency of questions, for example, the dependency between a certain question and another question. For example, Q3 sent by DJSI needs to be answered if the answer to Q2 also sent by DJSI is "YES", and Q4 sent by DJSI needs to be answered if the answer to Q2 also sent by DJSI is "NO". Furthermore, the "input format" included in the tag information 302 indicates the input format of the answer to the question. Examples of the input format include "natural language input", "button", "checkbox", and "numeric input". The question-type information configured as above is generated by, for example, a service provider or a user belonging to a company. Also, the question information database 160 configured as above is created and managed by, for example, a service provider for each company to which a user belongs, as described above. Meanwhile, a user can access the information processing device 100, which is a server, from a user terminal UT via the Internet and appropriately edit the question information and question-type information held in the question information database 160 of his / her company.

[0025] Reception Section 110 The receiving unit 110 receives, from the user terminal UT, information capable of identifying a question designated by the user as a question to be answered (hereinafter referred to as "question designation information"). A user can operate the user terminal UT to access the information processing device 100 and display a screen for using the answer creation support service on a display (not shown) of the user terminal UT. For example, when the answer creation support service is provided in the form of SaaS, the screen is displayed on a browser. For example, a question specification window can be displayed on the screen. The user can operate the user terminal UT to input an answer to any question into this question designation window. When the user inputs an answer to any question into the question designation window, the user terminal UT generates information capable of identifying the question designated by the user (question designation information) and transmits the generated question designation information to the information processing device 100.

[0026] For example, when answering question "Q1" sent by "DJSI", an ESG rating organization, the user operates the user terminal UT to input "answer to Q1 from DJSI" into the question designation window. When the user inputs the above into the question designation window, the user terminal UT generates question designation information indicating that the ESG rating organization is "DJSI" and the question number is "Q1", and transmits the generated question designation information to the information processing device 100. In this case, the question identified by "DJSI" designated by the user as the ESG rating organization and "Q1" designated as the question number is the "question designated by the user". The reception unit 110 of the information processing device 100 receives question specifying information transmitted from the user terminal UT. Upon receiving the question specifying information transmitted from the user terminal UT, the reception unit 110 outputs the received question specifying information to the information acquisition unit 120. Note that the question specifying information also includes information (company specifying information) that identifies the company to which the user who transmitted the question specifying information belongs.

[0027] 《Information acquisition section 120》 The information acquiring unit 120 acquires question designating information from the accepting unit 110. When the information acquiring unit 120 acquires the question designating information from the accepting unit 110, the information acquiring unit 120 identifies what the specific question designated by the user is, based on the acquired question designating information. Furthermore, when the information acquiring unit 120 acquires question specifying information from the accepting unit 110, the information acquiring unit 120 identifies the company to which the user who has transmitted the question specifying information belongs, based on the company identifying information included in the acquired question specifying information. Then, the information acquiring unit 120 searches the question information database 160 managed by the identified company, and acquires information (question information) related to the question specified by the user from the question information database 160. The information acquiring unit 120 also acquires question-style information from the question information database 160. Note that when acquiring the question-style information from the question information database 160, the information acquiring unit 120 may acquire the entire question-style information exemplified in FIG. 5, may acquire part of the question-style information on an ESG rating agency basis, or may acquire part of the question-style information on a question-by-question basis.

[0028] Next, the information acquisition unit 120 searches the disclosed information of a plurality of companies, which is held in the disclosed information database 150, for disclosed information necessary to answer the above-specified question. For example, the information acquiring unit 120 generates a search query for searching for disclosure information necessary to answer the question, based on text data indicating the question text and the auxiliary answer text, which are included in the question information acquired from the question information database 160. After generating the search query, the information acquiring unit 120 performs a vector search of the disclosure information database 150 based on the generated search query, and acquires the disclosure information necessary to answer the question as a search result. The information acquiring unit 120 may arbitrarily set an upper limit number of pieces of disclosure information to be acquired from the disclosure information database 150 based on the generated search query as a search condition when searching the disclosure information database 150. For example, the information acquiring unit 120 may set an upper limit number of pieces of disclosure information to be acquired as a search condition, such as acquiring, as a search result, up to the top 10 pieces of disclosure information having a high rate of matching the search query. As described above, the information acquiring unit 120 acquires question information and question format information from the question information database 160, acquires disclosure information necessary to answer the question from the disclosure information database 150, and outputs the acquired information to the information input unit 130. In the following explanation, for simplicity, the disclosure information necessary to answer the question acquired from the disclosure information database 150 is referred to as "specific disclosure information."

[0029] Information Input Unit 130 The information input unit 130 acquires the question information, question-format information, and specific disclosure information from the information acquisition unit 120. When the information input unit 130 acquires the question information, question-format information, and specific disclosure information from the information acquisition unit 120, the information input unit 130 inputs the acquired information to a machine learning model. Note that the question indicated by the question information is divided into a main text of the question and an auxiliary answer text for assisting in answering the question, as illustrated in FIG. 2. The machine learning model may be, for example, an artificial neural network, and there may be a plurality of machine learning models. In addition, when there are a plurality of machine learning models, the machine learning models may include a large language model (LLM). The information input unit 130 treats the question information, question-format information, and specific disclosure information acquired from the information acquisition unit 120 as an input instruction, and inputs the input instruction to one of the multiple machine learning models according to a predetermined condition. Note that the input instruction may include multiple pieces of question information and multiple pieces of question-format information. Furthermore, the input instruction may include, as the question-format information, information regarding the dependency relationship between each question indicated by the multiple pieces of question information and other questions.

[0030] For example, when the answer format to the question indicated by the tag information included in the question format information or the input format of the answer is an answer or input format in natural language (natural sentence), the information input unit 130 generates a prompt and inputs the generated prompt together with the specific disclosure information to the large scale language model (LLM). The prompt in this case includes the question information and the question format information. The prompt in this case also includes content instructing to answer the question indicated by the question information according to the answer format indicated by the question format information. If the specific disclosure information can be included in the prompt, the information input unit 130 may input only the prompt to the large-scale language model. The prompt may also include an instruction to cause the large-scale language model to output, when an appropriate answer to a question does not exist in the specific disclosure information, or when the question cannot be appropriately answered even with reference to the specific disclosure information, a message indicating that an appropriate answer does not exist in the specific disclosure information, or a message indicating that the question cannot be appropriately answered even with reference to the specific disclosure information. The prompt may also include an instruction to cause the large-scale language model to output, as location information, information indicating the location of specific disclosure information capable of explaining the reason for the selection, when the answer format of the question is a format in which a predetermined option is selected from a plurality of options. Furthermore, when the answer format to the question indicated by the tag information included in the question-format information or the answer input format is not a format for answering or inputting in natural language, such as a multiple-choice format, the information input unit 130 may input the above-mentioned input instructions to a machine learning model other than the LLM. In this case, the machine learning model is a machine learning model trained to output an appropriate numerical value or YES / NO as an answer when the above-mentioned input instructions (question information, question-format information, and specific disclosure information) are given as input.

[0031] The machine learning model including the large-scale language model may be provided in an external server or in the information processing device 100. As the large-scale language model, for example, an existing large-scale language model provided as a service by an external server may be used. In this case, the information processing device 100 can input a prompt to the large-scale language model through API (Application Programming Interface) cooperation and obtain a response to the input from the large-scale language model. The machine learning model generates information regarding the answer to the question (hereinafter referred to as “answer information”) based on the given question information, question format information, and specific disclosure information, and outputs the generated answer information to the information processing device 100.

[0032] Information output unit 140 The information output unit 140 acquires answer information from the machine learning model. When the information output unit 140 acquires the answer information from the machine learning model, the information output unit 140 performs a predetermined processing process on the acquired answer information so that the answer information can be displayed by a predetermined program. For example, the information output unit 140 performs a process of converting the format of the answer information acquired from the machine learning model into an appropriate format so that the answer information can be displayed by a predetermined program. The predetermined program is, for example, a web page for answer input designated by an ESG rating agency, or a program such as Excel that runs on the information processing device 100 and the user terminal UT. For example, depending on the ESG rating agency, the user may need to input the answers indicated by the answer information into a program such as Excel using the user terminal UT by manual operations such as copy and paste. In such a case, the information output unit 140 processes the answer information so that the format of the answer information is, for example, a table format in which instructions regarding the input location of each answer are described. This allows the user to easily understand where in the program each answer should be input, which reduces the user's workload. The information output unit 140 may determine how to process the answer information based on the tag information included in the question-format information. For example, if the "answer format" included in the tag information is "tabular," the information output unit 140 may convert the format of the answer information into a tabular format as described above. Alternatively, the information output unit 140 may store in advance in a storage unit (not shown) a table that associates ESG rating agencies with information about programs or Web pages used when responding to the ESG rating agencies, and process the response information by referring to the table. The information output unit 140 outputs the answer information processed as described above to the user terminal UT.

[0033] The user terminal UT acquires the processed answer information from the information output unit 140. When the user terminal UT acquires the processed answer information from the information output unit 140, the user terminal UT displays the answer indicated by the acquired answer information on a display of the user terminal UT. For example, the user terminal UT displays the answer on a screen for using the answer creation support service that is displayed on the display.

[0034] Next, an example of the operation of the information processing device 100 in the information processing system and the flow of each piece of information will be described with reference to Fig. 6. In the following description, it is assumed that questions have been sent in advance from multiple ESG rating agencies to the company to which the user belongs, and that a question information database 160 for each company has been constructed by the service provider. In the following description, it is assumed that a disclosure information database 150 has been constructed in advance by the service provider and the information processing device 100. First, the user operates the user terminal UT to input an answer to an arbitrary question in the above-mentioned question designation window. When the user inputs an answer to an arbitrary question in the question designation window, the user terminal UT generates question designation information and transmits the generated question designation information to the information processing device 100 (reference number (1) in FIG. 6). Next, in the information processing device 100, the receiving unit 110 receives question specifying information transmitted from the user terminal UT (reference number (2) in FIG. 6). Upon receiving the question specifying information from the user terminal UT, the receiving unit 110 outputs the received question specifying information to the information acquisition unit 120. Next, the information acquiring unit 120 acquires question specifying information from the receiving unit 110. The information acquiring unit 120 identifies the company to which the user who transmitted the question specifying information belongs based on the company identifying information included in the question specifying information acquired from the receiving unit 110, and acquires question information related to the question designated by the user from the question information database 160 managed by the identified company. The information acquiring unit 120 also acquires question format information from the question information database 160 (reference number (3) in FIG. 6). Next, the information acquisition unit 120 searches for disclosure information (specific disclosure information) necessary to answer the question from the disclosure information of multiple companies stored in the disclosure information database 150, and acquires the specific disclosure information as a search result (reference number (4) in FIG. 6). The information acquisition unit 120 outputs the acquired question information, question format information, and specific disclosure information to the information input unit 130. Next, the information input unit 130 acquires the question information, question-format information, and specific disclosure information from the information acquisition unit 120. When the information input unit 130 acquires the question information, question-format information, and specific disclosure information from the information acquisition unit 120, the information input unit 130 inputs the acquired information as an input instruction to the machine learning model 300 (reference number (5) in FIG. 6). Note that when the machine learning model 300 is a large-scale language model, a prompt including the question information and question-format information and the specific disclosure information, or a prompt including the question information, question-format information, and specific disclosure information, is input to the large-scale language model. Next, the machine learning model 300 generates information regarding the answer to the question (answer information) based on the question information, question format information, and specific disclosure information provided as input instructions, and outputs the generated answer information to the information processing device 100 (symbol (6) in Figure 6). Next, in the information processing device 100, the information output unit 140 acquires answer information from the machine learning model 300. When the information output unit 140 acquires the answer information from the machine learning model 300, the information output unit 140 performs a predetermined processing process on the acquired answer information so that the answer information can be displayed by a predetermined program, and outputs the processed answer information to the user terminal UT (reference number (7) in FIG. 6). As described above in (1) to (7), the information processing device 100 operates and transmits and receives various pieces of information. Here, the RAG mechanism is realized by (3) to (7).

[0035] In the above example, the information processing device 100 transmits and receives information to and from the machine learning model 300 once. However, when a large-scale language model is used as the machine learning model 300, the information processing device 100 may transmit and receive information to and from the large-scale language model multiple times (multi-turn). For example, in the first turn of the information processing device 100, the information input unit 130 generates a prompt as described above and inputs the generated prompt to the large-scale language model. Then, the information output unit 140 acquires answer information from the large-scale language model. In the second turn, the information input unit 130 generates a prompt for the large-scale language model, which describes the content asking about the probability or success or failure of the answer indicated by the answer information generated in the first turn, and inputs the generated prompt to the large-scale language model. The information output unit 140 then acquires information about the probability or success or failure of the answer output from the large-scale language model, and outputs the acquired information to the user terminal UT. In this case, the user can not only easily create an answer to the question, but also easily understand the probability or success or failure of the answer, and can appropriately modify the answer depending on the probability or success or failure of the answer.

[0036] In addition, when outputting the processed answer information to the user terminal UT in reference numeral (7) of Fig. 6, the information output unit 140 may output a plurality of pieces of answer information to the user terminal UT as answer candidates. For example, after outputting the processed answer information to the user terminal UT, the information processing device 100 collects feedback from the user regarding the answer information, and stores information regarding the collected feedback as a log in a storage unit (not shown) in advance. The feedback from the user includes, for example, corrected answer information or other answer information that is possible regarding the answer information. Then, when the information output unit 140 outputs the processed answer information to the user terminal UT in (7) of Fig. 6, it refers to the log stored in the storage unit, generates multiple pieces of answer information based on the log, and outputs the generated answer information to the user terminal UT as answer candidates. In this case, the information processing device 100 can present multiple answer candidates to the user at one time, and can reduce the number of times information is sent and received between the information processing device 100 and the user terminal UT, leading to a reduction in the amount of communication. Furthermore, the information processing device 100 may have a function for allowing the user to correct the answer. For example, in reference numeral (7) of FIG. 6, the information processing device 100 may accept corrections to the answer information from the user via the user terminal UT before the information output unit 140 outputs the processed answer information to the user terminal UT. Then, the information output unit 140 may output the answer information corrected by the user to the user terminal UT. In this case, the user can correct the answer information in advance on the information processing device 100 before acquiring the answer information, which leads to improved efficiency in the answer creation work. Furthermore, when the information processing device 100 receives a correction to the answer information from the user, the information processing device 100 may use the corrected answer information as re-learning data for the machine learning model 300. In this case, the machine learning model 300 becomes able to output highly accurate answer information that better reflects the user's intention, and the information processing device 100 can obtain, from the machine learning model 300, highly accurate answer information that better reflects the user's intention. In addition, when outputting the processed answer information to the user terminal UT in reference (7) of Fig. 6, the information output unit 140 may output location information indicating where the information on which the answer is based is located in the specific disclosure information by including it in the answer information. For example, when the answer format of a question is a format in which a predetermined option is selected from multiple options, the information output unit 140 may output information indicating the location of specific disclosure information that can explain the reason for the selection as location information, and may output this location information to the user terminal UT by including it in the answer information. In this case, the user can easily understand the reason for the selection in the answer.

[0037] <Experimental Results> The inventor of the disclosed technology conducted an experiment to see to what extent the information processing device 100 can generate answer information using specific questions from an ESG rating organization. In the experiment, a case was assumed in which the ESG rating organization was DJSI and a certain company answered questions from DJSI for 2023. In addition, in the experiment, the certain company used a 2023 report (Sustainability Data Book) that summarizes data and information on sustainable activities as the specific disclosure information of the company, and generated an answer by providing a prompt including question information and question format information and the specific disclosure information to a large-scale language model. Furthermore, when using the specific disclosure information of the company, the inventor of the disclosed technology used the information processing device 100 to convert the above-mentioned report into text data using optical character recognition (OCR), then divided the converted text data into groups of up to 400 characters, and converted the text data into vectors for each divided range. Here, overlapping of up to 100 characters was allowed before and after each divided range. The inventor of the disclosed technology then used the information processing device 100 to link the vector obtained by the conversion with the company-specific information of the company, and store the vector in the disclosure information database 150. In addition, in the experiment, the information acquisition unit 120 generated the question text and answer auxiliary sentence indicated by the question information as a search query, searched the disclosure information database 150 using the generated search query, and acquired specific disclosure information as the search result. Here, the information acquisition unit 120 acquired up to the top 10 specific disclosure information that had a high rate of matching the search query as the search result, and linked the acquired specific disclosure information. Then, the information input unit 130 inputted the specific disclosure information linked by the information acquisition unit 120 into the large-scale language model.

[0038] An example of the actual questions used in the experiment and the options given as answers is shown in FIG. 7, reference numeral 701 indicates the question body, and reference numeral 702 indicates the answer auxiliary sentence. Reference numeral 703 indicates an option shown as an answer. The information acquisition unit 120 generates the question body 701 and the answer auxiliary sentence 702 as a search query, searches the disclosure information database 150 using the generated search query, and acquires specific disclosure information as a search result. The specific disclosure information here is disclosure information necessary to answer a question including the question body 701 and the answer auxiliary sentence 702. Next, an example of a prompt generated by the information input unit 130 is shown in FIG. In Fig. 8, reference numeral 801 denotes the question body (reference numeral 701 in Fig. 7) and answer auxiliary text (reference numeral 702 in Fig. 7) embedded in the prompt. Reference numeral 802 denotes options (reference numeral 703 in Fig. 7) embedded in the prompt. Here, the three options are numbered 1 to 3. The information input unit 130 inputs the prompt generated in this way and the specific disclosure information to the large-scale language model. In response to these inputs, answer information is output from the large-scale language model. An example of an answer indicated by the answer information output from the large-scale language model is shown in FIG. 9. 9, as indicated by reference numeral 901, "1" is selected as an answer option. Reference numeral 902 indicates location information indicating the location of specific disclosure information that can explain the reason for the selection of "1." Then, the information processing device 100 presents the answer indicated by the answer information output from the large-scale language model to the inventor of the disclosed technology. In this way, the information processing device 100 selected option "1" as the answer. On the other hand, when the inventor of the disclosed technology checked with the company, the actual answer of the company was also "1." In this way, the information processing device 100 was able to select the same answer as the actual answer of the company by performing the above-mentioned processing. In addition, although detailed explanation is omitted here, the inventor of the disclosed technology conducted a similar experiment on a question with a multiple-choice answer format, which is different from the above question. As a result, the information processing device 100 failed to select only one of the options that the company actually selected as an answer, but selected the other options in the same way as the actual company's answer. From this example, it can be said that the information processing device 100 was able to present the user with an answer that was very close to the actual company's answer.

[0039] The information processing device 100 shown in the first embodiment may include at least the information acquisition unit 120, the information input unit 130, and the information output unit 140, and may omit the reception unit 110. When the reception unit 110 is omitted in the information processing device 100, the function of the reception unit 110 may be realized by the information acquisition unit 120, for example.

[0040] <<Variation>> Next, a modification of the information processing device 100 according to the first embodiment will be described. In addition, even in the modified example, the main operation example of the information processing device 100 and the flow of each piece of information are similar to those described in Fig. 6 with reference to embodiment 1. Therefore, in the following description, the modified example of the information processing device 100 will be described with reference to Fig. 6.

[0041] In the first embodiment, an example was described in which the receiving unit 110 receives, from the user terminal UT, information capable of identifying a question designated by the user as a question to be answered (question designation information). Also, in the above example, for example, when a user answers a question "Q1" sent from "DJSI", an ESG rating organization, the user operates the user terminal UT to input "answer to Q1 by DJSI" into the question designation window, and when the user inputs the above into the question designation window, the user terminal UT generates question designation information capable of identifying that the ESG rating organization is "DJSI" and that the question number is "Q1", and transmits the generated question designation information to the information processing device 100. In a modified example, the receiving unit 110 may receive, as question specification information from the user terminal UT, information capable of identifying the name of an ESG evaluation agency, or information capable of identifying the content of a specific question including the name of an ESG evaluation agency.

[0042] In this case, the user operates the user terminal UT to input, for example, "DJSI" as the name of the ESG assessment organization in the question designation window. When the user inputs "DJSI" in the question designation window, the user terminal UT generates question designation information capable of identifying that the name of the ESG assessment organization is "DJSI" and transmits the generated question designation information to the information processing device 100 (reference symbol (1) in FIG. 6 ). Alternatively, the user operates the user terminal UT to, for example, directly input the specific content of the question to be answered into the question designation window, or save a predetermined file describing the specific content of the question in the user terminal UT, and then input the save destination of the file in the user terminal UT into the question designation window. When the user inputs the specific content of the question into the question designation window or inputs the save destination of the file describing the specific content of the question into the question designation window, the user terminal UT generates question designation information capable of identifying the specific content of the question based on the input content, and transmits the generated question designation information to the information processing device 100.

[0043] Alternatively, the user may operate the user terminal UT to specify the name of the ESG rating agency or the specific content of the question, for example, by selecting it from a pull-down menu displayed in a question specification window. When the user selects the name of the ESG rating agency or the specific content of the question from the pull-down menu, the user terminal UT generates question specification information capable of identifying the name of the ESG rating agency or question specification information capable of identifying the specific content of the question based on the selected content, and transmits the generated question specification information to the information processing device 100. The pull-down menu is a menu that displays a list of multiple options when the user presses a specific position, and can be a menu that displays multiple options, such as the names of ESG rating agencies or the specific content of a question. Such a pull-down menu may be provided by the information processing device 100, for example, as follows. For example, when the information processing device 100 includes the question information database 160, the information processing device 100 extracts a list of names of ESG rating agencies included in the question information or a list of specific questions based on the question information stored in the question information database 160. Then, based on the extracted list of names of ESG rating agencies or the list of specific questions, the information processing device 100 generates a pull-down menu including a list menu that allows a user to select any one of options configured by these lists, and displays the generated pull-down menu in a question designation window, thereby providing the pull-down menu to the user. Note that, even if the question information database 160 is provided in an external device that can be connected to and communicated with by the information processing device 100, the information processing device 100 can generate a pull-down menu based on the question information stored in the question information database 160 in the same manner as described above, and display the generated pull-down menu in a question specification window, thereby providing the pull-down menu to the user.

[0044] Next, in the information processing device 100, the receiving unit 110 receives question specifying information transmitted from the user terminal UT (reference number (2) in FIG. 6). Upon receiving the question specifying information from the user terminal UT, the receiving unit 110 outputs the received question specifying information to the information acquisition unit 120. Next, the information acquiring unit 120 acquires question designating information from the receiving unit 110. When the name of the ESG evaluation organization can be identified from the question designating information, the information acquiring unit 120 identifies the company to which the user who transmitted the question designating information belongs, based on the company identifying information included in the question designating information acquired from the receiving unit 110. Then, the information acquiring unit 120 acquires question information related to a question posed by the ESG evaluation organization having the name identified by the question designating information, from the question information database 160 managed by the identified company. On the other hand, if the specific content of the question, including the name of the ESG evaluation agency, can be identified from the question specification information, the information acquisition unit 120 identifies the company to which the user who sent the question specification information belongs, based on the company identification information included in the question specification information acquired from the reception unit 110, and acquires question information regarding the content of the specific question identified by the question specification information from the question information database 160 managed by the identified company. In addition, if the content of the specific question specified by the question designation information is extremely detailed and it is determined that the content of the specific question specified by the question designation information can be substituted for question information without acquiring the question information stored in the question information database 160, the information acquisition unit 120 may omit acquiring the question information from the question information database 160.

[0045] Next, the information acquiring unit 120 acquires question-type information from the question information database 160 (reference number (3) in FIG. 6). When the name of an ESG rating agency can be identified from the question designation information, the information acquiring unit 120 may acquire question-type information for each ESG rating agency from the question information database 160. In addition, when the content of a specific question can be identified from the question designation information, the information acquiring unit 120 may acquire question-type information for each question from the question information database 160. Next, the information acquisition unit 120 searches for disclosure information (specific disclosure information) necessary to answer the question from the disclosure information of multiple companies stored in the disclosure information database 150, and acquires the specific disclosure information as a search result (reference number (4) in FIG. 6). The information acquisition unit 120 outputs the acquired question information, question format information, and specific disclosure information to the information input unit 130. Next, the information input unit 130 acquires the question information, question-format information, and specific disclosure information from the information acquisition unit 120. When the information input unit 130 acquires the question information, question-format information, and specific disclosure information from the information acquisition unit 120, the information input unit 130 inputs the acquired information as an input instruction to the machine learning model 300 (reference number (5) in FIG. 6). Note that when the machine learning model 300 is a large-scale language model, a prompt including the question information and question-format information and the specific disclosure information, or a prompt including the question information, question-format information, and specific disclosure information, is input to the large-scale language model.

[0046] 6, it is assumed that the information acquisition unit 120 cannot acquire specific disclosure information necessary to answer the question from the disclosure information database 150, or the amount of information is insufficient even if the information acquisition unit 120 can acquire the specific disclosure information necessary to answer the question from the disclosure information database 150. In such cases, when the machine learning model 300 is a large-scale language model, the information input unit 130 may include in the prompt an instruction to the large-scale language model to output a message such as "The answer may be insufficient because the disclosure information necessary to answer the question is insufficient. Please input to supplement the answer as necessary" to the corporate user, and input the prompt to the large-scale language model.

[0047] Next, the machine learning model 300 generates information (answer information) regarding the answer to the question based on the question information, question format information, and specific disclosure information given as input instructions, and outputs the generated answer information to the information processing device 100 (reference number (6) in FIG. 6). Note that when generating the answer information, the machine learning model 300 may generate the answer information by including information indicating the relevant portion of the specific disclosure information on which the answer is based. The relevant portion of the specific disclosure information may be, for example, the portion of the specific disclosure information that corresponds to the answer, or the page number of the portion. In addition, when the machine learning model 300 is a large-scale language model and the above-mentioned prompt is input to the large-scale language model, when generating answer information, the large-scale language model may generate answer information by including information indicating a message such as "The answer may be insufficient because the disclosure information necessary to answer the question is lacking. If necessary, please make input to supplement the answer."

[0048] Next, in the information processing device 100, the information output unit 140 acquires answer information from the machine learning model 300. When the information output unit 140 acquires the answer information from the machine learning model 300, the information output unit 140 performs a predetermined processing process on the acquired answer information so that the answer information can be displayed by a predetermined program, and outputs the processed answer information to the user terminal UT (reference number (7) in FIG. 6). In addition, if the answer format of the answer indicated by the answer information acquired from the machine learning model 300 is not the answer format specified in the question format information, the information output unit 140 may correct the answer format so that the answer format of the answer indicated by the answer information acquired from the machine learning model 300 matches the answer format specified in the question format information, and then output the answer information to the user terminal UT. Furthermore, when outputting the processed answer information or the answer information after the correction of the answer format to the user terminal UT, the information output unit 140 may output the answer information in a manner that allows the user to accept input from the user terminal UT for the answer indicated by the answer information. This is to leave room for the user to input (edit) the answer, since it is not always known whether the answer indicated by the answer information is the answer the user desires.

[0049] In this case, when the user uses the user terminal UT to make some input in response to the answer, the user terminal UT may transmit information indicating the content input in response to the answer (hereinafter referred to as "additional input information") to the information processing device 100. In the information processing device 100, the receiving unit 110 receives the additional input information transmitted from the user terminal UT and stores the received additional input information, for example, in an additional input information database (not shown). The additional input information database may be provided in the information processing device 100, or in an external device that can be communicatively connected to the information processing device 100. The additional input information stored in the additional input information database is used when the information output unit 140 outputs answer information in the next and subsequent times to a question that is the same as or similar to a question related to the additional input information. For example, when the information output unit 140 outputs the processed answer information or the answer information after the answer format has been corrected to the user terminal UT, the information output unit 140 refers to the additional input information database, and if additional input information related to the answer information to be output is stored in the additional input information database, the information output unit 140 reflects the contents of the additional input information in the answer information to be output, and outputs the answer information to the user terminal UT. For example, if additional input information related to the answer information to be output is stored in the additional input information database, the information output unit 140 may correct the answer information to be output based on the additional input information stored in the additional input information database, and output the corrected answer information, or may output the additional input information stored in the additional input information database together with the answer information to be output. This allows the information output unit 140 to present to the user an answer that is closer to the answer desired by the user.

[0050] The above-mentioned process is also applicable to the case where the answer information includes information indicating a message such as "The answer may be insufficient because disclosure information necessary to answer the question is insufficient. Please make an input to supplement the answer if necessary." In this case, it is assumed that the user checks the above message on the user terminal UT and, if necessary, inputs something to the answer using the user terminal UT. Even in this case, the user terminal UT transmits information indicating the content input to the answer (additional input information) to the information processing device 100, and in the information processing device 100, the reception unit 110 receives the additional input information transmitted from the user terminal UT and stores the received additional input information in an additional input information database (not shown). Hereinafter, the information output unit 140 uses the additional input information when outputting the answer information from the next time onwards in the same manner as described above, thereby being able to present the user with an answer closer to the answer desired by the user.

[0051] The functions of each unit of information processing device 100 described in the above-mentioned modified example of embodiment 1 are not realized only in the modified example, but can be selectively combined with the functions of each unit of information processing device 100 described in embodiment 1.

[0052] Embodiment 2 The second embodiment shows a case where the disclosed technology is realized as an information processing program. Unless otherwise specified, the same reference symbols as those used in the first embodiment are used in the second embodiment. Furthermore, in the second embodiment, explanations that overlap with those in the first embodiment are omitted as appropriate. The information processing program of embodiment 2 causes a computer to execute the following steps: a reception procedure for receiving from a user terminal question designation information that can identify a question used by an ESG evaluation agency in evaluating a specified organization and designated by a user as a question to be answered; an information acquisition procedure for acquiring from a database disclosure information to be disclosed by the specified organization, information regarding the question specified by the question designation information, and information regarding the answer format to the question based on the question designation information received in the reception procedure; an information input procedure for inputting the disclosure information acquired in the information acquisition procedure, information regarding the question, and information regarding the answer format to the question into a machine learning model; and an information output procedure for acquiring information regarding the answer to the question output from the machine learning model, and outputting the acquired information regarding the answer to the user terminal.

[0053] 10 is a diagram showing a hardware configuration 200 of an information processing device 100 according to the present disclosed technique. As shown in FIG. 10, the hardware configuration 200 of the information processing device 100 includes a communication interface 210, an input / output interface 220, a processor 230, and a memory 240.

[0054] 11 is a flowchart showing an information processing method realized by an information processing program according to the present disclosure as a processing procedure. As shown in FIG. 11, the information processing method realized by the information processing program includes a reception procedure ST110, an information acquisition procedure ST120, an information input procedure ST130, and an information output procedure ST140. The numbers attached to the respective processing steps correspond to the code of the entity that executes the processing step. For example, the reception step ST110 is executed by the reception unit 110, the information acquisition step ST120 by the information acquisition unit 120, the information input step ST130 by the information input unit 130, and the information output step ST140 by the information output unit 140. As shown in FIG. 11, a reception procedure ST110, an information acquisition procedure ST120, an information input procedure ST130, and an information output procedure ST140 are included in a loop process.

[0055] The information processing method realized by the information processing program according to the second embodiment is an information processing method executed by a computer. In this information processing method, when a loop is started, first, the reception unit 110 executes a reception procedure (ST110) for receiving question specification information transmitted from the user terminal UT. Then, the information acquisition unit 120 executes an information acquisition procedure (ST120) for acquiring from a database, based on the question specification information received in the reception procedure, disclosure information disclosed by a predetermined organization, information on a question used when an ESG evaluation organization evaluates the organization, and information on the answer format of the question. Then, the information input unit 130 executes an information input procedure (ST130) for inputting the disclosure information acquired in the information acquisition procedure, information on the question, and information on the answer format of the question to the machine learning model. Furthermore, the information output unit 140 executes an information output procedure (ST140) for acquiring information on the answer to the question output from the machine learning model, and outputting the acquired information on the answer. Note that, although FIG. 11 shows an example of a loop process, the disclosed technology is not limited to this.

[0056] The functions of the reception unit 110, the information acquisition unit 120, the information input unit 130, and the information output unit 140 of the information processing device 100 according to the disclosed technology are realized by processing circuits. That is, the information processing device 100 includes processing circuits for executing the reception procedure ST110, the information acquisition procedure ST120, the information input procedure ST130, and the information output procedure ST140. The processing circuit is a processor 230 (also referred to as a CPU, central processing unit, processing device, arithmetic unit, microprocessor, microcomputer, or DSP) that executes an information processing program stored in the memory 240.

[0057] Each function of the reception unit 110, the information acquisition unit 120, the information input unit 130, and the information output unit 140 is realized by software, firmware, or a combination of software and firmware. The software and firmware are written as an information processing program and stored in the memory 240. The processing circuit realizes the function of each unit by reading and executing the information processing program stored in the memory 240. That is, the information processing device 100 includes a memory 240 for storing an information processing program that results in the reception procedure ST110, the information acquisition procedure ST120, the information input procedure ST130, and the information output procedure ST140 being executed by the processing circuit. It can also be said that these information processing programs cause a computer to execute the procedures or methods of the reception unit 110, the information acquisition unit 120, the information input unit 130, and the information output unit 140. Here, the memory 240 may be, for example, a non-volatile or volatile semiconductor memory such as a RAM, a ROM, a flash memory, or an EPROM. The memory 240 may also be in the form of a disk, such as a magnetic disk, a floppy disk, an optical disk, a compact disk, a mini disk, a DVD, etc. The memory 240 may also be in the form of a HDD or SSD.

[0058] The information processing program shown in the second embodiment may be one that causes a computer to execute at least the information acquisition procedure ST120, the information input procedure ST130, and the information output procedure ST140, and the reception procedure ST110 is an optional procedure. When the reception procedure ST110 is omitted, the process in the reception procedure ST110 may be performed, for example, in the information acquisition procedure ST120.

[0059] As described above, the information processing program of embodiment 2 causes a computer to execute the following steps: a reception procedure for receiving, from a user terminal UT, question designation information capable of identifying a question to be answered that is a question used by an ESG evaluation agency in the evaluation of a specified organization and that is designated by a user; an information acquisition procedure ST120 for acquiring, from a database, disclosure information to be disclosed by the specified organization, information regarding the question specified by the question designation information, and information regarding the answer format to the question, based on the question designation information received in the reception procedure; an information input procedure ST130 for inputting the disclosure information acquired in the information acquisition procedure, information regarding the question, and information regarding the answer format to the question, into a machine learning model; and an information output procedure ST140 for acquiring information regarding the answer to the question output from the machine learning model, and outputting the acquired information regarding the answer to the user terminal UT. Therefore, the information processing program according to the second embodiment can cause a computer to execute a predetermined procedure to assist in creating answers to questions used by an ESG rating agency in rating a predetermined organization.

[0060] In addition, in the information processing program of embodiment 2, the information relating to the answer format of a question includes information relating to an identifier for identifying the question, and the information relating to the identifier for identifying the question may be provided with, for each question, one or more answer formats, dependencies between other questions, and tag information relating to the input format of the answer. As a result, the information processing program shown in embodiment 2 can cause a computer to execute a predetermined procedure and present to the user, for each question, an answer that appropriately takes into consideration the answer format, dependencies between other questions, and the answer input format.

[0061] In the information processing program according to the second embodiment, the question indicated by the information regarding the question may include a main text of the question and an auxiliary answer text for assisting in the answer to the question. As a result, the information processing program shown in embodiment 2 can cause a computer to execute a specified procedure to generate information regarding answers to questions with higher accuracy, and can present highly accurate answers to questions to the user.

[0062] In the information processing program according to the second embodiment, in the information input procedure ST130, the disclosure information, information related to the question, and information related to the answer format to the question may be input to the machine learning model as input instructions. As a result, the information processing program shown in embodiment 2 can cause a computer to execute a specified procedure to generate information regarding answers to questions in a machine learning model taking into account the answer format of the question, and can present the user with highly accurate answers that take into account the answer format of the question.

[0063] In addition, in the information processing program of embodiment 2, the input instructions may include multiple pieces of information related to questions and information related to the answer format of the questions, and may also include information related to the dependencies between each of the questions indicated by the information related to the multiple questions and other questions. As a result, the information processing program shown in embodiment 2 can cause a computer to execute a predetermined procedure to cause a machine learning model to generate answer-related information for each of a plurality of questions, in which the answer format and the dependencies between other questions are appropriately taken into consideration, and can present to a user answers for each question in which the answer format and the dependencies between other questions are appropriately taken into consideration.

[0064] In the information processing program according to the second embodiment, the input instruction may include information regarding an input format for an answer to a question. As a result, the information processing program shown in embodiment 2 can cause the machine learning model to generate information regarding answers to questions while taking into account the input format of the answer, and can present the user with highly accurate answers that take into account the input format of the answer.

[0065] In addition, in the information processing program according to the second embodiment, there are multiple machine learning models, and in the information input step ST130, the disclosure information, information related to the question, and information related to the answer format of the question may be input to any one of the multiple machine learning models according to predetermined conditions. As a result, the information processing program according to the second embodiment can cause a computer to execute a predetermined procedure, and can select and use an appropriate machine learning model according to the predetermined conditions. Also, the information processing program according to the second embodiment can present a highly accurate answer according to the predetermined conditions to a user.

[0066] Furthermore, in the information processing program according to the second embodiment, in the information input procedure ST130, if the answer format indicated by the information on the answer format to the question includes an answer in natural language, a prompt instructing the user to answer the question is generated, the prompt including information on the question and information on the answer format to the question, and the generated prompt and the disclosure information are input to a large-scale language model. If the answer format indicated by the information on the answer format to the question does not include an answer in natural language, the disclosure information, information on the question, and information on the answer format to the question may be input to a machine learning model other than the large-scale language model. As a result, the information processing program according to the second embodiment can execute a predetermined procedure on a computer to select and use an appropriate machine learning model depending on whether or not the answer format includes an answer in natural language. Also, the information processing program according to the second embodiment can present a highly accurate answer to a user depending on whether or not the answer format includes an answer in natural language.

[0067] In addition, in the information processing program according to the second embodiment, the prompt may include an instruction to cause the large-scale language model to output, when an appropriate answer to a question is not present in the disclosed information, a message indicating that an appropriate answer is not present in the disclosed information, or a message prompting the user of the user terminal to supplement the answer indicated by the information indicating the answer. As a result, the information processing program shown in embodiment 2 can cause the computer to execute a specified procedure and notify the user that if an appropriate answer to a question is not present in the disclosed information, the appropriate answer is not present in the disclosed information.

[0068] Furthermore, in the information processing program according to the second embodiment, in the information acquisition step ST120, a search query for searching for disclosure information relating to the question from the disclosure information stored in the database is generated based on the question indicated by the information relating to the question, and the database is searched using the generated search query to acquire disclosure information corresponding to the search query, and in the information input step ST130, the acquired disclosure information, information relating to the question, and information relating to the answer format of the question may be input to the machine learning model as input instructions. As a result, the information processing program shown in embodiment 2 can cause a computer to execute a specified procedure to appropriately acquire disclosure information related to a question as specific disclosure information, and can input the acquired specific disclosure information into a machine learning model as an input instruction. Moreover, the information processing program according to the second embodiment can present the user with an appropriate answer based on the specific disclosure information.

[0069] In the information processing program according to the second embodiment, in the information acquisition step ST120, the upper limit of the number of pieces of disclosure information to be acquired from the database using a search query can be set arbitrarily. As a result, the information processing program according to the second embodiment can cause a computer to execute a predetermined procedure, thereby preventing an excessive increase in the amount of specific disclosure information to be acquired.

[0070] In the information processing program according to the second embodiment, in the information output procedure ST140, information about the answer acquired from the machine learning model may be processed so as to be displayed in a predetermined program and then output. As a result, the information processing program according to the second embodiment can cause the computer to execute a predetermined procedure, making it possible to display information relating to the answer in a predetermined program.

[0071] In addition, in the information processing program according to the second embodiment, in the information output procedure ST140, the information regarding the answer obtained from the machine learning model may be processed so that the format of the information regarding the answer includes instructions regarding where to input the answer. As a result, the information processing program shown in embodiment 2 causes a computer to execute a predetermined procedure, thereby reducing the workload of a user when, for example, a user needs to use a user terminal UT to manually input an answer indicated by information relating to the answer into a predetermined program by copying and pasting or other means.

[0072] In the information processing program according to the second embodiment, the information output in the information output procedure ST140 may include location information indicating where in the disclosure information the information on which the answer to the question is based is located. In this way, the information processing program according to the second embodiment can cause a computer to execute a predetermined procedure to present to a user where in the disclosed information the information that is the basis for an answer to a question is located. Also, the user can easily grasp where in the disclosed information the information that is the basis for an answer to a question is located.

[0073] In addition, in the information processing program of embodiment 2, if the answer format of the question is a format in which a specific option is selected from a plurality of options, the location information is information indicating the location of disclosure information that can explain the reason for the selection. In this way, the information processing program shown in the second embodiment can cause a computer to execute a predetermined procedure to present to a user where in the disclosed information is information that is the basis for selecting an option for a question. Also, the user can easily grasp where in the disclosed information is information that is the basis for selecting an option for a question.

[0074] Furthermore, in the information processing program according to the second embodiment, in the reception procedure ST110, question specification information capable of identifying the name of an ESG evaluation agency is received as question specification information, and in the information acquisition procedure ST120, based on the question specification information received in the reception procedure ST110, disclosure information disclosed by a specified organization, information regarding the question posed by the ESG evaluation agency with the above name, and information regarding the answer format to the question are acquired from a database. As a result, the information processing program according to the second embodiment can cause the computer to execute a predetermined procedure, enabling the user to specify a question based on the name of an ESG evaluation organization.

[0075] Furthermore, in the information processing program according to the second embodiment, in the information output procedure ST140, when information regarding the answer is output to the user terminal UT, the information regarding the answer is output in a manner capable of accepting input from the user terminal UT with respect to the answer indicated by the information regarding the answer. As a result, the information processing program according to the second embodiment can cause the computer to execute a predetermined procedure and output information relating to the answer to the user in a form that leaves room for the user to input the answer.

[0076] The information processing device 100 of embodiment 1 includes a reception unit 110 that receives, from a user terminal UT, question designation information that can identify a question used by an ESG evaluation agency in evaluating a specified organization and designated by a user as a question to be answered; an information acquisition unit 120 that acquires, from a database, disclosure information of the specified organization, information regarding the question specified by the question designation information, and information regarding the answer format of the question based on the question designation information received by the reception unit 110; an information input unit 130 that inputs the disclosure information acquired by the information acquisition unit 120, information regarding the question, and information regarding the answer format of the question to a machine learning model; and an information output unit 140 that acquires information regarding the answer to the question output from the machine learning model and outputs the acquired information regarding the answer to the user terminal UT. Therefore, the information processing device 100 shown in the first embodiment can assist in creating answers to questions used by an ESG rating agency in rating a given organization.

[0077] The information processing method according to the first embodiment is an information processing method by an information processing device, in which the reception unit 110 receives, from the user terminal UT, question specification information capable of identifying a question that is used by an ESG rating agency in the evaluation of a predetermined organization and is specified by a user as a question to be answered, the information acquisition unit 120 acquires from a database, based on the question specification information received by the reception unit 110, disclosure information of the predetermined organization, information on the question specified by the question specification information, and information on the answer format of the question, the information input unit 130 inputs the disclosure information acquired by the information acquisition unit 120, information on the question, and information on the answer format of the question, to a machine learning model, the information output unit 140 acquires information on the answer to the question output from the machine learning model, and outputs the acquired information on the answer to the user terminal UT. The information processing device 100 executes the above method, thereby supporting the creation of answers to questions used by an ESG rating agency in the evaluation of a predetermined organization. [Explanation of symbols]

[0078] 100 information processing device, 110 reception unit, 120 information acquisition unit, 130 information input unit, 140 information output unit, 150 disclosure information database, 160 question information database, 200 hardware configuration, 201 question body, 202 answer auxiliary text, 210 communication interface, 220 input / output interface, 230 processor, 240 memory, 300 machine learning model, 301 question body, 302 answer auxiliary text, 303 options, 401 question body, 402 answer auxiliary text, 403 table of options, 501 question identification information, 502 tag information, 701 question body, 702 answer auxiliary text, 801 question body and answer auxiliary text embedded in prompt, 802 options embedded in prompt, 901 answer options, 902 basis for selection, UT user terminal.

Claims

1. A receiving step of receiving, from a user terminal, question specification information that can identify a question that is used by an ESG rating agency in rating a predetermined organization and that is specified by a user as a question to be answered; an information acquisition step of acquiring, from a database, disclosure information disclosed by the predetermined organization, information regarding the question specified by the question specification information, and information regarding the answer format of the question, based on the question specification information received in the reception step; an information input step of inputting the disclosure information acquired in the information acquisition step, information related to the question, and information related to the answer format of the question into a machine learning model; an information output step of acquiring information about an answer to a question output from the machine learning model, and outputting the acquired information about the answer to the user terminal; An information processing program for causing a computer to execute the above.

2. The information regarding the answer format of the question includes information regarding an identifier for identifying the question, The information on the identifier for identifying the question includes, for each question, one or more answer formats, dependencies between the questions and other questions, and tag information on an input format of the answer. The information processing program according to claim 1.

3. The question indicated by the information on the question is: The question includes a question body and an answer auxiliary text for assisting in answering the question.

3. The information processing program according to claim 1 or 2.

4. In the information input step, the disclosure information, information about the question, and information about an answer format to the question are input to the machine learning model as input instructions.

3. The information processing program according to claim 1 or 2.

5. The input instruction is: the system includes a plurality of pieces of information on the questions and a plurality of pieces of information on the answer formats of the questions, and includes information on dependencies between each of the questions indicated by the pieces of information on the questions and other questions; The information processing program according to claim 4.

6. The input instructions include information regarding an input format for an answer to a question. The information processing program according to claim 4.

7. There are multiple machine learning models, In the information input step, inputting the disclosure information, information about the question, and information about the answer format of the question into one of a plurality of machine learning models according to a predetermined condition; 3. The information processing program according to claim 1 or 2.

8. In the information input step, If the answer format indicated by the information on the answer format of the question includes an answer in natural language, a prompt is generated that instructs the user to answer the question, the prompt including information on the question and information on the answer format of the question, and the generated prompt and the disclosure information are input into a large-scale language model; If the answer format indicated by the information on the answer format of the question does not include an answer in natural language, inputting the disclosure information, information on the question, and information on the answer format of the question into a machine learning model other than a large-scale language model.

8. The information processing program according to claim 7.

9. The prompt may include: When an appropriate answer to a question is not present in the disclosed information, an instruction to cause the large-scale language model to output a message indicating that an appropriate answer is not present in the disclosed information, or a message prompting a user of the user terminal to supplement the answer indicated by the information indicating the answer.

9. The information processing program according to claim 8.

10. In the information acquisition step, generating a search query for searching for disclosure information related to the question from the disclosure information stored in the database based on the question indicated by the information related to the question, and searching the database using the generated search query to obtain disclosure information corresponding to the search query; In the information input step, the acquired disclosure information, information about the question, and information about an answer format to the question are input as input instructions to the machine learning model.

3. The information processing program according to claim 1 or 2.

11. In the information acquisition step, The upper limit of the number of pieces of disclosure information to be acquired from the database using a search query can be set arbitrarily. The information processing program according to claim 10.

12. In the information output step, Processing information about the answer obtained from the machine learning model so that it can be displayed in a predetermined program and then outputting the processed information.

3. The information processing program according to claim 1 or 2.

13. In the information output step, Processing information about the answer obtained from the machine learning model so that the format of the information about the answer includes instructions on where to enter the answer; 13. The information processing program according to claim 12.

14. The information to be output in the information output step is Including location information indicating where in the disclosed information the information on which the answer to the question is based is located; 3. The information processing program according to claim 1 or 2.

15. If the answer format of the question is to select a given option from multiple options, The location information is information indicating the location of the disclosure information capable of explaining the reason for the selection.

15. An information processing program according to claim 14.

16. In the receiving step, question specifying information capable of identifying a name of an ESG evaluation organization is received as the question specifying information, In the information acquisition step, based on the question designation information received in the reception step, disclosure information disclosed by the specified organization, information regarding questions from the ESG evaluation organization having the name, and information regarding the answer format of the questions are acquired from a database.

3. The information processing program according to claim 1 or 2.

17. In the information output step, When outputting the information about the answer to the user terminal, the information about the answer is output in a manner capable of accepting an input from the user terminal with respect to the answer indicated by the information about the answer.

3. The information processing program according to claim 1 or 2.

18. a reception unit that receives, from a user terminal, question specification information that can identify a question that is used by an ESG rating agency in rating a predetermined organization and that is specified by a user as a question to be answered; an information acquisition unit that acquires, based on the question specification information received by the reception unit, disclosure information disclosed by the predetermined organization, information regarding the question specified by the question specification information, and information regarding the answer format of the question from a database; an information input unit that inputs the disclosure information acquired by the information acquisition unit, information about a question, and information about an answer format to the question into a machine learning model; an information output unit that acquires information about an answer to a question output from the machine learning model and outputs the acquired information about the answer to the user terminal; An information processing device comprising:

19. An information processing method by an information processing device, the reception unit receives, from the user terminal, question specification information capable of identifying a question designated by the user as a question to be answered, which is a question used by an ESG rating agency in rating a predetermined organization; an information acquisition unit acquires, based on the question specification information received by the reception unit, disclosure information disclosed by the predetermined organization, information on the question specified by the question specification information, and information on the answer format of the question from a database; an information input unit inputs the disclosure information acquired by the information acquisition unit, information related to the question, and information related to the answer format of the question into a machine learning model; an information output unit acquires information about an answer to the question output from the machine learning model, and outputs the acquired information about the answer to the user terminal; Information processing methods.

Images