Information processing program, information processing device, and information processing method

Abstract

This program provides an information processing program that can assist ESG rating agencies in creating answers to questions they use to evaluate specific organizations. [Solution] The information processing program related to the disclosed technology causes a computer to execute the following steps: reception procedure (ST110) which receives question designation information from a user terminal that can identify questions used by an ESG rating agency to evaluate a specified organization and which have been designated by the user as questions to be answered; information acquisition procedure (ST120) which, based on the question designation information, obtains from a database disclosure information to be disclosed by the specified organization, information related to the questions identified by the question designation information, and information related to the format of the answers to the questions; information input procedure (ST130) which inputs each of the acquired pieces of information into a machine learning model; and information output procedure (ST140) which outputs information related to the answers to the questions output from the machine learning model to the user terminal.

Description

Technical Field

[0001] The disclosed technology relates to an information processing program, an information processing apparatus, and an information processing method.

Background Art

[0002] In recent years, as an essential business policy for the sustainable growth of organizations such as companies, ESG management has attracted global attention, and the number of organizations working on its introduction is increasing. Also, in various countries or regions around the world, various disclosure systems have been established that require organizations involved in activities within their jurisdiction to disclose information in accordance with disclosure criteria from the perspective of ESG (the initials of Environment, Social, and Governance). There are also ESG evaluation institutions that evaluate whether an organization is conducting activities considering ESG. Thus, ESG has come to be as notable as SDGs (Sustainable Development Goals). Also, as a term, in addition to the above-mentioned "ESG management," it generally has become widely used in the form of idiomatic expressions including ESG such as "ESG investment," "ESG report," "ESG analytics," etc.

[0003] ESG evaluation institutions collect information on ESG of organizations such as companies and evaluate whether the organizations are conducting activities considering ESG based on the collected information. ESG evaluation institutions may send questions to these organizations for the purpose of collecting information on their ESG. Generally, the questions sent by ESG evaluation institutions to organizations differ in question format and content depending on each evaluation institution. Therefore, in order for each organization to answer these questions, the person in charge of each organization needs to understand the question format and content of each question, search for the answer visually, and then create the answer.

[0004] Incidentally, while Patent Document 1 does not show any application to ESG, it discloses a technology that accepts questions about public gambling, uses a RAG (Retrieval Augmented Generation) mechanism to acquire data related to the questions, and generates answers to the questions using the accepted questions, acquired data, and a trained model. [Prior art documents] [Patent Documents]

[0005] [Patent Document 1] Patent No. 7550329 [Overview of the Initiative] [Problems that the invention aims to solve]

[0006] Technologies that enable the automatic generation of answers to questions using a trained model and the provision of the generated answers to the user have existed for some time, such as the technology disclosed in Patent Document 1. However, conventional technologies have had difficulty supporting the generation of answers to questions written in various formats, such as questions used by multiple ESG rating agencies to evaluate a given organization.

[0007] The disclosed technology aims to solve the above-mentioned problems and provides an information processing program, information processing device, and information processing method that can assist ESG rating agencies in creating answers to questions used in evaluating specific organizations. [Means for solving the problem]

[0008] The information processing program relating to this disclosed technology causes a computer to execute the following steps: a reception procedure that receives question designation information from a user terminal that can identify questions used by an ESG rating agency to evaluate a designated organization and which have been designated by the user as questions to be answered; an information acquisition procedure that, based on the question designation information received in the reception procedure, obtains from a database disclosure information to be disclosed by the designated organization, information about the questions identified by the question designation information, and information about the format of the answers to the questions; an information input procedure that inputs the disclosure information, information about the questions, and information about the format of the answers to the questions obtained in the information acquisition procedure into a machine learning model; and an information output procedure that obtains information about the answers to the questions output from the machine learning model and outputs the obtained information about the answers to the questions to the user terminal. [Effects of the Invention]

[0009] The information processing program relating to this disclosed technology has the above-described technical characteristics and has the effect of supporting ESG rating agencies in creating answers to questions used in evaluating designated organizations. [Brief explanation of the drawing]

[0010] [Figure 1] This is a block diagram showing an example configuration of an information processing system related to the disclosed technology. [Figure 2] This figure shows an example of a question in the disclosed technology, including the main text of the question and supplementary answer text. [Figure 3] This figure shows an example of a question format in which the user selects the appropriate option from multiple choices, as described in this disclosed technology. [Figure 4] This figure shows an example of question information in the disclosed technology. [Figure 5] This figure shows an example of question-format information in the disclosed technology. [Figure 6] This diagram illustrates an example of the operation of the information processing device related to the disclosed technology and the flow of each piece of information. [Figure 7]This figure shows an example of the questions used in the experiment in the disclosed technology, and the answer choices provided. [Figure 8] This figure shows an example of a prompt generated by the information acquisition unit in an experiment using the disclosed technology. [Figure 9] This figure shows an example of a response output from a large-scale language model in an experiment using the disclosed technology. [Figure 10] This figure shows the hardware configuration of the information processing device related to the disclosed technology. [Figure 11] This is a flowchart showing the processing procedure of the information processing method related to the disclosed technology. [Modes for carrying out the invention]

[0011] Next, some important terms used in the disclosed technology are defined as follows.

[0012] ESG Evaluation Criteria ESG evaluation items are defined as a single, identical item when they share common concepts or ideas, even if the wording of these evaluation items differs among different ESG rating agencies. ESG evaluation items include, for example, four items: ESG classification, code (number), major category, and minor category. In ESG evaluation criteria, the ESG classification indicates whether each data point falls under Environment, Social, or Governance. The letters used to represent the ESG classification are, for example, "E," "S," and "G." The codes (numbers) in ESG evaluation items are used to distinguish each subcategory, and each subcategory is assigned a different code. These codes that distinguish subcategories are also called "classification codes." The major categories and minor categories in the ESG evaluation items are the names expressed in characters that make it easier for people to understand what kind of content the data indicating ESG-related information is respectively. As the names of these major categories and minor categories, general or standardized expressions are used for the ESG evaluation items used by each ESG evaluation institution. The major categories include, for example, "Overall Environment", "Climate Change", "Water", "Resource Recycling", "Biodiversity", "Supplier (Environment)", "Environmental Opportunities and Impacts", "Human Rights", "Labor Practices", "Diversity", "Talent Development", "Employee Safety and Health", "Corporate Citizenship", "Product Quality and Safety", "Corporate Governance", "Risk Management", "Information Security and Privacy", etc. Also, the minor categories include, for example, as those belonging to "Overall Environment", "Policies Regarding Environmental Management", as those belonging to "Climate Change", "Policies to Address Climate Change", "Policies Regarding Reduction of Energy Consumption", "Carbon Pricing", etc.

[0013] 《Designated Organization》 A designated organization is an organization that receives an evaluation by an ESG evaluation institution by providing ESG-related information to the ESG evaluation institution. Designated organizations include, for example, companies, government agencies, non-profit organizations, industry associations, or academic institutions, etc. The ESG-related information provided by these organizations serves as a reference for the ESG evaluation of, for example, companies, and is also an important element for understanding the environmental and social efforts of such companies in the whole society.

[0014] 《Disclosure Materials and Disclosure Information》 Disclosure materials are materials in which each organization discloses the actual situation regarding its own ESG. Disclosure materials include, for example, integrated reports, corporate governance reports, etc. In addition, the disclosed information refers to the information disclosed by each organization as information corresponding to each indicator defined by various disclosure systems that require disclosure in accordance with the disclosure criteria from the perspective of ESG. That is, the disclosed information can be said to be the information described in the part corresponding to any of the ESG evaluation items (for example, its sub-classifications) among the contents disclosed in the disclosure materials.

[0015] Embodiment 1. Embodiment 1 shows the case where the disclosed technology is realized as the information processing device 100. FIG. 1 is a block diagram showing a configuration example of an information processing system according to the disclosed technology. As shown in FIG. 1, the information processing system is a system in which the information processing device 100 and the user terminal UT are connected via the Internet. In FIG. 1, three user terminals UT are shown, but the number of user terminals UT may be any number of one or more. Also, in FIG. 1, the information processing device 100 and the user terminal UT are connected via the Internet, but this connection is not limited to the Internet, and any telecommunications line may be used.

[0016] 《Information Processing Device 100》 The information processing device 100 supports the answer creation work when a predetermined organization to which the user belongs creates an answer to a question from an ESG evaluation institution by using the RAG mechanism, and includes a reception unit 110, an information acquisition unit 120, an information input unit 130, and an information output unit 140. In the following description, for the sake of specificity, it is assumed that the predetermined organization to which the user belongs is a company. In addition, the information processing device 100 includes a disclosed information database 150 and a question information database 160. Note that the information processing device 100 is not limited to this configuration, and the disclosed information database 150 and the question information database 160 may be provided by an external device that can be communicatively connected to the information processing device 100. The information processing device 100 provides the user with a service related to the disclosed technology, namely, a service that supports the user's company in preparing answers to questions from ESG rating agencies using the RAG mechanism (hereinafter referred to as the "answer preparation support service"). The information processing device 100 is any computer, such as a server or a terminal used by a user. Here, the response creation support service may be provided, for example, in the form of SaaS (Software as a Service). In this case, the information processing device 100 is owned or managed by the provider of the response creation support service (hereinafter referred to as the "service provider"). This specification provides details regarding the case where the information processing device 100 related to the disclosed technology is, for example, a server owned or managed by a service provider.

[0017] Disclosure Information Database The disclosure information database 150 holds the disclosure information of the company to which the user belongs. The disclosure information held by the disclosure information database 150 is ESG-related information of the company in question, relating to ESG evaluation items. The disclosure information held in the disclosure information database 150 consists of the disclosure information of one or more companies. The disclosure information of each company is obtained, for example, by crawling. Crawling is a well-known technique in which a program called a crawler periodically visits multiple websites to obtain and store information.

[0018] Here, for example, first, the information processing device 100 periodically visits the websites of multiple companies designated in advance by the service provider to acquire disclosure information from those companies. The information processing device 100 first converts the acquired disclosure information from the companies into text data using optical character recognition (OCR), then divides the converted text data into predetermined ranges, and converts the text data into vectors for each divided range. The information processing device 100 then links the vectors obtained by the conversion with information that identifies the company disclosing the information (hereinafter referred to as "company identification information") and stores them in the disclosure information database 150. Company-specific information includes, for example, the company's official name, abbreviation, or a unique ID or code. Hereafter, company-specific information will be defined as the company's official name or abbreviation. If the disclosed information is created in PDF format, the disclosed information may already contain text data. In that case, the information processing device 100 may not need to convert the disclosed information into text data using optical character recognition (OCR) as described above. Furthermore, the disclosed information of multiple companies acquired by the information processing device 100 may be created in HTML format. When the information processing device 100 divides the aforementioned text data into predetermined ranges, it can divide it, for example, by page, by paragraph, or by the number of characters. When dividing the text data by the number of characters, the information processing device 100 can set any number of characters, such as a maximum of 400 characters, as the unit of division. In this case, the information processing device 100 may also divide the text data so that there is an overlap of any number of characters, such as 100 characters, before and after the divided range.

[0019] 《Question Information Database》 The Question Information Database 160 holds information (hereinafter referred to as "Question Information") regarding questions used by ESG rating agencies when evaluating companies. For the purposes of the following explanation, the questions used by ESG rating agencies when evaluating companies will be assumed to be, for example, questions sent from ESG rating agencies to companies. The Question Information Database 160 is created and managed by the service provider for each company to which the user belongs. For example, a service provider may request a company in advance to answer questions sent by an ESG rating agency in a specified data format, such as Excel. The service provider then reads the file returned by the company using an information processing device 100 and stores the question information contained in the file in a question information database 160. Furthermore, the question information may be stored in the question information database 160 after the content of the questions has been generalized by the service provider, for example. Alternatively, the question information may be stored in the question information database 160 after the writing style of multiple questions has been standardized by the service provider, for example.

[0020] Furthermore, the questions indicated by the question information may be divided in advance into the question text and supplementary answer text to assist in answering the question, and then stored in the question information database 160. Figure 2 shows an example of a question, including the main text and supporting text for the answer. Typically, questions sent to companies by ESG rating agencies consist of multiple sentences per question. Of these, for example, the sentence at the beginning of the question that explains the general outline of the question can be called the question text, and the sentences that follow the question to support the answer can be called the answer supplementary sentence. The answer supplementary sentence explains, for example, what the question text specifically asks or what kind of answer should be given. In the example in Figure 2, "Material Issues for..." with the code 201 is the question text, and "Does your company..." with the code 202 is the answer supplementary sentence. Furthermore, if the question information indicates a question in which the respondent must select the appropriate option from multiple choices, the question information may include information about the options presented as answers.

[0021] Figure 3 shows an example of a question format in which the respondent selects the appropriate option from multiple choices. In Figure 3, "Climate Governance," labeled with code 301, is the main body of the question, and "Is your...," labeled with code 302, is the answer support text. Each sentence with a checkbox labeled with code 303 is one of the options that the company can select. For example, a company representative reads the main body of the question and the answer support text, and selects the appropriate option as the answer by checking the checkbox. Here, options 303 are indented by level. The company representative can select one or more appropriate options from the options at the same level for each level. Note that while Figure 3 shows an example where the question includes only option 303, some questions may include an open-ended response field corresponding to option 303. In this case, the company representative can, for example, select a predetermined option and then write any answer they wish in the open-ended response field corresponding to that option.

[0022] Figure 4 shows an example of question information stored in the question information database 160. In Figure 4, the question information shows a question that, like the question exemplified in Figure 3, consists of the question text, a supporting answer sentence, and multiple answer choices. However, the content of the question in Figure 4 is different from the content of the question in Figure 3. In Figure 4, "Material Issues..." labeled with reference numeral 401 is the main text of the question, and "Does your company..." labeled with reference numeral 402 is the supplementary answer text. The table labeled with reference numeral 403 organizes the options that companies can select in a tabular format. In this way, question information can be divided for each question into the main text of the question 401, the supplementary answer text 402, and the table 403 concerning the options, and then stored in the question information database 160.

[0023] Furthermore, the question information database 160 holds information regarding the format of the answers to the questions (hereinafter referred to as "question format information"). The question format information includes information regarding identifiers for identifying the questions (hereinafter referred to as "question identification information"). The identifier for identifying a question is an identifier that can uniquely identify the question, and is composed of a combination of the name of the ESG rating agency and the question number, such as "DJSI (S&P Dow Jones Indices); Q1" or "DJSI; Q2". Furthermore, the question identification information includes tag information for each question, such as one or more answer formats, dependencies between questions, and the format of the answer input. Figure 5 shows an example of question format information stored in the question information database 160. In Figure 5, the column labeled with code 501 shows the question identification information described above, and the column labeled with code 502 shows the tag information described above. The "answer format" included in tag information 502 indicates the answer format to the question. Answer formats include, for example, "free response," "YES / NO," "multiple choice," and "table format." Furthermore, "Answer Format 2" included in tag information 502 indicates a second answer format for the question. For example, Q3 and Q4 sent from DJSI indicate that the answer should first be given using a "multiple-choice" format, followed by a "free-text" response after selection. Thus, some questions require answers in multiple different formats. While only up to the second answer format is shown here, some questions may have "Answer Format 3 (third answer format)" and beyond.

[0024] The "dependency" included in tag information 502 indicates the dependency between questions, for example, the dependency between one question and another. For example, Q3 sent from DJSI needs to be answered if the answer to Q2, also sent from DJSI, is "YES," and Q4 sent from DJSI needs to be answered if the answer to Q2, also sent from DJSI, is "NO." Furthermore, the "input format" included in tag information 302 indicates the input format for the answer to the question. Input formats include, for example, "natural language input," "button," "checkbox," and "numerical input." The question format information, configured as described above, is generated, for example, by a service provider or a user belonging to a company. Furthermore, the question information database 160, configured as described above, is created and managed by, for example, a service provider for each company to which a user belongs. Meanwhile, a user can access the information processing device 100, which is a server, from their user terminal UT via the internet and edit the question information and question format information held in their company's question information database 160 as needed.

[0025] Reception Desk 110 The reception unit 110 receives information from the user terminal UT that allows the user to identify the question to be answered (hereinafter referred to as "question specification information"). The user can operate the user terminal UT to access the information processing device 100 and display a screen for using the answer creation support service on the user terminal UT's display (not shown). This screen will be displayed in a browser, for example, if the answer creation support service is provided in the form of SaaS. This screen may display, for example, a question selection window. The user can operate the user terminal UT and input an answer to any question into this question specification window. When the user inputs an answer to any question into the question specification window, the user terminal UT generates information that identifies the question specified by the user (question specification information) and transmits the generated question specification information to the information processing device 100.

[0026] For example, when a user answers question "Q1" sent by the ESG rating agency "DJSI", the user operates the user terminal UT and enters "Answering Q1 from DJSI" into the question specification window. When the user enters the above into the question specification window, the user terminal UT generates question specification information indicating that the ESG rating agency is "DJSI" and the question number is "Q1", and transmits the generated question specification information to the information processing device 100. In this case, the question identified by "DJSI" as the ESG rating agency specified by the user and "Q1" as the question number specified is the "question specified by the user". The reception unit 110 of the information processing device 100 receives question specification information transmitted from the user terminal UT. Upon receiving the question specification information transmitted from the user terminal UT, the reception unit 110 outputs the received question specification information to the information acquisition unit 120. The question specification information also includes information that identifies the company to which the user who transmitted the question specification information belongs (company identification information).

[0027] 《Information acquisition section 120》 The information acquisition unit 120 acquires question specification information from the reception unit 110. Upon acquiring the question specification information from the reception unit 110, the information acquisition unit 120 identifies what the question specified by the user is based on the acquired question specification information. Furthermore, when the information acquisition unit 120 acquires question specification information from the reception unit 110, it identifies the company to which the user who sent the question specification information belongs, based on the company identification information contained in the acquired question specification information. The information acquisition unit 120 then searches the question information database 160 managed by the identified company and obtains information (question information) related to the question specified by the user from the question information database 160. The information acquisition unit 120 also obtains question format information from the question information database 160. When the information acquisition unit 120 obtains question format information from the question information database 160, it may obtain the entire question format information as exemplified in Figure 5, or it may obtain the question format information partially on an ESG rating agency basis, or it may obtain the question format information partially on a question basis.

[0028] Next, the information acquisition unit 120 searches the disclosure information of multiple companies held in the disclosure information database 150 for the disclosure information necessary to answer the questions identified above. For example, the information acquisition unit 120 generates a search query to find the disclosure information necessary to answer a question, based on the text data indicating the question body and supplementary answer text included in the question information obtained from the question information database 160. Once the information acquisition unit 120 generates a search query, it performs a vector search of the disclosure information database 150 based on the generated search query and obtains the disclosure information necessary to answer the question as a search result. Furthermore, the information acquisition unit 120 may arbitrarily set an upper limit on the number of disclosure information items to be acquired from the disclosure information database 150 based on the generated search query as a search condition when searching the disclosure information database 150. For example, the information acquisition unit 120 may set an upper limit on the number of disclosure information items to be acquired as a search condition, such as acquiring up to the top 10 disclosure information items with a high percentage of matching the search query as search results. As described above, the information acquisition unit 120 acquires question information and question format information from the question information database 160, and acquires disclosure information necessary to answer the question from the disclosure information database 150, and outputs this acquired information to the information input unit 130. For the sake of simplicity in the following explanation, the disclosure information necessary to answer the question, acquired from the disclosure information database 150, will be referred to as "specific disclosure information."

[0029] Information input section 130 The information input unit 130 acquires question information, question format information, and specific disclosure information from the information acquisition unit 120. Once the information input unit 130 acquires the question information, question format information, and specific disclosure information from the information acquisition unit 120, it inputs this acquired information into the machine learning model. The question information is divided into the question text and a supporting sentence to assist in answering the question, as illustrated in Figure 2. A machine learning model may consist of, for example, an artificial neural network, and there may be multiple such models. Furthermore, if there are multiple machine learning models, they may include a Large Language Model (LLM). The information input unit 130 uses the question information, question format information, and specific disclosure information obtained from the information acquisition unit 120 as input instructions, and inputs these input instructions to one of several machine learning models according to predetermined conditions. The input instructions may include multiple sets of question information and multiple sets of question format information. Furthermore, the input instructions may include information regarding the dependencies between each of the multiple sets of question information and other questions as part of the question format information.

[0030] For example, if the information input unit 130 determines that the answer format to the question indicated by the tag information included in the question format information, or the answer input format, is in a format where the answer or input is in natural language (natural sentence), it generates a prompt and inputs the generated prompt into the Large-Scale Language Model (LLM) along with the specified disclosure information. In this case, the prompt includes question information and question format information. Furthermore, the prompt in this case includes content instructing the user to answer the question indicated by the question information in accordance with the answer format indicated by the question format information. Furthermore, if the prompt can include specific disclosure information, the information input unit 130 only needs to input the prompt into the large-scale language model. Furthermore, the prompt may include instructions to cause the large-scale language model to output a message indicating that a suitable answer to the question is not found in the specified disclosure information, or that the question cannot be adequately answered even after referring to the specified disclosure information, if the appropriate answer to the question is not found in the specified disclosure information, or if the question cannot be adequately answered even after referring to the specified disclosure information. Furthermore, if the answer format of the question is to select a predetermined option from multiple choices, the prompt may include instructions to cause the large-scale language model to output information indicating the location of the specified disclosure information that can explain the reason for the selection, as location information. Furthermore, if the answer format to the question indicated by the tag information included in the question format information, or the answer input format, is not in a format that allows for answering or inputting in natural language, such as a multiple-choice format, the information input unit 130 may input the above-mentioned input instructions to a machine learning model other than an LLM. In this case, the machine learning model is a machine learning model that has been trained to output an appropriate numerical value or YES / NO as an answer when given the above-mentioned input instructions (question information, question format information, and specific disclosure information) as input.

[0031] The machine learning model, including the large-scale language model, may be provided on an external server or on the information processing device 100. Furthermore, as the large-scale language model, for example, an existing large-scale language model provided as a service by an external server may be used. In this case, the information processing device 100 can input prompts to the large-scale language model and obtain responses to the input from the large-scale language model through API (Application Programming Interface) communication. The machine learning model generates information relating to the answer to the question (hereinafter referred to as "answer information") based on the given question information, question format information, and specific disclosure information, and outputs the generated answer information to the information processing device 100.

[0032] Information output unit 140 The information output unit 140 acquires response information from the machine learning model. Once the information output unit 140 acquires response information from the machine learning model, it performs predetermined processing on the acquired response information so that it can be displayed by a predetermined program. For example, the information output unit 140 performs a process to convert the format of the response information obtained from the machine learning model into an appropriate format so that it can be displayed by a predetermined program. The predetermined program is, for example, a web page for response input designated by an ESG rating agency, or a program such as Excel that runs on the information processing device 100 and the user terminal UT. For example, depending on the ESG rating agency, users may need to manually input the answers provided by the response information into a program such as Excel using a user terminal (UT), such as by copying and pasting. In such cases, the information output unit 140 processes the response information so that the format of the response information becomes, for example, a table format that includes instructions on where to input each answer. This makes it easy for the user to understand where to input each answer into the program, thus reducing the user's workload. The information output unit 140 may determine what processing should be performed on the answer information based on the tag information included in the question format information. For example, if the "answer format" included in the tag information is "table format", the information output unit 140 may convert the format of the answer information to a table format, as described above. Alternatively, the information output unit 140 may pre-store a table in a storage unit (not shown) or similar that associates ESG rating organizations with information about the programs or web pages used when responding to those ESG rating organizations, and process the response information by referring to this table. The information output unit 140 outputs the processed response information to the user terminal UT as described above.

[0033] The user terminal UT obtains the processed response information from the information output unit 140. Once the user terminal UT obtains the processed response information from the information output unit 140, it displays the response indicated by the obtained response information on the display of the user terminal UT. For example, the user terminal UT displays the response on the screen used to access the response creation support service, which is displayed on the display.

[0034] Next, we will explain an example of the operation of the information processing device 100 in the information processing system and the flow of each piece of information, referring to Figure 6. In the following explanation, it is assumed that questions have been sent to the user's company from multiple ESG rating agencies in advance, and that a question information database 160 for each company has been constructed by the service provider. Furthermore, in the following explanation, it is assumed that a disclosure information database 150 has been constructed in advance by the service provider and the information processing device 100. First, the user operates the user terminal UT and enters an indication in the question specification window described above that they will answer any question. Once the user enters an indication in the question specification window that they will answer any question, the user terminal UT generates question specification information and sends the generated question specification information to the information processing device 100 (indicated by (1) in Figure 6). Next, in the information processing device 100, the reception unit 110 receives the question specification information transmitted from the user terminal UT (indicated by (2) in Figure 6). Upon receiving the question specification information from the user terminal UT, the reception unit 110 outputs the received question specification information to the information acquisition unit 120. Next, the information acquisition unit 120 acquires question specification information from the reception unit 110. Based on the company identification information contained in the question specification information acquired from the reception unit 110, the information acquisition unit 120 identifies the company to which the user who sent the question specification information belongs, and acquires question information related to the question specified by the user from the question information database 160 managed by the identified company. The information acquisition unit 120 also acquires question format information from the question information database 160 (indicated by (3) in Figure 6). Next, the information acquisition unit 120 searches the disclosure information of multiple companies stored in the disclosure information database 150 for the disclosure information necessary to answer the question (specific disclosure information), and acquires the specific disclosure information as a search result (indicated by the symbol (4) in Figure 6). The information acquisition unit 120 outputs the acquired question information, question format information, and specific disclosure information to the information input unit 130. Next, the information input unit 130 acquires question information, question format information, and specific disclosure information from the information acquisition unit 120. Once the information input unit 130 acquires the question information, question format information, and specific disclosure information from the information acquisition unit 120, it inputs this acquired information as input instructions to the machine learning model 300 (indicated by (5) in Figure 6). If the machine learning model 300 is a large-scale language model, a prompt containing question information and question format information along with specific disclosure information, or a prompt containing question information, question format information, and specific disclosure information, is input to the large-scale language model. Next, the machine learning model 300 generates information related to the answer to the question (answer information) based on the question information, question format information, and specific disclosure information given as input instructions, and outputs the generated answer information to the information processing device 100 (indicated by (6) in Figure 6). Next, in the information processing device 100, the information output unit 140 acquires response information from the machine learning model 300. When the information output unit 140 acquires response information from the machine learning model 300, it performs a predetermined processing on the acquired response information so that it can be displayed by a predetermined program, and outputs the processed response information to the user terminal UT (indicated by (7) in Figure 6). As explained by the symbols (1) to (7) above, the information processing device 100 operates and each piece of information is transmitted and received. Here, the RAG mechanism is realized by symbols (3) to (7).

[0035] In the example above, the information processing device 100 is shown to perform one instance of sending and receiving information with the machine learning model 300. However, if the information processing device 100 uses a large-scale language model as the machine learning model 300, it may perform multiple instances of sending and receiving information with the large-scale language model (multi-turn). For example, in the first turn, the information input unit 130 of the information processing device 100 generates a prompt as described above and inputs the generated prompt to the large-scale language model. Then, the information output unit 140 obtains response information from the large-scale language model. In the second turn, the information input unit 130 generates a prompt for the large-scale language model that asks about the probability or success of the answer indicated by the answer information generated in the first turn, and inputs the generated prompt into the large-scale language model. The information output unit 140 then acquires the information regarding the probability or success of the answer output from the large-scale language model and outputs the acquired information to the user terminal UT. In this case, the user not only finds it easier to create answers to questions, but can also easily understand the probability or success of the answer, and can modify the answer as appropriate according to the probability or success of the answer.

[0036] Furthermore, when the information output unit 140 outputs the processed response information to the user terminal UT (indicated by (7) in Figure 6), it may output multiple response options to the user terminal UT. For example, after the information processing device 100 outputs the processed response information to the user terminal UT, it collects feedback from the user regarding the response information and stores the collected feedback information as a log in a storage unit (not shown). User feedback may include, for example, corrected response information or other response information that could be considered in relation to the response information. Then, the information output unit 140, indicated by reference numeral (7) in Figure 6, refers to the log stored in the storage unit when outputting the processed answer information to the user terminal UT. Based on the log, it generates multiple answer information items and outputs these generated answer information items to the user terminal UT as answer candidates. In this case, the information processing device 100 can present multiple answer candidates to the user at once and reduce the number of information transmissions and receptions with the user terminal UT, leading to a reduction in communication volume. Furthermore, the information processing device 100 may also include a function for user-defined response corrections. For example, as shown by reference numeral (7) in Figure 6, the information processing device 100 may accept corrections to the response information from the user via the user terminal UT before the information output unit 140 outputs the processed response information to the user terminal UT. The information output unit 140 may then output the user-defined response information to the user terminal UT. In this case, the user can correct the response information on the information processing device 100 before acquiring the response information, which leads to improved efficiency in the response creation process. Furthermore, if the information processing device 100 receives a request from the user to modify the response information, it may use the modified response information as retraining data for the machine learning model 300. In this case, the machine learning model 300 will be able to output more accurate response information that better reflects the user's intent, and the information processing device 100 can obtain more accurate response information that better reflects the user's intent from the machine learning model 300. Furthermore, when the information output unit 140 outputs the processed response information to the user terminal UT (indicated by the numeral (7) in Figure 6), it may include location information indicating where in the specific disclosure information the information that forms the basis of the response is located. For example, if the format of the question is such that the user selects a predetermined option from multiple choices, the information output unit 140 may use location information indicating the location of the specific disclosure information that can explain the reason for the selection, and include this location information in the response information before outputting it to the user terminal UT. In this case, the user can easily understand the reason for their selection in the response.

[0037] Experimental Results The inventors of this disclosed technology conducted an experiment to determine the extent to which the information processing device 100 could generate response information using specific questions from an ESG rating agency. The experiment assumed a scenario where a company responds to DJSI's 2023 questions, with DJSI being the ESG rating agency. The experiment also used the company's 2023 Sustainability Data Book, which compiles data and information on sustainable activities, as the company's specific disclosure information. Responses were generated by providing a large-scale language model with prompts containing question information and question format information, along with the specific disclosure information. Furthermore, when using the specific disclosure information of the company in question, the inventor of this disclosure technology first converted the aforementioned report into text data using optical character recognition (OCR) with the information processing device 100, then divided the converted text data into sections of up to 400 characters each, and converted the text data into vectors for each divided section. Here, a maximum of 100 characters of overlap was allowed before and after each divided section. The inventor of this disclosure technology then used the information processing device 100 to link the vectors obtained by the conversion with the company's specific information and stored them in the disclosure information database 150. In the experiment, the information acquisition unit 120 generated the question text and answer support text indicated by the question information as search queries, and used the generated search queries to search the disclosure information database 150 and obtain specific disclosure information as search results. Here, the information acquisition unit 120 obtained the top 10 specific disclosure information items with a high percentage of matching the search query as search results, and concatenated the obtained specific disclosure information. Then, the information input unit 130 input the specific disclosure information concatenated by the information acquisition unit 120 into a large-scale language model.

[0038] Figure 7 shows an example of the actual questions used in the experiment and the answer choices provided. In Figure 7, reference numeral 701 indicates the main text of the question, and reference numeral 702 indicates the supplementary answer text. Reference numeral 703 indicates the options presented as answers. The information acquisition unit 120 generates the main text of the question 701 and the supplementary answer text 702 as a search query, searches the disclosure information database 150 using the generated search query, and obtains specific disclosure information as a search result. The specific disclosure information here is the disclosure information necessary to answer the question, which includes the main text of the question 701 and the supplementary answer text 702. Next, Figure 8 shows an example of a prompt generated by the information input unit 130. In Figure 8, reference numeral 801 denotes the main text of the question embedded in the prompt (reference numeral 701 in Figure 7) and the supplementary answer text (reference numeral 702 in Figure 7). Reference numeral 802 denotes the options embedded in the prompt (reference numeral 703 in Figure 7). Here, the three options are numbered from 1 to 3. The information input unit 130 input the generated prompt and the specific disclosure information into the large-scale language model. In response to these inputs, the large-scale language model outputted response information. An example of the response indicated by the response information output from the large-scale language model is shown in Figure 9. In Figure 9, as shown by reference numeral 901, "1" is selected as the answer choice. Reference numeral 902 indicates location information showing the location of specific disclosure information that can explain the reason for selecting "1". The information processing device 100 then presented the answer indicated by the answer information output from the large-scale language model to the inventor of the disclosed technology. Thus, the information processing device 100 selected option "1" as the answer. Meanwhile, when the inventor of the disclosed technology confirmed with the company, the company's actual answer was also "1". In this way, the information processing device 100 was able to select the same answer as the company's actual answer by performing the processing described above. Furthermore, although a detailed explanation is omitted here, the inventor of the disclosed technology conducted a similar experiment with a question in a multiple-choice format, separate from the one described above. As a result, the information processing device 100 missed selecting only one of the options that the company actually chose as its answer, but selected the other options in the same way as the company's actual answers. This example also shows that the information processing device 100 was able to present the user with answers that were very close to the company's actual answers.

[0039] The information processing device 100 shown in Embodiment 1 only needs to include at least an information acquisition unit 120, an information input unit 130, and an information output unit 140, and the receiving unit 110 may be omitted. If the receiving unit 110 is omitted in the information processing device 100, the functions of the receiving unit 110 may be realized, for example, by the information acquisition unit 120.

[0040] Variant form Next, a modified example of the information processing device 100 according to Embodiment 1 will be described. In the modified configuration, the main operation examples of the information processing device 100 and the flow of each piece of information are the same as those described in Figure 6, which was referenced in Embodiment 1. Therefore, the following description will explain the modified configuration of the information processing device 100 with reference to Figure 6.

[0041] In Embodiment 1, an example was described in which the reception unit 110 receives information that allows the user to identify a question (question specification information) that the user has specified as a question to be answered from the user terminal UT. Furthermore, in the above example, for example, when the user answers question "Q1" sent from the ESG rating agency "DJSI", the user terminal UT operates and enters "Answering Q1 from DJSI" into the question specification window, and when the user terminal UT receives the above information from the user terminal UT, it generates question specification information that allows the user to identify that the ESG rating agency is "DJSI" and the question number is "Q1", and transmits the generated question specification information to the information processing device 100. In a modified version, the reception unit 110 may receive from the user terminal UT as question specification information, which may include information that identifies the name of an ESG rating agency, or information that identifies the content of a specific question that includes the name of an ESG rating agency.

[0042] In this case, the user operates the user terminal UT and enters, for example, "DJSI" as the name of the ESG rating agency into the question specification window. When the user enters "DJSI" into the question specification window, the user terminal UT generates question specification information that identifies the name of the ESG rating agency as "DJSI" and transmits the generated question specification information to the information processing device 100 (code (1) in Figure 6). Alternatively, the user operates the user terminal UT to directly input the specific content of the question to be answered into the question specification window, or to save a predetermined file containing the specific content of the question into the user terminal UT and then input the location where the file is saved into the question specification window. When the user inputs the specific content of the question into the question specification window, or inputs the location where the file containing the specific content of the question is saved into the question specification window, the user terminal UT generates question specification information that identifies the specific content of the question based on the input, and transmits the generated question specification information to the information processing device 100.

[0043] Alternatively, the user may specify the name of an ESG rating agency or the specific content of the question by operating the user terminal UT, for example by selecting from a pull-down menu displayed in the question specification window. When the user selects the name of an ESG rating agency or the specific content of the question from the pull-down menu, the user terminal UT generates question specification information that identifies the name of the ESG rating agency or the specific content of the question based on the selected information, and transmits the generated question specification information to the information processing device 100. A pull-down menu is a system that displays a list of options when a user presses a designated location. For example, it could display multiple options for the names of ESG rating agencies or specific questions. Such a pull-down menu may be provided by, for example, the information processing device 100 in the following manner. For example, if the information processing device 100 has a question information database 160, the information processing device 100 extracts a list of the names of ESG rating agencies or a list of the specific content of the questions based on the question information held in the question information database 160. Then, based on the extracted list of names of ESG rating agencies or the list of the specific content of the questions, the information processing device 100 generates a pull-down menu that includes a list menu from which any one of the options composed of these lists can be selected, and provides the pull-down menu to the user by displaying the generated pull-down menu in the question selection window. Furthermore, even if the question information database 160 is located on an external device that can be communicated to from the information processing device 100, the information processing device 100 can provide the pull-down menu to the user by generating a pull-down menu based on the question information held in the question information database 160, and displaying the generated pull-down menu in the question selection window, in the same manner as described above.

[0044] Next, in the information processing device 100, the reception unit 110 receives the question specification information transmitted from the user terminal UT (indicated by (2) in Figure 6). Upon receiving the question specification information from the user terminal UT, the reception unit 110 outputs the received question specification information to the information acquisition unit 120. Next, the information acquisition unit 120 acquires question specification information from the reception unit 110. If the name of an ESG rating agency can be identified from the question specification information, the information acquisition unit 120 identifies the company to which the user who submitted the question specification information belongs, based on the company identification information contained in the question specification information acquired from the reception unit 110. Then, the information acquisition unit 120 acquires question information related to the question from the ESG rating agency whose name is identified by the question specification information from the question information database 160 managed by the identified company. On the other hand, if the specific content of the question, including the name of the ESG rating agency, can be identified from the question specification information, the information acquisition unit 120 identifies the company to which the user who submitted the question specification information belongs, based on the company identification information contained in the question specification information acquired from the reception unit 110, and acquires question information relating to the specific content of the question identified by the question specification information from the question information database 160 managed by the identified company. Furthermore, if the information acquisition unit 120 determines that the content of the specific question identified by the question designation information is extremely detailed, and that the content of the specific question identified by the question designation information can substitute for the question information stored in the question information database 160, it may omit acquiring the question information from the question information database 160.

[0045] Next, the information acquisition unit 120 acquires question format information from the question information database 160 (indicated by (3) in Figure 6). If the name of an ESG rating agency can be identified from the question specification information, the information acquisition unit 120 may acquire the question format information from the question information database 160 on a per-ESG rating agency basis. Alternatively, if the content of a specific question can be identified from the question specification information, the information acquisition unit 120 may acquire the question format information from the question information database 160 on a per-question basis. Next, the information acquisition unit 120 searches the disclosure information of multiple companies stored in the disclosure information database 150 for the disclosure information necessary to answer the question (specific disclosure information), and acquires the specific disclosure information as a search result (indicated by the symbol (4) in Figure 6). The information acquisition unit 120 outputs the acquired question information, question format information, and specific disclosure information to the information input unit 130. Next, the information input unit 130 acquires question information, question format information, and specific disclosure information from the information acquisition unit 120. Once the information input unit 130 acquires the question information, question format information, and specific disclosure information from the information acquisition unit 120, it inputs this acquired information as input instructions to the machine learning model 300 (indicated by (5) in Figure 6). If the machine learning model 300 is a large-scale language model, a prompt containing question information and question format information along with specific disclosure information, or a prompt containing question information, question format information, and specific disclosure information, is input to the large-scale language model.

[0046] In some cases, as indicated by the symbol (4) in Figure 6, the information acquisition unit 120 may be unable to acquire the specific disclosure information necessary to answer the question from the disclosure information database 150, or it may acquire the information but the amount of information is insufficient. In such cases, if the machine learning model 300 is a large-scale language model, the information input unit 130 may include a prompt instructing the large-scale language model to output a message to the corporate user such as, "The answer may be insufficient because there is insufficient disclosure information necessary to answer the question. Please input additional information to supplement the answer as needed," and input this prompt to the large-scale language model.

[0047] Next, the machine learning model 300 generates information regarding the answer to the question (answer information) based on the question information, question format information, and specific disclosure information given as input instructions, and outputs the generated answer information to the information processing device 100 (indicated by (6) in Figure 6). The machine learning model 300 may also include information indicating the relevant section of the specific disclosure information that forms the basis of the answer. The relevant section of the specific disclosure information could be, for example, the part of the specific disclosure information that corresponds to the answer, or the page number of that section. Furthermore, if the machine learning model 300 is a large-scale language model, and the above-mentioned prompt is input to the large-scale language model, the large-scale language model may include information in the response information that indicates a message such as, "The answer may be insufficient because there is insufficient disclosure information necessary to answer the question. Please provide input to supplement the answer as needed."

[0048] Next, in the information processing device 100, the information output unit 140 acquires response information from the machine learning model 300. When the information output unit 140 acquires response information from the machine learning model 300, it performs a predetermined processing on the acquired response information so that it can be displayed by a predetermined program, and outputs the processed response information to the user terminal UT (indicated by (7) in Figure 6). Furthermore, if the answer format of the answer information obtained from the machine learning model 300 is not the answer format specified in the question format information, the information output unit 140 may modify the answer format so that the answer format of the answer information obtained from the machine learning model 300 matches the answer format specified in the question format information, and then output the answer information to the user terminal UT. Furthermore, when the information output unit 140 outputs the processed response information or the response information after the response format has been modified to the user terminal UT, it may output the response information in a manner that allows the user to input (edit) the response from the user terminal UT. This is because it is not possible to know whether the response indicated by the response information is necessarily the response the user desires, and therefore it leaves room for the user to input (edit) the response.

[0049] In this case, if the user inputs any information regarding the answer using the user terminal UT, the user terminal UT should transmit information indicating the content of the input regarding the answer (hereinafter referred to as "additional input information") to the information processing device 100. In the information processing device 100, the reception unit 110 receives additional input information transmitted from the user terminal UT and stores the received additional input information in, for example, an additional input information database (not shown). The additional input information database may be provided by the information processing device 100 or by an external device that can communicate with the information processing device 100. The additional input information stored in the additional input information database is used by the information output unit 140 when outputting answer information for the same or similar questions related to the said additional input information in subsequent instances. For example, when the information output unit 140 outputs the processed response information or the response information after the response format has been modified to the user terminal UT, it refers to the additional input information database. If additional input information related to the response information to be output is stored in the additional input information database, it reflects the content of the additional input information in the response information to be output before outputting the response information to the user terminal UT. For example, if additional input information related to the response information to be output is stored in the additional input information database, the information output unit 140 may modify the response information to be output based on the additional input information stored in the additional input information database and output the modified response information, or it may output the additional input information stored in the additional input information database together with the response information to be output. This allows the information output unit 140 to present the user with an answer that is closer to the answer the user desires.

[0050] The above processing also applies when the response information includes a message such as, "The response may be insufficient because the necessary disclosure information to answer the question is missing. Please provide additional input to supplement the response as needed." In this case, the user is expected to see the above message on the user terminal UT and, if necessary, use the user terminal UT to provide some input to the response. Even in that case, the user terminal UT will send information indicating the content entered for the response (additional input information) to the information processing device 100, and the information processing device 100's reception unit 110 will receive the additional input information sent from the user terminal UT and save the received additional input information to an additional input information database (not shown). Subsequently, the information output unit 140 will use the additional input information when outputting response information in the same manner as above, thereby providing the user with a response that is closer to the response the user desires.

[0051] Furthermore, the functions of each part of the information processing device 100 described in the modified example of Embodiment 1 are not limited to this modified example, but can be selectively combined with the functions of each part of the information processing device 100 described in Embodiment 1.

[0052] Embodiment 2. Embodiment 2 shows the implementation of the disclosed technology as an information processing program. Unless otherwise specified, the same reference numerals used in Embodiment 1 are used in Embodiment 2. In Embodiment 2, explanations that overlap with those in Embodiment 1 are omitted as appropriate. The information processing program of Embodiment 2 causes a computer to execute the following steps: a reception procedure that receives question designation information from a user terminal that can identify questions used by an ESG rating agency to evaluate a predetermined organization and which have been designated by the user as questions to be answered; an information acquisition procedure that, based on the question designation information received in the reception procedure, obtains from a database disclosure information to be disclosed by the predetermined organization, information about the questions identified by the question designation information, and information about the format of the answers to the questions; an information input procedure that inputs the disclosure information, information about the questions, and information about the format of the answers to the questions obtained in the information acquisition procedure into a machine learning model; and an information output procedure that obtains information about the answers to the questions output from the machine learning model and outputs the obtained information about the answers to the questions to the user terminal.

[0053] Figure 10 shows the hardware configuration 200 of the information processing device 100 according to the disclosed technology. As shown in Figure 10, the hardware configuration 200 of the information processing device 100 includes a communication interface 210, an input / output interface 220, a processor 230, and a memory 240.

[0054] Figure 11 is a flowchart showing the information processing method implemented by the information processing program relating to the disclosed technology as a processing procedure. As shown in Figure 11, the information processing method implemented by the information processing program includes a reception procedure ST110, an information acquisition procedure ST120, an information input procedure ST130, and an information output procedure ST140. The number assigned to each processing procedure corresponds to the code of the entity that executes that procedure. For example, the reception procedure ST110 is executed by the reception unit 110, the information acquisition procedure ST120 is executed by the information acquisition unit 120, the information input procedure ST130 is executed by the information input unit 130, and the information output procedure ST140 is executed by the information output unit 140. As shown in Figure 11, the reception procedure ST110, the information acquisition procedure ST120, the information input procedure ST130, and the information output procedure ST140 are included within a loop process.

[0055] The information processing method implemented by the information processing program according to Embodiment 2 is an information processing method executed by a computer. In this information processing method, when the loop is started, first, the reception unit 110 executes a reception procedure (ST110) to receive question specification information sent from the user terminal UT. Then, the information acquisition unit 120 executes an information acquisition procedure (ST120) to acquire from the database the disclosure information disclosed by a designated organization, the information about the questions used by ESG rating agencies when evaluating the organization, and the information about the answer format to the questions, based on the question specification information received in the reception procedure. Next, the information input unit 130 executes an information input procedure (ST130) to input the disclosure information acquired in the information acquisition procedure, the information about the questions, and the information about the answer format to the questions into a machine learning model. Furthermore, the information output unit 140 executes an information output procedure (ST140) to acquire the information about the answers to the questions output from the machine learning model and output the information about the acquired answers. Note that Figure 11 shows an example of loop processing, but this disclosed technology is not limited to this.

[0056] The functions of the receiving unit 110, information acquisition unit 120, information input unit 130, and information output unit 140 of the information processing device 100 according to the disclosed technology are realized by processing circuits. That is, the information processing device 100 includes processing circuits for executing the receiving procedure ST110, the information acquisition procedure ST120, the information input procedure ST130, and the information output procedure ST140. The processing circuit is a processor 230 (also called a CPU, central processing unit, processing unit, arithmetic unit, microprocessor, microcomputer, or DSP) that executes an information processing program stored in the memory 240.

[0057] The functions of the reception unit 110, information acquisition unit 120, information input unit 130, and information output unit 140 are realized by software, firmware, or a combination of software and firmware. The software and firmware are written as information processing programs and stored in memory 240. The processing circuit realizes the functions of each unit by reading and executing the information processing programs stored in memory 240. In other words, the information processing device 100 includes memory 240 for storing information processing programs that, when executed by the processing circuit, result in the execution of reception procedure ST110, information acquisition procedure ST120, information input procedure ST130, and information output procedure ST140. It can also be said that these information processing programs cause a computer to execute the procedures or methods of reception unit 110, information acquisition unit 120, information input unit 130, and information output unit 140. Here, memory 240 may be, for example, a non-volatile or volatile semiconductor memory such as RAM, ROM, flash memory, or EPROM. Furthermore, the memory 240 may be configured to include a disk such as a magnetic disk, flexible disk, optical disk, compact disk, minidisc, or DVD. In addition, the memory 240 may be configured as an HDD or SSD.

[0058] The information processing program shown in Embodiment 2 only needs to cause the computer to execute at least the information acquisition procedure ST120, the information input procedure ST130, and the information output procedure ST140, and the reception procedure ST110 is an optional procedure. If the reception procedure ST110 is omitted, the processing in the reception procedure ST110 may be performed, for example, in the information acquisition procedure ST120.

[0059] As described above, the information processing program according to Embodiment 2 causes a computer to execute the following steps: a reception procedure that receives question designation information from a user terminal UT that can identify questions used by an ESG rating agency to evaluate a predetermined organization and which have been designated by the user as questions to be answered; an information acquisition procedure ST120 that, based on the question designation information received in the reception procedure, obtains from a database disclosure information to be disclosed by the predetermined organization, information regarding the questions identified by the question designation information, and information regarding the format of the answers to the questions; an information input procedure ST130 that inputs the disclosure information, information regarding the questions, and information regarding the format of the answers to the questions obtained in the information acquisition procedure into a machine learning model; and an information output procedure ST140 that obtains information regarding the answers to the questions output from the machine learning model and outputs the obtained information regarding the answers to the questions to the user terminal UT. Therefore, the information processing program shown in Embodiment 2 can cause a computer to execute predetermined procedures to assist in creating answers to questions used by ESG rating agencies for evaluating specific organizations.

[0060] Furthermore, in the information processing program according to Embodiment 2, the information regarding the answer format of a question includes information regarding an identifier for identifying the question, and the information regarding the identifier for identifying the question may be accompanied by tag information for each question relating to one or more answer formats, dependencies between questions, and the input format of the answer. As a result, the information processing program shown in Embodiment 2 can cause a computer to execute predetermined procedures and present the user with answers for each question that appropriately consider the answer format, dependencies between questions, and the input format of the answers.

[0061] Furthermore, in the information processing program according to Embodiment 2, the question indicated by the information regarding the question may include the body of the question and a supporting statement for providing an answer to the question. As a result, the information processing program shown in Embodiment 2 can cause a computer to execute predetermined procedures to generate information related to the answer to a question with higher accuracy, and can also present the user with an answer to the question with higher accuracy.

[0062] Furthermore, in the information processing program according to Embodiment 2, the information input procedure ST130 may input the disclosed information, information related to the question, and information related to the answer format to the question as input instructions to the machine learning model. As a result, the information processing program shown in Embodiment 2 can cause a computer to execute predetermined procedures to generate information regarding the answer to a question in a machine learning model, taking into account the format of the answer to the question, and can also present the user with a highly accurate answer that takes into account the format of the answer to the question.

[0063] Furthermore, in the information processing program according to Embodiment 2, the input instruction may include multiple pieces of information regarding a question and multiple pieces of information regarding the format of the answer to the question, and may also include information regarding the dependencies between each question indicated by the multiple pieces of information regarding the question and other questions. As a result, the information processing program shown in Embodiment 2 can cause a computer to execute predetermined procedures to generate information about answers to multiple questions, for each question, that appropriately considers the answer format and dependencies between questions, using a machine learning model, and can also present the user with answers that appropriately consider the answer format and dependencies between questions for each question.

[0064] Furthermore, in the information processing program according to Embodiment 2, the input instruction may include information regarding the input format of the answer to the question. As a result, the information processing program shown in Embodiment 2 can generate information regarding the answer to a question using a machine learning model, taking into account the input format of the answer, and can also present the user with a highly accurate answer that takes into account the input format of the answer.

[0065] Furthermore, in the information processing program according to Embodiment 2, there are multiple machine learning models, and in the information input procedure ST130, the disclosed information, the information related to the question, and the information related to the answer format of the question may be input to one of the multiple machine learning models according to predetermined conditions. As a result, the information processing program shown in Embodiment 2 can have a computer execute predetermined procedures to select and use an appropriate machine learning model according to predetermined conditions. Furthermore, the information processing program shown in Embodiment 2 can provide the user with highly accurate answers according to predetermined conditions.

[0066] Furthermore, in the information processing program according to Embodiment 2, in the information input procedure ST130, if the answer format indicated by the information regarding the answer format of the question includes an answer in natural language, a prompt is generated that instructs the user to answer the question and includes information regarding the question and information regarding the answer format of the question. The generated prompt and the disclosed information are then input into the large-scale language model. If the answer format indicated by the information regarding the answer format of the question does not include an answer in natural language, the disclosed information, the information regarding the question, and the information regarding the answer format of the question may be input into a machine learning model other than the large-scale language model. As a result, the information processing program shown in Embodiment 2 can cause a computer to execute predetermined procedures and select and use an appropriate machine learning model depending on whether or not the response format includes a natural language response. Furthermore, the information processing program shown in Embodiment 2 can provide the user with highly accurate answers depending on whether or not the response format includes a natural language response.

[0067] Furthermore, in the information processing program according to Embodiment 2, the prompt may include an instruction to cause the large-scale language model to output a statement indicating that an appropriate answer to a question is not present in the disclosed information, or a statement prompting the user of the user terminal to provide supplementary information to the answer indicated by the information showing the answer. As a result, the information processing program shown in Embodiment 2 can cause a computer to execute a predetermined procedure and notify the user that if an appropriate answer to a question is not present in the disclosed information, it indicates that an appropriate answer is not present in the disclosed information.

[0068] Furthermore, in the information processing program according to Embodiment 2, in the information acquisition procedure ST120, a search query is generated to search for disclosure information related to the question from the disclosure information stored in the database based on the question indicated by the information related to the question, and the database is searched using the generated search query to obtain disclosure information corresponding to the search query. In the information input procedure ST130, the obtained disclosure information, the information related to the question, and the information related to the answer format of the question may be input to the machine learning model as input instructions. As a result, the information processing program shown in Embodiment 2 can cause a computer to execute predetermined procedures to appropriately acquire disclosure information related to a question as specific disclosure information, and can input the acquired specific disclosure information as input instructions into a machine learning model. Furthermore, the information processing program shown in Embodiment 2 can provide the user with an appropriate response based on the specific disclosed information.

[0069] Furthermore, in the information processing program according to Embodiment 2, the information acquisition procedure ST120 allows for the arbitrary setting of the upper limit of the amount of disclosed information to be acquired from the database using a search query. As a result, the information processing program shown in Embodiment 2 can cause the computer to execute predetermined procedures and suppress an excessive increase in the amount of specific disclosure information to be acquired.

[0070] Furthermore, in the information processing program according to Embodiment 2, the information output procedure ST140 may output the information related to the response obtained from the machine learning model after processing it so that it can be displayed in a predetermined program. As a result, the information processing program shown in Embodiment 2 can cause a computer to execute a predetermined procedure and display information related to the answer within the predetermined program.

[0071] Furthermore, in the information processing program according to Embodiment 2, the information output procedure ST140 may process the information regarding the answer obtained from the machine learning model so that the format of the information regarding the answer is in a format that includes instructions regarding the input location of the answer. As a result, the information processing program shown in Embodiment 2 reduces the workload on the user when, for example, the user needs to manually input the answer indicated by the answer information into a predetermined program using a user terminal UT, by copying and pasting, by having the computer execute predetermined procedures.

[0072] Furthermore, in the information processing program according to Embodiment 2, the information output in the information output procedure ST140 may include location information indicating where within the disclosed information the information that forms the basis for the answer to the question is located. As a result, the information processing program shown in Embodiment 2 can cause a computer to execute predetermined procedures and show the user where in the disclosed information the information that forms the basis for the answer to a question is located. Furthermore, the user can easily understand where in the disclosed information the information that forms the basis for the answer to a question is located.

[0073] Furthermore, in the information processing program according to Embodiment 2, if the format of the question response is to select a predetermined option from multiple options, the location information is information indicating the location of disclosed information that can explain the reason for the selection. As a result, the information processing program shown in Embodiment 2 can cause a computer to execute predetermined procedures and present to the user where within the disclosed information the information that forms the basis for selecting an option in response to a question is located. Furthermore, the user can easily grasp where within the disclosed information the information that forms the basis for selecting an option in response to a question is located.

[0074] Furthermore, in the information processing program according to Embodiment 2, the reception procedure ST110 receives question specification information that identifies the name of an ESG rating agency, and the information acquisition procedure ST120 acquires from the database, based on the question specification information received in the reception procedure ST110, the disclosure information to be disclosed by a specified organization, information regarding the question from the ESG rating agency with the above name, and information regarding the format of the answer to the question. As a result, the information processing program shown in Embodiment 2 can cause a computer to execute predetermined procedures, enabling the user to specify questions based on the name of an ESG rating agency.

[0075] Furthermore, in the information processing program according to Embodiment 2, when the information output procedure ST140 outputs information related to the answer to the user terminal UT, it outputs the information in a manner that allows the user terminal UT to accept input regarding the answer indicated by the information related to the answer. As a result, the information processing program shown in Embodiment 2 can cause a computer to execute a predetermined procedure and output information about the answer to the user in a way that leaves room for input regarding the answer.

[0076] The information processing device 100 according to Embodiment 1 includes: a reception unit 110 that receives question designation information from a user terminal UT that can identify questions used by an ESG rating agency to evaluate a predetermined organization and which have been designated by a user as questions to be answered; an information acquisition unit 120 that, based on the question designation information received by the reception unit 110, acquires disclosure information of a predetermined organization, information regarding the questions identified by the question designation information, and information regarding the format of the answers to the questions from a database; an information input unit 130 that inputs the disclosure information, information regarding the questions, and information regarding the format of the answers to the questions acquired by the information acquisition unit 120 into a machine learning model; and an information output unit 140 that acquires information regarding the answers to the questions output from the machine learning model and outputs the acquired information regarding the answers to the questions to the user terminal UT. Therefore, the information processing device 100 shown in Embodiment 1 can assist in creating answers to questions used by ESG rating agencies for evaluating a given organization.

[0077] The information processing method according to Embodiment 1 is an information processing method using an information processing device, wherein the receiving unit 110 receives question designation information from a user terminal UT that can identify questions used by an ESG rating agency to evaluate a predetermined organization and which have been designated by a user as questions to be answered; the information acquisition unit 120 acquires disclosure information of a predetermined organization, information about the questions identified by the question designation information, and information about the answer format of the questions from a database based on the question designation information received by the receiving unit 110; the information input unit 130 inputs the disclosure information, information about the questions, and information about the answer format of the questions acquired by the information acquisition unit 120 into a machine learning model; and the information output unit 140 acquires information about the answers to the questions output from the machine learning model and outputs the acquired information about the answers to the questions to the user terminal UT. By executing the above method, the information processing device 100 can support the creation of answers to questions used by an ESG rating agency to evaluate a predetermined organization. [Explanation of Symbols]

[0078] 100 Information processing device, 110 Reception unit, 120 Information acquisition unit, 130 Information input unit, 140 Information output unit, 150 Disclosure information database, 160 Question information database, 200 Hardware configuration, 201 Question text, 202 Answer support text, 210 Communication interface, 220 Input / output interface, 230 Processor, 240 Memory, 300 Machine learning model, 301 Question text, 302 Answer support text, 303 Choices, 401 Question text, 402 Answer support text, 403 Table of choices, 501 Question identification information, 502 Tag information, 701 Question text, 702 Answer support text, 801 Question text and answer support text embedded in prompt, 802 Choices embedded in prompt, 901 Answer choices, 902 Basis for selection, UT User terminal.

Claims

1. A reception procedure for receiving question specification information from a user terminal that allows the user to identify questions that an ESG rating agency uses to evaluate a designated organization and that the user has specified as questions to be answered, and An information acquisition procedure for obtaining from a database the following information based on the question designation information received in the aforementioned acceptance procedure: the information to be disclosed by the designated organization, information regarding the question identified by the question designation information, and information regarding the format of the answer to the question; An information input procedure for inputting the disclosed information obtained in the aforementioned information acquisition procedure, information regarding the question, and information regarding the answer format of the question into a machine learning model, Information output procedure for obtaining information regarding the answer to a question output from the machine learning model and outputting the obtained information regarding the answer to the user terminal, An information processing program that causes a computer to execute something.

2. The information regarding the format of the answers to the aforementioned questions includes information regarding an identifier for identifying the questions, The identifier information used to identify a question includes, for each question, tag information relating to one or more answer formats, dependencies between questions, and the format of the answer input. The information processing program according to claim 1.

3. The questions indicated by the information regarding the aforementioned questions are: The question includes the main text of the question and supporting statements to assist in answering that question. The information processing program according to claim 1 or claim 2.

4. In the aforementioned information input procedure, the disclosed information, the information related to the question, and the information related to the answer format to the question are input to the machine learning model as input instructions. The information processing program according to claim 1 or claim 2.

5. The aforementioned input instruction is, This includes multiple pieces of information regarding the aforementioned question and multiple pieces of information regarding the format of the answer to that question, and also includes information regarding the dependencies between each question indicated by the information regarding the multiple questions and other questions. The information processing program according to claim 4.

6. The aforementioned input instructions include information regarding the format of the input for the answer to the question. The information processing program according to claim 4.

7. There are multiple machine learning models, In the aforementioned information input procedure, The disclosed information, the information relating to the question, and the information relating to the answer format to the question are input into one of several machine learning models according to predetermined conditions. The information processing program according to claim 1 or claim 2.

8. In the aforementioned information input procedure, If the answer format indicated by the information regarding the answer format of the aforementioned question includes an answer in natural language, a prompt is generated that instructs the user to answer the question, and the prompt includes the information regarding the question and the information regarding the answer format of the question, and the generated prompt and the disclosed information are input into a large-scale language model. If the answer format indicated by the information regarding the answer format of the aforementioned question does not include an answer in natural language, the disclosed information, the information regarding the question, and the information regarding the answer format of the question are input into a machine learning model other than a large-scale language model. The information processing program according to claim 7.

9. The aforementioned prompt is, If the appropriate answer to the question is not present in the disclosed information, the instruction includes causing the large-scale language model to output a statement indicating that the appropriate answer is not present in the disclosed information, or a statement prompting the user of the user terminal to provide supplementary information to the answer indicated by the information indicating the answer. The information processing program according to claim 8.

10. In the aforementioned information acquisition procedure, Based on the question indicated by the information relating to the aforementioned question, a search query is generated to search for disclosure information relating to the question from the disclosure information stored in the database, and the database is searched using the generated search query to obtain the disclosure information corresponding to the search query. In the aforementioned information input procedure, the acquired disclosure information, the information related to the question, and the information related to the answer format to the question are input to the machine learning model as input instructions. The information processing program according to claim 1 or claim 2.

11. In the aforementioned information acquisition procedure, The maximum number of disclosed pieces of information to be retrieved from the database using a search query can be arbitrarily set. The information processing program according to claim 10.

12. In the above information output procedure, The information regarding the answers obtained from the aforementioned machine learning model is processed so that it can be displayed in a predetermined program and then output. The information processing program according to claim 1 or claim 2.

13. In the above information output procedure, The information regarding the response obtained from the aforementioned machine learning model is processed so that the format of the information regarding the response includes instructions on where to input the response. The information processing program according to claim 12.

14. The information to be output in the aforementioned information output procedure is: This includes location information indicating where within the disclosed information the information that forms the basis for the answer to the question can be found, The information processing program according to claim 1 or claim 2.

15. If the question format requires selecting a specific option from multiple choices, The aforementioned location information is information indicating the location of the disclosed information that can explain the reason for the aforementioned selection. The information processing program according to claim 14.

16. In the aforementioned reception procedure, the question specification information accepted is question specification information that can identify the name of the ESG evaluation organization. In the aforementioned information acquisition procedure, based on the question designation information received in the aforementioned reception procedure, the following information is obtained from the database: the disclosure information to be disclosed by the designated organization, information regarding the questions from the ESG evaluation organization of the aforementioned name, and information regarding the format of the answers to those questions. The information processing program according to claim 1 or claim 2.

17. In the above information output procedure, When outputting information regarding the aforementioned answer to the user terminal, the output is provided in a manner that allows the user terminal to accept input regarding the answer indicated by the information regarding the answer. The information processing program according to claim 1 or claim 2.

18. A reception unit that receives question designation information from a user terminal that allows the user to identify questions that an ESG rating agency uses to evaluate a designated organization and that the user has designated as questions to be answered, An information acquisition unit obtains from a database the following information from a database based on the question designation information received by the reception unit: the information to be disclosed by the designated organization, information regarding the question identified by the question designation information, and information regarding the format of the answer to the question. An information input unit inputs the disclosed information obtained by the information acquisition unit, information related to the question, and information related to the answer format of the question into a machine learning model. An information output unit that acquires information regarding the answer to a question output from the machine learning model and outputs the acquired information regarding the answer to the user terminal, Information processing device including

19. An information processing method using an information processing device, The reception department receives question designation information from the user terminal that allows the user to identify the questions that ESG rating agencies use to evaluate a designated organization and that the user has designated as questions to be answered. The information acquisition unit acquires from the database, based on the question designation information received by the reception unit, the disclosure information to be disclosed by the designated organization, information regarding the question identified by the question designation information, and information regarding the format of the answer to the question. The information input unit inputs the disclosure information acquired by the information acquisition unit, the information related to the question, and the information related to the answer format of the question into the machine learning model. The information output unit acquires information regarding the answer to the question output from the machine learning model and outputs the acquired information regarding the answer to the user terminal. Information processing methods.

Images