Prompt-based ESG report text analysis method and system

Abstract

The invention discloses an ESG report text analysis method and system based on prompt, and the method comprises the steps: S1, extracting the content of each chapter in an ESG report, and obtaining a plurality of keywords of the content of each chapter based on the content of each chapter; s2, extracting substantive topics in the ESG report, calculating the similarity between each substantive topic and the keyword of each chapter, and screening the substantive topics based on the similarity; and S3, constructing a prompt template, inputting the screened substantive issue into the prompt template to obtain a cue word text, splicing the cue word text and the content of the corresponding chapter, and inputting the spliced cue word text and the content of the corresponding chapter into the large language model to obtain a text analysis result of the corresponding chapter of the ESG report.

Description

Technical Field

[0001] The present invention relates to the field of large language model application technology, and in particular to a prompt-based ESG report text analysis method and system. Background Art

[0002] ESG is the abbreviation of "Environmental, Social, Governance", which is a set of standards to measure the performance of enterprises or organizations in terms of environment, society and corporate governance. ESG standards are used by investors to screen potential investment targets to ensure that these companies take environmental protection, social responsibility and good governance structure into account in their business activities. ESG reports can help improve corporate transparency and help external stakeholders understand how companies manage these non-financial risks and opportunities and their impact on the long-term sustainable development of the company. As the world pays more attention to sustainable development, ESG reports play an increasingly important role in investment decisions and corporate evaluations.

[0003] The unstructured nature and length of ESG reports often increase the workload of experts in reviewing and examining ESG reports. Due to the scattered content of the reports and the lack of a unified format, the extraction of key data and indicators becomes complicated and time-consuming. This format problem not only increases the difficulty of understanding and analyzing the ESG performance of enterprises, but also reduces work efficiency. Therefore, more experts are required to spend more time screening and organizing important information in ESG reports. In addition, the length of the report may also lead to information overload, making key performance indicators less prominent, further increasing the difficulty of obtaining valuable insights from the report. At the same time, the manual process of processing large amounts of text data also increases the risk of errors, which may affect the accuracy and reliability of the report, thereby increasing the operating costs and time costs of the enterprise. However, with the rise of large language models, we can use the text analysis capabilities of large language models to help experts review and examine ESG reports.

[0004] Patent CN115204143B proposes a method for using prompt to optimize cosine value to calculate text similarity, but does not involve how to design a prompt to better summarize text; Patent CN117743340A invents a prompt acquisition method, a task execution method and device based on a large model, but this invention only proposes how to design a prompt template, but this template is not suitable for the text summary of ESG reports, and lacks the screening of ESG substantive issues before prompt construction; Patent CN117744804A invents a reasoning method, terminal and medium for financial analysis tasks based on a large language model, but This invention only involves the knowledge base and program interface calls, and does not mention how to design a prompt to summarize the text of the program's unreadable report; Patent CN117931982A invented a prompt optimization method based on knowledge enhancement. This method mainly extracts keywords from the original prompt and then forms a more authoritative prompt through a certain method. It does not involve how to deal with it before the prompt; Patent CN117973518A invented a contract risk reasoning method based on prompt design and knowledge graph, which mainly involves the field of contract risk. Due to different application fields, its method is not applicable to the field of ESG reports. In summary, this requires us to design an effective prompt to guide the large language model to perform text analysis on the ESG report so that the required text content can be summarized. Summary of the invention

[0005] In order to solve the above technical problems, the present invention provides a prompt-based ESG report text analysis method, the method comprising:

[0006] Step S1, extracting the content of each chapter in the ESG report, and obtaining several keywords of the content of each chapter based on the content of each chapter;

[0007] Step S2: extracting substantive issues in the ESG report, calculating the similarity between each substantive issue and the keywords of each chapter, and screening the substantive issues based on the similarity;

[0008] Step S3: construct a prompt template, input the screened substantive issues into the prompt template to obtain a prompt word text, concatenate the prompt word text with the content of the corresponding chapter, and input it into the large language model to obtain the text analysis result of the corresponding chapter of the ESG report.

[0009] Optionally, in step S1, the process of extracting the content of each chapter in the ESG report specifically includes:

[0010] Enter the page range for each section of the ESG report;

[0011] Dividing the ESG report into chapters according to the page number range;

[0012] The text content of each chapter in the ESG report is extracted and stored in an empty list.

[0013] Optionally, in step S2, based on the content of each chapter, obtaining the contents of several keywords of the content of each chapter specifically includes:

[0014] Use the large language model to extract keywords from the list of text content of each chapter:

[0015] Read the text content of each chapter in the list and get the keywords of each chapter.

[0016] Optionally, in step S2, the process of screening the substantive topics based on the similarity specifically includes:

[0017] Construct a list based on the substantive topics;

[0018] Use word embedding models to extract features of keywords and substantive issues;

[0019] Based on the feature extraction results, the text similarity between each substantive topic and each chapter keyword is calculated, the average value of the text similarity is calculated, and the texts are sorted from high to low based on the average value, and several substantive topics with high similarity are selected.

[0020] Optionally, in S3, obtaining the ESG report text analysis results corresponding to the substantive issues also includes using the reasoning acceleration framework vllm to batch process the substantive issues and spliced ​​content.

[0021] The present invention also discloses a prompt-based ESG report text analysis system, the system comprising: a keyword extraction module, an indicator screening module and an ESG analysis module;

[0022] The keyword extraction module is used to extract the content of each chapter in the ESG report, and based on the content of each chapter, obtain a number of keywords of the content of each chapter;

[0023] The indicator screening module is used in step S2 to extract substantive issues in the ESG report, calculate the similarity between each substantive issue and the keywords of each chapter, and screen the substantive issues based on the similarity;

[0024] The ESG analysis module is used to construct a prompt template, input the screened substantive issues into the prompt template to obtain a prompt word text, splice the prompt word text with the content of the corresponding chapter, and input it into the large language model to obtain the text analysis result of the corresponding chapter of the ESG report.

[0025] Optionally, in the keyword extraction module, the process of extracting the content of each chapter in the ESG report specifically includes:

[0026] Enter the page range for each section of the ESG report;

[0027] Dividing the ESG report into chapters according to the page number range;

[0028] Extract the text content of each chapter in the ESG report and store it in an empty list

[0029] Optionally, in the keyword extraction module, based on the content of each chapter, obtaining the contents of several keywords of the content of each chapter specifically includes:

[0030] Use the large language model to extract keywords from the list of text content of each chapter:

[0031] Read the text content of each chapter in the list and get the keywords of each chapter.

[0032] Optionally, in the indicator screening module, the process of screening the initial substantive issues based on similarity specifically includes:

[0033] Construct a list based on the substantive topics;

[0034] Use word embedding models to extract features of keywords and substantive issues;

[0035] Based on the feature extraction results, the text similarity between each substantive topic and each chapter keyword is calculated, the average value of the text similarity is calculated, and the texts are sorted from high to low based on the average value, and several substantive topics with high similarity are selected.

[0036] Compared with the prior art, the present invention has the following beneficial effects:

[0037] The prompt design method of the present invention can enable the large language model to more accurately locate the key information in the text, avoid some unnecessary reasoning, and summarize the key points in the ESG report more completely and accurately. Compared with throwing all ESG substantive issues to the large language model at once, this method not only reduces the overhead, but also can accurately locate the text content, which improves the large language model's ability to extract and summarize the key points in the ESG report. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the technical solution of the present invention, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0039] Figure 1 A diagram of the steps of a method for performing a comprehensive text summary of an ESG report according to an embodiment of the present invention;

[0040] Figure 2 A step diagram of a method for processing an ESG report document into program-readable documents by dividing the document into chapters according to an embodiment of the present invention;

[0041] Figure 3 A method step diagram of calculating similarity between keywords of each chapter of a cut ESG report and all ESG substantive issues in an embodiment of the present invention;

[0042] Figure 4 A schematic diagram of a process of constructing a prompt input into a large language model according to an embodiment of the present invention. DETAILED DESCRIPTION

[0043] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0044] Embodiment 1

[0045] A prompt-based ESG report text analysis method, such as Figure 1 As shown, the method includes:

[0046] Step S1: extract the content of each chapter in the ESG report, and based on the content of each chapter, obtain several keywords of the content of each chapter.

[0047] like Figure 2-Figure 3 As shown, the initial ESG report is processed; first, an empty list Chapter_list = [] is constructed to store the text content of each chapter extracted from the 2021 company ESG report; the page range of each chapter in the 2021 company ESG report is input; the ESG report is divided into chapters according to the input page range; all the text of each chapter in the ESG report is extracted and added to the list, and the content of each chapter is an element in the list; after processing the ESG report in this way, the plain text information based on the ESG report is obtained;

[0048] Send the processed ESG report to the big language model for keyword extraction, extract the keywords of each chapter; put the processed files into a folder named process_txt; upload the files to the big language model kimi in turn; design a prompt to let the big language model kimi extract keywords from the text content. The designed prompt is as follows: prompt: Please extract all the keywords in this chapter; perform text processing on the obtained keywords and process them into the following example: Key_list = ["Sustainable Development\Company\Project\Technology\Digital\Environment\Education\Safety\Supply Chain\Risk"].

[0049] Step S2: extracting substantive issues from the ESG report, calculating the similarity between each keyword and the substantive issue, and screening the substantive issues based on the similarity.

[0050] Construct a list of ESG substantive issues, ESG_list = ["Greenhouse gas emissions", "Water resource management", "Environment and natural resources",,,,,"Indicator X1",,,"Indicator Xn"]; select a word embedding model to extract the keywords of each chapter and the features of ESG substantive issues, and obtain the keyword vector Key_E and ESG substantive issue vector ESG_E of each chapter to facilitate text similarity calculation, calculate the text similarity between each substantive issue and the keyword of each chapter based on the feature extraction results, calculate the average value of the text similarity, and sort from high to low based on the average value, and select several substantive issues with high similarity.

[0051] In this example, cosine distance is selected to calculate the semantic proximity between two text vectors, which can be expressed by formula (1):

[0052]

[0053] In addition, you can also use methods such as Euclidean distance, Jaccard similarity, and Hamming distance to calculate the semantic proximity between two text vectors, as shown below:

[0054]

[0055] Calculate the similarity between the keywords of each chapter of the ESG report and the substantive issues of ESG; calculate the similarity between each element of Key_list and all indicators in ESG_list; calculate the similarity between texts and sort the similarities from high to low, so as to screen out 3-5 substantive issues with the highest similarity to the chapter text; loop through the Key_list list and repeat the operation of S232; output the final result list similarity_list = ["Chapter 1 Keywords: Indicator 1, Indicator 2..., Indicator 5", "Chapter 2 Keywords: Indicator 1, Indicator 2..., Indicator 5",... "Chapter X Keywords: Indicator X1, Indicator X2..., Indicator X5",].

[0056] Step S3: construct a prompt template, input the screened substantive issues into the prompt template to obtain a prompt word text, concatenate the prompt word text with the content of the corresponding chapter, and input it into the large language model to obtain the text analysis result of the corresponding chapter of the ESG report.

[0057] Process the similarity_list list and separate the ESG material issues from the list to facilitate the construction of our prompt.

[0058] like Figure 4 As shown, prompt design: prompt initial template design: Background: ESG is the abbreviation of the three English words Environment, Society and Governance. It refers to measuring the sustainability of corporate development from the perspectives of environment, society and corporate governance. Role: You are an expert in the field of evaluating corporate ESG reports. You can accurately find important scoring points related to ESG in a text; Responsibilities: You now need to fully summarize the content related to {ESG material issues} in this text. If there are numerical descriptions, you must also summarize them, including their advantages and disadvantages, limited to the content in this text. Combine the final prompts sent to the big model: Splice the three material issues with the highest similarity between the chapter and the pre-set prompt template in turn, write each spliced ​​prompt into a readable file, and finally form a readable file containing all prompts as the input of the big language model in this example. The big language model performs reasoning: prompt input: In order to improve the reasoning speed, this example chooses to use batch processing to reason about the prompts in an entire readable file.

[0059] Adding the vllm reasoning acceleration framework: In order to further improve the reasoning speed in this example, the vllm reasoning acceleration framework is used. Text summary result processing: In this example, the final output result of the text summary of the large language model is stored in the form of a program-readable document.

[0060] The output results of the embodiment of the present invention are shown in Table 1 (taking Huawei's ESG report as an example, Table 1 is a partial output effect display diagram based on the first chapter of Huawei's ESG report).

[0061] Table 1

[0062]

[0063] In order to illustrate that the prompt design method proposed in this example is more effective in processing text, this example uses a common prompt for comparison. The design of the common prompt is to stack all the substantive issues together without index screening and directly send them to the large language model to summarize the chapter text, and write the final result into a readable document. In order to further illustrate the effectiveness of the prompt design method in this example, this example also selects a packaged large model that can process files to summarize the ES G report text. The large models used are "Tong Yi" under Tong Yi Qian Wen, "Zhi Pu Qing Yan" under Zhi Pu Hua Zhang, and "Wan Zhi" under Ling Yi Wan Wu. In this example, rouge is selected as the evaluation indicator, mainly calculating the rouge score between the text summarized by the large language model and the original text. The higher the rouge score, the more comprehensive the summary. From this example, we can see that this method can effectively improve the comprehensiveness of the large language model's summary of ESG reports. In each experiment in this example, three base models are used to illustrate the effectiveness of the design of this prompt method. These three base models are Qwen1.5-14B-Chat-GPTQ-Int4, Yi-34B-Chat-4bits and Chatglm3-6b. In order to ensure the rigor of the experiment, this example repeated the experiment three times for each prompt method with each base model. The results of Qwen1.5-14B-Chat-GPTQ-Int4 are shown in Appendix 2:

[0064] Table 2

[0065]

[0066] The results of Yi-34B-Chat-4bits are shown in Appendix 3:

[0067] Table 3

[0068]

[0069] The results of Chatglm3-6b are shown in Supplementary Table 4.

[0070] Table 4

[0071]

[0072] Through these groups of comparative experiments, it can be clearly seen that the effect of the prompt design method for ESG in this example is significantly higher than the previous two. Among them, noprompt represents directly sending the entire ESG report to the already integrated models "Tongyi", "Wanzhi" and "Zhipu Qingyan", prompt1 represents putting all substantive issues into the prompt at one time, and prompt2 represents the prompt obtained by the prompt design method proposed in this example.

[0073] Embodiment 2

[0074] A prompt-based ESG report text analysis system, the system comprising: a keyword extraction module, an indicator screening module and an ESG analysis module;

[0075] The keyword extraction module is used to extract the content of each chapter in the ESG report, and based on the content of each chapter, obtain several keywords of the content of each chapter.

[0076] Process the initial ESG report: first construct an empty list Chapter_list = [] to store the text content of each chapter extracted from the 2021 company ESG report; enter the page number range of each chapter in the 2021 company ESG report; divide the ESG report by chapter according to the input page number range; extract all the text in each chapter in the ESG report and add it to the list, with the content of each chapter as an element in the list; after processing the ESG report, we get pure text information; send the processed ESG report to the big language model for keyword extraction, and extract the keywords of each chapter; put the processed files into a folder named process_txt; upload the files to the big language model kimi in turn; design a prompt to let the big language model kimi extract keywords from the text content. The designed prompt is as follows: prompt: Please extract all the keywords in this chapter: process the obtained keywords into text and process them into the following example: Key_list = ["Sustainable Development\Company\Project\Technology\Digital\Environment\Education\Safety\Supply Chain\Risk"].

[0077] The indicator screening module is used to extract substantive issues in the ESG report, calculate the similarity between each keyword and the substantive issue, and screen the substantive issues based on the similarity.

[0078] Construct a list of ESG substantive issues, ESG_list = ["Greenhouse gas emissions", "Water resources management", "Environment and natural resources",,,,,"Indicator X1",,,"Indicator Xn"]; select a word embedding model to extract the keywords and features of ESG substantive issues in each chapter, and obtain the keyword vector Key_E and ESG substantive issue vector ESG_E of each chapter to facilitate text similarity calculation. In this example, cosine distance is selected to calculate the semantic proximity between two text vectors, which can be expressed by formula (5):

[0079]

[0080] Calculate the similarity between the keywords of each chapter of the ESG report and the substantive issues of ESG; Calculate the similarity between each element of Key_list and all indicators in ESG_list; Calculate the similarity between texts and sort the similarities from high to low to filter out the 3-5 substantive issues with the highest similarity to the chapter text; Loop through the Key_list list and repeat the above operations; Output the final result list similarity_list = ["Chapter 1 Keywords: Indicator 1, Indicator 2..., Indicator 5", "Chapter 2 Keywords: Indicator 1, Indicator 2..., Indicator 5", ... "Chapter X Keywords: Indicator X1, Indicator X2..., Indicator X5",].

[0081] The ESG analysis module is used to construct the prompt template. The screened substantive issues are input into the prompt template to obtain the prompt text. After the prompt text is spliced ​​with the content of the corresponding chapter, it is input into the large language model to obtain the text analysis results of the corresponding chapter of the ESG report. The ESG text analysis module is used by the large language model to realize the text summary and summary of the ESG report by reading the prompt. Process the similarity_list list and split the ESG substantive issues from the list to facilitate the construction of our prompt. Prompt design: prompt initial template design: Background: ESG is the abbreviation of the three English words environment (Environ mental), society (Social) and governance (Governance). It refers to measuring the sustainability of corporate development from the perspectives of environment, society and corporate governance. Role: You are an expert in the field of evaluating corporate ESG reports. You can accurately find important scoring points related to ESG in a text. Responsibilities: You now need to fully summarize the content related to {ESG substantive issues} in this text, and summarize it if there is a numerical description, including its advantages and disadvantages, limited to the content in this text.

[0082] Combine the final prompts that are fed into the large model: splice the three substantive topics with the highest similarity between the chapter and the pre-set prompt template in turn, write each spliced ​​prompt into a readable file, and finally form a readable file containing all prompts as the input of the large language model in this example. The large language model performs reasoning: prompt input: In order to improve the reasoning speed, this example chooses to use batch processing to reason about the prompts in an entire readable file. Add the vllm reasoning acceleration framework: In order to further improve the reasoning speed in this example, the vllm reasoning acceleration framework is adopted. Text summary result processing: In this example, the final output result of the text summary of the large language model is stored in the form of a program-readable document.

[0083] The embodiments described above are only descriptions of the preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Without departing from the design spirit of the present invention, various modifications and improvements made to the technical solutions of the present invention by ordinary technicians in this field should all fall within the protection scope determined by the claims of the present invention.

Claims

1. A prompt-based ESG report text analysis method, characterized in that: The method comprises: Step S1, extracting the content of each chapter in the ESG report, and obtaining several keywords of the content of each chapter based on the content of each chapter; Step S2: extracting substantive issues in the ESG report, calculating the similarity between each substantive issue and the keywords of each chapter, and screening the substantive issues based on the similarity; Step S3: construct a prompt template, input the screened substantive issues into the prompt template to obtain a prompt word text, concatenate the prompt word text with the content of the corresponding chapter, and input it into the large language model to obtain the text analysis result of the corresponding chapter of the ESG report.

2. The prompt-based ESG report text analysis method according to claim 1, characterized in that: In step S1, the process of extracting the content of each chapter in the ESG report specifically includes: Enter the page range for each section of the ESG report; Divide the ESG report into chapters according to the page number range; The text content of each chapter in the ESG report is extracted and stored in an empty list.

3. The prompt-based ESG report text analysis method according to claim 2, characterized in that: In step S2, based on the content of each chapter, obtaining the contents of several keywords of the content of each chapter specifically includes: Use the large language model to extract keywords from the list of text content of each chapter: Read the text content of each chapter in the list and get the keywords of each chapter.

4. The prompt-based ESG report text analysis method according to claim 1, characterized in that: In step S2, the process of screening the substantive topics based on the similarity specifically includes: Construct a list based on the substantive topics; Use word embedding models to extract features of keywords and substantive issues; Based on the feature extraction results, the text similarity between each substantive topic and the keywords of each chapter is calculated, the average value of the text similarity is calculated, and the topics are sorted from high to low based on the average value, and several substantive topics with high similarity are selected.

5. The prompt-based ESG report text analysis method according to claim 1, characterized in that: In S3, obtaining the ESG report text analysis results corresponding to the substantive issues also includes using the reasoning acceleration framework vllm to batch process the substantive issues and spliced ​​content.

6. A prompt-based ESG report text analysis system, the ESG report text analysis system is used to implement the ESG report text analysis method according to any one of claims 1 to 5, characterized in that: The system includes: keyword extraction module, indicator screening module and ESG analysis module; The keyword extraction module is used to extract the content of each chapter in the ESG report, and based on the content of each chapter, obtain a number of keywords of the content of each chapter; The indicator screening module is used to extract substantive issues in the ESG report, calculate the similarity between each keyword and the substantive issue, and screen the substantive issues based on the similarity; The ESG analysis module is used to construct a prompt template, input the screened substantive issues into the prompt template to obtain a prompt word text, splice the prompt word text with the content of the corresponding chapter, and input it into the large language model to obtain the text analysis result of the corresponding chapter of the ESG report.

7. The prompt-based ESG report text analysis system according to claim 6, characterized in that: In the keyword extraction module, the process of extracting the content of each chapter in the ESG report specifically includes: Enter the page range for each section of the ESG report; Dividing the ESG report into chapters according to the page number range; The text content of each chapter in the ESG report is extracted and stored in an empty list.

8. The prompt-based ESG report text analysis system according to claim 6, characterized in that: In the keyword extraction module, based on the content of each chapter, obtaining the contents of several keywords of the content of each chapter specifically includes: Use the large language model to extract keywords from the list of text content of each chapter: Read the text content of each chapter in the list and get the keywords of each chapter.

9. The prompt-based ESG report text analysis system according to claim 6, characterized in that: In the indicator screening module, the process of screening initial substantive issues based on similarity specifically includes: Construct a list based on the substantive topics; Use word embedding models to extract features of keywords and substantive issues; Based on the feature extraction results, the text similarity between each substantive topic and each chapter keyword is calculated, the average value of the text similarity is calculated, and the texts are sorted from high to low based on the average value, and several substantive topics with high similarity are selected.

Images