Index extraction method and device and storage medium

Abstract

The embodiment of the invention provides an index extraction method and device and a storage medium, and the method comprises the steps: obtaining a target report and a preset index disclosure rule, and the target report comprises to-be-extracted indexes needing to be disclosed in the preset index disclosure rule; according to a preset index disclosure rule, meta query information corresponding to the to-be-extracted index is obtained, and the meta query information comprises index description information corresponding to the to-be-extracted index, a search word and a prompt word; performing information extraction on the target report to generate a knowledge base corresponding to the target report; according to the index description information and the search term, obtaining target content corresponding to the to-be-extracted index from a knowledge base; and inputting the meta query information and the target content into the large language model to obtain the to-be-extracted index extracted from the target content by the large language model. According to the scheme, any to-be-extracted index can be extracted by utilizing the corresponding meta query information, so that batch automatic extraction of the to-be-extracted indexes can be realized, and the index extraction efficiency is effectively improved.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to an indicator extraction method, device and storage medium. Background Art

[0002] Environmental, Social and Governance (ESG) reports are used to evaluate companies across three dimensions: environment, society, and governance. They include multiple indicators that companies are required to disclose, such as greenhouse gas emissions and the gender ratio of their workforce. Because ESG disclosure guidelines developed by different organizations require companies to disclose different indicators, and because ESG reports contain a large amount of information and lack a uniform expression of indicators, manual extraction is often used to extract the indicators required for further analysis from ESG reports, which is labor-intensive and inefficient. Summary of the Invention

[0003] The embodiments of the present invention provide an indicator extraction method, device and storage medium to improve the extraction efficiency of indicators.

[0004] In a first aspect, an embodiment of the present invention provides an indicator extraction method, the method comprising:

[0005] Obtaining a target report and preset indicator disclosure rules, wherein the target report includes the indicators to be extracted that need to be disclosed in the preset indicator disclosure rules;

[0006] According to the preset indicator disclosure rules, obtaining meta-query information corresponding to the indicator to be extracted, wherein the meta-query information includes: indicator description information, search terms, and prompt terms corresponding to the indicator to be extracted;

[0007] Extracting information from the target report to generate a knowledge base corresponding to the target report;

[0008] Acquire target content corresponding to the indicator to be extracted from the knowledge base according to the indicator description information and the search term;

[0009] The meta-query information and the target content are input into a large language model to obtain the to-be-extracted indicator extracted by the large language model from the target content.

[0010] In a second aspect, an embodiment of the present invention provides an indicator extraction device, the device comprising:

[0011] An acquisition module, configured to acquire a target report and preset indicator disclosure rules, wherein the target report includes the indicators to be extracted that need to be disclosed in the preset indicator disclosure rules;

[0012] a processing module configured to obtain, according to the preset indicator disclosure rules, meta-query information corresponding to the indicator to be extracted, wherein the meta-query information includes indicator description information, search terms, and prompt terms corresponding to the indicator to be extracted; and extract information from the target report to generate a knowledge base corresponding to the target report;

[0013] An extraction module is used to obtain the target content corresponding to the indicator to be extracted from the knowledge base based on the indicator description information and the search term; and input the meta-query information and the target content into a large language model to obtain the indicator to be extracted from the target content by the large language model.

[0014] In a third aspect, an embodiment of the present invention provides an electronic device comprising: a memory, a processor, and a communication interface; wherein, the memory stores executable code, and when the executable code is executed by the processor, the processor can at least implement the indicator extraction method described in the first aspect.

[0015] In a fourth aspect, an embodiment of the present invention provides a non-temporary machine-readable storage medium having executable code stored thereon. When the executable code is executed by a processor of an electronic device, the processor can at least implement the indicator extraction method described in the first aspect.

[0016] In a fifth aspect, an embodiment of the present invention provides a computer program product, comprising: a computer program, which, when executed by a processor of an electronic device, enables the processor to at least implement the indicator extraction method described in the first aspect.

[0017] In the solution provided by an embodiment of the present invention, when it is necessary to extract indicators from a certain report (i.e., a target report), first, the preset indicator disclosure rules corresponding to the target report and the indicators to be extracted are obtained, wherein the target report includes the indicators to be extracted that need to be disclosed in the preset indicator disclosure rules. Then, according to the preset indicator disclosure rules, the meta-query information corresponding to the indicators to be extracted is obtained, and the meta-query information includes: indicator description information, search terms, and prompt terms corresponding to the indicators to be extracted; and information extraction is performed on the target report to generate a knowledge base corresponding to the target report. Thereafter, according to the indicator description information and the search terms, the target content corresponding to the indicators to be extracted is obtained from the knowledge base. Finally, the meta-query information and the target content are input into the large language model to obtain the indicators to be extracted extracted from the target content by the large language model. In this solution, the corresponding meta-query information for any indicator to be extracted can be determined based on the preset indicator disclosure rules. The meta-query information can be used to batch query the target content related to the indicator to be extracted from the knowledge base containing the target report information, and the indicator to be extracted can be extracted from the target content through a large language model. In the process of extracting the indicator to be extracted, no human intervention is required. It is an automated extraction process that can effectively improve the extraction efficiency of the indicator to be extracted. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following is a brief introduction to the drawings required for use in the description of the embodiments. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0019] Figure 1 A flowchart of an indicator extraction method provided by an embodiment of the present invention;

[0020] Figure 2 A schematic diagram of an indicator extraction process provided by an embodiment of the present invention;

[0021] Figure 3 A flowchart of a knowledge base generation process provided by an embodiment of the present invention;

[0022] Figure 4 A flowchart of another knowledge base generation process provided by an embodiment of the present invention;

[0023] Figure 5 A flowchart of another knowledge base generation process provided by an embodiment of the present invention;

[0024] Figure 6 A flowchart of a target content determination process provided by an embodiment of the present invention;

[0025] Figure 7A flowchart of a process for extracting an index to be extracted provided by an embodiment of the present invention;

[0026] Figure 8 A schematic diagram of an indicator extraction device provided by an embodiment of the present invention;

[0027] Figure 9 A schematic structural diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0028] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0029] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in the embodiments of the present invention are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0030] In addition, the step sequence in the following method embodiments is only an example and not a strict limitation.

[0031] Before introducing the indicator extraction method provided by the embodiment of the present invention, the following concepts are first explained.

[0032] Environmental, Social and Governance (ESG) reports are used to evaluate companies from the three dimensions of environment, society and governance. They include a number of indicators that companies need to disclose, such as greenhouse gas emissions and the male / female employee ratio.

[0033] Pre-set indicator disclosure rules are the reference standards for indicator disclosure in reports when generating a report, such as the types of indicators required to be disclosed in the report and the hierarchical relationship between the disclosed indicator types. For example, in the application scenario of ESG reporting, the pre-set indicator disclosure rules can be ESG disclosure guidelines developed by different organizations (such as the Global Reporting Initiative (GRI) and the Task Force on Climate-related Financial Disclosures (TCFD)). The ESG disclosure guidelines specified by different organizations may vary in the number and types of ESG indicators required to be disclosed.

[0034] The knowledge base is a database used to store the vectorized results of the information contained in the target report. The target report is the report from which indicators are extracted, including but not limited to the ESG report.

[0035] Meta-query information, that is, the structured expression result of the preset indicator disclosure rules, includes indicator description information, search terms, prompt terms, etc., and is used to obtain the indicators to be extracted from the knowledge base of the target report.

[0036] Large Language Models (LLMs) are artificial intelligence models that use machine learning technology to understand and generate human language. They can be used in various application scenarios such as question answering, classification, summarization, translation, and chat. They include but are not limited to BERT (Bidirectional Encoder Representations from Transformers), T5 (Text-to-Text Transfer Transformer), and Chat GPT (Generative Pre-trained Transformer).

[0037] A prompt is an input instruction used to guide or inspire an LLM to complete a specific task. Prompts typically contain context related to the input, helping the LLM better understand the input's intent and generate the correct output.

[0038] The index extraction method provided in the embodiments of the present invention can be performed by an electronic device, which can be a terminal device such as a PC, a laptop, a smartphone, or a server. The server can be a physical server including an independent host, a virtual server, a cloud server, or a server cluster.

[0039] Figure 1 A flowchart of an indicator extraction method provided by an embodiment of the present invention is shown in FIG. Figure 1 As shown, the method includes the following steps:

[0040] 101. Obtain the target report and preset indicator disclosure rules. The target report includes the indicators to be extracted that need to be disclosed in the preset indicator disclosure rules.

[0041] 102. According to the preset indicator disclosure rules, obtain meta-query information corresponding to the indicator to be extracted, where the meta-query information includes: indicator description information, search terms, and prompt terms corresponding to the indicator to be extracted.

[0042] 103. Extract information from the target report to generate a knowledge base corresponding to the target report.

[0043] 104. According to the indicator description information and the search term, the target content corresponding to the indicator to be extracted is obtained from the knowledge base.

[0044] 105. Input the meta-query information and the target content into the large language model to obtain the to-be-extracted indicators extracted by the large language model from the target content.

[0045] For ease of understanding, the present invention uses the ESG report as an example for illustration. It should be understood that the indicator extraction method provided in the present invention is also applicable to other application scenarios for extracting indicators from reports, such as extracting indicators from corporate annual reports, and is not limited to extracting indicators from ESG reports.

[0046] In practice, different companies often use inconsistent methods to disclose required indicators based on pre-set disclosure rules when generating ESG reports. For example, some companies use text descriptions while others use tables. Furthermore, ESG reports often contain a wealth of other information beyond the required indicators. Therefore, extracting indicators from ESG reports is often difficult, and manual extraction is labor-intensive and inefficient.

[0047] In order to improve the efficiency of extracting indicators to be extracted from ESG reports, in an embodiment of the present invention, LLM and meta-query information obtained based on preset indicator disclosure rules are introduced to realize the automatic extraction of indicators to be extracted.

[0048] Figure 2 A schematic diagram of an indicator extraction process provided by an embodiment of the present invention. Figure 2As shown in Figure 2, the indicator extraction process can be summarized into two stages: preprocessing and extraction. In the preprocessing stage, meta-query information corresponding to the target indicator is obtained based on the pre-set indicator disclosure rules for the target indicator. Furthermore, information is extracted from ESG reports to generate a knowledge base corresponding to the ESG report. In the indicator extraction stage, the indicator description and search terms corresponding to the target indicator in the meta-query information are first used to retrieve the target content from the knowledge base. The meta-query information and target content are then input into the LLM, which then extracts the target indicator from the target content.

[0049] Among them, the meta-query information corresponding to the indicator to be extracted is the result of disassembling the preset indicator disclosure rules and expressing them in a structured manner. The meta-query information contains multiple fields, which are used to describe the indicators to be extracted from multiple different dimensions, so as to accurately extract the indicators to be extracted from the ESG report. Optionally, the meta-query information may include a first field, a second field, and a third field. Among them, the first field contains the indicator description information of the indicator to be extracted, such as: the indicator name of the indicator to be extracted, the hierarchical relationship between the indicator to be extracted and other indicators (that is, the indicator subordination relationship), etc.; the second field contains the search term corresponding to the indicator to be extracted, which is used to retrieve the target content related to the indicator to be extracted from the knowledge base; the third field contains the prompt word corresponding to the indicator to be extracted, which is expressed in the form of natural language that LLM can process, and is used to guide LLM to correctly output the indicator to be extracted from the target content.

[0050] The following describes in detail the process of obtaining meta-query information.

[0051] The preset indicator disclosure rules can also be generally understood as the indicator system corresponding to the indicators that need to be disclosed in the ESG report. This indicator system reflects the hierarchical relationship between different indicators.

[0052] For example, the indicator system corresponding to a certain preset indicator disclosure rule includes three categories: environment, society, and governance. Each category includes multiple levels. For example, the "environment" category includes emissions, resource use, environment, and natural resources. Each level includes multiple key indicators. For example, the "emissions" level includes key indicators such as total greenhouse gas emissions and density, and total waste emissions and density. Each key indicator corresponds to multiple topics. For example, the key indicator "total greenhouse gas emissions and density" includes topics such as direct greenhouse gas emissions and indirect greenhouse gas emissions. Each topic has its own corresponding numerical value. For example, the "direct greenhouse gas emissions" topic is measured in tons.

[0053] For any to-be-extracted metric, based on the hierarchical relationship "category - aspect - key metric - topic - value" reflected by the above-mentioned preset metric disclosure rules, the metric description information of the to-be-extracted metric included in the first field of the meta-query information can be determined.

[0054] As an optional implementation method, at least one content corresponding to the to-be-extracted metric in the hierarchical relationship can be selected and filled into the corresponding field position in the first field as the metric description information of the to-be-extracted metric. For example: select "key metric" and "value" and fill them into the corresponding field positions in the first field as the metric description information of the to-be-extracted metric; or, select "aspect", "key metric", "topic", and "value" and fill them into the corresponding field positions in the first field as the metric description information of the to-be-extracted metric.

[0055] Among them, the content types that can be filled in different field positions in the first field can be predefined. For example: in the first field <aspect, kpi, topic, quantity>, the aspect field is used to fill in aspect-related content, the kpi field is used to fill in key metric-related content, the topic field is used to fill in topic-related content, and the quantity field is used to fill in value-related content. Thus, for different to-be-extracted metrics, the first field in the corresponding meta-query information containing the metric description information has a unified field identifier.

[0056] Furthermore, according to the metric description information of the to-be-extracted metric included in the first field, the search term corresponding to the to-be-extracted metric included in the second field can be determined. Among them, this search term is also the keyword used to search for information related to the to-be-extracted metric from the knowledge base and can be extracted from the metric description information. For example, assume that the metric description information of the to-be-extracted metric included in the first field is <aspect - emissions, kpi - total greenhouse gas emissions and density, topic - direct greenhouse gas emissions, quantity - unit of measurement is tons>, then the search terms can be: greenhouse gas, emissions, density, etc.

[0057] According to the metric description information of the to-be-extracted metric included in the first field, the prompt word corresponding to the to-be-extracted metric included in the third field can also be determined. Optionally, a corresponding prompt word template can be pre-configured in the third field, and then based on the metric description information of the to-be-extracted metric included in the first field and this prompt word template, the prompt word is determined. For example, this prompt word template can be "What is the {quantity} of the {topic} part corresponding to {kpi} regarding {aspect}", where {aspect}, {kpi}, {topic}, and {quantity} can be filled in with the metric description information of the to-be-extracted metric included in the first field.

[0058] In addition, in an optional embodiment, the meta-query information may also include prior knowledge information corresponding to the indicator to be extracted. This prior knowledge information is used to assist the LLM in better understanding the indicator to be extracted. For example, when the indicator to be extracted is related to direct greenhouse gases, the prior knowledge information may include the concept of greenhouse gases, the concept of direct greenhouse gases, and so on. By configuring the prior knowledge information in the meta-query information, the LLM can overcome its lack of professional knowledge in the field to which the indicator to be extracted belongs, better understand the task of extracting the indicator to be extracted, and ensure the accuracy of the indicator to be extracted from the target content. Optionally, the prior knowledge information corresponding to the indicator to be extracted can be configured in some fields within the second field.

[0059] After determining the relevant contents of the first field, the second field, and the third field corresponding to a certain indicator to be extracted, the combination result of the first field, the second field, and the third field can be used as meta-query information of the indicator to be extracted.

[0060] In practical applications, to improve the efficiency of extracting indicators, meta-query information corresponding to various indicators in different pre-set indicator disclosure rules can be pre-generated. When extracting indicators from an ESG report, meta-query information for all indicators in a specific pre-set indicator disclosure rule can be obtained from the dimensions of the pre-set indicator disclosure rule. Alternatively, meta-query information can be obtained only for the indicators to be extracted from the dimensions of the indicators to be extracted. This improves both the efficiency and flexibility of obtaining meta-query information.

[0061] After obtaining the meta-query information corresponding to the indicator to be extracted, the indicator description information and search terms in the meta-query information are used to retrieve the target content corresponding to the indicator to be extracted from the knowledge base corresponding to the ESG report.

[0062] In practice, the knowledge bases corresponding to ESG reports may have different interface specifications. To ensure that the target content can be retrieved from the knowledge base corresponding to the ESG report, the indicator description information and search terms in the meta-query information can be processed into a query statement that conforms to the interface specifications of the knowledge base corresponding to the ESG report. This query statement can then be used to retrieve the target content corresponding to the indicator to be extracted from the knowledge base corresponding to the ESG report.

[0063] Finally, the meta-query information and target content are input into the LLM to extract the indicators to be extracted from the target content through the LLM, thereby completing the indicator extraction of the ESG report.

[0064] In this solution, when it is necessary to extract the indicators to be extracted from a target report (such as the ESG report mentioned above), first, the target report and the preset indicator disclosure rules corresponding to the indicators to be extracted are obtained, wherein the target report includes the indicators to be extracted that are required to be disclosed in the preset indicator disclosure rules. Then, the target report is processed to extract the information of the target report to generate a knowledge base corresponding to the target report; and according to the preset indicator disclosure rules, the meta-query information corresponding to the indicators to be extracted is obtained, wherein the meta-query information includes: indicator description information, search terms, and prompt terms corresponding to the indicators to be extracted. The search terms are used to retrieve target content related to the indicators to be extracted from the knowledge base. Therefore, based on the indicator description information and search terms in the meta-query information corresponding to different indicators to be extracted, target content related to the indicators to be extracted can be automatically and batch-queried from the knowledge base containing the target report information. The prompt terms are used to guide the large language model to extract the indicators to be extracted from the target content. Therefore, after the meta-query information and target content are input into the large language model, the large language model can more accurately extract the indicators to be extracted from the target content. In this solution, by introducing the meta-query information corresponding to the indicators to be extracted and the large language model, an automated extraction process for the indicators to be extracted is designed, which can efficiently and accurately extract the indicators to be extracted from the target report.

[0065] The above examples provide an overview of the indicator extraction method provided by the present invention. As described above, during the indicator extraction process, the target content related to the indicator to be extracted is first obtained from the knowledge base corresponding to the target report. The LLM is then used to extract the indicator from the target content. Therefore, the accuracy of the knowledge base constructed based on the target report will affect the accuracy of the ultimately extracted indicator.

[0066] To ensure the accuracy of the knowledge base constructed based on the target report, this embodiment provides at least one of the following knowledge base generation methods corresponding to the target report, which ensures the accuracy of the generated knowledge base by ensuring the accuracy of the extracted information.

[0067] Figure 3 A flow chart of a knowledge base generation process provided by an embodiment of the present invention is as follows: Figure 3 As shown, the following steps are included:

[0068] 301. Locate text paragraphs in the target report.

[0069] 302. Extract multiple text paragraphs contained in the target report based on the text paragraph positioning result.

[0070] 303. Perform vectorization processing on the multiple text paragraphs to obtain a first sub-knowledge base.

[0071] It's understandable that complete sentences or paragraphs can more accurately express semantics. However, due to limitations such as typesetting, not every line of text in the target report is necessarily a complete sentence or paragraph. Direct text extraction makes it difficult to guarantee the integrity and independence of the extracted text content.

[0072] To this end, in this embodiment, before performing text extraction, the target report is first positioned for text paragraphs using, for example, a visual text detection model (such as: LayoutLMv3, etc.) to locate the regions in the target report corresponding to the multiple text paragraphs contained in the target report. Afterwards, based on the regions corresponding to the text paragraphs in the target report, the multiple text paragraphs contained in the target report are extracted, thereby ensuring the integrity and independence of the extracted text content. Specifically, the content corresponding to each paragraph is complete, and the contents corresponding to different paragraphs are independent of each other. Finally, multiple text paragraphs are vectorized using, for example, a text vectorization model (such as: m3e, etc.), and the text contents corresponding to the text paragraphs are converted into numerical vectors, which are stored in a vector database as a first knowledge base.

[0073] In this solution, by locating the text paragraphs of the target report, the integrity and independence of the text content of the extracted text paragraphs are improved, thereby ensuring the accuracy of the first knowledge base generated based on the text content corresponding to the text paragraphs.

[0074] Figure 4 A flowchart of another knowledge base generation process provided by an embodiment of the present invention is shown in FIG. Figure 4 As shown, the following steps are included:

[0075] 401. Perform table recognition on the target report.

[0076] 402. Determine a target text corresponding to the table based on the table recognition result. The target text includes a mapping relationship between different table data in the table.

[0077] 403. Perform vectorization processing on the target text to obtain a second sub-knowledge base.

[0078] In addition to text content, the target report also contains tables. Although tables can be used for more efficient data analysis, they are prone to information confusion when extracting information.

[0079] To this end, in this embodiment, before extracting the content of the table, the target report is first identified by using, for example, a visual table detection and recognition model (such as Table le-Transformer, etc.) to identify the position of the table in the target report, as well as the corresponding position of each cell in the table and the corresponding content of each cell.

[0080] Afterwards, based on the corresponding position and content of each cell in the identified table, the target text corresponding to the table is determined, that is, the table is converted into plain text, such as plain text in formats such as Comma-Separated Values ​​(CSV), JavaScript Object Notation (JSON), and Extensible Markup Language (XML).

[0081] Among them, the target text includes the mapping relationship between different table data in the table to ensure the accuracy of the meaning corresponding to the table data. For example, suppose a table contains two columns of name and value and several rows, in which there is a row: the name corresponds to carbon dioxide gas emissions, and the value corresponds to 1000 (tons), which means that carbon dioxide gas emissions are 1000 (tons). In order to ensure the accuracy of its expression, when determining the target text corresponding to the table, for this row, the target text will not only contain information such as "carbon dioxide gas emissions" and "1000 (tons)", but also include the mapping relationship between "carbon dioxide gas emissions" and "1000 (tons)".

[0082] Finally, the target text is vectorized by using a text vectorization model (such as m3e, etc.), and the text content corresponding to the target text is converted into a numerical vector and stored in a vector database as a second knowledge base.

[0083] In this solution, by performing table recognition on the target report and generating the target text corresponding to the table, the accuracy of the meaning of the table data when expressing the text is guaranteed, thereby ensuring the accuracy of the second knowledge base generated based on the table.

[0084] Figure 5 A flowchart of another knowledge base generation process provided by an embodiment of the present invention is as follows: Figure 5 As shown, the following steps are included:

[0085] 501. Perform text visual feature recognition on the target report.

[0086] 502. Determine a report content outline tree corresponding to the target report based on the text visual feature recognition result.

[0087] 503. Perform vectorization processing on the report content outline tree to obtain a third sub-knowledge base.

[0088] It is understandable that while extracting the indicators to be extracted from the full information corresponding to the target report can ensure complete extraction of the indicators to be extracted, it is inefficient. To this end, this implementation provides a third sub-knowledge base to extract the indicators to be extracted from the partial information corresponding to the target report, thereby improving the extraction efficiency of the indicators to be extracted.

[0089] Specifically, the visual features of the text in the target report can be identified through, for example, a visual text detection model (such as: LayoutLMv3, etc.), such as: identifying the font size, font, etc. of the text in the target report. Afterwards, the report content outline tree corresponding to the target report is determined based on the results of the text visual feature recognition. For example, based on the correspondence between the pre-defined font size, font and the titles and text at each level, the contents corresponding to the titles and text at each level in the target report can be determined; then, based on the hierarchical relationship between the titles and text at each level, the report content outline tree corresponding to the target report can be determined. Finally, through, for example, a text vectorization model (such as: m3e, etc.), the report content outline tree is vectorized, and the text content corresponding to the report content outline tree is converted into a numerical vector and stored in a vector database as a third knowledge base.

[0090] Optionally, when obtaining the target content corresponding to the indicator to be extracted from the knowledge base corresponding to the target report, the target content range related to the indicator to be extracted (for example, the text content under a certain title) can be determined based on the third knowledge base, and then a query can be performed within the target content range to obtain the target content corresponding to the indicator to be extracted. Since the amount of information corresponding to the target content range is less than the full amount of information corresponding to the target report, the extraction efficiency of the indicator to be extracted can be effectively improved. In addition, since the content corresponding to the target content range is related to the indicator to be extracted, the method of performing a detailed query within the target content range can effectively improve the accuracy of extracting the indicator to be extracted.

[0091] Optionally, the knowledge base corresponding to the target report may include at least one of a first knowledge base, a second knowledge base, and a third knowledge base.

[0092] The above describes the process of determining the knowledge base corresponding to the target report. The following describes the process of obtaining the target content corresponding to the indicator to be extracted from the knowledge base.

[0093] Figure 6 A flowchart of a target content determination process provided by an embodiment of the present invention is shown in FIG. Figure 6 As shown, the following steps are included:

[0094] 601. Generate a query vector based on the indicator description information and search terms included in the meta-query information corresponding to the indicator to be extracted.

[0095] 602. Determine a target numerical vector from the numerical vectors whose vector similarity is greater than a first similarity threshold based on the vector similarity between the query vector and the numerical vectors included in the knowledge base, wherein the knowledge base includes numerical vectors corresponding to different contents extracted from the target report.

[0096] Among them, determining a target numerical vector whose vector similarity is greater than a first similarity threshold from the numerical vector includes: determining a numerical vector whose vector similarity is greater than a customized first similarity threshold from the numerical vector as the target numerical vector; or determining the top k (k is an integer greater than 0) numerical vectors related to the query vector in the numerical vector as the target numerical vector, that is, the top k numerical vectors with the largest vector similarity after sorting the vector similarities in order from large to small are used as the target numerical vector.

[0097] In addition, based on the vector similarity between the query vector and the numerical vectors included in the knowledge base, determining a target numerical vector whose vector similarity is greater than a first similarity threshold from the numerical vectors, further comprising: based on the first vector similarity between the query vector and all the numerical vectors included in the knowledge base, and / or the second vector similarity between the query vector and some of the numerical vectors included in the knowledge base, determining a target numerical vector whose first vector similarity and / or second vector similarity is greater than a first similarity threshold from the numerical vectors. The some of the numerical vectors included in the knowledge base are numerical vectors corresponding to content that is relatively relevant to the indicator to be extracted, and can be determined based on the report content outline tree corresponding to the third knowledge base.

[0098] Optionally, the first content corresponding to the target numerical vector can be directly used as the target content corresponding to the indicator to be extracted; or the first content can be further processed to filter out the second content that is closer to the indicator to be extracted from the first content as the target content corresponding to the indicator to be extracted.

[0099] 603. According to the semantic similarity between the first content corresponding to the target numerical vector and the indicator description information, determine the second content having a semantic similarity greater than a second similarity threshold from the first content as the target content corresponding to the indicator to be extracted.

[0100] As an optional method for determining the second content, the semantic similarity between the first content corresponding to the target numerical vector and the indicator description information of the indicator to be extracted can be calculated using, for example, a semantic similarity calculation model (such as coROM). Subsequently, the second content whose semantic similarity is greater than a custom second similarity threshold is determined from the first content to be used as the target content; alternatively, the top n (n is an integer greater than 0, and n is less than or equal to k) first contents related to the indicator description information of the indicator to be extracted are determined as the second content, that is, the first contents corresponding to the largest n semantic similarities after sorting the semantic similarities in descending order are used as the second content.

[0101] In this embodiment, a query vector generated based on the indicator description information and search terms contained in the meta-query information corresponding to the indicator to be extracted is used to obtain target content that is more relevant to the indicator to be extracted in terms of vector expression and semantic expression from the knowledge base of the target report through vector similarity calculation and semantic similarity calculation, thereby ensuring the accuracy of the obtained target content.

[0102] After obtaining the target content corresponding to the to-be-extracted indicator from the knowledge base, the to-be-extracted indicator is extracted from the target content using a large language model, which is described in detail below.

[0103] Figure 7 A flowchart of a process for extracting an index to be extracted is provided in an embodiment of the present invention, such as Figure 7 As shown, the following steps are included:

[0104] 701. Generate an extraction question corresponding to the indicator to be extracted based on the meta-query information corresponding to the indicator to be extracted.

[0105] 702. Input the target content corresponding to the extraction question and the indicator to be extracted into the first language model, so that the first language model outputs an extraction answer based on the target content.

[0106] 703. Input the extraction question and the extraction answer into the second largest language model. If the second largest language model determines that the extraction question matches the extraction answer, then determine the extraction answer as the extraction result of the indicator to be extracted.

[0107] Optionally, before inputting the target content corresponding to the indicator to be extracted into the first large language model, the target content can also be input into other large language models to filter the target content through the other large language models, for example, filtering out content that cannot extract the indicator to be extracted. For example, assuming that a certain indicator to be extracted is carbon dioxide emissions, and the target content contains the content "carbon dioxide emissions are 1,000 tons" and "carbon dioxide emissions need to be controlled", the content "carbon dioxide emissions need to be controlled" in the target content can be filtered out through other large language models.

[0108] After extracting the answer corresponding to the extraction question from the target content through the first language model (i.e., extracting the indicator to be extracted), in order to ensure the accuracy of the extraction result, the extracted answer can be verified through the second language model. Specifically, the extraction question and the extraction answer are input into the second language model. If the second language model determines that the extraction question and the extraction answer match, the extraction answer is determined to be the extraction result of the indicator to be extracted; if the second language model determines that the extraction question and the extraction answer do not match (for example, the extraction answer cannot answer the extraction question), the extraction answer is determined not to be the extraction result of the indicator to be extracted.

[0109] In addition, after determining the extraction results of the indicators to be extracted, the extraction results can also be uniformly formatted. For example, the extraction results can be stored in a structured data table to facilitate subsequent data analysis of the extracted indicators. For example, in the application scenario corresponding to the ESG report, the extraction results of the indicators to be extracted can be stored in a structured data table; then, based on this data table, the disclosure completeness of the indicators corresponding to the ESG report can be determined, the ranking of the indicators to be extracted after sorting with other publicly available ESG indicators, and the construction of an ESG indicator database, etc., wherein the ESG indicator database can include indicators disclosed in the ESG reports of different companies.

[0110] In this solution, the indicators to be extracted are extracted from the target content through multiple different large language models, and the extracted indicators to be extracted are verified to ensure the accuracy of the extraction of the indicators to be extracted.

[0111] Figure 8 A schematic diagram of an indicator extraction device provided by an embodiment of the present invention is shown in FIG. Figure 8 As shown, it includes: an acquisition module 11, a processing module 12 and an extraction module 13.

[0112] The acquisition module 11 is used to obtain a target report and preset indicator disclosure rules, wherein the target report includes the indicators to be extracted that need to be disclosed in the preset indicator disclosure rules.

[0113] The processing module 12 is used to obtain the meta-query information corresponding to the indicator to be extracted according to the preset indicator disclosure rules, and the meta-query information includes: indicator description information, search terms and prompt terms corresponding to the indicator to be extracted; extract information from the target report to generate a knowledge base corresponding to the target report.

[0114] The extraction module 13 is configured to retrieve target content corresponding to the indicator to be extracted from the knowledge base based on the indicator description information and the search term; and input the meta-query information and the target content into a large language model to obtain the indicator to be extracted from the target content by the large language model. Optionally, the meta-query information also includes prior knowledge information corresponding to the indicator to be extracted.

[0115] Optionally, the processing module 12 is specifically used to locate text paragraphs in the target report; extract multiple text paragraphs contained in the target report based on the text paragraph location results; and vectorize the multiple text paragraphs to obtain a first sub-knowledge base.

[0116] Optionally, the processing module 12 is further specifically used to perform table recognition on the target report; determine the target text corresponding to the table based on the table recognition result, and the target text includes the mapping relationship between different table data in the table; and perform vectorization processing on the target text to obtain a second sub-knowledge base.

[0117] Optionally, the processing module 12 is further specifically used to perform text visual feature recognition on the target report; determine the report content outline tree corresponding to the target report based on the text visual feature recognition result; and vectorize the report content outline tree to obtain a third sub-knowledge base.

[0118] Optionally, the extraction module 13 is specifically used to generate a query vector based on the indicator description information and the search term; determine a target numerical vector from the numerical vector whose vector similarity is greater than a first similarity threshold based on the vector similarity between the query vector and the numerical vector; and determine the target content corresponding to the indicator to be extracted based on the first content corresponding to the target numerical vector.

[0119] Optionally, the extraction module 13 is further specifically configured to determine, from the first content, a second content having a semantic similarity greater than a second similarity threshold based on the semantic similarity between the first content and the indicator description information, as the target content corresponding to the indicator to be extracted.

[0120] Optionally, the extraction module 13 is further specifically used to generate an extraction question corresponding to the indicator to be extracted based on the meta-query information; input the extraction question and the target content into a first large language model, so that the first large language model outputs an extraction answer based on the target content; input the extraction question and the extraction answer into a second large language model; if it is determined through the second large language model that the extraction question matches the extraction answer, then the extraction answer is determined to be the extraction result of the indicator to be extracted.

[0121] Figure 8 The device shown can execute the steps introduced in the aforementioned embodiments. For detailed execution process and technical effects, please refer to the description in the aforementioned embodiments and will not be repeated here.

[0122] In a possible design, an embodiment of the present invention provides a structural diagram of an electronic device, such as Figure 9 As shown, the electronic device may include: a memory 21, a processor 22, and a communication interface 23. The memory 21 stores executable code, and when the executable code is executed by the processor 22, the processor 22 can at least implement the indicator extraction method provided in the above embodiment.

[0123] In addition, an embodiment of the present invention provides a non-temporary machine-readable storage medium, on which executable code is stored. When the executable code is executed by a processor of an electronic device, the processor can at least implement the indicator extraction method provided in the aforementioned embodiment.

[0124] An embodiment of the present invention provides a computer program product, comprising: a computer program, which, when executed by a processor of an electronic device, causes the processor to execute the indicator extraction method provided in the aforementioned embodiment.

[0125] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separate. Some or all of the modules may be selected based on actual needs to achieve the objectives of the present embodiment. Those skilled in the art can understand and implement the present invention without inventive effort.

[0126] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by adding a necessary general hardware platform, and of course can also be implemented by a combination of hardware and software. Based on this understanding, the above technical solution, or the part that contributes to the prior art, can be embodied in the form of a computer product. The present invention can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0127] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. An index extraction method, characterized in that: include: Obtaining a target report and preset indicator disclosure rules, wherein the target report includes the indicators to be extracted that need to be disclosed in the preset indicator disclosure rules; According to the preset indicator disclosure rules, obtaining meta-query information corresponding to the indicator to be extracted, wherein the meta-query information includes: indicator description information, search terms, and prompt terms corresponding to the indicator to be extracted; Extracting information from the target report to generate a knowledge base corresponding to the target report; Acquire target content corresponding to the indicator to be extracted from the knowledge base according to the indicator description information and the search term; The meta-query information and the target content are input into a large language model to obtain the to-be-extracted indicator extracted by the large language model from the target content.

2. The method according to claim 1, characterized in that The meta-query information also includes: prior knowledge information corresponding to the indicator to be extracted.

3. The method according to claim 1, characterized in that The extracting information from the target report to generate a knowledge base corresponding to the target report includes: Performing text paragraph positioning on the target report; Extracting multiple text paragraphs contained in the target report according to the text paragraph positioning result; Vectorization processing is performed on the multiple text paragraphs to obtain a first sub-knowledge base.

4. The method according to claim 1, wherein The extracting information from the target report to generate a knowledge base corresponding to the target report includes: performing table recognition on the target report; Determining a target text corresponding to the table according to the table recognition result, wherein the target text includes a mapping relationship between different table data in the table; Vectorization is performed on the target text to obtain a second sub-knowledge base.

5. The method according to claim 1, wherein The extracting information from the target report to generate a knowledge base corresponding to the target report includes: performing text visual feature recognition on the target report; Determine a report content outline tree corresponding to the target report based on the text visual feature recognition result; Vectorization is performed on the report content outline tree to obtain a third sub-knowledge base.

6. The method according to claim 1, wherein The knowledge base includes numerical vectors corresponding to the extracted different contents; and obtaining target content corresponding to the to-be-extracted indicator from the knowledge base according to the indicator description information and the search term includes: generating a query vector according to the indicator description information and the search term; determining, from the numerical vectors, a target numerical vector whose vector similarity is greater than a first similarity threshold, based on the vector similarity between the query vector and the numerical vector; The target content corresponding to the indicator to be extracted is determined according to the first content corresponding to the target numerical vector.

7. The method according to claim 6, characterized in that The determining, based on the first content corresponding to the target numerical vector, the target content corresponding to the indicator to be extracted includes: According to the semantic similarity between the first content and the indicator description information, second content whose semantic similarity is greater than a second similarity threshold is determined from the first content to serve as the target content corresponding to the indicator to be extracted.

8. The method according to any one of claims 1 to 7, characterized in that Inputting the meta-query information and the target content into a large language model to obtain the to-be-extracted indicator extracted by the large language model from the target content includes: Generating an extraction question corresponding to the indicator to be extracted according to the meta-query information; inputting the extraction question and the target content into a first large language model, so that the first large language model outputs an extraction answer based on the target content; Inputting the extracted question and the extracted answer into a second language model; If it is determined through the second largest language model that the extraction question matches the extraction answer, the extraction answer is determined to be the extraction result of the indicator to be extracted.

9. An electronic device, characterized in that: include: A memory, a processor, and a communication interface; wherein the memory stores executable code, and when the executable code is executed by the processor, the processor executes the indicator extraction method according to any one of claims 1 to 8.

10. A non-transitory machine-readable storage medium, characterized in that The non-transitory machine-readable storage medium stores executable code, and when the executable code is executed by a processor of an electronic device, the processor is caused to execute the indicator extraction method according to any one of claims 1 to 8.

11. A computer program product, characterized in that include: A computer program, when executed by a processor of an electronic device, causes the processor to execute the index extraction method according to any one of claims 1 to 8.

Images