Information extraction system and information extraction method

Abstract

To enable an information extraction system to generate appropriate summaries from data to be analyzed.SOLUTION: The information extraction system includes: an extraction unit configured to extract, from data to be analyzed, multiple pieces of information related to a subject matter explained by the data to be analyzed, as explanatory information; a storage unit configured to store the explanatory information; a combination unit configured to combine and structure the extracted explanatory information; and a generation unit configured to generate summary sentences that explain the combined units from the combination source explanatory information.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an information extraction system and an information extraction method. [Background technology]

[0002] There is a conventional technology for data processing of text. JP 2023-158992 A (Patent Document 1) describes the following: "The system includes a response generation unit 105 that generates a response text for an input text, a response correction unit 107 that corrects the response text generated by the response generation unit, a response expression appropriateness evaluation unit 109 that determines whether the expression of the response text is appropriate for the user for the response text in the response text group corrected by the response correction unit, a response content appropriateness evaluation unit 111 that determines whether the content of the response text in the response text group corrected by the response correction unit is appropriate for the content of the user's past interactions, and a selection unit 113 that determines the response text to be output from the corrected response text group based on the evaluation by the response expression appropriateness evaluation unit and the evaluation by the response content appropriateness evaluation unit." [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2023-158992 Summary of the Invention [Problem to be solved by the invention]

[0004] In the prior art, large language models (LLMs) are used to generate plausible response text for input text. Data processing using such large-scale language models (large-scale language processing) can be used not only to generate response text, but also to generate document summaries. If appropriate summaries can be generated from a large number of documents or documents with a large amount of text, it will greatly contribute to the utilization of documents. However, when generating document summaries using large-scale language models, it may not be possible to generate an appropriate summary due to reasons such as the information to be extracted being scattered throughout the text or referring to multiple topics. For example, when generating a summary from a business plan document that describes both an industry trend forecast and a business budget estimate, the summary generated may only cover the budget estimate, and the information about the industry trend forecast may be missing. In this case, the summary may be inappropriate for users who need the industry trend forecast information in the document. Similarly, when generating a summary from a manual that describes operation and repair instructions, if the repair instructions are missing, the summary may be inappropriate for users who want to know the repair instructions. This problem occurs not only when a summary is generated from a document, but also when a summary is generated from any data to be analyzed, such as audio data or video data. Therefore, an object of the present invention is to generate an appropriate summary from the data to be analyzed. [Means for solving the problem]

[0005] In order to achieve the above-mentioned object, one representative information extraction system of the present invention is characterized by having an extraction unit that extracts, from the data to be analyzed, multiple pieces of information related to the matters to be explained by the data to be analyzed as explanatory information; a memory unit that stores the explanatory information; a combination unit that combines and structures the extracted explanatory information; and a generation unit that generates summary sentences that explain the combined units from the original explanatory information that was combined. Furthermore, one representative information extraction method of the present invention is characterized by including an extraction step in which a computer extracts, from the data to be analyzed, a plurality of pieces of information relating to the matters to be explained by the data to be analyzed as explanatory information; a storage step in which the computer stores the explanatory information in a memory unit; a combination step in which the computer combines and structures the extracted explanatory information; and a generation step in which the computer generates summary sentences that explain the combined units from the explanatory information that was the source of the combination. [Effects of the Invention]

[0006] According to the present invention, it is possible to generate an appropriate summary from data to be analyzed. Problems, configurations, and effects other than those described above will become apparent from the following description of the embodiments. [Brief explanation of the drawings]

[0007] [Figure 1] Configuration diagram of information extraction system according to the first embodiment [Figure 2] Specific example of data (part 1) [Figure 3] Specific data example (part 2) [Figure 4] Flowchart showing the processing procedure of the computer [Figure 5] Flowchart showing the procedure for extracting matters of interest [Figure 6] Flowchart showing the information combination process [Figure 7] Flowchart showing the summary generation process [Figure 8] Flowchart showing the computer processing procedure when information combination by distillation is adopted [Figure 9] Flowchart showing the process of information merging by distillation [Figure 10] Example of displaying and selecting a hierarchical structure DETAILED DESCRIPTION OF THE INVENTION

[0008] Hereinafter, an embodiment will be described with reference to the drawings. [Example]

[0009] Fig. 1 is a configuration diagram of an information extraction system according to Example 1. A computer 100 shown in Fig. 1 includes a CPU (Central Processing Unit) 101, a memory 102, a storage 103, an input device 104, an output device 105, and a communication device 106.

[0010] The input device 104 is, for example, a keyboard or a pointing device. The output device 105 is, for example, a liquid crystal display. The communication device 106 is a communication interface for communicating with external devices.

[0011] The storage 103 is an auxiliary storage device and is realized by a hard disk drive, etc. The storage 103 stores information source information 131, prompt information 132, extraction information 133, threshold information 134, large-scale language processing information 135, embedded calculation information 136, connection information 137, programs 151, etc.

[0012] The CPU 101 reads the program 151 from the storage 103, expands it in the memory 102, and executes it sequentially to realize various functions as an information extraction system. The functions as an information extraction system include a subject-of-interest extraction unit 121, a large-scale language processing unit 122, an information combination unit 123, an embedding calculation unit 124, and a summary generation unit 125.

[0013] The information source information 131 indicates the data to be analyzed. The prompt information 132 is a template used as input for large-scale language processing using a large-scale language model (LLM). The extracted information 133 is data indicating a sentence as explanatory information extracted from the data to be analyzed. The threshold information 134 is a threshold for clustering the explanation information. The large-scale language processing information 135 is information such as weight vectors required for a large-scale language model. The embedding calculation information 136 is information necessary for the embedding calculation. The combined information 137 indicates the structure of the combined explanatory information and the generated summary sentence.

[0014] The matter of interest extraction unit 121 is an extraction unit that reads out the data to be analyzed by referring to the information source information 131, and extracts, as explanation information, a plurality of pieces of information relating to the matter to be explained that is explained by the data to be analyzed. Specifically, the interest extraction unit 121 divides the read document into chunks of a size suitable for the first large-scale language processing, performs the first large-scale language processing on each chunk to extract sentences related to the subject to be explained as explanatory information, and registers the extracted explanatory information in the extracted information 133.

[0015] The explanation subject is the subject that is explained by the analyzed data. In other words, the explanation subject indicates what interest the user seeks information about when using the analyzed data. For example, if the data to be analyzed is a business plan document, the matters to be explained may include industry trend forecasts and budget estimates. For example, if the data to be analyzed is investor relations (IR) information, business issues and corporate strategies can be explained. For example, if the data to be analyzed is a manual, the items to be explained may include operation methods and repair methods. In this way, it is possible to predict in advance what kind of explanation subject will be used depending on the type of data to be analyzed. Also, there may be multiple explanation subject matters for one piece of data to be analyzed. To configure the information extraction system simply, it may be configured to extract specific explanation subject matters from the data to be analyzed. Alternatively, it may be configured to associate one or multiple explanation subject matters with the type of data to be analyzed as an explanation subject matter set and store them in storage 103. When an explanation subject matter set is used, the matter of interest extraction unit 121 selects an explanation subject matter set based on the type of data to be analyzed, and further extracts explanation information for the explanation subject matter selected from the explanation subject matter set.

[0016] The information combining unit 123 is a combining unit that combines and structures the extracted explanatory information. Specifically, the information combination unit 123 reads multiple pieces of explanatory information from the extracted information 133, calculates the feature values ​​of each piece of explanatory information, clusters the multiple pieces of explanatory information based on the feature values, combines them into a tree structure, and registers them in the combined information 137. The summary generating unit 125 is a generating unit that generates a summary sentence that explains the combined unit from the explanation information of the combined source. Specifically, the summary generation unit 125 uses the second large-scale language processing to summarize the contents of the explanatory information located above a predetermined node in the tree structure, generate a summary sentence, and register the summary sentence in the combined information 137. Furthermore, the summary generation unit 125 can display hierarchical information of a tree structure on the output device 105, accept the designation of a node or a hierarchy through the input device 104, and output a corresponding summary sentence.

[0017] The large-scale language processing unit 122 performs large-scale language processing using a large-scale language model (LLM). It is used internally by the subject-of-interest extraction unit 121 and the summary generation unit 125. The large-scale language processing unit 122 loads large-scale language processing information 135 and performs first large-scale language processing and second large-scale language processing. The embedding calculation unit 124 is used internally by the information combination unit 123. The embedding calculation unit 124 loads the embedding calculation information 136 and calculates the feature amount of the explanation information. The large-scale language processor 122 and the embedded computing unit 124 may be external services.

[0018] 2 and 3 are specific examples of data. As shown in FIG. 2, the information source information 131 has the following items: "id" that identifies the data to be analyzed, "Name" that is the name of the data to be analyzed, and "Location" that indicates the location of the data to be analyzed. For example, the data to be analyzed indicated by "id:00001" in information source information 131 is "Company A's 2021 Medium-Term Management Plan" located in " / aaa / bbb / ccc.txt". Although the location of the data to be analyzed is managed here, the data to be analyzed itself may also be stored.

[0019] The prompt information 132 has the following items: "id" that identifies the prompt, "Purpose" that indicates the use of the prompt, and "Prompt" that indicates the content of the prompt. For example, the prompt indicated by "id:00001" in prompt information 132 has the purpose of information extraction, and the content of the prompt is "Please extract information about {aspect} from the text below and return it in bullet points. {text}." For example, the prompt indicated by "id:00002" in the prompt information 132 has a use of combined summary, and the content of the prompt is "Please summarize the information about the extracted {aspect} below. {text}". For example, the prompt indicated by "id:00003" in prompt information 132 has the purpose of distillation, and the content of the prompt is "For the extracted {aspect} below, if there are any that have similar perspectives and can be combined, please combine them and reduce the number of items. {text}." Note that {aspect} is an argument that specifies the subject of explanation, and {text} is the chunk to be analyzed.

[0020] As shown in FIG. 3, the extracted information 133 has the following items: "id" for identifying the explanatory information, "doc_id" for identifying the data to be analyzed, and "extracted_info" for indicating the content of the explanatory information. For example, "id:00002" of the extracted information 133 is a sentence extracted from the analysis target data "doc_id:00001", and "Development of unique elements of large-scale AI systems" is registered as "extracted_info". The extraction information 133 may include an item for identifying a chunk.

[0021] The threshold information 134 can be set arbitrarily depending on the clustering method, such as the minimum distance between clusters or the maximum number of clusters. The combined information 137 has the following items: "id" that identifies the node, "parent_id" that indicates the branching point of the node in the tree structure, "extract_id" that indicates the extracted information 133, "selected" that indicates whether a summary sentence is required, and "description" that indicates the content of the sentence. "extract_id" has a value if the node is the explanatory information itself of the extracted information 133, and is "N / A" if the node is created by structuring. "selected" is "True" if a summary sentence is required, and "False" if not. If the node is the explanatory information itself of the extracted information 133, "description" is the same sentence as "extracted_info" of the extracted information 133. If the node is a node created by structuring and "selected" is "True", the summary sentence generated by the summary generation unit 125 is registered. If the node is a node created by structuring and "selected" is "False", it becomes "N / A".

[0022] In the example of FIG. 3, the nodes "id:1" and "id:2" are the extracted information 133 themselves. The nodes "id:1" and "id:2" are connected by the node "id:3". Since the node "id:3" is "selected:True", the "description" of the nodes "id:1" and "id:2" are summarized by the second large-scale language processing, and the generated summary sentence is set as the "description" of the node "id:3". In this way, nodes that are the explanatory information of the extracted information 133 themselves are joined together to form a tree structure, but for convenience, the side of the node that is the explanatory information of the extracted information 133 itself is called the branch destination or branch side, and the side of the joined node is called the branch origin or trunk side.

[0023] 4 is a flowchart showing the processing procedure of the computer 100. The computer 100 sequentially executes interest extraction (step S101), information combination (step S102), and summary generation (step S103). These steps will be described in detail below.

[0024] 5 is a flowchart showing the procedure for extracting a matter of interest. The matter of interest extraction unit 121 executes the following steps S201 to S206. In step S201, the matter of interest extraction unit 121 acquires the storage location of the specified document from the information source information 131 and reads the document, and then proceeds to step S202. In step S202, the subject-of-interest extraction unit 121 performs chunking. In step S202, the input text is chunked into granularity that is sufficiently smaller than the number of tokens that can be handled in large-scale language processing. Then, the process proceeds to step S203. In step S203, the matter of interest extractor 121 extracts a template of an information extraction prompt from the prompt information 132. Thereafter, steps S204 to S206 are executed for each chunk.

[0025] In step S204, the matter of interest extractor 121 generates a prompt by inserting information into the argument of the information extraction prompt template, and then proceeds to step S205. In step S205, the large-scale language processing unit 122 inputs the prompt into the large-scale language model (LLM) and obtains the result, and then the process proceeds to step S206. In step S206, the matter-of-interest extractor 121 stores the acquired information in the extracted information 133. For each chunk, zero or more pieces of extracted information are obtained. After executing steps S204 to S206 for each chunk, the matter of interest extraction unit 121 ends the flow of matter of interest extraction.

[0026] 6 is a flowchart showing the procedure for information combination. The information combination unit 123 sequentially executes the following steps S301 to S305. In step S301, the information combination unit 123 acquires explanatory information derived from the relevant document from the extracted information 133. After that, the process proceeds to step S302. In step S302, the embedding calculation unit 124 calculates the text embedding of each piece of explanatory information. This allows the feature amount of each piece of explanatory information to be obtained. Then, the process proceeds to step S303. In step S303, the information combination unit 123 acquires the threshold information 134. Then, the process proceeds to step S304. In step S304, the information combination unit 123 performs clustering by embedding using the threshold information 134. As a result, the explanatory information is structured based on the feature amounts into a tree structure. Then, the process proceeds to step S305. In step S305, the information combination unit 123 stores the clustering results (tree structure) in the combination information 137. When the extracted information itself, that is, the explanatory information, is used as a node, the description is also stored. For cluster nodes, that is, nodes that require a summary sentence, True is stored in selected. At this point, description can be left blank for information other than the extracted information, and the structural information (columns up to selected) will be filled in. After step S305, the information combination unit 123 ends the information combination flow.

[0027] 7 is a flowchart showing the procedure for generating a summary. The summary generating unit 125 sequentially executes the following steps S401 to S406. In step S401, the summary generator 125 refers to the combined information 137 and acquires all rows in the document where selected is True, and then executes steps S402 to S406 for each row.

[0028] In step S402, the summary generation unit 125 follows the branch side, i.e., the branch destination, using a depth-first search or the like to acquire all of the shallowest nodes that contain descriptions, and obtains a list of descriptions. If there is a node on the branch side for which a summary sentence has already been generated, that node becomes the shallowest node, and no further search is necessary. If there is a node on the branch side for which a summary sentence has already been generated, the node will be traced back to the node containing the explanatory information. After obtaining the list of descriptions, the process proceeds to step S403.

[0029] In step S403, the summary generator 125 obtains the combined summary prompt template from the prompt information 132. After that, the process proceeds to step S404. In step S404, the summary generator 125 generates a prompt by inserting information into the arguments of the combined prompt template, and then proceeds to step S405. In step S405, the large-scale language processing unit 122 inputs the prompt into the large-scale language model (LLM) and obtains the result, and then the process proceeds to step S406. In step S406, the summary generator 125 stores the acquired information in the description of the corresponding line of the combined information 137. After executing steps S404 to S406 for each row, summary generation unit 125 ends the summary generation flow.

[0030] So far, an example has been shown in which multiple pieces of explanatory information are clustered and combined. As another example, information may be combined by distillation, which involves repeatedly combining multiple pieces of explanatory information.

[0031] 8 is a flowchart showing the processing procedure of the computer 100 when information combination by distillation is adopted. The computer 100 performs information combination by distillation (step S502) after extracting a matter of interest (step S501). The matter of interest extraction (step S501) is the same as the matter of interest extraction (step S101) shown in FIG. 4, so a description thereof will be omitted.

[0032] 9 is a flowchart showing the procedure for information combination by distillation. The information combination unit 123 executes the following steps S601 to S605. In step S601, the information combining unit 123 obtains explanatory information corresponding to the relevant document from the extracted information 133. After that, the process proceeds to step S602. In step S602, the information combining unit 123 acquires the distillation prompt template from the prompt information 132. Thereafter, steps S603 to S604 are repeatedly executed multiple times. The information combination unit 123 divides all the current explanatory information into a set of a certain number of explanatory information, and repeatedly executes step S603 for each set of explanatory information. In step S603, the information combining unit 123 inserts information into the distilled prompt template to generate a prompt. This prompt is input into the large-scale language model (LLM) by the large-scale language processing unit 122. After step S603 is executed for each explanation information set, the process proceeds to step S604. In step S604, the information combining unit 123 combines the generated explanation information lists to obtain the current entire explanation information. The information combination unit 123 executes the loop of steps S603 to S604 a plurality of times, and then proceeds to step S605. The number of times can be set appropriately, for example, to "execute 10 times" or "execute until the total number reaches 20." In step S605, the information combination unit 123 stores the distillation result in the combined information 137, and ends the information combination process by distillation. At this time, "Parent" is unnecessary, and "Selected" is set to "True."

[0033] FIG. 10 shows a specific example of display and selection of a hierarchical structure. In FIG. 10, the tree structure of the connection information 137 is displayed as a model. In the tree structure, the branch destination is on the right side and the branch source is on the left side. A bar is displayed on the screen, and a hierarchy in the tree can be specified by moving the bar left or right. When an operation to specify a hierarchy in the tree structure in this way or to directly specify a node is accepted, the computer 100 generates a summary sentence corresponding to the specified hierarchy or node. This type of control makes it possible to easily obtain summary sentences with the granularity desired by the user.

[0034] As described above, the disclosed information extraction system includes an interest extraction unit 121 as an extraction unit that extracts, from the data to be analyzed, multiple pieces of information related to the matters to be explained that are explained by the data to be analyzed as explanatory information, a storage 103 as a memory unit that stores the explanatory information, an information combination unit 123 as a combination unit that combines and structures the extracted explanatory information, and a summary generation unit 125 as a generation unit that generates summary sentences that explain the combined units from the explanatory information that is the source of the combination. This allows for the generation of an appropriate summary from the data being analyzed.

[0035] In addition, the extraction unit divides the document, which is the data to be analyzed, into chunks of a size suitable for first large-scale language processing, and performs the first large-scale language processing on each chunk to extract sentences related to the subject to be explained as the explanatory information, the combination unit calculates features for each of the multiple pieces of explanatory information, clusters the multiple pieces of explanatory information based on the features, and combines them into a tree structure, and the generation unit summarizes the contents of the explanatory information located beyond a specified node in the tree structure using second large-scale language processing to generate the summary sentence. Therefore, by utilizing LLM, it is possible to generate appropriate summaries from large amounts of text.

[0036] The generation unit also displays hierarchical information of the tree structure, accepts the designation of the node or hierarchy, and outputs a corresponding summary sentence. This allows the user to specify the granularity of the summary as appropriate.

[0037] The storage unit stores a set of explanation items corresponding to the type of the data to be analyzed, and the extraction unit extracts the explanation information for an explanation item selected from the set of explanation items. For data to be analyzed, such as documents, the content of inquiries (interests) can be narrowed down to a certain extent. In other words, which documents are used for what purposes can be narrowed down to a certain extent. Therefore, by narrowing down the items to be explained according to the type of document and structuring them in advance, it is possible to efficiently obtain summaries from the data to be analyzed.

[0038] The extraction unit extracts the explanatory information for a plurality of pieces of data to be analyzed, the storage unit stores the explanatory information for a plurality of pieces of data to be analyzed, and the combining unit performs the structuring for each piece of data to be analyzed. Therefore, it is possible to extract explanatory information from a large amount of data to be analyzed in advance, and obtain a summary of the necessary data to be analyzed. Furthermore, the data to be analyzed can be hierarchically organized to obtain summaries for each layer. For example, if multiple small, self-contained stories form a single large story, you can specify the data to be analyzed corresponding to each small story and generate summaries individually, or you can specify all the data to be analyzed included in the large story and generate a summary for the entire story.

[0039] The present invention is not limited to the above-described embodiments, but includes various modifications. For example, the above-described embodiments have been described in detail to clearly explain the present invention, and the present invention is not necessarily limited to those including all of the described configurations. Furthermore, not only can the configurations be deleted, but also replacements and additions of configurations are possible. For example, in the above embodiment, a configuration was shown in which one computer also performed LLM and text mining, but LLM and text mining may be outsourced to a separate system. [Explanation of symbols]

[0040] 100: Computer, 101: CPU, 102: Memory, 103: Storage, 104: Input device, 105: Output device, 106: Communication device, 121: Interest extraction unit, 122: Large-scale language processing unit, 123: Information combination unit, 124: Embedding calculation unit, 125: Summary generation unit, 131: Information source information, 132: Prompt information, 133: Extraction information, 134: Threshold information, 135: Information for large-scale language processing, 136: Information for embedding calculation, 137: Combination information, 151: Program

Claims

1. an extraction unit that extracts, from the analysis target data, a plurality of pieces of information related to an explanation target matter that is explained by the analysis target data as explanation information; a storage unit that stores the explanatory information; a combining unit that combines and structures the extracted explanatory information; a generating unit that generates a summary sentence explaining the combined unit from the combined unit's explanation information; An information extraction system comprising:

2. 2. The information extraction system according to claim 1, the extraction unit divides the document that is the analysis target data into chunks of a size suitable for a first large-scale language processing, and performs the first large-scale language processing on each chunk to extract sentences related to the matter to be explained as the explanation information; the combining unit calculates a feature amount for each of a plurality of pieces of explanatory information, clusters the plurality of pieces of explanatory information based on the feature amount, and combines the clustered pieces of explanatory information into a tree structure; The information extraction system is characterized in that the generation unit generates the summary sentence by summarizing the contents of explanatory information located beyond a predetermined node in the tree structure using a second large-scale language processing.

3. 3. The information extraction system according to claim 2, The information extraction system is characterized in that the generation unit displays hierarchical information of the tree structure, accepts specification of the node or hierarchy, and outputs a corresponding summary sentence.

4. 2. The information extraction system according to claim 1, the storage unit stores an explanation item set corresponding to the type of the analysis target data; The information extraction system is characterized in that the extraction unit extracts the explanation information for an explanation target item selected from the explanation target item set.

5. 2. The information extraction system according to claim 1, the extraction unit extracts the explanatory information for a plurality of pieces of analysis target data; the storage unit stores the explanatory information for a plurality of pieces of analysis target data; The combining unit performs the structuring for each of the analysis target data. An information extraction system comprising:

6. an extraction step in which a computer extracts, from the analysis target data, a plurality of pieces of information relating to an explanation target matter explained by the analysis target data as explanation information; a storing step of the computer storing the explanation information in a storage unit; a combining step in which the computer combines and structures the extracted explanatory information; a generating step in which the computer generates a summary sentence explaining the combined unit from the explanation information of the combined unit; An information extraction method comprising:

Images