Information extraction system and information extraction method

Abstract

This information extraction system addresses the problem of generating an appropriate summary from data to be analyzed. The information extraction system is characterized by comprising: an extraction part that extracts, as explanatory information items from analysis target data, a plurality of items of information regarding explanatory information target matters which are explained by the analysis target data; a storage part that stores the explanatory information items; a coupling part that couples the extracted explanatory information items and structures the same; and a generation part that generates, from the explanatory information items of a coupling source, a summary sentence explaining the coupled unit.

Description

Information extraction system and information extraction method

[0001] The present invention relates to an information extraction system and an information extraction method.

[0002] Japanese Patent Laid-Open Publication No. 2023-158992 (Patent Document 1) describes a technology for data processing of text, which includes a response generation unit 105 that generates a response text in response to an input text, a response correction unit 107 that corrects the response text generated by the response generation unit, a response expression appropriateness evaluation unit 109 that determines whether the expression of the response text in the response text group corrected by the response correction unit is appropriate for the user, a response content appropriateness evaluation unit 111 that determines whether the content of the response text in the response text group corrected by the response correction unit is appropriate for the content of past exchanges between the user, and a selection unit 113 that determines the response text to be output from the corrected response text group based on the evaluation by the response expression appropriateness evaluation unit and the evaluation by the response content appropriateness evaluation unit.

[0003] JP 2023-158992 A

[0004] In conventional technology, large-scale language models (LLMs) are used to generate appropriate response text for input text. Data processing using such large-scale language models (large-scale language processing) can be utilized not only for generating response text but also for generating document summaries. Generating appropriate summaries from a large number of documents or documents with a large amount of text would greatly contribute to document utilization. However, when generating document summaries using large-scale language models, an appropriate summary may not be generated due to reasons such as the information to be extracted being scattered throughout the text or the content referring to multiple contents. For example, when generating a summary from a business plan document that describes both an industry trend forecast and a business budget estimate, the summary generated may include only the budget estimate, but the content regarding the industry trend forecast may be missing. In this case, the summary may be inappropriate for users who need the content regarding the industry trend forecast in the document. Furthermore, when generating a summary from a manual that describes operation and repair methods, if the repair method is missing, the summary may be inappropriate for users who want to know the repair method. This problem occurs not only when generating summaries from documents, but also when generating summaries from any data to be analyzed, such as audio data, video data, etc. Therefore, the object of the present invention is to generate appropriate summaries from data to be analyzed.

[0005] To achieve the above object, one representative information extraction system of the present invention is characterized by having an extraction unit that extracts, from analysis target data, multiple pieces of information related to explanation targets explained by the analysis target data as explanatory information, a storage unit that stores the explanatory information, a combination unit that combines and structures the extracted explanatory information, and a generation unit that generates summary sentences explaining the combined units from the combination source explanatory information.Furthermore, one representative information extraction method of the present invention is characterized by including an extraction step in which a computer extracts, from analysis target data, multiple pieces of information related to explanation targets explained by the analysis target data as explanatory information, a storage step in which the computer stores the explanatory information in a storage unit, a combination step in which the computer combines and structures the extracted explanatory information, and a generation step in which the computer generates summary sentences explaining the combined units from the combination source explanatory information.

[0006] According to the present invention, it is possible to generate an appropriate summary from data to be analyzed. Problems, configurations, and effects other than those described above will become apparent from the following description of the embodiments.

[0007] Configuration diagram of the information extraction system of the first embodiment Specific example of data (part 1) Specific example of data (part 2) Flowchart showing the computer processing procedure Flowchart showing the processing procedure of interest extraction Flowchart showing the processing procedure of information combination Flowchart showing the processing procedure of summary generation Flowchart showing the computer processing procedure when information combination by distillation is adopted Flowchart showing the processing procedure of information combination by distillation Specific example of display and selection of hierarchical structure

[0008] Hereinafter, an embodiment will be described with reference to the drawings.

[0009] Fig. 1 is a configuration diagram of an information extraction system according to Example 1. A computer 100 shown in Fig. 1 includes a CPU (Central Processing Unit) 101, a memory 102, a storage 103, an input device 104, an output device 105, and a communication device 106.

[0010] The input device 104 is, for example, a keyboard or a pointing device. The output device 105 is, for example, a liquid crystal display. The communication device 106 is a communication interface for communicating with external devices.

[0011] The storage 103 is an auxiliary storage device and is realized by a hard disk drive, etc. The storage 103 stores information source information 131, prompt information 132, extraction information 133, threshold information 134, large-scale language processing information 135, embedding calculation information 136, connection information 137, programs 151, etc.

[0012] The CPU 101 reads the program 151 from the storage 103, expands it in the memory 102, and executes it sequentially to realize various functions of the information extraction system. The functions of the information extraction system include a subject-of-interest extraction unit 121, a large-scale language processing unit 122, an information combination unit 123, an embedding calculation unit 124, and a summary generation unit 125.

[0013] The information source information 131 indicates the data to be analyzed. The prompt information 132 is a template used for inputting large-scale language processing using a large-scale language model (LLM). The extraction information 133 is data indicating sentences as explanatory information extracted from the data to be analyzed. The threshold information 134 is a threshold for clustering the explanatory information. The large-scale language processing information 135 is information such as weight vectors required for the large-scale language model. The embedding calculation information 136 is information required for embedding calculation. The combination information 137 indicates the structure of the combined explanatory information and the generated summary sentence.

[0014] The matter-of-interest extraction unit 121 is an extraction unit that reads the data to be analyzed by referring to the information source information 131, and extracts, as explanatory information, a plurality of pieces of information related to the matters to be explained explained by the data to be analyzed. Specifically, the matter-of-interest extraction unit 121 divides the read document into chunks of a size suitable for first large-scale language processing, performs the first large-scale language processing on each chunk, and extracts sentences related to the matters to be explained as explanatory information, and registers the extracted explanatory information in the extracted information 133.

[0015] An explanation item is an item that is explained by the analysis target data. In other words, the explanation item indicates what interest a user seeks information about when using the analysis target data. For example, if the analysis target data is a business plan document, the explanation items may include industry trend forecasts and budget estimates. For example, if the analysis target data is investor relations (IR) information, the explanation items may include business issues and corporate strategies. For example, if the analysis target data is a manual, the explanation items may include operation methods and repair methods. In this way, it is possible to predict in advance what explanation items will be used depending on the type of analysis target data. Furthermore, one analysis target data may have multiple explanation items. A simple configuration of the information extraction system may be one that extracts specific explanation items from the analysis target data. Furthermore, one or more explanation items may be associated with a type of analysis target data as an explanation item set and stored in storage 103. When using a set of explanation items, the matter of interest extraction unit 121 selects a set of explanation items based on the type of data to be analyzed, and then extracts explanation information for the explanation items selected from the set of explanation items.

[0016] The information combining unit 123 is a combining unit that combines and structures the extracted explanatory information. Specifically, the information combining unit 123 reads multiple pieces of explanatory information from the extracted information 133, calculates feature quantities for each piece of explanatory information, clusters the multiple pieces of explanatory information based on the feature quantities, combines them into a tree structure, and registers the combined information 137. The summary generating unit 125 is a generating unit that generates summary sentences that explain the combined units from the combined explanatory information. Specifically, the summary generating unit 125 aggregates the content of the explanatory information located beyond a predetermined node in the tree structure using second large-scale language processing to generate summary sentences, and registers the summary sentences in the combined information 137. The summary generating unit 125 can also display hierarchical information of the tree structure on the output device 105, accept designation of a node or layer via the input device 104, and output the corresponding summary sentences.

[0017] The large-scale language processing unit 122 performs large-scale language processing using a large-scale language model (LLM). It is used internally by the subject-of-interest extraction unit 121 and the summary generation unit 125. The large-scale language processing unit 122 loads large-scale language processing information 135 and performs first large-scale language processing and second large-scale language processing. The embedding calculation unit 124 is used internally by the information combination unit 123. The embedding calculation unit 124 loads embedding calculation information 136 and calculates features of the explanation information. The large-scale language processing unit 122 and the embedding calculation unit 124 may be external services.

[0018] 2 and 3 are specific examples of data. As shown in FIG. 2, the information source information 131 has the following items: "id" that identifies the data to be analyzed, "Name" that is the name of the data to be analyzed, and "Location" that indicates the location of the data to be analyzed. For example, the data to be analyzed indicated by "id:00001" in the information source information 131 is "Company A's 2021 Medium-Term Management Plan" located in " / aaa / bbb / ccc.txt". Note that although the location of the data to be analyzed is managed here, the data to be analyzed itself may also be stored.

[0019] The prompt information 132 has fields for an "id" that identifies the prompt, a "Purpose" that indicates the purpose of the prompt, and a "Prompt" that indicates the content of the prompt. For example, the prompt indicated by "id:00001" in the prompt information 132 has the purpose of information extraction and the content of the prompt as follows: "Please extract information about {aspect} from the following sentence and return it in bullet points. {text}." For example, the prompt indicated by "id:00002" in the prompt information 132 has the purpose of combined summary and the content of the prompt as follows: "Please summarize the information about the extracted {aspect} below. {text}." For example, the prompt indicated by "id:00003" in the prompt information 132 has the purpose of distillation and the content of the prompt as follows: "For the extracted {aspect} below, if there are any that have similar perspectives and can be summarized, please summarize them to reduce the number of items. {text}." Note that {aspect} is an argument that specifies the matter to be explained, and {text} is the chunk to be analyzed.

[0020] 3, the extracted information 133 has the following fields: "id" for identifying the explanatory information, "doc_id" for identifying the data to be analyzed, and "extracted_info" for indicating the content of the explanatory information. For example, "id:00002" in the extracted information 133 is a sentence extracted from the data to be analyzed, which is "doc_id:00001," and "Development of unique elements for large-scale AI systems" is registered as "extracted_info." The extracted information 133 may also include a field for identifying chunks.

[0021] The threshold information 134 can be set arbitrarily depending on the clustering method. For example, it may be the minimum distance between clusters or the maximum number of clusters. The connection information 137 has the following items: "id" for identifying the node, "parent_id" indicating the branching point of the node in the tree structure, "extract_id" indicating the extracted information 133, "selected" indicating whether a summary sentence is required, and "description" indicating the content of the sentence. "extract_id" has a value if the node is the description information itself of the extracted information 133, and is "N / A" if the node is created by structuring. "selected" is "True" if a summary sentence is required, and "False" if it is not required. If the node is the description information itself of the extracted information 133, "description" is the same sentence as "extracted_info" of the extracted information 133. If the node is a node created by structuring and "selected" is "True", the summary sentence generated by the summary generation unit 125 is registered. If the node is a node created by structuring and "selected" is "False", "N / A" is registered.

[0022] In the example of FIG. 3 , nodes "id:1" and "id:2" are the extracted information 133 itself. Nodes "id:1" and "id:2" are connected by node "id:3." Because node "id:3" has "selected:True," the "descriptions" of nodes "id:1" and "id:2" are summarized by the second large-scale language processing, and the generated summary sentence is the "description" of node "id:3." In this way, nodes that are the explanatory information of the extracted information 133 itself are connected to form a tree structure. For convenience, the side of the node that is the explanatory information of the extracted information 133 itself is referred to as the branch destination or branch side, and the side of the connected node is referred to as the branch origin or trunk side.

[0023] 4 is a flowchart showing the processing procedure of the computer 100. The computer 100 sequentially executes interest extraction (step S101), information combination (step S102), and summary generation (step S103). These steps will be described in detail below.

[0024] 5 is a flowchart showing the procedure for extracting a matter of interest. The matter of interest extraction unit 121 executes the following steps S201 to S206. Step S201: The matter of interest extraction unit 121 acquires the storage location of the specified document from the information source information 131 and reads the document. Then, the process proceeds to step S202. Step S202: The matter of interest extraction unit 121 performs chunking. In step S202, the input text is chunked into granularity that is sufficiently smaller than the number of tokens that can be handled by large-scale language processing. Then, the process proceeds to step S203. Step S203: The matter of interest extraction unit 121 extracts a template for an information extraction prompt from the prompt information 132. Then, steps S204 to S206 are executed for each chunk.

[0025] In step S204, the subject of interest extraction unit 121 inserts information into the arguments of the information extraction prompt template to generate a prompt. Then, the process proceeds to step S205. In step S205, the large-scale language processing unit 122 inputs the prompt into the large-scale language model (LLM) and obtains the results. Then, the process proceeds to step S206. In step S206, the subject of interest extraction unit 121 stores the obtained information in the extracted information 133. Zero or more pieces of extracted information are obtained per chunk. After executing steps S204 to S206 for each chunk, the subject of interest extraction unit 121 ends the subject of interest extraction flow.

[0026] FIG. 6 is a flowchart showing the information combination process. The information combination unit 123 sequentially executes the following steps S301 to S305. Step S301: The information combination unit 123 acquires explanatory information derived from the relevant document from the extracted information 133. Then, the process proceeds to step S302. Step S302: The embedding calculation unit 124 calculates text embedding for each explanatory information. As a result, feature values ​​for each explanatory information are obtained. Then, the process proceeds to step S303. Step S303: The information combination unit 123 acquires threshold information 134. Then, the process proceeds to step S304. Step S304: The information combination unit 123 uses the threshold information 134 to perform clustering by embedding. As a result, the explanatory information is structured based on the feature values ​​and becomes a tree structure. Then, the process proceeds to step S305. Step S305: The information combination unit 123 stores the clustering results (tree structure) in the combined information 137. If the extracted information itself, i.e., the explanatory information itself, is used as a node, the description is also stored. For nodes of the cluster, i.e., nodes requiring summary sentences, True is stored in selected. At this point, description can be empty except for the extracted information, and the structural information (columns up to selected) will be filled in. After step S305, the information combining unit 123 ends the information combining flow.

[0027] 7 is a flowchart showing the summary generation process. The summary generation unit 125 sequentially executes the following steps S401 to S406. Step S401: The summary generation unit 125 references the combined information 137 and acquires all rows in the document where selected is True. Then, steps S402 to S406 are executed for each row.

[0028] In step S402, the summary generation unit 125 follows the branch side, i.e., the branch destination, using a depth-first search or the like, acquires all of the shallowest nodes described in the description, and obtains a list of descriptions. If there is a node on the branch side for which a summary sentence has already been generated, that node becomes the shallowest node, and no further search is necessary. If there is a node on the branch side for which a summary sentence has already been generated, the process will be traced to the node of the explanatory information. After obtaining the list of descriptions, proceed to step S403.

[0029] Step S403: The summary generation unit 125 obtains a combined summary prompt template from the prompt information 132. Then, proceed to step S404. Step S404: The summary generation unit 125 inserts information into the arguments of the combined prompt template to generate a prompt. Then, proceed to step S405. Step S405: The large-scale language processing unit 122 inputs the prompt into the large-scale language model (LLM) and obtains the results. Then, proceed to step S406. Step S406: The summary generation unit 125 stores the obtained information in the description of the corresponding row in the combined information 137. After executing steps S404 to S406 for each row, the summary generation unit 125 ends the summary generation flow.

[0030] So far, an example has been shown in which multiple pieces of explanatory information are clustered and combined. As another example, information may be combined by distillation, which involves repeatedly combining multiple pieces of explanatory information.

[0031] 8 is a flowchart showing the processing procedure of the computer 100 when information combination by distillation is adopted. The computer 100 performs information combination by distillation (step S502) after extracting a matter of interest (step S501). The matter of interest extraction (step S501) is similar to the matter of interest extraction (step S101) shown in FIG. 4, so a description thereof will be omitted.

[0032] FIG. 9 is a flowchart showing the processing procedure for information combination by distillation. The information combination unit 123 executes the following steps S601 to S605. Step S601: The information combination unit 123 acquires explanatory information corresponding to the relevant document from the extracted information 133. Then, the process proceeds to step S602. Step S602: The information combination unit 123 acquires a distilled prompt template from the prompt information 132. Then, the process repeats steps S603 to S604 multiple times. The information combination unit 123 divides the current entire explanatory information into a fixed number of sets and repeatedly executes step S603 for each explanatory information set. Step S603: The information combination unit 123 inserts information into the distilled prompt template to generate a prompt. This prompt is input into a large-scale language model (LLM) by the large-scale language processing unit 122. After executing step S603 for each explanatory information set, the process proceeds to step S604. Step S604: The information combination unit 123 combines the generated explanatory information lists to generate the current entire explanatory information. The information combining unit 123 executes the loop of steps S603 to S604 multiple times, and then proceeds to step S605. The number of times can be set appropriately, for example, to "10 times" or "until the total number reaches 20." In step S605, the information combining unit 123 stores the distillation result in the combined information 137, and ends the information combining process by distillation. At this time, "Parent" is unnecessary, and "Selected" is set to "True."

[0033] FIG. 10 shows a specific example of display and selection of a hierarchical structure. In FIG. 10, the tree structure of the connection information 137 is displayed as a model. In the tree structure, the branch destination is on the right side and the branch source is on the left side. A bar is displayed on the screen, and a hierarchy in the tree can be specified by moving the bar left or right. When an operation to specify a hierarchy in the tree structure in this way or to directly specify a node is accepted, the computer 100 generates a summary sentence corresponding to the specified hierarchy or node. This type of control makes it possible to easily obtain summary sentences with the granularity desired by the user.

[0034] As described above, the disclosed information extraction system includes the interest extraction unit 121 as an extraction unit that extracts, from the analysis target data, a plurality of pieces of information related to the explanation subject explained by the analysis target data as explanation information, the storage 103 as a memory unit that stores the explanation information, the information combination unit 123 as a combination unit that combines and structures the extracted explanation information, and the summary generation unit 125 as a generation unit that generates summary sentences that explain the combined units from the combination source explanation information. Thus, an appropriate summary can be generated from the analysis target data.

[0035] The extraction unit divides the document, which is the data to be analyzed, into chunks of a size suitable for first large-scale language processing, and performs the first large-scale language processing on each chunk to extract sentences related to the subject to be explained as the explanatory information, the combination unit calculates feature amounts for each of the multiple pieces of explanatory information, clusters the multiple pieces of explanatory information based on the feature amounts, and combines them into a tree structure, and the generation unit generates the summary sentence by summarizing the content of the explanatory information located beyond a predetermined node in the tree structure using second large-scale language processing. Thus, by utilizing LLM, it is possible to generate appropriate summaries from a large amount of text.

[0036] The generation unit displays hierarchical information of the tree structure, accepts the designation of the node or layer, and outputs a corresponding summary sentence, thereby allowing the user to appropriately designate the granularity of the summary.

[0037] The storage unit also stores a set of explanation items according to the type of data to be analyzed, and the extraction unit extracts the explanation information for an explanation item selected from the set of explanation items. For data to be analyzed, such as documents, the content of inquiries (interests) can be narrowed down to a certain extent. In other words, the purpose for which a document is used can be narrowed down to a certain extent. Therefore, by narrowing down the explanation items according to the type of document and structuring them in advance, summaries can be efficiently obtained from the data to be analyzed.

[0038] The extraction unit extracts the explanatory information for multiple pieces of analysis target data, the storage unit stores the explanatory information for multiple pieces of analysis target data, and the combining unit performs the structuring for each piece of analysis target data. Therefore, explanatory information can be extracted in advance for a large amount of analysis target data, allowing summaries to be obtained for the necessary analysis target data. Furthermore, the analysis target data can be hierarchically organized to obtain summaries for each layer. For example, if multiple small, self-contained stories form a single large story, the analysis target data corresponding to each small story can be specified and summaries can be generated individually, or all of the analysis target data included in the large story can be specified and an overall summary can be generated.

[0039] The present invention is not limited to the above-described embodiments, and various modifications are possible. For example, the above-described embodiments have been described in detail to clearly explain the present invention, and the present invention is not necessarily limited to those including all of the described configurations. Furthermore, not only can such configurations be deleted, but also replacement or addition of configurations is possible. For example, in the above-described embodiments, a configuration in which one computer performs both LLM and text mining is shown, but LLM and text mining may also be outsourced to a separate system.

[0040] 100: Computer, 101: CPU, 102: Memory, 103: Storage, 104: Input device, 105: Output device, 106: Communication device, 121: Interest extraction unit, 122: Large-scale language processing unit, 123: Information combination unit, 124: Embedding calculation unit, 125: Summary generation unit, 131: Information source information, 132: Prompt information, 133: Extraction information, 134: Threshold information, 135: Information for large-scale language processing, 136: Information for embedding calculation, 137: Combination information, 151: Program

Claims

1. An information extraction system comprising: an extraction unit that extracts, from the data to be analyzed, a plurality of pieces of information relating to the subject matter to be explained by the data to be analyzed as explanatory information; a storage unit that stores the explanatory information; a combination unit that combines and structures the extracted explanatory information; and a generation unit that generates summary sentences that explain the combined units from the explanatory information that was the source of the combination.

2. An information extraction system as described in claim 1, wherein the extraction unit divides the document, which is the data to be analyzed, into chunks of a size suitable for first large-scale language processing, and performs the first large-scale language processing on each chunk to extract sentences related to the subject to be explained as the explanatory information; the combination unit calculates features for each of the multiple pieces of explanatory information, clusters the multiple pieces of explanatory information based on the features, and combines them into a tree structure; and the generation unit summarizes the contents of the explanatory information located beyond a specified node in the tree structure using second large-scale language processing to generate the summary sentence.

3. An information extraction system as described in claim 2, characterized in that the generation unit displays hierarchical information of the tree structure, accepts the specification of the node or hierarchy, and outputs a corresponding summary sentence.

4. An information extraction system as described in claim 1, characterized in that the memory unit stores a set of items to be explained according to the type of data to be analyzed, and the extraction unit extracts the explanatory information for items to be explained selected from the set of items to be explained.

5. An information extraction system according to claim 1, characterized in that: the extraction unit extracts the explanatory information for a plurality of pieces of data to be analyzed; the storage unit stores the explanatory information for a plurality of pieces of data to be analyzed; and the combining unit performs the structuring for each piece of data to be analyzed.

6. An information extraction method comprising: an extraction step in which a computer extracts, from the data to be analyzed, a plurality of pieces of information relating to the matter to be explained by the data to be analyzed as explanatory information; a storage step in which the computer stores the explanatory information in a memory unit; a combination step in which the computer combines and structures the extracted explanatory information; and a generation step in which the computer generates summary sentences that explain the combined units from the explanatory information that was the source of the combination.

Images