Information processing device, information processing method, and program

Abstract

To suppress information deficiencies and prevent the generation of erroneous summary data when generating summary data from processed sentence data. [Solution] The information processing device comprises: a summary sentence data generation unit that performs summarization of the target sentence data multiple times according to processing instruction data and generates multiple summary sentence data for each of the multiple categories constituting the target sentence data; an extraction unit that extracts key information contained in the multiple summary sentence data for each of the categories constituting the target sentence data; and a summary sentence data selection unit that uses the key information for each category to select a summary result to be output from the multiple summary sentence data for each of the categories constituting the target sentence data, wherein the summary sentence data generation unit uses the key information contained in the selected summary result to regenerate the summary result using a large-scale language model.

Description

Technical Field

[0001] The present disclosure relates to an information processing apparatus, an information processing method, and a program.

Background Art

[0002] While large language models (LLMs) can generate high-quality summaries, the content may vary for each summary generation, and appropriate information may be missed. In addition, large language models tend to generate information not included in the input document, called hallucination. Conventionally, a step-by-step method (two-step summarization) has been proposed to generate a highly explainable summary by first extracting the text used for the summary from the input document and then synthesizing them (see, for example, Non-Patent Document 1).

Prior Art Documents

Non-Patent Documents

[0003]

Non-Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, in two-step summarization using a large language model, if appropriate information is not extracted in the first-step summary, appropriate information will not be extracted in the final result either. In addition, since texts extracted from different parts of the input document are synthesized, the language model performing the synthesis may not be able to grasp the original context and may generate incorrect information.

[0005] The purpose of this disclosure is to provide an information processing device, an information processing method, and a program that can suppress information deficiencies and prevent the generation of erroneous summary data when generating summary data from sentence data to be processed. [Means for solving the problem]

[0006] This disclosure comprises the following configuration.

[0007] [1] A summary sentence data generation unit that generates multiple summary sentence data from the sentence data to be processed according to the processing instruction data, An extraction unit that extracts key information contained in the plurality of summary sentence data for each category that constitutes the sentence data to be processed, A summary sentence data selection unit selects the summary sentence data to be output from the plurality of summary sentence data using the key information for each category, An information processing device having

[0008] [2] The summary sentence data generation unit generates a plurality of summary sentence data for each category from the sentence data to be processed according to the processing instruction data. The information processing apparatus described in [1], characterized by the following:

[0009] [3] The system further comprises an importance calculation unit that calculates the importance of the key information, The summary data selection unit selects the summary data to be output from the plurality of summary data using the importance level. The information processing apparatus according to [1] or [2], characterized by the following:

[0010] [4] The summary data selection unit, Using the importance of the aforementioned key information, the importance of each of the multiple summary sentence data is calculated. From the aforementioned multiple summary data, select the first summary data with the highest importance. Using the importance of the remaining key information after removing the key information contained in the first summary data, the importance of each of the remaining summary data after removing the first summary data is calculated. Select the second summary data with the highest importance from the remaining summary data. The information processing apparatus described in [3], characterized by the following:

[0011] [5] The summary sentence data generation unit generates multiple summary sentence data from the same sentence data to be processed according to different processing instruction data. An information processing device as described in any one of [1] to [4], characterized by [1].

[0012] [6] The summary sentence data generation unit generates multiple summary sentence data from different sentence data to be processed according to the same processing instruction data. An information processing device as described in any one of [1] to [4], characterized by [1].

[0013] [7] The summary sentence data generation unit generates a plurality of summary sentence data from different processing target sentence data according to different processing instruction data. An information processing device as described in any one of [1] to [4], characterized by [1].

[0014] [8] The system further includes a detection unit for detecting variations in the notation of the key information, The extraction unit treats the key information of the detected variations in notation as the same key information. An information processing device as described in any one of [1] to [7], characterized by [1].

[0015] [9] The importance calculation unit calculates the importance of the key information using one or more of the following: the number of occurrences of the key information, TF-IDF (Term Frequency - Inverse Document Frequency), or the co-occurrence relationship between words. The information processing apparatus according to [3] or [4], characterized by the following:

[0016]

[10] A program that causes an information processing apparatus to execute a summary sentence data generation step of generating a plurality of summary sentence data from processing target sentence data according to processing instruction data, an extraction step of extracting key information included in the plurality of summary sentence data for each category constituting the processing target sentence data, and a summary sentence data selection step of selecting the summary sentence data to be output from the plurality of summary sentence data using the key information for each category.

[0017]

[11] An information processing system comprising: an input reception unit that receives input of processing instruction data and processing target sentence data; a summary sentence data generation unit that generates a plurality of summary sentence data from the processing target sentence data according to the processing instruction data; an extraction unit that extracts key information included in the plurality of summary sentence data for each category constituting the processing target sentence data; a summary sentence data selection unit that selects the summary sentence data to be output from the plurality of summary sentence data using the key information for each category; a display control unit that displays the selected summary sentence data on a display device.

Advantages of the Invention

[0018] According to the present disclosure, when generating summary sentence data from processing target sentence data, it is possible to suppress lack of information and to suppress generation of incorrect summary sentence data.

Brief Description of the Drawings

[0019] [Figure 1] It is a configuration diagram of an example of an information processing system according to the present embodiment. [Figure 2] It is a hardware configuration diagram of an example of a computer according to the present embodiment. [Figure 3] It is an explanatory diagram of an example of a process of generating summary sentence data from processing target sentence data according to a prompt. [Figure 4] This diagram illustrates an example of the process by which the information processing system according to this embodiment generates summary data from data to be processed. [Figure 5] This is a functional configuration diagram of an example of an information processing system according to this embodiment. [Figure 6] This is an example flowchart illustrating the processing of the information processing system according to this embodiment. [Figure 7] This is an example flowchart illustrating the processing of the information processing system according to this embodiment. [Figure 8] This is an example flowchart illustrating the process in step S16. [Figure 9] This is an example flowchart illustrating the process in step S16. [Figure 10] Figure 9 is an explanatory diagram illustrating an example of the process in the flowchart. [Figure 11] This is an explanatory diagram illustrating an example of the process in step S12. [Figure 12] This is an explanatory diagram illustrating an example of the process in step S12. [Figure 13] This is an explanatory diagram illustrating an example of the process in step S12. [Figure 14] This is an explanatory diagram illustrating an example of the process in step S12. [Figure 15] This is a specific example of the processing performed by the information processing system according to this embodiment. [Modes for carrying out the invention]

[0020] Next, embodiments of the present invention will be described in detail. However, the present invention is not limited to the following embodiments.

[0021] <System Configuration> Figure 1 is a diagram illustrating an example of an information processing system 1 according to this embodiment. The information processing system 1 in Figure 1 provides a summarization function using a large-scale language model (hereinafter referred to as LLM). The LLM generates summary sentence data from the sentence data to be processed according to prompts. A prompt is an example of processing instruction data, and is data such as a string that instructs the LLM on the content of the summary sentence data to be generated. The information processing system 1 also selects the summary sentence data to be output from the multiple summary sentence data generated by the LLM, as described later. In this way, the information processing system 1 performs a process of selecting and outputting the summary sentence data selected from the multiple summary sentence data generated by the summarization function of the LLM.

[0022] Information processing system 1 comprises an information processing device 10 and a user terminal 12, which are connected via a network 18 for data communication. The network 18 is, for example, a local area network (LAN) or the internet. The information processing device 10 can be implemented as a workstation or a PC (Personal Computer). The user terminal 12 can be implemented as an information processing terminal operated by the user, such as a PC, tablet, or smartphone.

[0023] The user can operate the user terminal 12 and utilize the summarization function provided by the information processing device 10. The user operates the user terminal 12 and inputs the sentence data to be processed and prompts to the information processing device 10. By sending the input sentence data to be processed and prompts to the information processing device 10, the user terminal 12 can cause the information processing device 10 to output summarized sentence data of the sentence data to be processed, as described later.

[0024] The user terminal 12 receives the summary text data of the processed sentence data output from the information processing device 10 and displays it on a display device or the like. In this way, the user can check the summary text data of the processed sentence data displayed on the display device or the like.

[0025] Furthermore, the information processing device 10 receives the sentence data to be processed and prompts from the user terminal 12. The information processing device 10 performs a process to generate multiple summary sentence data from the received sentence data to be processed according to the received prompts. The information processing device 10 selects the summary sentence data to be output from the multiple summary sentence data generated, as described below, and sends the summary sentence data to the user terminal 12 for display on a display device or the like.

[0026] The functions of the information processing device 10 may also be provided as a cloud service. The information processing device 10 may use its own LLM or an LLM from another device. The LLM is a natural language processing model (a pre-trained large-scale language model) that has been trained using a large amount of text data. The LLM can use a finely tuned pre-trained model. The LLM can use GPT (Generative Pre-trained Transformer) (registered trademark), etc.

[0027] Note that the configuration of the information processing system 1 shown in Figure 1 is just one example. The configuration of the information processing system 1 can vary depending on the application and purpose. For example, the functions of the information processing device 10 and the user terminal 12 may be integrated and implemented by a single computer. Alternatively, the information processing device 10 may be implemented by multiple computers.

[0028] <Hardware Configuration> For example, the information processing device 10 and user terminal 12 in Figure 1 can be realized by a computer 500 with the hardware configuration shown in Figure 2.

[0029] Figure 2 is a hardware configuration diagram of an example of a computer 500 according to this embodiment. The computer 500 includes, for example, an input device 501, an output device 502, an external interface 503, RAM (Random Access Memory) 504, ROM (Read Only Memory) 505, a CPU (Central Processing Unit) 506, a communication interface 507, and an auxiliary storage device 508, all of which are interconnected via bus B. The input device 501 and the output device 502 may also be used by connecting them to the computer 500 via the external interface 503.

[0030] The input device 501 is a device that accepts user input, such as a touch panel, operation keys, buttons, keyboard, or mouse. The output device 502 has a device that displays a screen and a device that outputs sound. The device that displays a screen is, for example, a display (display device) such as an LCD. The device that outputs sound is, for example, a speaker. The communication I / F 507 is an interface for the computer 500 to perform data communication.

[0031] The auxiliary storage device 508 is an example of a non-volatile storage device that stores programs and data. The auxiliary storage device 508 is, for example, an HDD (Hard Disk Drive) or an SSD (Solid State Drive). The programs are, for example, the operating system (OS), which is basic software, and applications that provide various functions on the OS.

[0032] External I / F 503 is an interface to an external device. The external device is a recording medium 503a, etc. Computer 500 can read programs and data from the recording medium 503a via external I / F 503. The recording medium 503a is, for example, a flexible disk, CD, DVD, SD memory card, or USB memory.

[0033] ROM 505 stores the BIOS, OS settings, and network settings that are executed when the computer 500 starts up. RAM 504 is working memory for temporarily storing programs and data. The CPU 506 can realize the various functions described later by reading programs and data from ROM 505 or auxiliary storage device 508 into RAM 504 and executing processing.

[0034] <Processing Overview> Figure 3 is an explanatory diagram of an example of a process that generates summary sentence data 1004-1 to 1004-3 from the sentence data 1000 to be processed according to prompt 1002. In Figure 3, the sentence data 1000 to be processed is an example of a patent document.

[0035] Summary data 1004-1 is an example of summary data generated by LLM from the processed sentence data for the first time in accordance with prompt 1002. Summary data 1004-2 is an example of summary data generated by LLM from the processed sentence data for the second time in accordance with prompt 1002. Summary data 1004-3 is an example of summary data generated by LLM from the processed sentence data for the third time in accordance with prompt 1002.

[0036] Summary sentence data 1004-1 to 1004-3 show multiple output results using the same target sentence data 1000 and prompt 1002. The summary sentence data 1004-1 to 1004-3 generated by LLM will have different output results even when using the same target sentence data 1000 and prompt 1002. Therefore, summary sentence data 1004-1 to 1004-3 may lack sufficient information to provide an overview of the entire target sentence data 1000.

[0037] Therefore, the information processing system 1 according to this embodiment generates summary data 1008 from the text data 1000 to be processed, as shown in Figure 4. Figure 4 is an explanatory diagram of an example of the process by which the information processing system 1 according to this embodiment generates summary data 1008 from the text data 1000 to be processed. Figure 4 shows an example where the text data 1000 to be processed is a patent document.

[0038] Summary data 1004-1 is an example of summary data generated by LLM for the first time from the processed sentence data in accordance with prompt 1002. Summary data 1004-1 includes summary sentences of problems, solutions, and effects, which are examples of categories that make up the processed sentence data 1000.

[0039] Summary data 1004-2 is an example of summary data generated by LLM for the second time from the processed sentence data in accordance with prompt 1002. Summary data 1004-2 includes summary sentences of problems, solutions, and effects, which are examples of categories that make up the processed sentence data 1000.

[0040] Summary data 1004-3 is an example of summary data generated by LLM from the processed sentence data for the third time in accordance with prompt 1002. Summary data 1004-3 includes summary sentences of problems, solutions, and effects, which are examples of categories that make up the processed sentence data 1000.

[0041] Summary data 1006-1 is an example of summary data that collects summary sentences of problems, which are an example of the categories that make up the processed sentence data 1000, from summary data 1004-1 to 1004-3. Summary data 1006-2 is an example of summary data that collects summary sentences of solutions, which are an example of the categories that make up the processed sentence data 1000, from summary data 1004-1 to 1004-3. Summary data 1006-3 is an example of summary data that collects summary sentences of effects, which are an example of the categories that make up the processed sentence data 1000, from summary data 1004-1 to 1004-3.

[0042] In the information processing system 1 according to this embodiment, key information extraction processing is performed from the summary text data 1006-1 using LLM. The key information extraction processing using LLM extracts one or more key pieces of information (key information list) of issues, which are an example of categories that constitute the text data to be processed 1000. The key information may be, for example, a keyword or a key phrase. Alternatively, the key information may be a combination of a keyword and a key phrase.

[0043] In the information processing system 1 according to this embodiment, key information extraction processing is performed from summary sentence data 1006-2 using LLM. The key information extraction processing using LLM extracts key information (key information list) of solution means, which is an example of a category that constitutes the sentence data to be processed 1000. Furthermore, in the information processing system 1 according to this embodiment, key information extraction processing is performed from summary sentence data 1006-3 using LLM. The key information extraction processing using LLM extracts key information (key information list) of effect, which is an example of a category that constitutes the sentence data to be processed 1000.

[0044] In the information processing system 1 according to this embodiment, the key information of the task extracted from the summary text data 1006-1 by LLM is used to select the summary text data of the task to be output from the summary text data 1006-1, and this is added as the summary text of the task to be output summary text data 1008.

[0045] For example, in the information processing system 1 according to this embodiment, summary text data (text) of a problem that broadly covers the key information of the problem extracted by LLM from the summary text data 1006-1 is selected.

[0046] In the information processing system 1 according to this embodiment, key information of the solution extracted from the summary data 1006-2 by LLM is used to select the summary data of the solution to be output from the summary data 1006-2, and this is added as the summary of the solution to the output summary data 1008.

[0047] For example, in the information processing system 1 according to this embodiment, summary text data (text) of solution means that broadly covers the key information of solution means extracted by LLM from summary text data 1006-2 is selected.

[0048] In the information processing system 1 according to this embodiment, key information of the effect extracted from the summary data 1006-3 by LLM is used to select the summary data of the effect to be output from the summary data 1006-3, and this is added as the summary of the effect to be output summary data 1008.

[0049] For example, in the information processing system 1 according to this embodiment, summary text data (sentences) of effects that broadly cover the key information of effects extracted by LLM from the summary text data 1006-3 are selected.

[0050] <Functional Configuration> The functional configuration of the information processing system 1 according to this embodiment will be described.

[0051] Figure 5 is a functional configuration diagram of an example of the information processing system 1 according to this embodiment. Note that parts of the configuration diagram in Figure 10 that are not necessary for the explanation of this embodiment have been appropriately omitted. The information processing device 10 in Figure 5 has an input receiving unit 30, a summary text data generation unit 32, an extraction unit 34, a summary text data selection unit 36, a display control unit 38, an importance calculation unit 40, a detection unit 42, a communication unit 44, and an LLM storage unit 50. The user terminal 12 in Figure 5 has an operation receiving unit 60, a display unit 62, and a communication unit 64.

[0052] The operation reception unit 60 receives various operations from the user. The display unit 62 displays screens for the user to view. For example, the display unit 62 displays a screen for receiving requests from the user to execute processing on the information processing device 10, and a screen for presenting the user with the results of processing by the information processing device 10.

[0053] The communication unit 64 communicates data with the information processing device 10. For example, the communication unit 64 sends a request to the information processing device 10 and receives a response from the information processing device 10.

[0054] The input receiving unit 30 of the information processing device 10 receives input from the user terminal 12 operated by the user. For example, the input receiving unit 30 receives input of the sentence data to be processed 1000 and a prompt 1002.

[0055] The summary sentence data generation unit 32 uses LLM to generate summary sentence data 1004-1 to 1004-3 from the sentence data to be processed 1000 according to prompt 1002. Alternatively, the summary sentence data generation unit 32 may use LLM to generate summary sentence data 1004-1 to 1004-3 from the sentence data to be processed 1000 for each category (problem, solution, and effect) according to prompt 1002. The summary sentence data generation unit 32 generates summary sentence data 1006-1 to 1006-3 from the summary sentence data 1004-1 to 1004-3, which collects the summary sentences for each category that constitute the sentence data to be processed 1000.

[0056] The extraction unit 34 extracts key information contained in the summary sentence data 1006-1 to 1006-3 for each category that constitutes the sentence data 1000 to be processed. LLM is used for key information extraction. Alternatively, existing techniques for extracting important words may be used for key information extraction.

[0057] The summary data selection unit 36 ​​uses key information extracted for each category to select the summary data for the category to be output from the summary data 1006-1 to 1006-3 and adds it as the summary text of the output summary data 1008. Details of the processing of the summary data selection unit 36 ​​will be described later. The display control unit 38 performs display control to display the output summary data 1008 on the display unit 62 of the user terminal 12.

[0058] The importance calculation unit 40 calculates the importance of the key information. The importance calculation unit 40 may also calculate the importance of the key information extracted from the summary text data 1006-1 to 1006-3 using the number of occurrences of the key information.

[0059] Furthermore, the importance calculation unit 40 may use TF-IDF (Term Frequency - Inverse Document Frequency) to calculate the importance of key information extracted from summary data 1006-1 to 1006-3. TF-IDF is an effective method for extracting important words that characterize a given document. TF-IDF is calculated by multiplying tf and idf.

[0060] tf represents the frequency of a word's appearance in a given text, and is calculated using the following formula (1). The higher the frequency, the greater the importance of the word.

[0061] tf = Number of occurrences of word i in sentence d / Sum of occurrences of all words in sentence d ... (1) The IDF (International Classification Factor) represents how many sentences a given word appears in, and is calculated using the following formula (2). The less frequently a word appears, the higher its importance.

[0062] idf = log(total number of sentences / number of sentences containing a given word i) ... (2) The summary data selection unit 36 ​​may also select the summary data of the category to be output from the summary data 1006-1 to 1006-3 using the importance of the key information calculated by the importance calculation unit 40.

[0063] Furthermore, the importance calculation unit 40 may calculate the importance of key information extracted from summary sentence data 1006-1 to 1006-3 using word co-occurrence relationships. Word co-occurrence relationships refer to the simultaneous appearance of certain words in a text, and represent the strength of the connection between words. More specifically, a word co-occurrence matrix can be created within the summary sentence data, and the importance of key information can be calculated based on this co-occurrence matrix. As a method for calculating the importance of key information using word co-occurrence relationships, TextRank, TopicRank, etc., can be used. Using the key information and importance scores calculated by these methods, the summary sentence data of the category to be output from summary sentence data 1006-1 to 1006-3 may be selected.

[0064] The detection unit 42 detects variations in the notation of key information. For example, the detection unit 42 detects variations in the notation of key information by determining synonyms using a thesaurus or embedding. The extraction unit 34 may absorb variations in the notation of key information by treating the key information of the variations included in the summary data 1006-1 to 1006-3 as the same key information.

[0065] The communication unit 44 communicates data with the user terminal 12. For example, the communication unit 44 receives a request from the user terminal 12 and sends a response to the user terminal 12. The LLM storage unit 50 stores the LLM.

[0066] <Processing> The information processing system 1 according to this embodiment performs the following processes. Figure 6 is a flowchart illustrating an example of the processes of the information processing system 1 according to this embodiment.

[0067] In step S10, the input receiving unit 30 of the information processing device 10 receives input of the sentence data to be processed 1000 and the prompt 1002 from the user. The sentence data to be processed 1000 and the prompt 1002 received from the user may be one or more.

[0068] In step S12, the summary sentence data generation unit 32 of the information processing device 10 performs summarization multiple times on the processing target sentence data 1000 that it has received input from the user, in accordance with the prompt 1002 that it has received input from the user. For example, the summary sentence data generation unit 32 uses LLM to generate summary sentence data 1004-1 to 1004-3 from the processing target sentence data 1000 in accordance with the prompt 1002. The summary sentence data 1004-1 to 1004-3 include summary sentences of problems, solutions, and effects, which are examples of categories that make up the processing target sentence data 1000.

[0069] Furthermore, the summary sentence data generation unit 32 collects summary sentences for each category that constitute the sentence data to be processed 1000 from the summary sentence data 1004-1 to 1004-3, and generates summary sentence data 1006-1 to 1006-3, for example.

[0070] In step S14, the extraction unit 34 extracts key information contained in the summary sentence data 1006-1 to 1006-3 for each summary sentence of a category that constitutes the sentence data to be processed 1000, such as problem, solution, and effect. Note that the extraction of key information may be done not only from the extraction of important words, but also from multiple perspectives such as technology, application, material, method, problem, or effect.

[0071] In step S16, the summary data selection unit 36 ​​uses key information extracted for each category, such as problem, solution, and effect, as described below, to select summary data for each category from summary data 1006-1 to 1006-3, and adds them as summary texts to the output summary data 1008.

[0072] In step S18, the display control unit 38 performs display control to display the output summary data 1008 on the display unit 62 of the user terminal 12, thereby presenting the summary data selected for each category in step S16 to the user.

[0073] In addition to the processing shown in Figure 6, the information processing system 1 according to this embodiment may also include processing to absorb variations in the notation of key information, as shown in step S15 of Figure 7. Figure 7 is a flowchart showing an example of the processing of the information processing system 1 according to this embodiment. Note that the flowchart shown in Figure 7 has the processing of step S15 added to the flowchart in Figure 6.

[0074] In step S15, the detection unit 42 detects variations in the notation of the key information detected in step S14. The variations in the notation of the key information detected in step S15 are absorbed by treating the key information of the variations as the same key information. The processes other than step S15 are the same as the flowchart in Figure 6, so their explanation is omitted.

[0075] The process in step S16 in Figures 6 and 7 is performed, for example, as shown in the flowchart in Figure 8. Figure 8 is an example flowchart showing the process in step S16.

[0076] In step S30, the importance calculation unit 40 calculates the importance of the key information extracted in step S14 of Figure 6 or Figure 7. For example, the importance calculation unit 40 calculates the importance of the key information extracted from summary data 1006-1 to 1006-3.

[0077] The importance calculation unit 40 calculates the importance of the key information extracted in step S14 using the TF-IDF of the key information. Alternatively, the importance calculation unit 40 may calculate the importance of the key information using the number of occurrences of the key information. When using the number of applications filed for the key information, for example, the more occurrences the key information has, the higher its importance, and the less occurrences it has, the lower its importance.

[0078] In step S32, the importance calculation unit 40 calculates the importance of the summary data generated in step S12 in Figure 6 or Figure 7. For example, the importance calculation unit 40 calculates the importance of each sentence included in the summary data 1006-1 to 1006-3.

[0079] The importance calculation unit 40 calculates the importance of the summary text data generated in step S12 of Figure 6 or Figure 7 using the importance of each key information calculated in step S30. The importance of the summary text data is the sum of the importance of the key information contained in that summary text data. For example, the importance calculation unit 40 calculates the importance of each sentence (sentences generated by the first to third summaries shown in Figure 4) contained in the summary text data 1006-1 to 1006-3.

[0080] In step S34, the summary data selection unit 36 ​​selects summary data with high importance from the summary data generated in step S12 in Figure 6 or Figure 7.

[0081] For example, the summary data selection unit 36 ​​uses the importance of each sentence included in the summary data 1006-1 to 1006-3 to select the sentences with the highest importance from among the sentences included in the summary data 1006-1 to 1006-3 for each of the summary data 1006-1 to 1006-3. The number of summary data sentences selected for each of the summary data 1006-1 to 1006-3 only needs to be one or more. For example, among the sentences "[Task (1st time)]....[Task (2nd time)]....[Task (3rd time)]....." included in the summary data 1006-1 in Figure 4, if the importance of "[Task (1st time)]....." is higher than that of "[Task (2nd time)].....[Task (3rd time)].....", the summary data selection unit 36 ​​selects "[Task (1st time)].....".

[0082] The process in step S16 in Figures 6 and 7 may be performed, for example, as shown in the flowchart in Figure 9. Figure 9 is an example flowchart showing the process in step S16.

[0083] In step S50, the importance calculation unit 40 calculates the importance of the key information extracted in step S14 of Figure 6 or Figure 7. For example, the importance calculation unit 40 calculates the importance of the key information extracted from summary data 1006-1 to 1006-3.

[0084] The importance calculation unit 40 calculates the importance of the key information extracted in step S14 using the TF-IDF of the key information. Alternatively, the importance calculation unit 40 may calculate the importance of the key information using the number of occurrences of the key information.

[0085] In step S52, the importance calculation unit 40 calculates the importance of the summary data generated in step S12 of Figure 6 or Figure 7. For example, the importance calculation unit 40 calculates the importance of each sentence included in the summary data 1006-1 to 1006-3.

[0086] The importance calculation unit 40 calculates the importance of the summary data generated in step S12 of Figure 6 or Figure 7 using the importance of each key piece of information calculated in step S30. For example, the importance calculation unit 40 calculates the importance of each sentence included in the summary data 1006-1 to 1006-3.

[0087] In step S54, the summary data selection unit 36 ​​selects the first summary data with the highest importance from the summary data generated in step S12 in Figure 6 or Figure 7.

[0088] For example, the summary data selection unit 36 ​​uses the importance of each sentence included in the summary data 1006-1 to 1006-3 to select the sentence of the first summary data with the highest importance from among the sentences included in the summary data 1006-1 to 1006-3, for each of the summary data 1006-1 to 1006-3.

[0089] In step S56, the summary data selection unit 36 ​​uses the importance of the remaining key information, excluding the text of the first summary data selected in step S54, to calculate the importance of each sentence in the remaining summary data, excluding the text of the first summary data.

[0090] In step S58, the summary data selection unit 36 ​​selects the second summary data sentence, which has the highest importance, from the remaining summary data. The second summary data sentence will be a sentence with minimal overlap with the first summary data sentence.

[0091] The processing of the flowchart in Figure 9 will be further explained using Figure 10. Figure 10 is an explanatory diagram of an example of the processing of the flowchart in Figure 9. Note that Figure 10 shows an example where the key information is a keyword. Furthermore, Figure 10 explains an example where the importance of the keyword is a TF-IDF value.

[0092] For example, "Summary-1" in Figure 10 is the sentence "[Task (1st time)]..." included in summary data 1006-1 shown in Figure 4. "Summary-2" in Figure 10 is the sentence "[Task (2nd time)]..." included in summary data 1006-1 shown in Figure 4. "Summary-3" in Figure 10 is the sentence "[Task (3rd time)]..." included in summary data 1006-1 shown in Figure 4.

[0093] In Figure 10, KW-A to KW-F represent keywords A to F. The table shown in Figure 10 has checks indicating the keywords included in "Summary-1" to "Summary-3".

[0094] For example, the text in "Summary-1" contains keywords A, B, and D. The text in "Summary-2" contains keywords A, C, and E. The text in "Summary-3" contains keywords D and F.

[0095] In step S50 of Figure 9, the importance calculation unit 40 calculates the TF-IDF values ​​of keywords A to F included in "Summary Sentence-1" to "Summary Sentence-3" in Figure 10.

[0096] In step S52, the importance calculation unit 40 calculates the importance of "Summary Sentence-1" to "Summary Sentence-3" in Figure 10 using the TF-IDF values ​​of keywords A to F. The importance of "Summary Sentence-1" to "Summary Sentence-3" in Figure 10 is the sum of the TF-IDF values ​​of keywords A to F included in each of "Summary Sentence-1" to "Summary Sentence-3" in Figure 10.

[0097] Here, we will continue the explanation assuming that "Summary-1" had the highest importance. In step S54, the summary data selection unit 36 ​​selects "Summary-1" as the first summary data with the highest importance from "Summary-1" to "Summary-3" in Figure 10.

[0098] In step S56, the summary data selection unit 36 ​​excludes keywords A, B, and D contained in "Summary-1," which was selected as the first summary data. The lower table shown in Figure 10 shows "Summary-1," which was selected as the first summary data, and the result after excluding keywords A, B, and D contained in "Summary-1," which was selected as the first summary data.

[0099] The importance calculation unit 40 calculates the importance of "Summary Sentence-2" to "Summary Sentence-3" in Figure 10 using the TF-IDF values ​​of the remaining keywords C, E, and F, excluding keywords A, B, and D included in "Summary Sentence-1," which was selected as the first summary data.

[0100] Here, we will continue the explanation assuming that "Summary-2" had the highest importance. In step S58, the summary data selection unit 36 ​​selects "Summary-2" as the second summary data with the highest importance from "Summary-2" to "Summary-3" in Figure 10.

[0101] The second summary data selected, "Summary-2," contains different keywords than the first summary data selected, "Summary-1." Therefore, by combining "Summary-1" (selected as the first summary data) and "Summary-2" (selected as the second summary data), it is possible to select summary data sentences that broadly cover the keywords.

[0102] The process in step S12 in Figures 6 and 7 may also be performed as shown in Figures 11 to 14, for example. Figures 11 to 14 are explanatory diagrams illustrating an example of the process in step S12.

[0103] Figure 11 shows an example of a process that generates multiple summary sentence data from the same target sentence data according to different prompts. The summary sentence data generation unit 32 in Figure 11 accepts different prompts A and B and one target sentence data as input.

[0104] For example, Prompt A might be, "Please list three key points in 20-40 characters each regarding the problem described in the [Problem] section of the patent document abstract." Prompt B might be, "Please list three key points in 20-40 characters each regarding the problem described in the [Problem] section of the patent document abstract and the [Problem to be Solved by the Invention] section of the specification."

[0105] The summary sentence data generation unit 32 can generate multiple summary sentence data of the processed sentence data by performing summarization multiple times on the processed sentence data that has been received as input, according to different prompts A and B.

[0106] In Figure 11, multiple summary sentence data can be generated from the target sentence data according to prompt A, and multiple summary sentence data can be generated from the target sentence data according to prompt B.

[0107] Figure 12 shows an example of a process that generates multiple summary sentence data from different target sentence data according to the same prompt. The summary sentence data generation unit 32 in Figure 12 accepts one prompt and two different target sentence data A and B as input. For example, in the case of the patent document in Figure 4, target sentence data A may be the text described in the [Problem] section of the abstract. In the case of the patent document in Figure 4, target sentence data B may be the text described in the [Problem] section of the abstract and the [Problem to be Solved by the Invention] section of the specification.

[0108] The summary sentence data generation unit 32 can generate multiple summary sentence data for the processed sentence data A and B by performing summarization multiple times according to the same prompt for the processed sentence data A and B that it has received as input.

[0109] Figure 12 shows that multiple summary sentence data can be generated from the target sentence data A according to the prompt, and multiple summary sentence data can be generated from the target sentence data B according to the prompt.

[0110] Figure 13 shows an example of a process that generates multiple summary sentence data from different target sentence data according to different prompts. The summary sentence data generation unit 32 in Figure 13 accepts different prompts A and B and different target sentence data A and B as input.

[0111] The summary sentence data generation unit 32 can generate multiple summary sentence data for the processed sentence data A and B that it has received as input by performing summarization multiple times according to prompts A and B. In Figure 13, multiple summary sentence data can be generated, such as summary sentence data generated from processed sentence data A according to prompt A, summary sentence data generated from processed sentence data B according to prompt A, summary sentence data generated from processed sentence data A according to prompt B, and summary sentence data generated from processed sentence data B according to prompt B.

[0112] Figure 14 shows an example of a process that generates multiple summary sentence data from different target sentence data according to different prompts. The summary sentence data generation unit 32 in Figure 14 accepts different prompts A to C and different target sentence data A and B as input.

[0113] For example, Prompt A might be, "Please summarize the key points regarding the issues described in the texts of the two patent documents in 20 to 40 characters each." Prompt B might be, "Please summarize the similarities regarding the issues described in the texts of the two patent documents in 20 to 40 characters each." Prompt C might be, "Please summarize the differences regarding the issues described in the texts of the two patent documents in 20 to 40 characters each."

[0114] The summary sentence data generation unit 32 can generate multiple summary sentence data for the processed sentence data A and B, which it has received as input, by performing summarization multiple times according to prompts A to C.

[0115] For example, in Figure 14, multiple summary sentence data can be generated from the processed sentence data A or B according to prompt A. The summary sentence data generated from the processed sentence data A or B according to prompt A is, for example, a sentence that represents the main points of the processed sentence data A or B.

[0116] Furthermore, Figure 14 shows that multiple summary sentence data can be generated from the processed sentence data A and B according to prompt B. The summary sentence data generated from the processed sentence data A and B according to prompt B are, for example, sentences that describe the similarities between processed sentence data A or B.

[0117] Furthermore, Figure 14 shows that multiple summary sentence data can be generated from the processed sentence data A and B according to prompt C. The summary sentence data generated from the processed sentence data A and B according to prompt C are, for example, sentences that describe the differences between processed sentence data A or B.

[0118] Figure 15 shows a specific example of the processing of the information processing system 1 according to this embodiment. As shown in Figure 15, the information processing device 10 generates multiple summary sentence data of key points related to the "issue" using LLM from the patent document received as the sentence data to be processed, in accordance with the prompt.

[0119] The information processing device 10 extracts key information such as keywords and key phrases from multiple generated summary sentence data, calculates the TF-IDF value of the extracted key information, and calculates the TF-IDF value for each sentence included in the multiple generated summary sentence data.

[0120] The information processing device 10 selects the sentence with the highest TF-IDF value from among the sentences included in the multiple generated summary sentence data. In Figure 15, the sentence "Solving the variability in the reliability of organizational data due to measurement, observation, and analysis skills and subjectivity" included in the summary sentence data generated by the second summarization is selected.

[0121] The information processing device 10 selects the sentence with the highest TF-IDF value from among the sentences containing key information that are not included in the selected sentence with the highest TF-IDF value. The sentence "Provides a device that facilitates data acquisition and enables highly accurate estimation," which is included in the summary sentence data generated by the second summarization, is selected.

[0122] <Other Embodiments> The information processing system 1 according to this embodiment selects a summary result from multiple generated summary sentence data or multiple sentences contained in the generated summary sentence data according to their importance. However, the selected summary result may be regenerated using LLM. The regeneration of the summary result may be performed, for example, by using key information contained in the summary result sentences, so as to ensure that high-importance key information is not lost.

[0123] For example, the information processing system 1 according to this embodiment may regenerate summary data for the entire text to be processed using LLM, based on the summary results for each selected category.

[0124] In this embodiment, summarization results can be selected from multiple generated summary sentence data or multiple sentences contained in the generated summary sentence data according to their importance, thereby suppressing the generation of erroneous (low-accuracy) summary sentence data from the processed sentence data. Furthermore, in this embodiment, a summary result that includes a wide range of key information and provides an overview of the entire processed sentence data can be output.

[0125] Although this embodiment has been described above, it will be understood that various modifications to the form and details are possible without departing from the spirit and scope of the claims. Although the present invention has been described above based on examples, the present invention is not limited to the above examples, and various modifications are possible within the scope described in the claims. This application claims priority to Basic Application No. 2024-085146 filed with the Japan Patent Office on 24 May 2024, the entire contents of which are incorporated herein by reference. [Explanation of symbols]

[0126] 1. Information Processing System 10 Information Processing Devices 12 User terminals 18 Network 30 Input reception section 32 Summary Data Generation Unit 34 Extraction part 36 Summary text data selection section 38 Display Control Unit 40 Importance calculation part 42 Detection unit 44 Communications Department 50 LLM storage 60 Operation reception unit 62 Display section 64 Communications Department 500 Computers

Claims

1. A summary sentence data generation unit performs summarization of the target sentence data multiple times according to processing instruction data and generates multiple summary sentence data for each of the multiple categories that constitute the target sentence data, An extraction unit that extracts key information contained in the plurality of summary sentence data for each category that constitutes the sentence data to be processed, A summary sentence data selection unit selects a summary result to be output from the multiple summary sentence data using the key information for each category, for each category that constitutes the sentence data to be processed. It has, The summary data generation unit uses the key information included in the selected summary result to regenerate the summary result using a large-scale language model. An information processing device characterized by the following.

2. The summary data generation unit regenerates the summary result such that the key information with high importance among the key information included in the selected summary result is included in the regenerated summary result. The information processing apparatus according to claim 1, characterized in that

3. The summary data generation unit uses the summarization results selected for each category to regenerate the summary data for the entire set of processed sentence data using a large-scale language model. An information processing apparatus according to claim 1 or 2, characterized by the above.

4. The importance of the aforementioned key information is calculated using one or more of the following: the number of occurrences of the key information, TF-IDF (Term Frequency - Inverse Document Frequency), or the co-occurrence relationships between words. An information processing apparatus according to claim 1 or 2, characterized by the above.

5. A process of generating multiple summarized sentence data by performing summaries of the target sentence data multiple times according to the processing instruction data, A step of extracting key information contained in the plurality of summary sentence data for each category that constitutes the sentence data to be processed, A step of selecting a summary result from the multiple summary sentence data using the key information for each category, The process involves using key information contained in the selected summary result to regenerate the summary result using a large-scale language model, Information processing methods including

6. A program that causes a computer to execute the information processing method described in claim 5.

Images