Summarizing method, summarizing system, and program

Abstract

To provide a technique for summarizing a text in dialogue.SOLUTION: A server 10 includes: a division unit (121) performing a division step of dividing a dialogue text representing the contents of a dialogue into a plurality of blocks based on a topic included in the dialogue, using a text division model (M1); and a summarization unit (122) performing a summarization step of generating summary blocks obtained by summarizing the blocks using a summarization model (M2) to generate a summary text based on the summary blocks.SELECTED DRAWING: Figure 7

Description

[Technical Field]

[0001] TECHNICAL FIELD This disclosure relates to techniques for summarizing interactive text. [Background technology]

[0002] In recent years, technology that automatically converts speech into text using speech recognition technology has become widespread, and further improvements are being made. For example, Patent Document 1 discloses a summarization technology that uses the dialogue structure to correct dialogue text on a word-by-word basis to make it easier to read. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2017-111190 Summary of the Invention [Problem to be solved by the invention]

[0004] For example, transcription summaries using speech recognition technology may contain unnecessary backchannels or errors in key words, and because they are conversational summaries, they are difficult to read. On the other hand, summaries using generative AI (artificial intelligence), such as large-scale language models, can interpret the meaning of a sentence and produce summaries, but they have limitations on the number of characters that can be input and the risk of hallucination.

[0005] However, Patent Document 1 only discloses a technology for automatically correcting dialogue-style text on a word-by-word basis to create an easy-to-read summary. Therefore, there is room for further study on a technology for summarizing dialogue-style text using generative AI.

[0006] The non-limiting embodiments of the present disclosure have been made in view of the above background, and contribute to providing a technique for more appropriately summarizing dialogue-style text. [Means for solving the problem]

[0007] A summarization method according to one aspect of the present disclosure uses an information processing device to perform a segmentation step of segmenting dialogue text representing the content of a dialogue into multiple blocks based on topics included in the dialogue using a text segmentation model, and a summarization step of generating summary blocks that summarize each of the blocks using a summarization model, and generating summary text based on the summary blocks.

[0008] A summarization system according to one aspect of the present disclosure includes a segmentation unit that uses a text segmentation model to segment dialogue text representing the content of a dialogue into multiple blocks based on topics included in the dialogue, and a summarization unit that uses the summarization model to generate summary blocks that summarize each of the blocks, and generates summary text based on the summary blocks.

[0009] A computer program according to one aspect of the present disclosure is a program for causing a computer to function as the summarization system described above, and causes the computer to function as each of the above-mentioned parts.

[0010] These comprehensive or specific aspects may be realized as a system, a method, an integrated circuit, a computer program, or a recording medium, or may be realized as any combination of a system, an apparatus, a method, an integrated circuit, a computer program, and a recording medium. [Effects of the Invention]

[0011] According to one aspect of the present disclosure, a technique for more appropriately summarizing interactive text can be provided.

[0012] Further advantages and benefits of certain aspects of the present disclosure will become apparent from the specification and drawings. Such advantages and / or benefits may be provided by some of the embodiments and features described in the specification and drawings, respectively, but not necessarily all of them may be provided to obtain one or more identical features. [Brief explanation of the drawings]

[0013] [Figure 1] FIG. 2 is a block diagram showing an example of a functional configuration of the summarization system. [Figure 2] FIG. 10 is a diagram illustrating an example of a display screen of a user terminal. [Figure 3] FIG. 10 is a diagram illustrating an example of a display screen of a user terminal. [Figure 4] FIG. 10 is a diagram illustrating an example of a display screen of a user terminal. [Figure 5] FIG. 10 is a diagram illustrating an example of a display screen of a user terminal. [Figure 6] FIG. 10 is a diagram illustrating an example of a display screen of a user terminal. [Figure 7] FIG. 10 is a diagram illustrating an example of a display screen of a user terminal. [Figure 8] 10 is a flowchart illustrating an example of an outline of the operation of the summarization system. [Figure 9] FIG. 10 is a diagram showing an example of a result of text segmentation processing performed by a segmentation unit. [Figure 10] FIG. 10 is a diagram illustrating an example of summary text. [Figure 11] FIG. 10 is a diagram illustrating an image of EDU division processing in the key sentence extraction unit. [Figure 12] FIG. 10 is a diagram illustrating an image of a key sentence extraction process in a key sentence extraction unit. [Figure 13] FIG. 10 is a diagram showing an example of a summary text before processing for removing unnecessary sections. [Figure 14] FIG. 10 is a diagram showing an image of a dialogue text generation process. [Figure 15] 10 is a flowchart illustrating an example of an outline of the operation of the summarization system. [Figure 16] 10 is a flowchart illustrating an example of an outline of the operation of the summarization system. [Figure 17] 10 is a flowchart illustrating an example of an outline of the operation of the summarization system. [Figure 18] 10 is a flowchart illustrating an example of an outline of the operation of the summarization system. [Figure 19] 10 is a flowchart illustrating an example of an outline of the operation of the summarization system. [Figure 20] FIG. 2 is a block diagram illustrating the physical configuration of each computer that constitutes the summarization system. DETAILED DESCRIPTION OF THE INVENTION

[0014] Hereinafter, an embodiment of the present disclosure will be described in detail with appropriate reference to the drawings. However, more detailed description than necessary may be omitted. For example, detailed description of already well-known matters or redundant description of substantially identical configurations may be omitted. This is to avoid unnecessary redundancy in the following description and to facilitate understanding by those skilled in the art. Note that the accompanying drawings and the following description are provided to enable those skilled in the art to fully understand the present disclosure, and are not intended to limit the subject matter described in the claims.

[0015] [Embodiment] <<Overview of Summarization System 1>> First, an overview of the summarization system 1 according to this embodiment will be described. The summarization system 1 according to this embodiment is an information processing system that summarizes dialogue-style text. The summarization system 1 may have a function for generating dialogue-style text by recording the audio of a conversation and transcribing it using a voice recognition function. An example of the functional configuration of the summarization system 1 will be described in detail later using FIG. 1. As an example, the summarization system 1 may be configured to include a server 10 and a user terminal 20, as shown in FIG. 1.

[0016] As an example, summarization system 1 can be suitably introduced for creating minutes of conferences and meetings within a company, or business negotiations with other companies. Possible users of summarization system 1 include various business people who hold meetings and interviews on a daily basis, such as company employees and self-employed individuals. However, summarization system 1 can also be used in schools, homes, and the like, and can also be effectively used by people other than business people, so there are no particular limitations on the application scenarios or users.

[0017] As an example of how the summarization system 1 is used, the flow of a user creating minutes of a meeting (interview) will be outlined with reference to Figures 2 to 7. Note that the image of how the summarization system 1 is used explained with reference to Figures 2 to 7 is merely an example and is not particularly limited.

[0018] Figure 2 is a diagram showing an example of a display screen of the user terminal 20. When starting an interview, the user presses the "Start Interview" button displayed in Figure 2. As a result, in the summarization system 1, as an example, recording of the interview audio begins on the user terminal 20.

[0019] 3 is a diagram showing an example of a display screen of the user terminal 20. As shown in FIG. 3, after the user presses the "Start interview" button, "Interview in progress" is displayed, allowing the user to recognize that the interview is being recorded. When the interview ends, the user presses the "End interview" button. This causes the summarization system 1 to stop recording the interview audio.

[0020] 4 is a diagram showing an example of a display screen of the user terminal 20. As shown in FIG. 4, after the user presses the "End interview" button, "Interview Ended" is displayed, and at the same time, a "Create summary" button is displayed. When the user wants to create a summary of the contents of the conversation in the interview, the user presses the "Create summary" button.

[0021] FIG. 5 is a diagram illustrating an example of a display screen of the user terminal 20. As shown in FIG. 5, after the user presses the "Create Summary" button, a screen for selecting an interview type and an interview topic when creating a new summary is displayed. In the example of FIG. 5, the user selects "retail sales" as the interview type and "business situation interview" as the interview topic. When the user then presses the "Create Summary" button, the summarization system 1, for example, transmits audio data recorded on the user terminal 20 to the server 10. The server 10 then starts a process of creating a summary of the interview dialogue from audio text transcribed by a speech recognition function. This allows the summarization system 1 to create a more appropriate summary based on the interview type and interview topic selected by the user.

[0022] Fig. 6 is a diagram showing an example of a display screen of the user terminal 20. As shown in Fig. 6, after the user presses the "Create summary" button, once the summary creation process in the summarization system 1 is complete, "Summary creation complete" is displayed. Once the summary creation is complete, the user presses the "Summary list" button.

[0023] FIG. 7 is a diagram showing an example of a display screen of the user terminal 20. When the user presses the "Summary List" button and selects the summary result they want to display from the list, the "Summary Result" as shown in FIG. 7 is displayed. As shown in FIG. 7, the user can select the content to be displayed from "Short Summary," "Long Summary," "Transcription," etc. The user can also download the text from the server 20 by pressing the "Download Text" button.

[0024] <<Configuration of Summarization System 1>> The summarization system 1 according to this embodiment will be described in detail below with reference to the drawings. FIG. 1 is a block diagram showing an example of the functional configuration of the summarization system 1. The summarization system 1 is an information processing system configured with an information processing device. The summarization system 1 may be configured with one device or multiple devices. As an example of multiple devices, the summarization system 1 may be configured as an information processing system including a server 10 and a user terminal 20 (client terminal). Alternatively, the server 20 may be configured with multiple information processing devices. On the other hand, the information processing system may be configured as a complete system with a single user terminal, and is not particularly limited. Furthermore, when an information processing device is configured with multiple devices, the devices do not need to be installed in the same space, such as the same room, but may be installed in different rooms, different buildings, different regions, etc., and is not particularly limited.

[0025] 1, the summary system 1 includes a server 10 and a user terminal 20. The server 10 is an example of an information processing device in the present disclosure. The server 10 and the user terminal 20 are communicatively connected via a network N1. The network N1 connecting the server 10 and the user terminal 20 is a wired local area network (LAN), a wireless LAN, the Internet, a public line network, a mobile data communication network, or a combination thereof.

[0026] (Configuration of user terminal 20) 1, user terminal 20 includes a storage unit 21, a control unit 22, and a microphone 23. Control unit 22 acquires various information (input information) input by a user of summary system 1 via an input device. Input by a user via an input device may be simply referred to as "input by a user." Control unit 22 can also acquire voice input via microphone 23 as input information.

[0027] Furthermore, the control unit 22 presents to the user various pieces of information (output information) transmitted from the server 10 in response to the transmission of information input by the user by displaying them on a display device. Presenting to the user by displaying on a display device may be simply referred to as "presenting to the user." Furthermore, the storage unit 21 stores information transmitted and received between the server 10.

[0028] (Server 10 configuration) The server 10 includes a storage unit 11, a control unit 12, and an artificial intelligence model M. The storage unit 11 stores information to be transmitted and received between the server 10 and the user terminal 20, as well as information to be transmitted and received between the server 10 and other devices communicably connected via the network N1.

[0029] The control unit 12 includes a division unit 121 , a summarization unit 122 , a key sentence extraction unit 123 , a dialogue-related information reception unit 124 , a dialogue text generation unit 125 , and a learning unit 126 .

[0030] The artificial intelligence model M includes a text segmentation model M1, a summary model M2, a key sentence extraction model M3, and an unnecessary section removal model M4.

[0031] Note that this is merely one example of the functional configuration of the summary system 1, and it is not necessary for all functional blocks to be included. For example, if there are functions that do not need to be provided depending on the user's needs, the configuration may include only the necessary combination of functional blocks.

[0032] Furthermore, the configuration in which all the functional blocks included in the control unit 12 are arranged in one server 10 is merely an example, and each functional block may be distributed and arranged in multiple information processing devices, and is not particularly limited. Furthermore, each functional block is not limited to a configuration in which it is realized by one integrated program, and may be a configuration in which it is realized by multiple programs, and is not particularly limited.

[0033] Furthermore, the configuration in which all models included in the artificial intelligence model M are arranged in one server 10 is merely an example, and each model may be distributed and arranged in multiple information processing devices, or may use a function provided by an external device communicably connected via a network N1, etc., and is not particularly limited. Furthermore, each model may be realized as one function of a single, integrated, general-purpose artificial intelligence model, or may be realized as an independent, trained model for each model, and is not particularly limited.

[0034] <<Outline of operation of Summarization System 1>> Here, an example of the operation of the summarization system 1 according to this embodiment will be outlined. Fig. 8 is a flowchart illustrating an example of the operation of the summarization system 1. Note that the operation of each functional block will be outlined here, and details will be described later.

[0035] In step S801, the summarization system 1 executes a speech recognition process. For example, the summarization system 1 transcribes the audio recorded during a meeting between a customer and a sales representative. For example, the dialogue text generation unit 125 generates dialogue text from the audio data recorded during the meeting using a speech recognition technique.

[0036] In step S802, the summarization system 1 executes a text segmentation process. For example, the summarization system 1 segments a transcribed sentence into sentences for each theme (topic). For example, the segmentation unit 121 uses the text segmentation model M1 to segment the dialogue text into multiple blocks based on the topics.

[0037] In step S803, the summarization system 1 executes an important sentence extraction process. The summarization system 1 extracts, for example, sentences to be retained as a summary, i.e., key points in the conversation. For example, the important sentence extraction unit 123 uses an important sentence extraction model to generate important sentence blocks by extracting important sentences from each block of the dialogue text.

[0038] In step S804, the summarization system 1 executes summarization processing. For example, the summarization system 1 inputs each part of the extracted sentence into a large-scale language model, shapes it into itemized lists, and assigns headings to each part. In other words, the summarization system 1 shapes dialogue text, such as notes composed of transcriptions, into highly readable text information. For example, the summarization unit 122 uses the summary model M2 to generate summary blocks that summarize each important sentence block, and then integrates these summary blocks to generate a summary text.

[0039] In step S805, the summarization system 1 executes an unnecessary section removal process. For example, the summarization system 1 distinguishes and removes unnecessary sections from the summary text. That is, unnecessary conversation sections are removed from a negotiation history, etc. For example, the summarization unit 122 removes all or part of the unnecessary blocks from the summary text based on the importance of at least some of the summary blocks.

[0040] This concludes one example of the operation of the summarization system 1. Note that the flowchart shown in Figure 8 is merely one example of the processing flow, and the operation of the summarization system 1 is not limited to this. For example, other steps may be included depending on the user's request, the same steps may be executed repeatedly, the order of the steps may be changed, or some steps may not be executed, and there are no particular limitations.

[0041] <<Summary of dialogue text>> The following describes in detail the functions of the summarization system 1 for summarizing an audio interview text.

[0042] As an example of this embodiment, in the summarization system 1, the division unit 121 divides the dialogue text representing the content of the dialogue into multiple blocks based on the topics included in the dialogue using a text division model, and the summarization unit 122 uses the summarization model to generate summary blocks that summarize each block, and generates summary text based on the summary blocks.

[0043] (Dialogue text) The dialogue text may be, for example, a text transcribed from an actual conversation between people in a meeting or the like. In the summarization system 1, the dialogue text generation unit 125 may generate the dialogue text by transcribing speech recorded via a microphone 23 of a user terminal 20, such as a smartphone, using a voice recognition technology. Details of the dialogue text generation process by the dialogue text generation unit 125 will be described later.

[0044] Furthermore, the dialogue text is not necessarily limited to a transcription of an actual dialogue using a voice recognition function, but may also be, for example, a text such as a dialogue-style minutes manually created by meeting participants using word processing software. The dialogue text may also be a dialogue-style text transcribed using a voice recognition function and then manually corrected, or a text representing an exchange of messages in a chat format, or even an original dialogue-style text, and is not particularly limited.

[0045] The text to be processed in summarization system 1 is stored as a text file (text data) in, for example, storage unit 21 of user terminal 20 or storage unit 11 of server 10, and each text file is given identification information (identification ID) and a file name that allow summarization system 1 or a user to identify the text file. The format of the text file is not particularly limited as long as it allows summarization system 1 to refer to the text information contained in the text file.

[0046] (Text segmentation model M1) The text segmentation model M1 is an artificial intelligence model (AI model) that acquires dialogue-style text as training text and trains it to segment the text into multiple blocks based on the topic of the dialogue.

[0047] The text segmentation model M1 is configured as, for example, an AI model (NLP model) for natural language processing (NLP). For example, the text segmentation model M1 may be configured as a model that divides transcribed dialogue text into multiple blocks by vectorizing each sentence, grouping the documents according to semantic similarity, and separating sentences with different meanings.

[0048] The semantic similarity of documents may be evaluated, for example, by extracting features from each sentence, converting them into vectors consisting of multidimensional feature information, and using the distance between the vectors, etc. In other words, since a conversation contains multiple topics (topics), and if the similarity between sentences exceeds a certain threshold, this indicates a change in topic, an AI model may be configured, for example, to estimate division positions based on predetermined rules regarding such vector distances and divide the conversational text into multiple blocks based on the topics.

[0049] Note that the above configuration is merely an example, and the text segmentation model M1 may be configured as any statistical model that can probabilistically estimate the division positions of text, for example, it may be configured to output the estimated probability of the division positions. The text segmentation model M1 may be a machine learning model such as a deep learning model, a model configured to divide text into blocks using a statistical method, or a model configured to estimate the division positions of segments based on predetermined rules, such as when a specific word appears, the sentence is considered to be the end of the section, and is not particularly limited.

[0050] Furthermore, the output of the text segmentation model M1 may be the output of each divided block, or may be the output of information indicating the division point (such as a line break, a blank line, or a delimiter) inserted into the dialogue text, and is not particularly limited as long as it is possible to identify each block. Furthermore, the text segmentation model M1 may output an estimated probability of a candidate division position for a block. For example, the estimated probability may be compared with a predetermined threshold, and a candidate division position with an estimated probability higher than the threshold may be identified as the division position for the block.

[0051] Here, we will provide some additional information about AI models. An AI model is a model created by artificial intelligence learning input data. For example, it learns the rules and relationships in a set of training data from a huge number of sets of training data that have been input, and by optimizing the parameters within the model, it becomes possible to output an output that corresponds to the learning content in response to the input. In other words, an AI model is a mechanism for determining what output to return in response to input data.

[0052] It should be noted that a known learning algorithm can be applied as a learning algorithm for training an AI model. For example, deep learning (CNN: Convolutional Neural Network, RNN: Recurrent Neural Network, LSTM: Long short-term memory, GAN: Generative Adversarial Network, etc.) may be used as an example of an algorithm for training an AI model.

[0053] Deep learning is an algorithm that deepens the hierarchy of neural networks, a type of machine learning technology. Neural networks are modeled after biological nerves (neurons), and deep learning is a type of machine learning in which a computer automatically extracts features from input data using neural networks. Deep learning can be applied to both supervised and unsupervised learning. Neural networks may be trained using techniques such as gradient descent, stochastic gradient descent, or backpropagation.

[0054] Furthermore, the AI ​​model is not limited to supervised learning, and may be trained using unsupervised learning, semi-supervised learning, etc. The AI ​​learning model does not necessarily have to have a neural network structure, and may be, for example, an SVM (Support Vector Machine) or a decision tree, and is not particularly limited.

[0055] (Learning Section 126) The summarization system 1 may be configured to automate the learning of the text segmentation model M1 by the learning unit 126. The learning unit 126 may perform supervised learning of the text segmentation model M1 using training data.

[0056] The training data may be, for example, a pair of dialogue-style text and a group of blocks obtained by dividing the dialogue-style text into multiple blocks based on topics. For example, the text segmentation model M1 may acquire a pair of dialogue-style text and a group of blocks obtained by dividing the dialogue text based on topics, and use these to learn. Alternatively, the text segmentation model M1 may acquire from the learning unit 126 a pair of dialogue-style text and dialogue text into which information capable of identifying the blocks divided based on topics has been inserted, and use these to learn.

[0057] In addition, the text segmentation model M1 may not only be configured to be automatically learned by the learning unit 126, but may also be configured such that, instead of the learning unit 126, a user inputs, for example, a set of dialogue-style text and a group of blocks into which the dialogue text is divided based on topics into the summarization system 1, and is not particularly limited.

[0058] Then, in training the text segmentation model M1, as an example, a mechanism may be constructed that determines what kind of output to return for the input dialogue text (how to divide the dialogue text into blocks based on topics) by repeatedly updating (i.e., repeatedly learning) the parameters within the model to optimize them based on the rules and relationships, including information such as the similarity of the feature vectors of the sentences contained in the dialogue text, from a combination of a huge number of dialogue texts and blocks into which the dialogue texts are divided based on topics.

[0059] (Divided part 121) The segmentation unit 121, for example, reads out a dialogue text to be processed that is specified by a user from the storage unit 11. Then, the segmentation unit 121 segments the dialogue text into a plurality of blocks based on topics included in the dialogue text using the trained text segmentation model M1 described above. The segmentation unit 121 may, for example, output each of the segments. Alternatively, the segmentation unit 121 may insert information (such as a line break, a blank line, or a delimiter) in the dialogue text to indicate the point at which the dialogue text is to be segmented and output the inserted information; the information is not particularly limited as long as it allows each block to be identified. Furthermore, each segmented block and various pieces of information that can identify each block may be stored in the storage unit 11 together with information that identifies the dialogue text to be segmented (such as a dialogue text identification ID).

[0060] 9 is a diagram showing an example of the results of text segmentation processing by the segmentation unit 121. The original dialogue text did not contain line breaks or blank lines, but in the example shown in FIG. 9, the dialogue text after segmentation processing by the segmentation unit 121 is segmented into a "block for the topic of price increases," a "block for the topic of the number of employees," and a "block for the topic of employee ages," and is output as text with line breaks and blank lines inserted for each topic block, indicating that segmentation processing of the dialogue text is performed based on each topic. Note that the segmentation unit 121 may output each topic block as a separate text file, and this is not a particular limitation.

[0061] In summary model M2 (to be described later) and the like, due to a limit on the number of input characters, if the dialogue text is large, it may not be possible to input the dialogue text all at once, but by performing the above-mentioned division process by division unit 121, it becomes possible to deal with the limit on the number of input characters. Furthermore, the summary text that is finally output from summarization unit 122 is also divided based on topics, which improves readability and makes it easier for those who come across the summary text to understand the content.

[0062] (Summary model M2) The summary model M2 is a type of artificial intelligence used in the field of natural language processing, such as a large-scale language model, which statistically learns the probability distribution of words and sentences from huge amounts of text data, and can generate natural language output like a human (for example, a summary of each of the above blocks or a summary text that combines them into one) in response to a given input (for example, information that can identify each block into which the dialogue text described above is divided).

[0063] More specifically, the M2 summary model is a type of deep learning model based on a computational model called a neural network, which mimics the function of neurons in the human brain. A neural network has a multi-layer structure that receives a huge amount of input data and adjusts its parameters by learning patterns.

[0064] The summary model M2 is composed of a huge neural network consisting of a huge number of parameters, and is a model that has been trained using a large amount of text data. As a result, the summary model M2 has internal linguistic knowledge such as grammar, semantics, and context, and can generate appropriate output for a given input. The summary model M2 according to this embodiment is a model that is trained to generate summary blocks that are summarized for each block divided from the dialogue text by the division unit 121, and to generate a summary text based on the summary blocks. The summary model M2 may be trained by the training unit 126.

[0065] (Summary 122) The summarization unit 122, for example, acquires block-based text into which the dialogue text has been divided from the division unit 121. Then, the summarization unit 122 uses the trained summary model M2 described above to generate a summary block that summarizes each of the acquired blocks (block-based text).

[0066] A typical example of the summarization process by the summarization unit 122 is a process of shaping the text into a highly readable display style, such as by adding a heading (title) to each block or converting a typed transcript into, for example, a bulleted list. That is, as an example, the summary block is generated as a bulleted text with a heading added. The summarization process by the summarization unit 122 may include, in addition to the above-mentioned title-adding process, a process of extracting the main points of each sentence or the entire sentence and summarizing them into short sentences, and is not particularly limited thereto.

[0067] Furthermore, the summarization unit 122 generates summary text based on the summary blocks. For example, the summarization unit 122 may collect multiple individual summary blocks and generate the summary text by storing them in a single text file or a single display output. The generated summary text may be stored in the storage unit 11 together with information identifying the original dialogue text (for example, a dialogue text identification ID, etc.).

[0068] Alternatively, the summarization unit 122 may acquire text that identifies each block in which information indicating division points (such as line breaks, blank lines, or delimiters) has been inserted in the dialogue text, and is not particularly limited. By acquiring information that identifies each block along with the dialogue text, the summarization unit 122 can extract each block from the dialogue text and generate a summary block that summarizes them. The summarization process and the process of generating summary text from the summary block are the same as those described above.

[0069] Figure 10 is a diagram showing an example of summary text. Before the summarization process by the summarization unit 122, the divided text (i.e., the blocks for each topic) was not given a title for each block, and the text was simply typed out. However, in the example shown in Figure 10, the summary text after the summarization process by the summarization unit 122 has headings such as "#Information about customer requests 1," "#Discussion about restaurant profit margins," and "#Information about customer requests 2" for each summary block, and each block has a title and has been converted into a bulleted list style, indicating that summary block generation processing has been performed. Furthermore, multiple summary blocks are stored in a single file or a single display output, indicating that summary text generation processing is performed based on the summary blocks.

[0070] To elaborate on the summarization process, for example, a summary block titled "#Information about customer requests 1" contains the title and four sentences formatted in a bulleted style: "There are no documents for April, so they will need to be copied later," "The customer is asking about sales progress," "Sales progress has not increased much," and "We found that sales are 110% compared to last year." The block of text that forms the basis of this summary block is, for example, a typed-out text that is a transcription of the conversation on this topic before it is formatted.

[0071] (Key sentence extraction model M3) For example, the important sentence extraction model M3 may be configured as an AI model that extracts important sentences from a text when the text is input. Alternatively, the important sentence extraction model M3 may be configured as an AI model that outputs an importance score for each sentence included in the text, or may be configured as an AI model that outputs a flag indicating whether each sentence is important or not.

[0072] That is, the important sentence extraction model M3 can be trained in advance to receive text as an input and output important sentences contained in the text. Alternatively, the important sentence extraction model M3 can be trained in advance to receive text as an input and output a flag indicating the importance or importance of each sentence contained in the text.

[0073] The important sentence extraction model M3 may be trained using, for example, training data (a set of input and output data of the model) in which text is associated with important sentences contained in the text. Alternatively, the important sentence extraction model M3 may be trained using training data (a set of input and output data of the model) in which text is associated with a score indicating the importance of each sentence contained in the text or label information indicating whether the sentence is important (for example, "1" if the sentence is an important sentence and "0" if the sentence is not an important sentence). The important sentence extraction model M3 may be trained by the training unit 126.

[0074] (Important sentence extraction part 123) The key sentence extraction unit 123, for example, acquires text in units of blocks into which the dialogue text has been divided from the division unit 121. Then, the key sentence extraction unit 123 uses the trained key sentence extraction model M3 described above to generate key sentence blocks in which important sentences are extracted from the acquired block-unit text (each block divided from the dialogue text).

[0075] An example of the processing of the key sentence extraction unit 123 will be described in more detail below. The key sentence extraction unit 123 first divides the text into EDUs (Elementary Discourse Units). Here, an EDU is, for example, a unit of a phrase close to a subject-predicate, and is equivalent to a clause, which is the smallest structural unit of a tree in rhetorical structure theory. In rhetorical structure theory, an EDU is expressed as a rhetorical structure tree, which is a constituent tree with EDUs as terminal nodes. For example, Japanese Patent Application Laid-Open No. 2016-162198 discloses discourse structure analysis using EDU division.

[0076] Next, the important sentence extraction unit 123, for example, considers the divided EDUs as one sentence, scores the importance of each sentence (i.e., each EDU), and extracts important sentences by, for example, selecting a predetermined number of EDUs with the highest importance scores in order.

[0077] Figure 11 is a diagram illustrating the EDU division process in the key sentence extraction unit 123. As shown in Figure 11, text 1001 before EDU division by the key sentence extraction unit 123 includes three sentences: (1) "Today, I would like to explain about the xxx product. This product is capable of doing xxx, and thanks to you, it has been well received," (2) "It uses innovative xxx technology that is attracting attention in the industry," and (3) "We have received many comments from customers about its high quality and performance."

[0078] On the other hand, the text 1002 after EDU division by the key sentence extraction unit 123 is divided into five EDUs: (EDU1) "Today, I would like to explain the xxx product," (EDU2) "This product is a product that can xxx xxx," (EDU3) "Thanks to you, it has been very well received," (EDU4) "It uses innovative xxx technology that is attracting attention in the industry," and (EDU5) "We have received many comments from customers about its high quality and performance."

[0079] In the example shown in FIG. 11 , sentence (1) included in text 1001 before EDU segmentation is segmented into three EDUs, (EDU1), (EDU2), and (EDU3), in text 1002 after DU segmentation. The key sentence extraction unit 123 then uses the key sentence extraction model M3 to score the importance of each EDU. Here, the scoring of the importance of each EDU and the process of extracting important EDUs can be performed by treating each EDU as a single sentence, for example, using a key sentence extraction method using a classifier configured by Support Vector Machine, as disclosed in Japanese Patent Application Laid-Open No. 2003-36262. Note that the key sentence extraction process may be performed using the key sentence extraction model M3 trained to directly score the importance of each EDU without treating each EDU as a single sentence, and is not particularly limited.

[0080] Figure 12 is a diagram showing an image of the key sentence extraction process in the key sentence extraction unit 123. The dependency tree shown in Figure 12 represents the dependency of the first sentence (sentence (1) above) of text 1001 before EDU division in Figure 11. The dependency tree represents the dependency relationship in a tree structure, with phrases considered as nodes and the relationship between the dependency source phrase and the dependency destination phrase as edges. In the dependency tree shown in Figure 12, the dependency source EDU1 and the dependency destination EDU3 are connected by an edge, representing the dependency relationship of sentence (1) above.

[0081] 12 is a schematic representation of the result of dividing the pre-EDU division text 1001 shown in FIG. 11 into EDU1 to EDU5 shown in post-EDU division text 1002. In FIG.

[0082] 12 shows the results of sorting EDU1 to EDU5 in descending order of the score (EDU importance score) that represents the importance of each EDU. That is, as described above, the key sentence extraction unit 123 regards each divided EDU as a single sentence and scores the importance of each sentence (i.e., each EDU). In the example shown in processing image 2, of EDU1 to EDU5, EDU1 has the highest EDU importance score, followed by EDU4, EDU5, EDU3, and EDU2.

[0083] 12 shows that EDU1, EDU3, and EDU4 are selected when it is predetermined to select the top three EDUs with the highest EDU importance scores. In processing image 2, the top three EDUs with the highest EDU importance scores are EDU1, EDU4, and EDU5, in descending order of EDU importance score, but EDU1 forms a subtree with EDU3 as its dependency, as shown in the dependency tree.

[0084] That is, in the example of processing image 3, when the key sentence extraction unit 123 selects an EDU based on the EDU importance score, if the EDU to be selected constitutes a subtree of the dependency tree, a restriction is set to select an EDU so as to generate a sentence that conforms to the dependency relationship represented by the subtree. Therefore, when the key sentence extraction unit 123 selects EDU1 with the highest EDU importance score, it next selects EDU3 that has a dependency relationship with EDU1, and then selects EDU4 with the next highest EDU importance score.

[0085] As a result, as shown in processing image 3, the key sentence extraction unit 123 extracts the sentence composed of EDU1 and EDU3 and the sentence EDU4 as key sentences. That is, the key sentence extraction unit 123 extracts, from text 1001 in Fig. 11, the sentence combining EDU1 and EDU3, "Today, I would like to explain about the xxx product, which has been well received," and the sentence from EDU4, "It uses innovative xxx technology that is attracting attention in the industry."

[0086] In the above example, the important sentence extraction model M3 is described as being configured to output the EDU importance score and execute the processing selected by the important sentence extraction unit 123, but the important sentence extraction model M3 may also be configured to directly output important sentences based on the EDU importance score and the dependency tree, and is not particularly limited thereto.

[0087] In this way, for each block into which the dialogue text is divided, the key sentence extraction unit 123 performs a process of extracting key sentences to generate a key sentence block (for example, text in a typed state before being converted into each summary block).

[0088] Then, the summarization unit 122 acquires the key sentence blocks from the key sentence extraction unit 123 and generates the above-mentioned summary blocks (for example, each block for each topic shown in FIG. 10) based on the key sentence blocks. Furthermore, as described above, the summarization unit 122 generates summary text (for example, the entire text shown in FIG. 10) based on the summary blocks.

[0089] (Removal of unnecessary sections) Furthermore, the summarizing section 122 may be configured to remove unnecessary sections from the generated summary text based on the importance of at least a portion of each summary block that constitutes the summary text.

[0090] Here, a section refers to, for example, a range spanning multiple sentences. As a typical example, the summary unit 122 regards a summary block as one section, scores the importance of each section (i.e., each block), ranks each section by a score (section importance score) representing the importance of the section, and keeps only a predetermined number of top-ranked sections, eliminating the other sections.

[0091] It is not necessary to treat each summary block as a single section. For example, a block may be divided into multiple sub-blocks, and a section importance score may be evaluated for the importance of at least some sections (sub-blocks) within the block. This may result in removing not only the entire block, but also some sections within the summary block; this is not a particular limitation.

[0092] For example, the summarization unit 122 may treat each summary block as a single sentence, calculate a segment importance score using an algorithm called LexRank, and remove unnecessary segments. LexRank is an extractive summarization algorithm that uses the concept of PageRank in search engines.

[0093] PageRank regards the link structure on the Internet as a graph with each web page as a node and links as edges, and evaluates the importance of each web page, with important sites and web pages that are linked to by many sites being given a higher importance rating.Similarly, LexRank regards the structure of a document as a graph with each sentence as a node and links as the similarity between sentences, and evaluates the importance of each sentence, with important sentences and sentences that are similar to many other sentences being given a higher importance rating.

[0094] The summarization unit 122 may further apply this, for example, by treating all or part of a section (sub-block) of a summary block as a single sentence, and by applying the LexRank method described above, score the importance of each section, select a predetermined number of sections with the highest section importance scores in order, and remove the remaining sections.

[0095] Fig. 13 is a diagram showing an example of summary text before unnecessary section removal processing. For example, summarization section 122 first generates the summary text shown in Fig. 13, and then performs unnecessary section removal processing to generate the summary text shown in Fig. 10.

[0096] As shown in Figure 13, the summary text before removing unnecessary sections contains seven summary blocks with the titles "#Summary of conversation," "#Important information," "#Information about customer requests," "#Concerns about customer sales," "#Discussion about restaurant profit margins," "#Information about customer requests," and "#Conversation about customer requests."

[0097] Here, the summarization unit 122 regards each summary block as a single interval and performs the above-mentioned unnecessary interval removal process on these seven intervals, thereby leaving the three intervals (summary blocks) titled "#Information about customer requests 1," "#Discussion about restaurant profit margins," and "#Information about customer requests 2" shown in Figure 10, and removing the other four intervals (summary blocks) surrounded by dashed lines.As a result, the summarization unit 122 ultimately generates a summary text consisting of the three intervals (summary blocks) shown in Figure 10.

[0098] The summarization unit 122 may be configured to remove unnecessary sections from the input text using, for example, an unnecessary section removal model M4. For example, the unnecessary section removal model M4 may output text from which unnecessary sections have been removed. Alternatively, the unnecessary section removal model M4 may be configured to output information for evaluating, for each section included in the text, whether or not a functional block such as the summarization unit 122 is an unnecessary section, and is not particularly limited.

[0099] For example, the unnecessary section removal model M4 may be configured as an AI model that, when a text consisting of multiple sections is input, removes unnecessary sections from the text. Alternatively, the unnecessary section removal model M4 may be configured as an AI model that outputs an importance score for each section included in the text, or may be configured as an AI model that outputs a flag that determines whether each section is a removal target.

[0100] That is, the unnecessary section removal model M4 can be trained in advance to receive, for example, text that can be identified for a plurality of sections as input, remove unnecessary sections included in the text, and output the result. Alternatively, the unnecessary section removal model M4 can be trained in advance to receive, for example, text that can be identified for a plurality of sections as input, and output, for each section included in the text, a flag indicating the importance or whether or not the section is to be removed.

[0101] The unnecessary section removal model M4 may be trained using, for example, training data (a set of input / output data of the model) in which text capable of specifying multiple sections is associated with sections to be removed (or sections not to be removed) included in the text. Alternatively, the unnecessary section removal model M4 may be trained using, for example, training data (a set of input / output data of the model) in which text capable of specifying multiple sections is associated with scores indicating the importance of each section included in the text and label information indicating whether or not the section is to be removed (for example, "1" if the section corresponds to the section to be removed, and "0" if the section does not correspond). The unnecessary section removal model M4 may be trained by the training unit 126.

[0102] It should be noted that the process of extracting important sentences by the above-mentioned important sentence extraction unit 123 is a process for extracting important sentences mainly in units of one sentence or less, such as each sentence included in the text or units obtained by further breaking down each sentence, such as EDU, into phrases, whereas the process of removing unnecessary sections by the summarization unit 122 etc. is a process for determining the importance of sections mainly consisting of multiple sentences, and the units for evaluating the importance are different.

[0103] That is, the key sentence extraction process in the key sentence extraction unit 123 extracts important points in the conversation for each topic on a sentence or phrase basis to generate key sentences, thereby extracting the main points of each topic and compressing the text. On the other hand, unnecessary section removal deletes entire sections of topics that are not necessarily required to remain in the final summary text, such as casual conversation before getting into serious business negotiations or topics that deviate from the main topic, and is different from the process in the key sentence extraction unit 123.

[0104] <<Select a theme>> (Processing according to dialogue-related information) In the summary system 1, the dialogue-related information receiving unit 124 may receive input of information related to the dialogue (dialogue-related information), and the summarization unit 122 may generate a summary block or summary text in a predetermined format for the dialogue-related information input by the dialogue-related information receiving unit 124.

[0105] For example, in the summarization system 1, as shown in Fig. 5, a screen for selecting an interview type and an interview theme is presented via the user terminal 20. Then, when the user selects an interview type and an interview theme (dialogue-related information), the dialogue-related information receiving unit 124 receives input of the dialogue-related information selected by the user.

[0106] In the example shown in Figure 5, the dialogue-related information receiving unit 124 accepts input of "corporate sales" and "retail sales" as interview types, and input of "on-the-spot interview," "financial status interview," "interview on other issues," "deadline management for current loans and property loans," "solution proposal (loans)," "solution proposal (other than loans)," and "loan / guarantee offer from customer" as interview themes.

[0107] For example, as a format for "live hearing," which is an example of dialogue-related information, for example, "item style" is defined, and is stored in the memory unit 11 of the server 10 in association with data representing the format (output format, etc.).

[0108] For example, when the dialogue-related information receiving unit 124 receives input of "live hearing," the summarizing unit 122 reads format data relating to the "bullet style" of the format predetermined for "live hearing" when generating a summary block or summary text, and shapes the summary block or summary text into a bullet style in accordance with the format determined by the format data.

[0109] Furthermore, when other dialogue-related information is selected, for example, if a "tabular style" is defined as the format for the dialogue-related information, the summarization unit 122 formats the summary block and summary text in a tabular style. Note that the dialogue-related information is not limited to the above-mentioned interview type and interview theme, and may be, for example, the dialogue agenda or the participants in the dialogue, and is not particularly limited.

[0110] Furthermore, for example, the text segmentation model M1 may be configured as a model that has been trained on dialogue-related information. For example, the text segmentation model M1 may be an AI model trained specifically on "live interviews," which is an example of dialogue-related information. That is, in this case, the text segmentation model M1 is constructed as an AI model trained using training data specialized for live interviews (for example, a set of dialogue text from live interviews and segmentation results of the dialogue text), and can accurately and appropriately process the segmentation of text related to live interviews. Alternatively, the text segmentation model M1 may be configured as an AI model trained using training data specialized for "financial status interviews," "corporate sales," etc., and constructed separately and independently, or may be constructed as an AI model in which these are integrated into one; this is not particularly limited.

[0111] For example, when the dialogue-related information receiving unit 124 receives an input of "live commentary hearing", the dividing unit 121 divides the dialogue text based on the topic using the text division model M1 that has been trained on "live commentary hearing".

[0112] Furthermore, for example, the important sentence extraction model M3 may be configured as a model trained on dialogue-related information. For example, the important sentence extraction model M3 may be an AI model trained specifically on "live interviews," an example of dialogue-related information. In other words, in this case, the important sentence extraction model M3 is constructed as an AI model trained using training data specialized for live interviews (e.g., pairs of blocks into which the dialogue text of the live interviews is divided and the results of key sentence extraction from those blocks), and can accurately and appropriately process the extraction of important sentences from blocks of text related to live interviews. Alternatively, the important sentence extraction model M3 may be configured as an independent model trained using training data specialized for "financial status interviews," "corporate sales," etc., or as an integrated AI model; this is not particularly limited.

[0113] For example, when the dialogue-related information receiving unit 124 receives input of "live commentary hearing," the key sentence extraction unit 123 extracts key sentences from the blocks into which the dialogue text is divided, using the key sentence extraction model M3 that has been trained on "live commentary hearing."

[0114] Furthermore, with regard to the removal of the unnecessary sections described above, the summarization unit 122 may remove the unnecessary sections based on the importance of the dialogue-related information received by the dialogue-related information receiving unit 124. For example, when scoring the importance of the sections using the LexRank algorithm described above, a weighting parameter for calculating the similarity of sentences that is preset for each piece of dialogue-related information may be used. That is, the weighting parameter for calculating the similarity of sentences may be stored in the storage unit 11 according to each piece of dialogue-related information, and may be read out when scoring the importance.

[0115] For example, when the dialogue-related information receiving unit 124 receives an input of "live commentary hearing", the summarizing unit 122 removes unnecessary sections based on the importance of "live commentary hearing".

[0116] For example, the unnecessary section removal model M4 may be configured as a model that has been trained on dialogue-related information. For example, the unnecessary section removal model M4 may be an AI model trained specifically on "live interviews," which is an example of dialogue-related information. That is, in this case, the unnecessary section removal model M4 is constructed as an AI model trained using training data specialized for live interviews (e.g., a pair of a block in which the dialogue text of the live interview is summarized and the result of unnecessary section removal for that block), and can accurately and appropriately remove unnecessary sections from blocks of text related to live interviews. Alternatively, the unnecessary section removal model M4 may be configured as an AI model trained using training data specialized for "financial status interviews," "corporate sales," etc., and independently constructed, or may be constructed as a single integrated AI model; this is not particularly limited.

[0117] For example, when the dialogue-related information receiving unit 124 receives input of "live commentary hearing," the summarizing unit 122 uses the unnecessary section removal model M4 that has been trained on "live commentary hearing" to remove unnecessary sections based on the importance of the blocks that summarize the dialogue text.

[0118] <<Transcription>> In the summarization system 1, the dialogue text generation unit 125 may generate a dialogue text based on the audio of the dialogue. For example, the terminal 20 may be equipped with a recording function, and when a user holds a meeting, the audio of the dialogue in the meeting may be recorded via the microphone 23.

[0119] Then, when the audio file of the dialogue recorded by the terminal 20 is sent to the server 10, the dialogue text generation unit 125 performs a speech recognition process and executes a process of transcribing the dialogue audio to generate a dialogue text. The dialogue text generated by the dialogue text generation unit 125 is stored in the storage unit 11 together with information such as identification information, title, and processing time, and is saved as data in a state that can be read by the summarization system 1. Publicly known technology may be used for the voice recognition.

[0120] Furthermore, in the summarization system 1, the dialogue text generation unit 125 may detect and remove fillers when generating dialogue text. Fillers include transitional expressions and hesitations, such as "um," "um," "hmm," and "you see." The dialogue text generation unit 125 may, for example, execute a process of searching for words corresponding to fillers from preliminarily generated dialogue text. For example, words corresponding to fillers may be pre-registered in the storage unit 11, and the dialogue text generation unit 125 may determine whether the dialogue text contains any of the corresponding words. Alternatively, the dialogue text generation unit 125 may, for example, be configured to remove fillers using an AI model that has been trained to input dialogue text and output dialogue text from which fillers have been removed.

[0121] Fig. 14 is a diagram illustrating an example of the dialogue text generation process. As shown in Fig. 14, the text before filler removal contains fillers such as "desu ne" (well, right) and "eh" (well). The dialogue text generation unit 125 detects and removes these fillers. In the text after filler removal, the fillers have been removed, but the punctuation is not appropriate.

[0122] Therefore, in the summarization system 1, the dialogue text generation unit 125 may detect punctuation errors and correct punctuation detected as an error when generating dialogue text. In speech recognition processing, processing is performed for each speech segment, so appropriate punctuation is not necessarily used in the transcribed text, which may affect the performance of each subsequent process. Therefore, correcting the punctuation to the correct one improves the accuracy of the summarization processing in the summarization system 1.

[0123] The dialogue text generation unit 125 may, for example, execute a process of detecting incorrect punctuation marks from a preliminarily generated dialogue text. For example, the dialogue text generation unit 125 may execute an existing algorithm for detecting incorrect punctuation marks, such as that implemented in word processing software. Alternatively, the dialogue text generation unit 125 may be configured to correct punctuation marks using an AI model that has been trained to output dialogue text in which incorrect punctuation marks have been corrected, when dialogue text is input.

[0124] In the example shown in FIG. 14, the text before punctuation correction contains improper punctuation, such as a period after a subject or object phrase, such as "Today," or "Explain the xxx product." The dialogue text generation unit 125 detects and corrects such improper punctuation. In the text after punctuation correction shown in FIG. 14, incorrect periods have been replaced with commas, such as "Today," or "Explain the xxx product," and unnecessary periods have been removed, thereby correcting the improper punctuation.

[0125] <<Variation 1>> As mentioned above, an example of the operation of the summarization system 1 has been described using Figure 8, but the operation of the summarization system 1 can also be modified as follows. Figure 15 is a flowchart illustrating an example of the outline of the operation of the summarization system 1. In the example shown in Figure 15, the summarization system 1 has the key sentence extraction unit 123 generate important dialogue text in which important sentences are extracted from dialogue text using an important sentence extraction model, and the division unit 121 obtains the important dialogue text as dialogue text and divides it into a plurality of blocks.

[0126] Step S1501 is the same process as step S801 in FIG. 8, and therefore a description thereof will be omitted.

[0127] In step S1502, the summarization system 1 executes an important sentence extraction process. For example, the important sentence extraction unit 123 generates an important dialogue text by extracting important sentences from the entire dialogue text. For example, the important sentence extraction unit 123 may generate the important sentence text using an important sentence extraction model M3 that has been trained to extract important sentences from the entire dialogue text. As an example, the important sentence extraction model may be configured as an AI model trained using pairs of dialogue text and important dialogue text as training data. Alternatively, the important sentence extraction model may be configured as an AI model that outputs the importance of each sentence included in the dialogue text, and the important sentence extraction unit 123 may generate the important dialogue text by extracting a predetermined number of sentences in order of increasing importance.

[0128] In step S1503, the summarization system 1 executes a text segmentation process. For example, the segmentation unit 121 uses the text segmentation model M1 to segment the important dialogue text into a plurality of blocks based on topics.

[0129] In step S1504, the summarization system 1 executes summarization processing. For example, the summarization unit 122 uses the summary model M2 to generate summary blocks that summarize each block divided from the important dialogue text, and then integrates these summary blocks to generate a summary text.

[0130] This completes one example of the operation of the summarization system 1. Note that the flowchart shown in Fig. 15 is merely an example, and for example, when summarizing dialogue text that has already been transcribed, step S1501 is not necessary, or alternatively, after step S1504, an unnecessary section removal process similar to step S805 shown in Fig. 8 may be performed.

[0131] <<Variation 2>> The operation of summarization system 1 can also be modified as follows. Figure 16 is a flowchart illustrating an example of the outline of the operation of summarization system 1. In the example shown in Figure 16, in summarization system 1, key sentence extraction unit 123 uses a key sentence extraction model to extract key sentences from the summary blocks generated by summarization unit 122.

[0132] Steps S1601 and S1602 are the same as steps S801 and S802 in FIG. 8, and therefore a description thereof will be omitted.

[0133] In step S1603, the summarization system 1 executes summarization processing. This processing is similar to step S804 in Fig. 8; for example, the summarization unit 122 uses summary model M2 to generate summary blocks that summarize each block divided from the dialogue text, and then integrates these summary blocks to generate summary text. While step S804 generated summary blocks that summarize key sentence blocks, the example in Fig. 16 differs in that summary processing is performed on blocks that have not undergone key sentence extraction processing to generate summary blocks.

[0134] In step S1604, the summary system 1 executes an important sentence extraction process. For example, the important sentence extraction unit 123 extracts important sentences from each summary block. That is, the important sentence extraction unit 123 generates a summary block consisting of important sentences. As a result, the summary text is composed of summary blocks consisting of important sentences. For example, the important sentence extraction unit 123 may generate a summary block consisting of important sentences using an important sentence extraction model M3 that has been trained to extract important sentences from summary blocks. As an example, the important sentence extraction model may be configured as an AI model trained using pairs of summary blocks and summary blocks consisting of important sentences as training data. Alternatively, the important sentence extraction model may be configured as an AI model that outputs the importance of each sentence included in the summary block, and the important sentence extraction unit 123 may generate a summary block consisting of important sentences by extracting a predetermined number of sentences in order of decreasing importance.

[0135] This completes one example of the operation of the summarization system 1. Note that the flowchart shown in Fig. 16 is merely an example, and for example, when summarizing dialogue text that has already been transcribed, step S1601 is not necessary, or alternatively, after step S1604, an unnecessary section removal process similar to step S805 shown in Fig. 8 may be performed.

[0136] <<Variation 3>> The operation of summarization system 1 can also be modified as follows. Figure 17 is a flowchart outlining an example of the operation of summarization system 1. In the example shown in Figure 17, summarization unit 122 in summarization system 1 removes unnecessary sections from blocks based on the importance of at least some of the blocks, and generates summary text based on the blocks from which the unnecessary sections have been removed.

[0137] Steps S1701 and S1702 are the same as steps S801 and S802 in FIG. 8, and therefore a description thereof will be omitted.

[0138] In step S1703, the summarization system 1 executes an unnecessary section removal process. For example, the summarization system 1 distinguishes and removes unnecessary sections from each block divided from the dialogue text. For example, the summarization unit 122 removes unnecessary sections from each block based on the importance of at least a part of each block.

[0139] In step S1704, the summarization system 1 executes summarization processing. For example, the summarization unit 122 uses the summary model M2 to generate a summary block that summarizes each block remaining after the unnecessary sections have been removed, and then integrates the summary blocks to generate a summary text.

[0140] This completes one example of the operation of the summarization system 1. Note that the flowchart shown in Figure 17 is merely an example, and for example, when summarizing dialogue text that has already been transcribed, step S1701 is not necessary, or key sentence extraction processing may be performed as appropriate.

[0141] <<Variation 4>> The operation of summarization system 1 can also be modified as follows: Figure 18 is a flowchart outlining an example of the operation of summarization system 1. In the example shown in Figure 18, summarization unit 122 in summarization system 1 removes unnecessary sections from within a summary block based on the importance of at least a portion of the summary block.

[0142] Steps S1801 and S1802 are the same as steps S801 and S802 in FIG. 8, and therefore a description thereof will be omitted.

[0143] In step S1803, the summarization system 1 executes summarization processing. For example, the summarization unit 122 uses the summary model M2 to generate summary blocks that summarize each block divided from the dialogue text, and then integrates these summary blocks to generate summary text.

[0144] In step S1804, the summarization system 1 executes unnecessary section removal processing. For example, the summarization unit 122 removes all or part of unnecessary blocks from the summary text based on the importance of at least some of the summary blocks.

[0145] This completes one example of the operation of the summarization system 1. Note that the flowchart shown in Fig. 18 is merely an example, and for example, when summarizing dialogue text that has already been transcribed, step S1801 is not necessary, or key sentence extraction processing may be performed as appropriate.

[0146] <<Variation 5>> The operation of summarization system 1 can also be modified as follows: Fig. 19 is a flowchart illustrating an example of the outline of the operation of summarization system 1. In the example shown in Fig. 19, in summarization system 1, division unit 121 divides dialogue text, from which unnecessary sections have been removed, into multiple blocks based on the importance of at least a portion of the dialogue text.

[0147] Step S1901 is the same process as step S801 in FIG. 8, and therefore a description thereof will be omitted.

[0148] In step S1902, the summarization system 1 executes an unnecessary section removal process. In the summarization system 1, for example, the division unit 121 removes unnecessary sections from the dialogue text. For example, the unnecessary section removal model M4 may be trained to remove unnecessary sections from input dialogue text and output the resulting text, and the division unit 121 may use the unnecessary section removal model M4 to remove the unnecessary sections. In this case, for example, a large number of pairs of dialogue text before and after unnecessary section removal may be prepared as training data, and the above-mentioned unnecessary section removal model M4 may be constructed by repeatedly training using the training data.

[0149] In step S1903, the summarization system 1 executes text segmentation processing. For example, the segmentation unit 121 uses the text segmentation model M1 to segment the dialogue text from which unnecessary sections have been removed into a plurality of blocks based on topics.

[0150] In step S1904, the summarization system 1 executes summarization processing. For example, the summarization unit 122 uses the summary model M2 to generate summary blocks that summarize each block, and then integrates these summary blocks to generate summary text.

[0151] This completes one example of the operation of the summarization system 1. Note that the flowchart shown in Fig. 19 is merely an example, and for example, when summarizing dialogue text that has already been transcribed, step S1901 is not necessary, or key sentence extraction processing may be performed as appropriate.

[0152] [Software implementation example] The control block of the server 10 may be realized by a logic circuit (hardware) formed on an integrated circuit (IC chip) or the like, or may be realized by software. In the latter case, each of the server 10 and the user terminal 20 is configured using, for example, a computer (electronic calculator).

[0153] (Physical configuration of server 10) FIG. 20 is a block diagram illustrating the physical configuration of a computer used as the server 10 and the user terminal 20. As shown in FIG.

[0154] 20, the server 10 can be configured by a computer including a bus 110, a processor 101, a main memory 102, an auxiliary memory 103, and a communication interface 104. The processor 101, the main memory 102, the auxiliary memory 103, and the communication interface 104 are connected to one another via the bus 110.

[0155] The processor 101 may be, for example, a CPU (Central Processing Unit), a microprocessor, a digital signal processor, a microcontroller, or a combination of these.

[0156] The main memory 102 may be, for example, a semiconductor RAM (random access memory).

[0157] For example, a flash memory, a hard disk drive (HDD), a solid state drive (SSD), or a combination of these may be used as the auxiliary memory 103. The auxiliary memory 103 stores a program for causing the processor 101 to execute the above-described operations of the server 10. The processor 101 loads the program stored in the auxiliary memory 103 onto the main memory 102 and executes each instruction included in the loaded program.

[0158] The communication interface 104 is an interface that connects to the network N1.

[0159] In this example, the processor 101 and the communication interface 104 are examples of hardware elements that realize the control unit 12. The main memory 102 and the auxiliary memory 103 are examples of hardware elements that realize the storage unit 11.

[0160] (Physical configuration of user terminal 20) 20, the user terminal 20 can be configured by a computer including a bus 210, a processor 201, a main memory 202, an auxiliary memory 203, a communication interface 204, and an input / output interface 205. The processor 201, the main memory 202, the auxiliary memory 203, the communication interface 204, and the input / output interface 205 are connected to one another via the bus 210. An input device 206 and an output device 207 are connected to the input / output interface 205.

[0161] The processor 201 may be, for example, a CPU, a microprocessor, a digital signal processor, a microcontroller, or a combination of these.

[0162] The main memory 202 may be, for example, a semiconductor RAM.

[0163] The auxiliary memory 203 may be, for example, a flash memory, an HDD, an SSD, or a combination thereof. The auxiliary memory 203 stores a program for causing the computer to operate as the user terminal 20. The processor 201 loads the program stored in the auxiliary memory 203 onto the main memory 202 and executes each instruction included in the loaded program. The auxiliary memory 203 also stores various data referenced by the processor 201 to cause the computer to operate as the user terminal 20.

[0164] The communication interface 204 is an interface for connecting to a network.

[0165] The input / output interface 205 may be, for example, a USB interface, a short-range communication interface such as infrared or Bluetooth (registered trademark), or a combination of these.

[0166] The input device 206 may be, for example, a keyboard, a mouse, a touchpad, a microphone, or a combination thereof. The output device 207 may be, for example, a display, a printer, a speaker, or a combination thereof.

[0167] In this example, the processor 201 and the communication interface 204 are examples of hardware elements that realize the control unit 22. The main memory 202 and the auxiliary memory 203 are examples of hardware elements that realize the storage unit 21.

[0168] Instead of storing the above-described programs in auxiliary memories 103 and 203, the programs may be recorded on an external recording medium and supplied to the corresponding computer by being read from the external recording medium. The external recording medium may be a computer-readable "non-transitory tangible medium," such as a tape, a disk, a card, a semiconductor memory, or a programmable logic circuit. The above-described programs may also be supplied to the computer via any transmission medium (such as a communications network or broadcast waves). Another aspect of the present invention may be realized in the form of a data signal embedded in a carrier wave, in which the programs are embodied by electronic transmission.

[0169] The present invention is not limited to the above-described embodiments, and various modifications are possible within the scope of the claims. Embodiments obtained by appropriately combining the technical means disclosed in different embodiments are also included in the technical scope of the present invention.

[0170] <<Summary of this Disclosure>> The following description will focus on the actions and effects of the summary system 1 according to one embodiment of the present disclosure. All of the configurations described below can also be used as configurations of this embodiment.

[0171] A summarization method according to one aspect of the present disclosure uses an information processing device to perform a segmentation step of segmenting dialogue text representing the content of a dialogue into multiple blocks based on topics included in the dialogue using a text segmentation model, and a summarization step of generating summary blocks summarized for each of the blocks using the summary model, and generating summary text based on the summary blocks.

[0172] According to the above configuration, summarization is performed for each divided block using a summarization model. The summarization process may include, for example, adding titles to the blocks and converting the style to bullet points. This allows summarization using a summarization model even when the dialogue text is large in volume. Furthermore, because the dialogue text is divided based on topics, it is possible to provide highly readable summary text that is appropriately summarized for each topic.

[0173] A summarization method according to one aspect of the present disclosure further includes an important sentence extraction step of using an important sentence extraction model to generate an important sentence block from which important sentences are extracted for each block, and the summarization step uses the summary model to generate the summary block based on the important sentence block.

[0174] According to the above configuration, since key sentences are extracted before summarizing, it is possible to provide a more appropriately summarized summary text.

[0175] In a summarizing method according to one aspect of the present disclosure, the summarizing step removes unnecessary sections from the summary text based on the importance of at least a portion of the generated summary blocks.

[0176] According to the above configuration, unnecessary sections are removed from all or part of each summary block that makes up the summary text, so that unnecessary information is removed from the summary text that is ultimately provided, making it easier to understand the content and providing a more appropriate summary text.

[0177] A summarizing method according to one aspect of the present disclosure further includes a dialogue-related information receiving step of receiving input of dialogue-related information regarding the dialogue, and the summarizing step generates the summary block or the summary text based on a predetermined format for the input dialogue-related information.

[0178] According to the above configuration, it is possible to generate a summary text in an appropriate format according to dialogue-related information such as the theme of the dialogue.

[0179] A summarization method according to one aspect of the present disclosure further includes a dialogue-related information receiving step for receiving input of dialogue-related information regarding the dialogue, and the segmentation step divides the input dialogue-related information into the plurality of blocks using the text segmentation model that has been trained on the input dialogue-related information.

[0180] According to the above configuration, the dialogue text is divided optimally according to dialogue-related information such as the theme of the dialogue, so that a more appropriate summary text can be generated.

[0181] A summarization method according to one aspect of the present disclosure further includes a dialogue-related information receiving step for receiving input of dialogue-related information regarding the dialogue, and the summarization step generates the key sentence block using the key sentence extraction model that has been trained on the input dialogue-related information.

[0182] According to the above configuration, an optimum key sentence extraction process is performed according to dialogue-related information such as the theme of the dialogue, so that a more appropriate summary text can be generated.

[0183] A summarization method according to one aspect of the present disclosure further includes a dialogue-related information receiving step for receiving input of dialogue-related information related to the dialogue, and the summarization step removes the unnecessary sections based on the importance of the dialogue-related information that has been input.

[0184] According to the above configuration, the optimal unnecessary section removal process is performed according to dialogue-related information such as the theme of the dialogue, so that a more appropriate summary text can be generated.

[0185] The summarization method according to one aspect of the present disclosure further includes a dialogue text generation step of generating the dialogue text based on the speech of the dialogue.

[0186] According to the above configuration, if the voice of a conversation during a business meeting or the like is recorded, a summary text that appropriately summarizes the content of the conversation is automatically generated, thereby significantly improving user convenience.

[0187] In the summarization method according to one aspect of the present disclosure, the dialogue text generating step detects fillers when generating the dialogue text and removes the detected fillers.

[0188] According to the above configuration, when a summary text is generated from a speech, it is possible to provide a summary text that is easy to read and does not contain fillers.

[0189] In the summarization method according to one aspect of the present disclosure, the dialogue text generation step detects punctuation errors when generating the dialogue text, and corrects the punctuation errors detected as punctuation errors.

[0190] According to the above configuration, when generating a summary text from speech, it is possible to provide a summary text that is easy to read and uses punctuation appropriately.

[0191] A summarization method according to one aspect of the present disclosure further includes an important sentence extraction step of generating an important dialogue text in which important sentences are extracted from the dialogue text using an important sentence extraction model, and the division step obtains the important dialogue text as the dialogue text and divides it into the plurality of blocks.

[0192] According to the above configuration, an important dialogue text consisting of key sentences extracted from the dialogue text is divided into a plurality of blocks, and a summary block is generated for each block. This allows the division process to be performed based on the important dialogue text, improving the accuracy of text division and making it possible to provide an appropriate summary text.

[0193] In a summarizing method according to one aspect of the present disclosure, the summarizing step extracts key sentences from the generated summary block using a key sentence extraction model.

[0194] According to the above configuration, after summary blocks are generated, key sentences are extracted, and summary text is generated based on the summary blocks consisting of the key sentences. Alternatively, once the summary text is generated, key sentences are extracted for each summary block contained in the summary text. As a result, the summary text is composed of summary blocks consisting of key sentences, making it possible to provide an appropriate summary text from which key sentences have been extracted with high accuracy.

[0195] In a summarizing method according to one aspect of the present disclosure, the summarizing step removes unnecessary sections from the blocks based on the importance of at least a portion of the blocks, and generates the summary text based on the blocks from which the unnecessary sections have been removed.

[0196] According to the above configuration, since the summarization process is performed after removing unnecessary sections, there is no need to perform summarization on unnecessary sections. This makes it possible to provide appropriate summary text while reducing the burden of summarization process.

[0197] In a summarization method according to one aspect of the present disclosure, the summarization step removes unnecessary sections from the summary block based on the importance of at least a portion of the summary block.

[0198] According to the above configuration, once a summary text is generated, unnecessary sections are removed from all or part of each summary block that makes up the summary text, making it possible to provide an appropriate summary text from which unnecessary sections have been accurately removed.

[0199] In a summarizing method according to one aspect of the present disclosure, the dividing step divides the dialogue text, from which unnecessary sections have been removed, into the plurality of blocks based on the importance of at least a portion of the dialogue text.

[0200] According to the above configuration, segmentation processing is performed after unnecessary sections are removed, improving the accuracy of text segmentation and making it possible to provide an appropriate summarized text.

[0201] A summarization system according to one aspect of the present disclosure includes a segmentation unit that uses a text segmentation model to segment dialogue text representing the content of a dialogue into multiple blocks based on topics included in the dialogue, and a summarization unit that uses the summarization model to generate summary blocks that summarize each of the blocks, and generates summary text based on the summary blocks.

[0202] According to the above configuration, the same effects as those of the summarizing method described above can be achieved.

[0203] A program according to one aspect of the present disclosure is a program for causing a computer to function as the summarization system described above, and causes the computer to function as each of the above-described units.

[0204] According to the above configuration, the same effects as those of the summarization system described above can be achieved. [Industrial Applicability]

[0205] One aspect of the present disclosure is useful in an information processing system for summarizing interactive text. [Explanation of symbols]

[0206] 1. Summary System 10 Servers 20 User terminal 11, 21 Storage section 12, 22 Control section 23. Mike 101, 201 processors 102, 202 Main memory 103, 203 Auxiliary memory 104, 204 communication interface 110, 210 buses 121 Division 122 Summary 123 Important sentence extraction part 124 Dialogue-related Information Reception Department 125 Dialogue Text Generation Unit 126 Learning Department 205 Input / Output Interface 206 Input Device 207 Output Device M Artificial Intelligence Model M1 text classification model M2 summary model M3 Key Sentence Extraction Model M4 dead zone removal model

Claims

1. Using an information processing device, a segmentation step of segmenting a dialogue text representing the content of the dialogue into a plurality of blocks based on topics included in the dialogue using a text segmentation model; a summarization step of generating a summary block that summarizes each of the blocks using the summary model, and generating a summary text based on the summary block.

2. further performing an important sentence extraction step of generating an important sentence block from which important sentences are extracted for each block using an important sentence extraction model; The method of claim 1 , wherein the summarizing step uses the summary model to generate the summary block based on the key sentence block.

3. The summarizing method according to claim 2 , wherein the summarizing step removes unnecessary sections from the summary text based on at least a degree of importance of the generated summary blocks.

4. a dialogue-related information receiving step of receiving input of dialogue-related information about the dialogue; The summarizing method according to claim 1 , wherein the summarizing step generates the summary block or the summary text based on a predetermined format for the inputted dialogue-related information.

5. a dialogue-related information receiving step of receiving input of dialogue-related information about the dialogue; The summarization method according to claim 1 , wherein the dividing step divides the input text into the plurality of blocks using the text division model that has been trained on the input dialogue-related information.

6. a dialogue-related information receiving step of receiving input of dialogue-related information about the dialogue; The summarizing method according to claim 2 , wherein the summarizing step generates the key sentence block using the key sentence extraction model that has been trained on the input dialogue-related information.

7. a dialogue-related information receiving step of receiving input of dialogue-related information about the dialogue; The summarizing method according to claim 3 , wherein the summarizing step removes the unnecessary sections based on the importance of the inputted dialogue-related information.

8. The method of claim 1 further comprising the step of generating a dialogue text based on the speech of the dialogue.

9. 9. The summarization method according to claim 8, wherein the dialogue text generating step detects fillers when generating the dialogue text and removes the detected fillers.

10. 9. The summarization method according to claim 8, wherein said dialogue text generating step detects imperfect punctuation when generating said dialogue text, and corrects the punctuation detected as imperfect.

11. further performing an important sentence extraction step of generating an important dialogue text by extracting important sentences from the dialogue text using an important sentence extraction model; The summarization method according to claim 1 , wherein the dividing step obtains the important dialogue text as the dialogue text and divides it into the plurality of blocks.

12. The summarizing method according to claim 1 , wherein the summarizing step extracts key sentences from the generated summary block using a key sentence extraction model.

13. 2. The summarizing method according to claim 1, wherein the summarizing step removes unnecessary sections from the blocks based on the importance of at least a part of the blocks, and generates the summary text based on the blocks from which the unnecessary sections have been removed.

14. 2. The summarization method according to claim 1, wherein said summarizing step removes unnecessary sections from said summary block based on at least a degree of importance of said summary block.

15. The summarization method according to claim 1 , wherein the dividing step divides the dialogue text from which unnecessary sections have been removed into the plurality of blocks based on the importance of at least a portion of the dialogue text.

16. a segmentation unit that segments a dialogue text representing the content of the dialogue into a plurality of blocks based on topics included in the dialogue using a text segmentation model; a summarization unit that uses the summary model to generate a summary block that summarizes each of the blocks, and generates a summary text based on the summary block.

17. 17. A program for causing a computer to function as the summarization system according to claim 16, the program causing the computer to function as each of said units.

Images