Information processing device, information processing method, and information processing program

Abstract

This invention provides an information processing device that optimally displays information related to speech recognition results, even when there are many topics and the conversation is lengthy. [Solution] The information processing device includes: a text acquisition unit that acquires speech recognition text converted from speech into text; a topic acquisition unit that estimates topic boundaries in the speech recognition text using a topic boundary estimation model that estimates topic boundaries in the text and acquires topics included between said boundaries; a summary generation unit that generates summaries and important words for each range of the speech recognition text separated by boundaries; and a display processing unit that displays at least two of at least three display areas in parallel on a display screen, and in each of the at least two display areas, displays them separately according to different topics.

Description

Technical Field

[0004]

[0001] The present disclosure relates to an information processing apparatus, an information processing method, and an information processing program.

Background Art

[0002] A technique for converting the content of a spoken utterance into text from data recording the spoken voice is known. As an example of this technique, for example, the voice recognition result display device described in Patent Document 1 can be cited. This device includes a display unit that displays, in a display area for the content of a spoken utterance within a screen, a voice recognition result text that is the text of the voice recognition result of the latest spoken utterance, and a summary text that is the text obtained by summarizing the voice recognition result texts of spoken utterances older than the latest spoken utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In the voice recognition result display device described in Patent Document 1, a voice recognition result text and a summary text that is the text obtained by summarizing the voice recognition result texts of spoken utterances older than that are displayed. However, it is difficult to find where the summary text corresponds to the voice recognition result text. In particular, when the number of topics in a conversation increases or the conversation becomes longer, it is difficult to understand the correspondence between the two texts.

[0005] The present disclosure has been made in view of the above problems, and an exemplary object thereof is to provide a technique for suitably displaying information regarding voice recognition results even when there are many topics and the conversation is long.

Means for Solving the Problems

[0006] An information processing device relating to an exemplary aspect of this disclosure includes: a text acquisition unit that acquires speech recognition text converted from speech into text; a topic acquisition unit that estimates topic boundaries in the speech recognition text using a topic boundary estimation model that estimates topic boundaries in the text and acquires topics included between said boundaries; a summary generation unit that generates summaries and important phrases for each range of the speech recognition text separated by the boundaries; and a display processing unit that displays at least two of at least three display areas in parallel on a display screen, and in each of the at least two of the display areas, divides and displays the topics according to different ranges.

[0007] An information processing method relating to an illustrative aspect of this disclosure includes: obtaining speech recognition text converted from speech into text; estimating the boundaries of topics in the speech recognition text using a topic boundary estimation model that estimates the boundaries of topics in the text, and obtaining the topics included between the boundaries; generating summaries and key phrases for each range of the speech recognition text separated by the boundaries; and displaying at least two of at least three display areas in parallel on a display screen, each displaying the speech recognition text, the summaries, and the key phrases, respectively, and displaying each of the at least two display areas separately according to different ranges of topics.

[0008] An information processing program relating to an exemplary aspect of this disclosure causes a computer to perform the following: a text acquisition process to acquire speech recognition text converted from speech into text; a topic acquisition process to estimate the topic boundaries in the speech recognition text using a topic boundary estimation model that estimates the topic boundaries in the text and acquire the topics included between the boundaries; a generation process to generate summaries and important phrases for each range of the speech recognition text separated by the boundaries; and a display process to display at least two of at least three display areas in parallel on a display screen, and to display each of the at least two display areas separately for different ranges of topics. [Effects of the Invention]

[0009] According to an illustrative aspect of this disclosure, one exemplary effect is that it is possible to provide a technology that can suitably display information related to speech recognition results even when there are many topics and the conversation is long. [Brief explanation of the drawing]

[0010] [Figure 1] This is a block diagram showing the configuration of the information processing device 1 related to this disclosure. [Figure 2] This flowchart shows the flow of the information processing method S1 related to this disclosure. [Figure 3] This is a block diagram showing the configuration of the information processing device 1A related to this disclosure. [Figure 4] This is a schematic diagram showing an example of an image displayed on the display unit related to this disclosure. [Figure 5] This is a schematic diagram showing an example of another image that can be displayed on the display unit. [Figure 6] This is a schematic diagram showing an example of another image that can be displayed on the display unit. [Figure 7] This is a schematic diagram showing an example of another image that can be displayed on the display unit. [Figure 8] This is a block diagram showing the configuration of the information processing device 1B related to this disclosure. [Figure 9]This is a block diagram showing the configuration of a computer that functions as an information processing device related to this disclosure. [Modes for carrying out the invention]

[0011] The following are examples of embodiments of the present invention. However, the present invention is not limited to the exemplary embodiments shown below, and various modifications are possible within the scope of the claims. For example, embodiments obtained by appropriately combining some or all of the technologies (things or methods) employed in each of the exemplary embodiments shown below may also be included in the scope of the present invention. Furthermore, embodiments obtained by appropriately omitting some of the technologies employed in each of the exemplary embodiments shown below may also be included in the scope of the present invention. In addition, the effects mentioned in each of the exemplary embodiments shown below are examples of effects that can be expected in that exemplary embodiment and do not define the scope of the present invention. That is, embodiments that do not produce the effects mentioned in each of the exemplary embodiments shown below may also be included in the scope of the present invention.

[0012] [First Exemplary Embodiment] A first exemplary embodiment, which is an example of an embodiment of the present invention, will be described in detail with reference to the drawings. This exemplary embodiment is the basic form for each of the exemplary embodiments described later. The scope of application of each technology adopted in this exemplary embodiment is not limited to this exemplary embodiment. That is, each technology adopted in this exemplary embodiment can also be adopted in other exemplary embodiments included in this disclosure, to the extent that no particular technical problems occur. Furthermore, each technology shown in the drawings referenced to explain this exemplary embodiment can also be adopted in other exemplary embodiments included in this disclosure, to the extent that no particular technical problems occur.

[0013] (Configuration of Information Processing Device 1) The configuration of the information processing device 1 according to this exemplary embodiment will be described with reference to Figure 1. Figure 1 is a block diagram showing the configuration of the information processing device 1. The information processing device 1 is a device that displays text, summaries, etc., of speech-to-text data, categorized by topic, on a single display screen. The information processing device 1 can analyze speech recognition text in specialized technical fields. For example, the information processing device 1 may be a device that displays text containing specialized terminology from a conversation between a patient and a doctor in a hospital, as well as its summary, important terms, etc. As shown in Figure 1, the information processing device 1 includes a text acquisition unit 11, a topic acquisition unit 12, a summary generation unit 13, and a display processing unit 14.

[0014] The text acquisition unit 11 acquires speech recognition text, which is a transcription of speech. Speech recognition text can be generated from recorded speech data using known techniques. Speech recognition text (hereinafter also simply referred to as "text") may be recorded in any memory or database, and the text acquisition unit 11 may acquire pre-recorded speech recognition text and record it in the memory of the information processing device 1. Alternatively, the text acquisition unit 11 may use a program that generates speech recognition text from speech data to generate speech recognition text from speech data, record it in the memory of the information processing device 1, and acquire it. The text acquisition unit 11 may also correct errors contained in the acquired speech recognition text.

[0015] The topic acquisition unit 12 estimates the topic boundaries in the speech recognition text using a topic boundary estimation model that estimates the topic boundaries in the text, and acquires the topics contained between those boundaries. The topic boundary estimation model is any known machine model. The topic boundary estimation model calculates, for example, the degree of similarity of the topics (themes, domains) represented by the words contained in the text. Then, it processes the topic boundaries by defining the range in which words with similar topics follow as one topic range.

[0016] The topic acquisition unit 12, for example, inputs the speech recognition text acquired by the text acquisition unit 11 into a topic boundary estimation model provided in the information processing apparatus 1, and acquires the output topic boundary. Alternatively, the topic acquisition unit 12 may use a topic boundary estimation model provided outside the information processing apparatus 1. In that case, the topic acquisition unit 12 can input the speech recognition text into a topic boundary estimation model provided outside via the Internet, and acquire the output topic boundary (hereinafter, also simply referred to as "boundary") via the Internet. The boundary may be before and after one sentence, or may be before and after a plurality of sentences. When it is determined that the boundary is before and after one sentence, it corresponds to the case where there is only one sentence regarding a certain topic.

[0017] Also, the topic acquisition unit 12 acquires (extracts) a topic from one or a plurality of text sentences between two boundaries. The method of extracting a topic from the text is arbitrary. For example, the topic boundary estimation model outputs the common topic represented by the words most included in a certain range delimited by the boundary as the topic of that range. The topic acquisition unit 12 may acquire the topic output from the topic boundary estimation model as the topic in that range. When there is only one sentence between two boundaries, the topic acquisition unit 12 may acquire the subject of the conversation included in that sentence as the topic. The topic acquisition unit 12 records the acquired topic in the memory of the information processing apparatus 1.

[0018] The summary generation unit 13 generates a summary of the topic and important phrases for each range of the speech recognition text delimited by boundaries. The summary generation unit 13 acquires the text between the boundaries acquired by the topic acquisition unit 12, generates a summary from the text, and extracts and edits important phrases. Also, the summary generation unit 13 may create a summary using a large language model from the text or important phrase candidates. The important phrase candidates are phrases that are presumed to be considered important by the conversation participants of the text, especially experts (e.g., doctors). The technique for generating the summary is arbitrary. For example, the summary generation unit 13 may input the text into a large language model to create a summary. The summary generation unit 13 may generate a summary for multiple or all sentences (one topic) included between two topic boundaries, or may generate a summary for one sentence.

[0019] Also, the summary generation unit 13 may generate important phrase candidates. The summary generation unit 13 may generate important phrase candidates for each sentence. The summary generation unit 13 may generate important phrase candidates from the summary. The summary generation unit 13 can generate, as important phrases, the phrases specified as important phrases by experts from among the important phrase candidates. In this case, important phrases can be listed without omission, and the possibility of regarding unnecessary phrases as important phrases can be eliminated. Alternatively, the summary generation unit 13 may generate, as important phrases, phrases that are presumed to be considered important by experts (e.g., doctors) instead of important phrase candidates. The summary generation unit 13 records the generated summary, important phrase candidates, and / or important phrases in the memory of the information processing apparatus 1 together with the topic and the speech recognition text.

[0020] The method by which the summary generation unit 13 extracts important phrase candidates is arbitrary. For example, a user (such as an expert) may create a keyword list of important phrases in advance for each specialized field, and the summary generation unit 13 may select the phrases containing the keywords. Alternatively, the summary generation unit 13 may extract important phrases using a machine model trained with learning data labeled with important phrases and their labels.

[0021] The display processing unit 14 displays at least two of the at least three display areas that display the speech recognition text, summary, and key words, respectively, in parallel on the display screen, and in each of the at least two display areas, it displays the topics acquired by the topic acquisition unit 12 divided into different ranges.

[0022] A display area is a dedicated area for displaying three types of content, such as speech recognition text, a summary, and key terms, each tagged with its respective name. The number (content) of display areas is not limited to the three of speech recognition text, summary, and key terms. For example, an additional display area for key term candidates may be provided. Alternatively, a display area may be used to display related medical data (electronic medical records, test results, and other related medical documents). The following describes an example of displaying the three types of content mentioned above (speech recognition text, summary, and key terms), but the combination of display content is arbitrary. These areas are displayed side-by-side on the display screen of the display device (display) by the user selecting at least two of them. Displaying them side-by-side means arranging them horizontally or vertically. In the case of a landscape display, arranging them horizontally is preferable from the standpoint of readability. Note that display areas do not need to be displayed within a single screen or window. For example, they may be displayed as a pop-up window partially overlapping with other display areas based on user operation, or they may be displayed in a separate window.

[0023] Displaying content by topic means displaying each topic within the range of speech recognition text separated by boundaries in a recognizable manner. The display processing unit 14 generates such display data and outputs it to an external or internal display device of the information processing device 1. Furthermore, if the user specifies a topic, the display processing unit 14 may display the range containing that topic in each display area. Alternatively, if the user scrolls in one display area and changes topics, the display processing unit 14 may follow and scroll the display content of other display areas to the new topic.

[0024] The text acquisition unit 11, topic acquisition unit 12, summary generation unit 13, and display processing unit 14 can be implemented, for example, by a processor reading and executing a program that describes each of their functions.

[0025] (Effects of information processing equipment) As described above, the information processing device 1 employs a configuration comprising: a text acquisition unit that acquires speech recognition text converted from speech into text; a topic acquisition unit that estimates topic boundaries in the speech recognition text using a topic boundary estimation model that estimates topic boundaries in the text and acquires topics included between said boundaries; a summary generation unit that generates a summary and important words for each range of the speech recognition text separated by boundaries; and a display processing unit that displays at least two of at least three display areas in parallel on the display screen, and displays the content of each of the at least two display areas in correspondence with each topic. Therefore, with the information processing device 1, even when there are many topics and the conversation is long, the effect is obtained that information related to the speech recognition result can be displayed appropriately by dividing the ranges in which different topics are described.

[0026] For example, in the medical field, attempts are being made to efficiently support doctors in creating documents by mechanically transcribing conversations between doctors and patients into text. However, simply transcribing long conversations containing various topics makes it difficult to quickly grasp the content. Therefore, by using the technology described above, doctors (professionals) can quickly grasp the content of conversations, and their documentation work can be efficiently supported.

[0027] (Information processing flow) The flow of the information processing method S1 will be explained with reference to Figure 2. Figure 2 is a flowchart showing the flow of the information processing method S1. As shown in Figure 2, the information processing method S1 includes a text acquisition process S11, a topic acquisition process S12, a generation process S13, and a display process S14.

[0028] Text acquisition process S11 is the process of acquiring speech recognition text, which is the text generated from speech. Text acquisition process S11 is executed, for example, by the text acquisition unit 11 (one processor). The content of text acquisition process S11 is as described with respect to the text acquisition unit 11.

[0029] The topic acquisition process S12 is a process that estimates the topic boundaries in the speech recognition text using a topic boundary estimation model that estimates the topic boundaries in the text, and acquires the topics contained between those boundaries. The topic acquisition process S12 is executed, for example, by the topic acquisition unit 12 (one processor). The contents of the topic acquisition process S12 are as described with respect to the topic acquisition unit 12.

[0030] Generation process S13 is the process of generating topic summaries and key words for each range of speech recognition text separated by boundaries. Generation process S13 is executed, for example, by the summary generation unit 13 (one processor). The contents of generation process S13 are as described with respect to the summary generation unit 13.

[0031] Display process S14 is a process that displays at least two of the at least three display areas, each for displaying speech recognition text, a summary, and key phrases, in parallel on the display screen, and further divides each of the at least two display areas into sections based on different topics. Display process S14 is executed, for example, by a display processing unit 14 (one processor). The content of display process S14 is as described with respect to the display processing unit 14.

[0032] (Effects of information processing methods) As described above, the information processing method S1 employs a configuration in which at least one processor performs the following actions: acquires speech recognition text converted from speech to text; estimates topic boundaries in the speech recognition text using a topic boundary estimation model that estimates topic boundaries in the text and acquires topics included between said boundaries; generates topic summaries and important words for each range of speech recognition text separated by boundaries; displays at least two of the at least three display areas that display the speech recognition text, summaries, and important words respectively, in parallel on the display screen, and displays the content corresponding to each topic in each of the at least two display areas. Therefore, according to the information processing method S1, even when there are many topics and the conversation is long, the effect is obtained that information related to the speech recognition result can be displayed appropriately by dividing the ranges in which different topics are described.

[0033] [Second exemplary embodiment] A second exemplary embodiment, which is an example of an embodiment of the present invention, will be described in detail with reference to the drawings. Components having the same function as those described in the above-described exemplary embodiment are denoted by the same reference numerals, and their descriptions are omitted as appropriate. The scope of application of each technology adopted in this exemplary embodiment is not limited to this exemplary embodiment. That is, each technology adopted in this exemplary embodiment can also be adopted in other exemplary embodiments included in this disclosure, to the extent that no particular technical problems arise. Furthermore, each technology shown in the drawings referenced to describe this exemplary embodiment can also be adopted in other exemplary embodiments included in this disclosure, to the extent that no particular technical problems arise.

[0034] (Configuration of Information Processing Device 1A) The configuration of the information processing device 1A will be described with reference to Figure 3. Figure 3 is a block diagram showing the configuration of the information processing device 1A. In addition to the text acquisition unit 11, topic acquisition unit 12, summary generation unit 13, and display processing unit 14 provided by the information processing device 1, the information processing device 1A includes an input / output interface (input / output IF) 20, at least one processor 30, and at least one memory 40. The information processing device 1A may also be connected to a display unit (display) 70.

[0035] The processor 30 can be configured using at least one general-purpose processor such as an MPU (Micro Processing Unit) or CPU (Central Processing Unit). The processor 30 may also include a dedicated processor composed of an ASIC (Application Specific Integrated Circuit), FPGA (Field Programmable Gate Array), or PLD (Programmable Logic Device).

[0036] Memory 40 may include multiple types of memory, such as ROM (Read Only Memory) and RAM (Random Access Memory). Memory 40 may also include internal or external memory such as an HDD (Hard Disk Drive) or SSD (Solid State Drive). For example, the processor 20 implements the functions of a text acquisition unit 11, a topic acquisition unit 12, a summary generation unit 13, and a display processing unit 14 by loading various control programs recorded in the ROM of memory 40 into RAM and executing them. Furthermore, data such as various programs, speech recognition text, summaries, and important keywords may be recorded in an externally located database 60 or similar.

[0037] The input / output IF20 is an interface for sending and receiving data to and from the outside. Communication between the input / output IF20 and the outside (e.g., database 60) may be performed via, for example, the Internet 90. The input / output IF20 may be equipped with a short-range communication device such as WiFi® or Bluetooth® that can connect wirelessly to an internet connection point. Alternatively, it may be a wired connection interface such as a USB connector.

[0038] The display processing unit 14 may, for example, display at least two of the following in relation to each other in the display area: speech recognition text, summary, and key phrases. Relational means, for example, displaying items on the same topic in parallel in different display areas within a single screen. For instance, if the same topic is displayed in at least two display areas on a single screen, the user can easily compare them, making it easier to understand even long texts. Alternatively, the same phrases in different display areas can be displayed by scrolling. Or, the same highlighting (marking, highlighting, etc.) may be applied, or a combination of these may be used. Such display methods allow the user to easily see which key phrases appeared in which topics.

[0039] (Specific example of the displayed image) Figure 4 is a schematic diagram showing the image 100 displayed on the display unit 70. Image 100 shows an example where display area 110 and display area 120 are displayed by default. Display area 110 displays speech recognition text (voice memo), and a heading 111 indicating that it is a "voice memo" is also displayed. Display area 120 displays important keywords, and a heading 121 indicating that it is an "important keywords" is also displayed. The speech recognition text in this exemplary embodiment is a transcript of a conversation between a doctor and a patient in a hospital. Figure 4 shows an example where nothing is displayed in area 130. A selection button 140 is displayed in the upper right of Figure 4, and the user can tap it to switch between displaying or not displaying. In the illustrated example, three options are available: "voice memo," "important keywords," and "summary." The buttons for "voice memo" and "important keywords," which have content displayed, are highlighted, while the "summary" button, which does not have content displayed, is faintly displayed. The patient ID 150 is displayed in the upper left of Figure 4.

[0040] The display area 110 displays, for example, the time 112 and the content of the conversation 113 at that time in chronological order. The display processing unit 14 of the information processing device 1A further displays the topic item names for each display area. For example, in the display area 110, the topic item 114 of the conversation is highlighted with a frame or the like and placed where the topic begins. In the example shown in Figure 4, the topic item "pathological condition" is displayed, and the range in which the topic of "pathological condition" continues is further separated and displayed with a dotted line. The way in which the topic item and the range in which it is displayed are separated and displayed is arbitrary.

[0041] Furthermore, the display processing unit 14 may change the display mode for each of the multiple topics displayed. For example, the range of the item "pathological condition" may be marked in blue, and the range of the item "treatment" may be marked in yellow. This allows the user to easily recognize the range of a single topic.

[0042] The topics may be specified in advance by the user, for example. The topic acquisition unit 12 of the information processing device 1A acquires a topic if the user has provided one or more items related to that topic in advance, and assigns at least one of the items to the acquired topic. If the user has not specified any topics, the topic acquisition unit 12 can use the topics it has acquired. For example, the item 114 "Pathological condition" displayed in display areas 110 and 120, and the item "Treatment" displayed in display area 120 are items specified in advance by the user.

[0043] Words identified as important are underlined (115). Important words may be selected by the user (e.g., a doctor) from candidate words or voice memos, as described in the exemplary embodiment, or they may be generated by the summary generation unit 13. The summary generation unit 13 may also switch the method of extracting summaries or important words based on the speaker or topic. In this case, the text acquisition unit 11 records the speech recognition text in a way that distinguishes it for each speaker of the speech. For example, the summary generation unit 13 may extract summaries or important words only from the words spoken by experts. If there is a technical term corresponding to an important word, the summary generation unit 13 may replace the important word with the corresponding technical term. Since conversations between doctors and patients, or their speech recognition texts, may use simple language, replacing such terms with technical terms can make them easier for experts to read. For example, a technical term conversion rule or dictionary such as "cold → common cold" may be prepared in advance, and the summary generation unit 13 may be configured to perform the conversion when the corresponding string is found.

[0044] The display processing unit 14 may scroll to the location of important words or topics in other display areas when a mouse operation (such as clicking) is performed on an important word or topic in one of the displayed display areas. For example, when the cursor is placed on an important word and clicked, that important word is scrolled and displayed in other display areas. In the example shown in Figure 4, the user clicks on "endoscope" in display area 110, and "endoscope" is scrolled to the top and displayed in display area 120. The clicked word (endoscope) is enclosed in a frame (highlighted).

[0045] Display area 120 displays a collection of "important terms." Clicking on an "important term" displayed in display area 120 will also bring up the corresponding "important term" displayed in display area 110. Additionally, clicking on the audio mark 122 will allow you to listen to audio data related to that item.

[0046] Figure 5 is a schematic diagram showing other images 200 displayed on the display unit 70. Image 200 shows an example where "Voice Memo," "Keywords," and "Summary" are displayed in display areas 210, 220, and 230, respectively. The "Summary" is generated mainly from key words. It is not necessary to display all three; only "Voice Memo" and "Summary" may be displayed.

[0047] Figure 6 is a schematic diagram showing other images 300 displayed on the display unit 70. Image 300 shows an example where "important keyword candidates" are displayed in the display area 340. In this case, the summary generation unit 13 generates important keyword candidates, and the display processing unit 14 can further display a fourth display area (display area 340) that displays the important keyword candidates.

[0048] At the "Keyword Candidate" stage, the words are not yet "Keywords," so no association has been made between the underlined words and other display areas. Users can select words they deem important from the "Keyword Candidate" by clicking or other means. In Figure 6, the words selected by the user from the "Keyword Candidate" are shown in bold. The words selected by the user are recorded as "Keywords," and as explained in Figure 4, the association of these words is made in different display areas.

[0049] Figure 6 shows the "Voice Memo" with two additional tags: "Summary" and "Notes." "Summary" is a summarized version of the voice memo. "Notes" is the full version of the voice memo. Thus, the "Voice Memo" may be configured to allow users to select between full display and summary display. Figure 6 shows the screen when "Notes" (full version) of the "Voice Memo" is selected. On the other hand, Figure 7 shows the screen when "Summary" (summary version) of the "Voice Memo" is selected. The "Summary" of the voice memo may be a compilation or reorganization of the text without disrupting the chronological display. For example, it may be a compilation of redundant expressions removed from a single utterance (or sentence), a summary of the text, a bulleted list, or a conversion of spoken language into written language.

[0050] (Effects of Information Processing Device 1A) As described above, the information processing device 1A employs a configuration in which the display processing unit displays at least two of the speech recognition text, summary, and key terms in relation to each other. Therefore, the information processing device 1A has the effect of displaying at least two of the speech recognition text, summary, and key terms in comparison to sections on the same topic.

[0051] Furthermore, the information processing device 1A employs a configuration in which the summary generation unit generates further candidate important terms, and the display processing unit displays a fourth display area for displaying the candidate important terms. Therefore, the information processing device 1A has the effect of displaying and comparing locations where the same candidate important terms are described.

[0052] Furthermore, in the information processing device 1A, the text acquisition unit 11 records speech recognition text in a way that distinguishes it from the speech utterance, and the summary generation unit 13 switches the method of summarizing or extracting important words based on the utterance or topic. As a result, the information processing device 1A can, for example, extract summaries or important words from the speech content of experts.

[0053] Furthermore, in the information processing device 1A, the topic acquisition unit 12 is configured to assign at least one item to the acquired topic when the user has provided items related to the topic in advance. Therefore, the information processing device 1A has the effect of being able to separate topics according to the topic items specified by the user.

[0054] Furthermore, in the information processing device 1A, the display processing unit 14 is configured to change the display mode for each of the multiple topics that are displayed. Therefore, the information processing device 1A has the effect of making it easy to recognize the scope of the topic.

[0055] Furthermore, in the information processing device 1A, the display processing unit 14 is configured to scroll to the location of important words or topics in other display areas when a mouse operation (such as a click) is performed on an important word or topic in one of the displayed display areas. Therefore, the information processing device 1A has the effect of allowing users to compare and refer to the content of common important words or topics in multiple display areas on a single display screen.

[0056] Furthermore, in the information processing device 1A, the display processing unit 14 is configured to display the topic name for each display area. Therefore, the information processing device 1A has the effect of allowing the content of the topic to be understood at a glance.

[0057] In this way, the information processing device 1A has the effect of providing an improved user interface for electronic devices.

[0058] [Third Exemplary Embodiment] A third exemplary embodiment, which is an example of an embodiment of the present invention, will be described in detail with reference to the drawings. Components having the same function as those described in the above-described exemplary embodiments are denoted by the same reference numerals, and their descriptions are omitted as appropriate. The scope of application of each technology adopted in this exemplary embodiment is not limited to this exemplary embodiment. That is, each technology adopted in this exemplary embodiment can also be adopted in other exemplary embodiments included in this disclosure, to the extent that no particular technical hindrance occurs. Furthermore, each technology shown in the drawings referenced to describe this exemplary embodiment can also be adopted in other exemplary embodiments included in this disclosure, to the extent that no particular technical hindrance occurs.

[0059] (Configuration of Information Processing Unit 1B) The configuration of the information processing device 1B will be described with reference to Figure 8. Figure 8 is a block diagram showing the configuration of the information processing device 1B. In addition to the text acquisition unit 11, topic acquisition unit 12, summary generation unit 13, and display processing unit 14 provided by the information processing device 1, the information processing device 1B also includes an error correction unit 50. The summary generation unit 13 also includes a trained model 131. The information processing device 1B includes an input / output IF 20, at least one processor 30, and at least one memory 40. The information processing device 1B may also be connected to a display unit 70. Furthermore, communication between the input / output IF 20 and an external source (e.g., a database 60) may be performed via, for example, the Internet 90.

[0060] The error correction unit 50 corrects errors in the speech recognition text acquired by the text acquisition unit 11. The error correction unit 50 includes an error detection unit 51, a phoneme distance calculation unit 52, and a text correction unit 53.

[0061] The error detection unit 51 inputs, for example, speech recognition text and a prompt to detect an error word in speech recognition from the speech recognition text to the first large-scale language model, and obtains the error word output from the first large-scale language model.

[0062] The phoneme distance calculation unit 52 obtains one or more phoneme sequences of the incorrect word's pronunciation and outputs a word correction candidate for which the phoneme distance between two phonemes in the phoneme sequence is less than or equal to a predetermined threshold.

[0063] The text correction unit 53 inputs the erroneous word, the word correction candidate output for that erroneous word, and a prompt instructing the second large-scale language model to select a word correction candidate to replace the erroneous word, and then reflects the word correction candidate output from the second large-scale language model into the speech recognition text and outputs it.

[0064] [Examples of implementation using software] Some or all of the functions of the information processing devices 1, 1A, and 1B (hereinafter also referred to as "the above devices") may be implemented by hardware such as integrated circuits (IC chips) or by software.

[0065] In the latter case, each of the above devices is implemented, for example, by a computer that executes instructions for a program, which is software that realizes each function. An example of such a computer (hereinafter referred to as Computer C) is shown in Figure 9. Figure 9 is a block diagram showing the hardware configuration of Computer C, which functions as each of the above devices.

[0066] Computer C comprises at least one processor C1 and at least one memory C2. Memory C2 stores a program P for operating Computer C as each of the above-mentioned devices. In Computer C, the processor C1 reads and executes the program P from memory C2, thereby realizing each of the above-mentioned information processing devices.

[0067] For processor C1, for example, a CPU (Central Processing Unit), GPU (Graphics Processing Unit), DSP (Digital Signal Processor), MPU (Micro Processing Unit), FPU (Floating Point Number Processing Unit), PPU (Physics Processing Unit), TPU (Tensor Processing Unit), quantum processor, microcontroller, or a combination thereof can be used. For memory C2, for example, flash memory, HDD (Hard Disk Drive), SSD (Solid State Drive), or a combination thereof can be used.

[0068] Computer C may also be equipped with RAM (Random Access Memory) for loading program P at runtime and for temporarily storing various data. Furthermore, computer C may be equipped with communication interfaces for sending and receiving data with other devices. Additionally, computer C may be equipped with input / output interfaces for connecting input / output devices such as keyboards, mice, displays, and printers.

[0069] Furthermore, program P can be recorded on a non-temporary, tangible recording medium M that is readable by computer C. Such a recording medium M could be, for example, tape, disk, card, semiconductor memory, or programmable logic circuitry. Computer C can acquire program P via such a recording medium M. Program P can also be transmitted via a transmission medium. Such a transmission medium could be, for example, a communication network or broadcast waves. Computer C can also acquire program P via such a transmission medium.

[0070] Furthermore, each of the above functions of each of the above devices may be implemented by a single processor in a single computer, by multiple processors in a single computer working together, or by multiple processors in each of multiple computers working together. In addition, the programs for implementing each of the above functions in each of the above devices may be stored in a single memory in a single computer, distributed and stored in multiple memories in a single computer, or distributed and stored in multiple memories in each of multiple computers.

[0071] [Additional Note 1] This disclosure includes the technologies described in the following appendices. However, the present invention is not limited to the technologies described in the following appendices, and various modifications are possible within the scope of the claims.

[0072] (Note 1) A text acquisition unit that obtains speech recognition text by converting speech into text, A topic acquisition unit estimates the topic boundaries in the speech recognition text using a topic boundary estimation model that estimates the topic boundaries in the text, and acquires the topics included between those boundaries. A summary generation unit that generates summaries of topics and important words for each range of the speech recognition text demarcated by the aforementioned boundary, A display processing unit that displays at least two of the at least display areas that display the speech recognition text, the summary, and the key words, respectively, in parallel on the display screen, and displays each of the at least two display areas in sections that cover different topics, An information processing device equipped with the following features.

[0073] (Note 2) The information processing apparatus according to Appendix 1, wherein the display processing unit displays at least two of the speech recognition text, the summary, and the important phrases in relation to each other.

[0074] (Note 3) The information processing apparatus according to Appendix 1 or 2, wherein the summary generation unit further generates candidate important terms, and the display processing unit further displays a fourth display area for displaying the candidate important terms.

[0075] (Note 4) The information processing apparatus according to any one of the appendices 1 to 3, wherein the text acquisition unit records the speech recognition text in a manner that distinguishes it for each speaker of the speech, and the summary generation unit switches the method of extracting the summary or important words based on the speaker or the topic.

[0076] (Note 5) The information processing apparatus according to any one of the appendices 1 to 4, wherein the topic acquisition unit assigns at least one of the items to the acquired topic when the user has provided items related to the topic in advance.

[0077] (Note 6) The display processing unit is an information processing device according to any one of the appendices 1 to 5, which changes the display mode for each of the multiple topics that are displayed.

[0078] (Note 7) The information processing apparatus according to any one of the appendices 1 to 6, wherein when a mouse operation (such as clicking) is performed on the important word or topic in one of the displayed display areas, the display processing apparatus scrolls to the location of the important word or topic in another displayed display area.

[0079] (Note 8) The display processing unit further displays the topic item name for each of the display areas, as described in any one of the appendices 1 to 7.

[0080] (Note 9) Obtaining speech recognition text by converting speech into text, The process involves estimating the topic boundaries in the speech recognition text using a topic boundary estimation model that estimates the topic boundaries within the text, and obtaining the topics included between those boundaries. To generate a summary of the topic and key words for each range of the speech recognition text delimited by the aforementioned boundary, The display screen will show at least two of the at least two display areas that display the speech recognition text, the summary, and the key words, respectively, in parallel, and each of the at least two display areas will be divided and displayed according to different ranges of topics. Information processing methods including

[0081] (Note 10) On the computer, A text acquisition process that obtains speech recognition text by converting speech into text, A topic acquisition process that estimates the topic boundaries in the speech recognition text using a topic boundary estimation model that estimates the topic boundaries in the text, and obtains the topics included between those boundaries, A generation process that generates a summary of the topic and important words for each range of the speech recognition text delimited by the aforementioned boundary, Display processing that displays at least two of the at least display areas that display the speech recognition text, the summary, and the key words, respectively, in parallel on the display screen, and that in each of the at least two display areas, displays the topics in separate ranges. An information processing program that executes [something].

[0082] (Note 11) An error detection unit inputs the speech recognition text and a prompt for detecting an error word in the speech recognition from the speech recognition text into a first large-scale language model, and obtains the error word output from the first large-scale language model. A phoneme distance calculation unit obtains one or more phoneme sequences of the pronunciation of the aforementioned erroneous word and outputs a word correction candidate in which the phoneme distance between two phonemes in the aforementioned phoneme sequence is less than or equal to a predetermined threshold. A text correction unit inputs the aforementioned erroneous word, the word correction candidate output for the erroneous word, and a prompt instructing the user to select the word correction candidate to replace the erroneous word, and then reflects the word correction candidate output from the second large-scale language model into the speech recognition text and outputs it. An information processing device described in any one of the appendices 1 to 8, further comprising the above.

[0083] (Note 12) The summary generation unit selects the important words from the summary using a trained model, as described in any one of the appendices 1 to 9.

[0084] [Additional Note 2] This disclosure includes the technologies described in the following appendices. However, the present invention is not limited to the technologies described in the following appendices, and various modifications are possible within the scope of the claims. (Note 21) An information processing device comprising at least one processor, the processor performing: a text acquisition process to acquire speech recognition text converted from speech into text; a topic acquisition process to estimate topic boundaries in the speech recognition text using a topic boundary estimation model that estimates topic boundaries in the text and acquire topics included between said boundaries; a summary generation process to generate summaries and important phrases for each range of the speech recognition text separated by the boundaries; and a display process to display at least two of at least three display areas in parallel on a display screen, and in each of the at least two of the display areas, the topics are divided and displayed according to different ranges. The information processing device may also include memory. Furthermore, the memory may store a program that causes at least one processor to execute each of the aforementioned processes.

[0085] (Note 22) The information processing apparatus according to Appendix 21, wherein the processor causes at least two of the speech recognition text, the summary, and the important phrases to be displayed in relation to each other in the display processing.

[0086] (Note 23) The information processing apparatus according to Appendix 21, wherein the processor further generates candidate important terms in the summary generation process, and further displays a fourth display area for displaying the candidate important terms in the display process.

[0087] (Note 24) The information processing apparatus according to Appendix 21, wherein the processor, in the text acquisition process, records the speech recognition text in a manner that distinguishes it for each speaker of the speech, and in the summary generation process, switches the method of extracting the summary or important words based on the speaker or the topic.

[0088] (Note 25) The information processing apparatus according to Appendix 21, wherein, in the topic acquisition process, if the user has provided items related to the topic in advance, the processor assigns at least one of the items to the acquired topic.

[0089] (Note 26) The information processing apparatus according to Appendix 21, wherein the processor changes the display mode for each of the multiple topics displayed in the display processing.

[0090] (Note 27) The information processing apparatus according to Appendix 21, wherein, in the display processing, when a mouse operation (such as clicking) is performed on the important word or topic in one of the displayed display areas, the processor scrolls to the location of the important word or topic in another displayed display area.

[0091] (Note 28) The information processing apparatus described in Appendix 21, wherein the processor further displays the topic item name for each display area in the display processing.

[0092] [Additional Note 3] This disclosure includes the technologies described in the following appendices. However, the present invention is not limited to the technologies described in the following appendices, and various modifications are possible within the scope of the claims. (Note 31) An information processing method comprising: a text acquisition process in which at least one processor acquires speech recognition text converted from speech into text; a topic acquisition process that estimates topic boundaries in the speech recognition text using a topic boundary estimation model for estimating topic boundaries in the text and acquires topics included between said boundaries; a summary generation process that generates summaries and important phrases for each range of the speech recognition text separated by the boundaries; and a display process that displays at least two of at least three display areas in parallel on a display screen, and in each of the at least two display areas, divides and displays the topics according to different ranges.

[0093] (Note 32) The information processing method according to Appendix 31, wherein the processor displays at least two of the speech recognition text, the summary, and the important words in relation to each other during the display process.

[0094] (Note 33) The information processing method according to Appendix 31 or 32, wherein the processor further generates candidate important terms in the summary generation process, and further displays a fourth display area for displaying the candidate important terms in the display process.

[0095] (Note 34) The information processing method according to any one of appendices 31 to 33, wherein the processor, in the text acquisition process, records the speech recognition text in a manner that distinguishes it for each speaker of the speech, and in the summary generation process, switches the method for extracting the summary or important words based on the speaker or the topic.

[0096] (Note 35) The information processing method according to any one of appendices 31 to 34, wherein the processor, in acquiring a topic, assigns at least one of the items to the acquired topic if the user has provided items related to the topic in advance.

[0097] (Note 36) The information processing method according to any one of appendices 31 to 35, wherein the processor changes the display mode for each of the multiple topics displayed in the display processing.

[0098] (Note 37) The information processing method according to any one of appendices 31 to 36, wherein the processor, in the display processing, scrolls to the location of the important word or topic in another display area when a mouse operation (such as a click) is performed on the important word or topic in one of the displayed display areas.

[0099] (Note 38) The information processing method according to any one of appendices 31 to 37, wherein the processor further displays the topic item name for each display area in the display processing. [Explanation of symbols]

[0100] 1, 1A, 1B... Information processing device 11. Text acquisition section 12... Topic acquisition section 13...Summary generator 131... Pre-trained model 14. Display Processing Unit 20. Input / Output Interface 30 Processors 40...memory 50...Error correction section 51. Error detection unit 52... Phoneme distance calculation section 53... Sentence correction department 60... Database 70...Display section 90...Internet

Claims

1. A text acquisition unit that obtains speech recognition text by converting speech into text, A topic acquisition unit estimates the topic boundaries in the speech recognition text using a topic boundary estimation model that estimates the topic boundaries in the text, and acquires the topics included between those boundaries. A summary generation unit that generates summaries of topics and important words for each range of the speech recognition text demarcated by the aforementioned boundary, A display processing unit that displays at least two of the at least three display areas for displaying the speech recognition text, the summary, and the key words, respectively, in parallel on the display screen, and displays each of the at least two display areas in sections according to different topics, An information processing device equipped with the following features.

2. The information processing apparatus according to claim 1, wherein the display processing unit displays at least two of the speech recognition text, the summary, and the important phrases in relation to each other.

3. The information processing apparatus according to claim 1, wherein the summary generation unit further generates candidate important terms, and the display processing unit further displays a fourth display area for displaying the candidate important terms.

4. The information processing apparatus according to claim 1, wherein the text acquisition unit records the speech recognition text in a manner that distinguishes it for each speaker of the speech, and the summary generation unit switches the method of extracting the summary or important words based on the speaker or the topic.

5. The information processing apparatus according to any one of claims 1 to 4, wherein the topic acquisition unit assigns at least one of the items to the acquired topic when the user has provided items related to the topic in advance.

6. The information processing apparatus according to any one of claims 1 to 4, wherein the display processing unit changes the display mode for each of the multiple topics that have been displayed.

7. The information processing apparatus according to any one of claims 1 to 4, wherein the display processing unit scrolls to the location of the important word or topic in another display area when a mouse operation is performed on the important word or topic in one of the displayed display areas.

8. The information processing apparatus according to any one of claims 1 to 4, wherein the display processing unit further displays the topic item name for each of the display areas.

9. Obtaining speech recognition text by converting speech into text, The process involves estimating the topic boundaries in the speech recognition text using a topic boundary estimation model that estimates the topic boundaries within the text, and obtaining the topics included between those boundaries. To generate a summary of the topic and key words for each range of the speech recognition text delimited by the aforementioned boundary, The at least two of the at least three display areas for displaying the speech recognition text, the summary, and the key terms are displayed side-by-side on the display screen, and each of the at least two display areas is divided and displayed according to different ranges of topics. Information processing methods including

10. On the computer, A text acquisition process that obtains speech recognition text by converting speech into text, A topic acquisition process that estimates the topic boundaries in the speech recognition text using a topic boundary estimation model that estimates the topic boundaries in the text, and obtains the topics included between those boundaries, A generation process that generates a summary of the topic and important words for each range of the speech recognition text delimited by the aforementioned boundary, Display processing that displays at least two of the at least three display areas for displaying the speech recognition text, the summary, and the key words, respectively, in parallel on the display screen, and that in each of the at least two display areas, displays the topics in separate ranges according to their respective categories. A program that executes the command.

Images