Explanatory material presentation auxiliary system

Abstract

[Problem] To provide a system that can, on the basis of the interests of a user, display desired information and perform presentation based on the desired information. [Solution] An explanatory material presentation auxiliary system 1 including: a material storage unit 3 in which explanatory material is stored; a related word storage unit 5 in which one or a plurality of related words are stored in association with pages of the explanatory material; a topic storage unit 7 in which the flow of a plurality of topics related to the explanatory material is stored in association with the related words; an input term analysis unit 11 which analyzes an inputted term; a subsequent page output unit 13 which, on the basis of the inputted term, analyzes the topic at the present time in the flow of the plurality of topics and outputs the explanatory material page that is related to a topic to be subsequent to that at the present time; and a selected page output unit 15 which, when there are a plurality of topics to be subsequent to the topic at the present time, outputs the explanatory material page based on an inputted selection instruction.

Description

Explanation Material Presentation Assistance System

[0001] This invention relates to an explanation material presentation assistance system.

[0002] Japanese Patent No. 7102035 describes an explanation support system. This system includes a material storage unit that stores explanation materials and a plurality of related words related to the explanation materials, and stores display information to be displayed on a display unit based on combinations of related words. It also analyzes voice terms, which are terms included in voice information related to the explanation materials, and the main body of the voice terms, identifies which of the plurality of related words the voice terms of a specific main body are, and reads out the display information from the display information storage unit using the information related to the identified related words and causes it to be displayed on the display unit. According to this system, based on voice, after analyzing a specific main body, display information suitable for the specific main body can be displayed.

[0003] This explanation support system is preferable because it can analyze a specific main body in a situation where a plurality of voices are input. However, displaying desirable information based on the user's interesting points of a specific main body and conducting a presentation based on the desirable information are not necessarily intended.

[0004] Japanese Patent No. 7102035

[0005] An object is to provide a system that can display desirable information based on the user's interesting points and conduct a presentation based on the desirable information.

[0006] The explanation material presentation assistance system 1 that solves the above problems includes a material storage unit 3 that stores explanation materials, a related word storage unit 5 that stores one or more related words in association with each page of the explanation materials, a topic storage unit 7 that stores the flow of a plurality of topics related to the explanation materials in association with the related words, an input term analysis unit 11 that analyzes the input terms, a next page output unit 13 that analyzes the current topic in the flow of a plurality of topics based on the input terms and outputs the page of the explanation material related to the next topic after the current time, and a selection page output unit 15 that outputs the page of the explanation material based on the input selection instruction when there are a plurality of next topics for the current topic.

[0007] According to this invention, when there are branches in the topics based on the pages of a document, the system allows the user to select a branch, thereby displaying desirable information based on the user's interests and enabling a presentation based on that desirable information.

[0008] Figure 1 is a block diagram illustrating an example of a presentation assistance system for explanatory materials. Figure 2 is a conceptual diagram showing a hypothetical use case of the presentation assistance system for explanatory materials. Figure 3 is a conceptual diagram illustrating the flow of topics on a particular page of explanatory materials. Figure 4 is a conceptual diagram showing an example where thumbnails are displayed. Figure 5 is a conceptual diagram illustrating the user learning section.

[0009] Figure 1 is a block diagram illustrating an example of a system for assisting in the presentation of explanatory materials. As shown in Figure 1, the system for assisting in the presentation of explanatory materials 1 includes a material storage unit 3, a related word storage unit 5, a topic storage unit 7, an input term analysis unit 11, a next page output unit 13, and a selected page output unit 15. In the example shown in Figure 1, the system 1 further includes a user learning unit 17. This system can output pages selected based on the speaker's interests. This system typically includes a server and multiple terminals. In other words, this system may be implemented by one or more information processing terminals (computers). A conceivable use case of this system is that the speaker's terminal is connected to one or more audience terminals so that information can be exchanged. The speaker's terminal receives assistance in presenting explanatory materials based on this system 1. The speaker's terminal is registered with this system 1, and it is preferable that the information provided for assistance is customized according to the speaker by a trained model, which will be described later.

[0010] A computer has an input unit, an output unit, a control unit, an arithmetic unit, and a memory unit, and each element is connected by a bus or the like to enable the exchange of information. For example, the memory unit may store a control program or various kinds of information. When predetermined information is input from the input unit, the control unit reads the control program stored in the memory unit. The control unit then reads the information stored in the memory unit as appropriate and transmits it to the arithmetic unit. The control unit also transmits the input information to the arithmetic unit as appropriate. The arithmetic unit performs calculations using the received information and stores it in the memory unit. The control unit reads the calculation results stored in the memory unit and outputs them from the output unit. In this way, various processes and steps are executed. Each unit and each means is responsible for executing these various processes. A computer may have a processor, and the processor may implement various functions and steps. A computer may be standalone. A computer may have some of its functions distributed between a server and terminals. In that case, it is preferable that the server and terminals can exchange information via a network such as the internet or an intranet. A computer may include a processor and memory connected to the processor. The memory may store instructions, and when executed by the processor, these instructions may cause the computer to perform various processes or to function as various components. The computer may build a learning model by providing various training data and perform various calculations through machine learning. In this case, the computer may perform various analyses and interpretations using the learning model created by AI (artificial intelligence) machine learning and deep learning.

[0011] The system typically includes a display unit, which is not shown in the diagram. The display unit is an element for displaying various types of information based on a computer. A monitor or display, which is a type of computer output unit, or a smartphone's touch panel can function as the display unit. The display unit may also be a projector. When giving a presentation, the monitor of a computer or tablet can function as the display unit, and presentation materials may be projected using a projector. In this case, as will be described later, the monitor may display not only the presentation materials but also information regarding the word order of related terms, or both, along with explanatory text.

[0012] Figure 2 is a conceptual diagram showing a hypothetical use case of the explanatory material presentation assistance system. In this example, the speaker's terminal 21 is connected to the audience's terminal 23 via the internet. This use case could be a web conference, answering questions from the audience (users), or an example where a medical representative (MR) explains materials to a doctor (audience member). The speaker's terminal 21 may display the speaker's face as captured by the camera, or a page from the materials. These may also be displayed on the audience's terminal 23. On the other hand, the speaker's terminal 21 may also display various information for selecting the next page, as will be described later. In the example in Figure 2, a thumbnail 25 indicating a page from the presentation materials is displayed on the speaker's terminal 21, and related words are also displayed in the related words display field 27 for that page. If the speaker generates a related word or processes the output of the page displayed as a thumbnail, that page may also be output to the audience's terminal 23 and displayed on the audience's terminal 23.

[0013] Document Storage Unit 3 Document Storage Unit 3 is an element for storing explanatory materials. Document Storage Unit 3 functions as a database for storing materials prepared in advance by System 1, for example. Document Storage Unit 3 stores the entire presentation material and each page of the presentation material. The computer's memory unit functions as Document Storage Unit 3. Examples of presentation materials are materials created in PowerPoint® or PDF®. Presentation material can mean the entire series of materials (a file) created with software such as PowerPoint®, or a specific page. For example, each presentation material is assigned an identification number or ID. Document Storage Unit 3 stores multiple related words in association with the assigned information (identification number or ID), or the identification number and page number (slide number). In this way, each presentation material or each page (each slide) of each material is associated with and stored with multiple related words related to each presentation material.

[0014] An example of a presentation document is a PowerPoint presentation about a new drug X for diabetes. Examples of related terms for that presentation document are the related terms for that PowerPoint document: "diabetes," "X," "dosage," "side effects," "dizziness," "drowsiness," "pregnant women" (who should not be administered the drug), and "under 19 years old" (who should not be administered the drug). These are stored in the document storage unit 3, for example, in relation to the presentation document's identification number (and each page number).

[0015] Related Word Memory Unit 5: The related word memory unit 5 is an element for storing one or more related words in relation to each page of the explanatory material. The related word memory unit 5 stores related words corresponding to each page, thereby enabling the association of each page of the material with a topic. The order in which the related words appear may also be stored for each page. Related words may be keywords or topics, or synonyms of keywords.

[0016] Topic Memory Unit 7 The topic memory unit 7 is an element for storing a flow of multiple topics related to the explanatory materials, in relation to related words. The topic memory unit 7 structurally records a flow of multiple topics, for example. It is preferable that the transitions between topics are clearly defined in order and branching based on related words. Furthermore, it is preferable that each topic included in the flow of topics is stored in relation to related words and each page. In other words, it is preferable that the corresponding page or topic is retrieved based on related words.

[0017] Figure 3 is a conceptual diagram illustrating the flow of topics related to a particular page of explanatory material. For example, the topic memory unit 7 may store multiple topic flows in relation to the entire explanatory material. Alternatively, for example, the topic memory unit 7 may store multiple topic flows in relation to each page of the explanatory material. In the topic flows, for example, related words may exist for each story that constitutes a topic. The topic flows may then be formed based on the order in which the related words are registered. In the example in Figure 3, an example is shown where, for a page called up in relation to a certain related word, there are four major topic flows, each with its own branch. For example, if the term based on the input audio is a related word from A4, and the term analyzed based on subsequent input audio is also a related word after A4, the topic flows are uniform, and the next page after this page is also determined. Therefore, for example, when an explanation is given based on this page and the input audio is based on the flow of A4, the next page may be read and output at a predetermined timing after the last related word and related term are input. On the other hand, for example, if the first related word based on the input voice is like A1, a list of options may be presented, and the next page may be displayed based on the selection information of the presented options.

[0018] The Input Term Analysis Unit 11 is an element for analyzing the audio input to the system and obtaining the terms contained in the audio. The Input Term Analysis Unit 11 is implemented, for example, by a control program stored in a memory unit. The Input Term Analysis Unit 11 can be implemented using a known speech recognition device. The audio input to the system is usually analog information. The Input Term Analysis Unit 11 should convert the analog information into digital information and store it in a form that can be processed by a computer as terms. For example, it reads the audio information input to system 1. Then, the control unit of the computer reads the control program stored in the memory unit and has the calculation unit analyze the read audio information. At this time, it is sufficient to read multiple terms and the audio information of those terms stored in the memory unit and analyze the terms. Then, the analyzed terms should be stored in the memory unit as appropriate. In this way, the Input Term Analysis Unit 11 can analyze the terms contained in the sound input to the system.

[0019] Next Page Output Unit 13 The next page output unit 13 is an element that analyzes the current topic in a flow of multiple topics based on the input term, and outputs the next topic and the page of the explanatory material related to that topic. For example, the next page output unit 13 understands how far the discussion has progressed regarding the current topic based on the input term. Then, if the page of the explanatory material related to the next topic has been determined, the next page output unit 13 outputs the next page when the explanation of the current topic on the current page is finished. If the current page is page A1, and the flow of related words included in the topic on that page is A1A, A1B, A1C, and A1D, the next page output unit 13 may consider whether the term analyzed by the input term analysis unit 11 is related to any of the related words, and then understand how far the discussion has progressed regarding the current topic. Then, if the next page output unit 13 determines that the terms entered into the system are related in the order of A1A, A1B, A1C, and A1D, it may display page A2, which is the next page of the explanatory material following page A1, because the current topic is being spoken in accordance with the expected flow of the conversation. It may also read out related words stored in relation to page A2 that are in accordance with the flow of the conversation. Furthermore, even if the topic is being output automatically, the predetermined next page may be automatically displayed when the current topic ends, and audio related to the next page may be automatically output.

[0020] The Selected Page Output Unit 15 is an element for outputting pages of explanatory materials based on the input selection instruction when there are multiple pages of explanatory materials related to the next topic following the current topic (the page currently displayed in the display unit). When the topic branches into multiple categories, it outputs the next page based on the selection instruction. The Selected Page Output Unit 15 may also output related words related to each of the next topics when there are multiple topics following the current topic, and output pages of explanatory materials based on the input selection instruction related to the related words. For example, multiple related words may be displayed in the related word display field 27. If, after the topic about the current page, there are related words such as B1, B2, and B3 and related topics (next pages), those related words may be displayed in the related word display field 27 of the speaker's terminal 21 while the page is being displayed on the speaker's terminal 21 and the audience's terminal 23. Then, when the speaker says something like "B1", the page related to B1 may be displayed promptly after the topic about the current page has finished. Furthermore, System 1 may store voice commands such as "Be One," and when a related word displayed on the display unit is input from the speaker's terminal 21, it may process the audio so that the voice related to that word is not output to the listener's terminal 23. In this way, the appropriate page can be selected without the listener realizing that the next page has been indicated by voice.

[0021] The selected page output unit 15 may output thumbnails for each of the following pages if there are multiple subsequent topics following the current topic, and if there are corresponding explanatory material pages for each of the multiple subsequent topics, and then output the explanatory material page based on the selection instruction entered in relation to a specific page among those pages.

[0022] Figure 4 is a conceptual diagram showing an example of thumbnail display. In this example, multiple thumbnails 25 are displayed on the display unit of the speaker's terminal 21. In this example, the thumbnails display the next page. In this example, the thumbnails also display information for reading the pages associated with these thumbnails. For example, if the voice input is "at mark 1", the slide in the upper left will be output to the audience's terminal (and the speaker's terminal). For thumbnails, for example, the photo information contained in each slide can be reduced in resolution and reduced to a predetermined reduced size. Alternatively, for example, characters smaller than a predetermined font size can be deleted, and characters larger than the predetermined font size can be reduced to a predetermined size by a predetermined magnification before being displayed on the display unit. In this way, a deformed image of each slide can be obtained. If the obtained deformed images can be displayed on multiple screens, thumbnail display can be performed. Thus, the images related to multiple slides that the slide images display on the screen may be a collection of reduced images of each slide.

[0023] The input selection instructions may be voice inputs related to pre-registered selections. Examples of selections include returning to the previously displayed page, moving to the next page, ending the presentation, and returning to the first page. It is preferable that the voice input to the speaker's input unit is canceled so that it is not heard by the audience. For example, if the speaker wants to move to the next page instead of the page displayed on the display unit, they may say "at sign, at sign," and when this voice is input to the system, the input term analysis unit 11 should analyze the term "at sign, plus." The presentation assistance system 1 can then read the instruction stored in association with "at sign, plus" from the instruction storage unit and perform the predetermined control. For example, in this case, the next page after the current page will be displayed. Furthermore, the voice output for this selection instruction "at sign, plus" may be canceled out so that it is not heard by the audience. The memory unit should store a selection command, "Proceed to the next page," in association with the voice command "at sign, plus sign." When "at sign, plus sign" is input via voice, the memory unit should read the stored selection command associated with that voice command and perform the corresponding processing. Preferably, this voice command is one that is not included in a normal explanation.

[0024] The following describes examples of commands entered via voice input (voice commands). When using voice commands during speech recognition, it is preferable that the trigger for the voice command be a term that does not exist in Japanese, as there are no breaks in the Japanese language. An example of a voice command that takes this into consideration is the "@" symbol. The "@" symbol may be changed as appropriate. However, terms that appear in the explanation of the document, related words, keywords, etc., are not suitable as triggers. It is preferable to be able to construct selection commands as appropriate, including the term that follows the trigger of the voice command.

[0025] Examples of voice commands are as follows: @ ~ (~ means no voice input) After a predetermined time (e.g., 3 seconds) following the voice command @, a RAG lookup is performed and the result is displayed. @ + number (e.g., one) A thumbnail preview is displayed by specifying a number. The page that was displayed as a thumbnail is also displayed as the next page. When implementing this function, it is preferable not to include numbers in related words and to prevent RAG lookup from being performed for utterances such as "number ~~~". @ plus display the next page of the preview. @ minus display the previous page of the preview. @ @ close the preview. The above are examples of voice commands. By using such voice commands, it becomes possible to display pages based on the user's emotions. Furthermore, in this case, the user learning unit 17, which will be described later, can suggest candidates for the next page based on the flow of the topic, so that materials that better reflect the user's emotions can be output.

[0026] The explanatory material presentation assistance system 1 preferably further includes a user learning unit 17 that learns the user's selection instruction information. The user learning unit 17 is involved in the processing of the next page output unit 13 and the selected page output unit 15. Figure 5 is a conceptual diagram illustrating the user learning unit. Information including the user's selection instruction information of system 1 is used as learning data and input into an artificial intelligence model to construct a trained model that can estimate candidates for the next page. The learning data of the trained model may include the flow of topics (order of appearance of related words) and selection instruction information at each branch. In addition, the learning data may further include either or both speaker information and audience information. By inputting various data as learning data into the artificial intelligence model, it becomes possible to effectively output materials based on the user's emotions (emotions such as which page they want to display as the next page) of system 1. At that time, by including the flow of topics (order of appearance of related words) before and after the input of selection instruction information as learning data, the relationship between the flow of topics and selection at branches can be effectively learned. For example, the flow of topics shown in Figure 3 may be modified as appropriate by the user learning unit 17. Then, when data corresponding to the training data (for example, information about the flow of topics before branching) is input to the user learning unit 17 constructed in this manner, it becomes possible to effectively output information about the next page. In this way, by using the flow of topics based on the order in which related words appear, associated with multiple pages, as training data to construct a trained model, it becomes possible to construct a user learning unit 17 that can accurately grasp the user's thoughts or feelings and suggest (output) the next page in accordance with the flow of topics.

[0027] Information about the speaker and / or audience may be stored in the memory of System 1. Examples of such information include employee number, name, gender, age, sales performance, position, place of origin, presentation evaluation, years of service, and years in charge. Examples of information about the audience for the presentation include the size of the hospital, the location of the hospital, whether it is a lecture or for a single doctor, and (if for a doctor) information about the doctor. Other examples of information about the audience for the presentation include the region of the lecture, the level of the attendees, the academic year of the attendees, the number of attendees, the occupation of the attendees, the job description of the attendees, the years of service of the attendees, and the position of the attendees.

[0028] When building an artificial intelligence model, it is preferable to use RAG (Search Augmented Generator). Using Search Augmented Generator, an artificial intelligence model can be built as follows: Step 1: Data preparation Document corpus collection First, collect the knowledge base (document corpus) to be used for searching. Examples of the knowledge base include the flow of past topics and branching selection information at branching points. The collected information may be divided as needed. Step 2: Search model (Retriever) preparation For example, a Dense Retriever may be used to calculate embedding vectors between training data and output. Specifically, pairs of training data and output data may be used to train the model to maximize the similarity between question embeddings and correct answer embeddings. However, vector calculation is not required. The model may be trained by feeding back the correct and incorrect suggestions for the next page or candidate for the next page from System 1 as answers. Step 3: Generator model (Generator) preparation Selection of a pre-trained generator model The model is trained to receive search results as input and generate answers corresponding to questions. Step 4: Integration of the RAG model's search and generative models

[0029] One invention described in this specification relates to a computer program and a computer-readable non-temporary information recording medium (such as a CD-ROM) that stores the program. This program is basically a computer-readable program that enables the computer to function as one of the above-described explanatory material presentation assistance systems. For example, when this program is installed on a computer, it can enable the computer to function as an explanatory material presentation assistance system 1 having a data storage unit 3, a related word storage unit 5, a topic storage unit 7, an input term analysis unit 11, a next page output unit 13, and a selected page output unit 15.

[0030] The following describes an example of the process for providing explanatory materials using the explanatory material presentation assistance system. As previously explained, the explanatory material presentation assistance system 1 includes a material storage unit 3, a related word storage unit 5, a topic storage unit 7, an input term analysis unit 11, a next page output unit 13, and a selected page output unit 15. Furthermore, it is preferable that the system 1 further includes a user learning unit 17. The user learning unit 17 may be constructed using the method described above.

[0031] Suppose a speaker is explaining page A using their terminal 21. The user learning unit 17 has learned the flow of topic A on page A. The speaker's pronunciation is input to system 1, which analyzes it and analyzes the flow of conversation about page A, including related words. System 1 inputs information about the flow of conversation to the user learning unit 17. The user learning unit 17 then outputs only one candidate for the next page. It is likely that in the past, when a similar flow of conversation occurred, a specific next page was often used. In this case, the next page output unit 13 of system 1 outputs the next page relating to topic B, which is the next topic after topic A. This next page (page B) is also output to the audience's terminal. Page B is then displayed on the audience's terminal monitor. If this selection is correct, the speaker, for example, inputs that the selection is correct. This further improves the accuracy of the user learning unit 17. On the other hand, if page B is displayed as a thumbnail in the speaker's display area and page B is incorrect, the speaker will pronounce something like "at sign minus". System 1 will then perform processing related to the selection instruction "at sign minus". In this case, since the thumbnail of the next page candidate was incorrect, the system may display multiple related words for the next page, or display thumbnails for multiple candidates for the next page.

[0032] Furthermore, suppose a speaker is explaining page C using the speaker's terminal 21. Suppose page C is about topic C. For simplicity, in this example, each page is assumed to be about only one topic. The flow of the conversation then branches into topics D and E after topic C. Then, the next pages after page C are candidates for page D and page E. The selected page output unit 15 may further display thumbnails 25 of page D and page E on the display unit of the speaker's terminal 21 where page C is displayed, or it may display related words related to page D and page E in the related word display field 27. In this way, the speaker can be effectively allowed to select the next page. As explained earlier, the candidates for page D and page E may also be output data by inputting the flow of a speaker's conversation (and therefore the selection information) as input data to the user learning unit 17.

[0033] Furthermore, by storing explanatory sounds related to each page in the memory unit, it becomes possible to provide appropriate audio explanations based on the user's page selection command. For example, when trying to understand something (on one's own) based on the materials, the user can select the appropriate page by speaking a selection command based on the selection information displayed at branching points (e.g., thumbnails, related words), and receive an appropriate explanation that follows the flow of the conversation and corresponds to the user's emotions (the emotions of wanting this kind of information).

[0034] This invention can be used in information-related industries because it learns the flow of past data utilization and proposes established data based on the current flow of data utilization.

[0035] 1. Explanatory Material Presentation Assistance System 3. Material Storage Unit 5. Related Word Storage Unit 7. Topic Storage Unit 11. Input Term Analysis Unit 13. Next Page Output Unit 15. Selected Page Output Unit 17. User Learning Unit 21. Speaker's Terminal 23. Audience Terminal 25. Thumbnail 27. Related Word Display Field

Claims

1. An explanatory material presentation assistance system (1) comprising: a material storage unit (3) for storing explanatory materials; a related word storage unit (5) for storing one or more related words in relation to each page of the explanatory materials; a topic storage unit (7) for storing a plurality of topic flows related to the explanatory materials in relation to the related words, wherein the topic flows relate to the order in which the related words appear; an input term analysis unit (11) for analyzing input terms; a next page output unit (13) for analyzing the current topic in the plurality of topic flows based on the input terms and outputting a page of the explanatory material related to the next topic at the current time; and a selected page output unit (15) for outputting a page of the explanatory material based on an input selection instruction if there are multiple pages of the explanatory material related to the topic following the current topic.

2. An explanatory material presentation assistance system according to claim 1, wherein the selected page output unit (15) displays related words related to each of the following topics when there are multiple topics following the current topic, and outputs the explanatory material page based on the selection instruction entered in relation to the related words.

3. An explanatory material presentation assistance system according to claim 1, wherein the selected page output unit (15) outputs a thumbnail of each page when there are multiple subsequent topics following the current topic and pages of the explanatory material corresponding to each of the multiple subsequent topics, and outputs a page of the explanatory material based on a selection instruction entered in relation to a specific page among the pages.

4. An explanatory material presentation assistance system according to claim 1, wherein the input selection instruction is a pre-registered voice input relating to the selection.

5. An explanatory material presentation assistance system according to claim 1, comprising a user learning unit that learns user selection instruction information, wherein the user learning unit, when the user selection instruction information is input, outputs a page of the explanatory material based on the input selection instruction.

6. An explanatory material presentation assistance system according to claim 1, further comprising an audio output unit that outputs audio information corresponding to each of the plurality of topics.

7. A program to cause a computer to function as an explanatory material presentation assistance system (1), comprising: a material storage unit (3) for storing explanatory materials; a related word storage unit (5) for storing one or more related words in relation to each page of the explanatory materials; a topic storage unit (7) for storing a plurality of topic flows related to the explanatory materials in relation to the related words, wherein the topic flows relate to the order in which the related words appear; an input term analysis unit (11) for analyzing input terms; a next page output unit (13) for analyzing the current topic in the plurality of topic flows based on the input terms and outputting a page of the explanatory material related to the next topic at the current time; and a selected page output unit (15) for outputting a page of the explanatory material based on an input selection instruction if there are multiple pages of the explanatory material related to the topic following the current topic.

8. A computer-readable information recording medium storing the program described in claim 7.

Images