System for Assisting the Presentation of Explanatory Materials

Abstract

The system provides the ability to display desirable information based on the user's interests and to deliver presentations based on that desirable information. [Solution] An explanatory material presentation assistance system 1 comprising: a material storage unit 3 for storing explanatory materials; a related word storage unit 5 for storing one or more related words in relation to each page of the explanatory materials; a topic storage unit 7 for storing a flow of multiple topics related to the explanatory materials in relation to the related words; an input term analysis unit 11 for analyzing input terms; a next page output unit 13 for analyzing the current topic in a flow of multiple topics based on the input terms and outputting the page of the explanatory material related to the next topic at the current time; and a selected page output unit 15 for outputting the page of the explanatory material based on an input selection instruction if there are multiple topics following the current topic.

Description

Technical Field

[0001] This invention relates to an auxiliary system for presenting explanatory materials.

Background Art

[0002] Japanese Patent No. 7102035 describes an explanation support system. This system includes a material storage unit that stores explanatory materials and a plurality of related words related to the explanatory materials, and stores display information to be displayed on a display unit based on combinations of related words. It also analyzes voice terms, which are terms included in the voice information related to the explanatory materials, and the subject of the voice terms, identifies which of the plurality of related words the voice terms of a specific subject are, reads out the display information from the display information storage unit using the information related to the identified related words, and causes it to be displayed on the display unit. According to this system, based on voice, after analyzing a specific subject, display information suitable for the specific subject can be displayed.

[0003] This explanation support system is preferable because it can analyze a specific subject in a situation where a plurality of voices are input. However, displaying desirable information based on the interesting points of a specific subject or performing a presentation based on the desirable information is not necessarily intended.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] An object is to provide a system that can display desirable information based on the user's interesting points and perform a presentation based on the desirable information.

Means for Solving the Problems

[0006] The explanatory material presentation assistance system 1, which solves the above problems, comprises: a material storage unit 3 for storing explanatory materials; a related word storage unit 5 for storing one or more related words in relation to each page of the explanatory materials; a topic storage unit 7 for storing a flow of multiple topics related to the explanatory materials in relation to the related words; an input term analysis unit 11 for analyzing input terms; a next page output unit 13 for analyzing the current topic in a flow of multiple topics based on the input terms and outputting the page of the explanatory material related to the next topic at the current time; and a selected page output unit 15 for outputting the page of the explanatory material based on an input selection instruction if there are multiple topics following the current topic. [Effects of the Invention]

[0007] According to this invention, if there are branches in the topics based on the pages of the document, the system can display desirable information based on the user's interests and provide a presentation based on that desirable information by allowing the user to select a branch. [Brief explanation of the drawing]

[0008] [Figure 1] Figure 1 is a block diagram illustrating an example of a system for assisting in the presentation of explanatory materials. [Figure 2] Figure 2 is a conceptual diagram illustrating one anticipated use case of the explanatory material presentation support system. [Figure 3] Figure 3 is a conceptual diagram illustrating the flow of topics on a particular page of the explanatory material. [Figure 4] Figure 4 is a conceptual diagram showing an example of thumbnail display. [Figure 5] Figure 5 is a conceptual diagram illustrating the user learning section. [Modes for carrying out the invention]

[0009] Figure 1 is a block diagram illustrating an example of a system for assisting in the presentation of explanatory materials. As shown in Figure 1, the system for assisting in the presentation of explanatory materials 1 includes a material storage unit 3, a related word storage unit 5, a topic storage unit 7, an input term analysis unit 11, a next page output unit 13, and a selected page output unit 15. In the example shown in Figure 1, system 1 further includes a user learning unit 17. This system can output pages selected based on the speaker's interests. This system typically includes a server and multiple terminals. In other words, this system may be implemented by one or more information processing terminals (computers). A conceivable use case of this system is that the speaker's terminal is connected to one or more audience terminals so that information can be exchanged. The speaker's terminal receives assistance in presenting explanatory materials based on this system 1. The speaker's terminal is registered with this system 1, and it is preferable that the information provided for assistance is customized according to the speaker by a trained model described later.

[0010] A computer has an input unit, an output unit, a control unit, an arithmetic unit, and a memory unit, and each element is connected by a bus or the like to enable the exchange of information. For example, the memory unit may store a control program or various kinds of information. When predetermined information is input from the input unit, the control unit reads the control program stored in the memory unit. The control unit then reads the information stored in the memory unit as appropriate and transmits it to the arithmetic unit. The control unit also transmits the input information to the arithmetic unit as appropriate. The arithmetic unit performs calculations using the received information and stores it in the memory unit. The control unit reads the calculation results stored in the memory unit and outputs them from the output unit. In this way, various processes and steps are executed. Each unit and each means is responsible for executing these various processes. A computer may have a processor, and the processor may implement various functions and steps. A computer may be standalone. A computer may have some of its functions distributed between a server and terminals. In that case, it is preferable that the server and terminals can exchange information via a network such as the internet or an intranet. A computer may include a processor and memory connected to the processor. The memory may store instructions, which, when executed by the processor, cause the computer to perform various processes or to function as various components. The computer may build a learning model by providing various training data and perform various calculations through machine learning. In this case, the computer may perform various analyses and interpretations using the learning model created by AI (artificial intelligence) machine learning and deep learning.

[0011] The system typically includes a display unit, which is not shown in the diagram. The display unit is an element for displaying various types of information based on a computer. A monitor or display, which is a type of computer output unit, or a smartphone's touch panel can function as the display unit. The display unit may also be a projector. When giving a presentation, the monitor of a computer or tablet can function as the display unit, and presentation materials may be projected using a projector. In this case, as will be described later, the monitor may display not only the presentation materials but also information regarding the word order of related terms, or both, along with explanatory text.

[0012] Figure 2 is a conceptual diagram showing a hypothetical use case of the presentation assistance system. In this example, the speaker's terminal 21 is connected to the audience's terminal 23 via the internet. This use case could be a web conference, answering questions from the audience (users), or a medical representative (MR) explaining materials to a doctor (audience member) in the audience. The speaker's terminal 21 may display the speaker's face as captured by the camera, or a page from the materials. These may also be displayed on the audience's terminal 23. On the other hand, the speaker's terminal 21 may also display various information for selecting the next page, as will be described later. In the example in Figure 2, a thumbnail 25 indicating a page from the presentation materials is displayed on the speaker's terminal 21, and related words are also displayed in the related words display field 27 for that page. If the speaker generates a related word or processes the output of the page displayed as a thumbnail, that page may also be output to the audience's terminal 23 and displayed on the audience's terminal 23.

[0013] Data Storage Unit 3 The data storage unit 3 is an element for storing explanatory materials. The data storage unit 3 functions as a database for storing materials prepared in advance by System 1, for example. The data storage unit 3 stores both the entire presentation material and each page of the presentation material. The computer's memory unit functions as the data storage unit 3. Examples of presentation materials are materials created in PowerPoint® or PDF®. Presentation material can mean the entire series of materials (a file) created with software such as PowerPoint®, or a specific page. For example, each presentation material is assigned an identification number or ID. The data storage unit 3 stores multiple related words in association with the assigned information (identification number or ID), or the identification number and page number (slide number). In this way, each presentation material or each page (each slide) of each material is associated with and stored with multiple related words related to each presentation material.

[0014] An example of a presentation document is a PowerPoint presentation about a new drug X for diabetes. Examples of related terms for that presentation document are the related terms for that PowerPoint document: "diabetes," "X," "dosage," "side effects," "dizziness," "drowsiness," "pregnant women" (who should not be administered the drug), and "under 19 years old" (who should not be administered the drug). These are stored in the document storage unit 3, for example, in relation to the presentation document's identification number (and each page number).

[0015] Related word memory section 5 The related word memory unit 5 is an element for storing one or more related words in relation to each page of the explanatory material. The related word memory unit 5 stores related words corresponding to each page, thereby enabling the association of each page of the material with its topic. For each page, the order in which the related words appear may also be stored. The related words may be keywords or topics, or synonyms of keywords.

[0016] Topic memory section 7 The topic memory unit 7 is an element for storing a sequence of multiple topics related to the explanatory materials, in relation to related words. For example, the topic memory unit 7 structurally records a sequence of multiple topics. It is preferable that the transitions between topics are clearly defined in order and branching based on related words. Furthermore, it is preferable that each topic included in the sequence of topics is stored in relation to related words and each page. In other words, it is preferable that the corresponding page or topic is retrieved based on related words.

[0017] Figure 3 is a conceptual diagram illustrating the flow of topics related to a particular page of explanatory material. For example, the topic memory unit 7 may store multiple topic flows in relation to the entire explanatory material. Alternatively, for example, the topic memory unit 7 may store multiple topic flows in relation to each page of the explanatory material. The topic flows may include, for example, related words for each topic that constitutes a topic. The topic flows may then be formed based on the order in which the related words are registered. In the example in Figure 3, for a page that is called up in relation to a certain related word, there are four major topic flows, each with its own branch. For example, if the term based on the input audio is a related word from A4, and the term analyzed based on subsequent input audio is also a related word after A4, the topic flows are uniform, and the next page after this page is also determined. Therefore, for example, if an explanation is given based on this page and the input audio is based on the flow of A4, the next page may be read and output at a predetermined timing after the last related word and related term have been input. On the other hand, for example, if the first related word based on the input speech is like A1, a list of options may be presented, and the next page may be displayed based on the selection information of the presented options.

[0018] Input term analysis unit 11 The input term analysis unit 11 is an element for analyzing the voice input to the system and obtaining the terms included in the voice. The input term analysis unit 11 is implemented, for example, by a control program stored in the storage unit. The input term analysis unit 11 can be implemented using a known voice recognition device. The voice input to the system is usually analog information. The input term analysis unit 11 only needs to convert the analog information into digital information and store it in a form that can be processed by a computer as a term. For example, the voice information input to the system 1 is read. Then, the control unit of the computer reads the control program stored in the storage unit and causes the read voice information to be analyzed by the arithmetic unit. At this time, a plurality of terms stored in the storage unit and the voice information of those terms are read and the terms are analyzed. Then, the analyzed terms may be appropriately stored in the storage unit. In this way, the input term analysis unit 11 can analyze the terms included in the voice input to the system.

[0019] Next page output unit 13 The next page output unit 13 is an element for analyzing the current topic in the flows of a plurality of topics based on the input terms and outputting a page of an explanatory material related to the next topic after the current time. For example, the next page output unit 13 grasps, based on the input terms, up to which part the conversation has progressed regarding the current topic. Moreover, when the page of the explanatory material related to the next topic is determined, the next page output unit 13 outputs the next page at the timing when the explanation regarding the current topic related to the current page has ended. When the current page is page A1 and the flow of related terms included in the topic related to that page is A1A, A1B, A1C, and A1D, the next page output unit 13 may examine whether the term analyzed by the input term analysis unit 11 is related to any of the related terms and grasp up to which part the conversation has progressed regarding the current topic. And when the next page output unit 13 determines that the terms input to the system are related in the order of A1A, A1B, A1C, and A1D, since the conversation is being carried out along the assumed conversation flow regarding the current topic, page A2, which is the next page of page A1 among the explanatory materials, is displayed, and related terms along the conversation flow memorized in association with page A2 may be read out. Also, even when the voice is automatically outputting the topic, the next page determined in advance at the timing when the current topic ends may be automatically displayed, and the voice regarding the next page may be automatically outputted.

[0020] Selected page output unit 15 The selected page output unit 15 is an element for outputting the page of the explanatory material based on the input selection instruction when there are multiple pages of explanatory material related to the next topic of the current topic (the page currently displayed in the display unit). When the topic branches into multiple parts, it outputs the next page based on the selection instruction. The selected page output unit 15 may also output related words related to each of the next topics when there are multiple topics following the current topic, and output the page of the explanatory material based on the input selection instruction related to the related words. For example, multiple related words may be displayed in the related word display field 27. If, after the topic about the current page, there are related words such as B1, B2, and B3 and related topics (next pages), those related words may be displayed in the related word display field 27 of the speaker's terminal 21 while the page is being displayed on the speaker's terminal 21 and the audience's terminal 23. Then, when the speaker says something like "B1", the page related to B1 may be displayed immediately after the topic about the current page has finished. Furthermore, System 1 may store voice commands such as "Be One," and when a related word displayed on the display unit is input from the speaker's terminal 21, it may process the audio so that the voice related to that word is not output to the listener's terminal 23. In this way, the appropriate page can be selected without the listener realizing that the next page has been indicated by voice.

[0021] The selected page output unit 15 is, If there are multiple subsequent topics following the current topic, and each of these subsequent topics has corresponding explanatory material pages, the system may output thumbnails for each page and then output explanatory material pages based on selection instructions entered in relation to a specific page within those pages.

[0022] Figure 4 is a conceptual diagram showing an example of thumbnail display. In this example, multiple thumbnails 25 are displayed on the speaker's terminal 21. In this example, the thumbnails display the next page. In this example, the thumbnails also display information for reading the pages associated with these thumbnails. For example, if the voice input is "at mark 1", the upper left slide will be output to the audience's terminal (and the speaker's terminal). For thumbnails, for example, the photo information contained in each slide can be reduced in resolution and scaled down to a predetermined reduced size. Alternatively, for example, characters smaller than a predetermined font size can be deleted, and characters larger than the predetermined font size can be scaled down to a predetermined size before being displayed on the display. In this way, a stylized image of each slide can be obtained. If the obtained stylized images can be displayed on multiple screens, thumbnail display can be performed. Thus, the images related to multiple slides that the slide images display on the screen may be a collection of scaled-down images of each slide.

[0023] The input selection instructions may be voice inputs related to pre-registered selections. Examples of selections include returning to the previously displayed page, moving to the next page, ending the presentation, and returning to the first page. It is preferable that the voice input to the speaker's input unit is canceled so that it is not heard by the audience. For example, if the speaker wants to move to the next page instead of the page displayed on the display unit, they may say "at sign, at sign," and when this voice is input to the system, the input term analysis unit 11 should analyze the term "at sign, plus." The presentation assistance system 1 can then read the instruction stored in association with "at sign, plus" from the instruction storage unit and perform the predetermined control. For example, in this case, the next page after the current page will be displayed. In addition, the voice output for this selection instruction "at sign, plus" may be canceled out so that it is not heard by the audience. The storage unit may store the selection command "move to the next page" in association with the voice "at sign, plus," and when "at sign, plus" is input by voice, it should read the selection command stored in association with that voice and perform the processing corresponding to the selection command. These voice instructions should preferably be ones not included in the usual explanation.

[0024] The following describes examples of commands (voice commands) that can be entered via voice input. When using voice commands during speech recognition, it is preferable that the trigger for the voice command be a term that does not exist in Japanese, as there are no breaks in the Japanese language. An example of a voice command that takes this into consideration is the "@" symbol. The "@" symbol may be changed as appropriate. However, terms that appear in the explanation of the document, related words, keywords, etc., are not suitable as triggers. It is preferable to be able to construct selection commands as appropriate, including the term that follows the trigger of the voice command.

[0025] Examples of voice commands are as follows: @ (The ~ symbol indicates that voice input is unavailable.) After a predetermined time (for example, 3 seconds) following the voice command "at mark", a RAG lookup is performed and the result is displayed. @ symbol + number (for example, one) This feature displays a thumbnail preview based on a specified number. It also displays the page that was previously shown as a thumbnail as the next page. When implementing this feature, it is preferable to avoid including numbers in related words and to prevent RAG lookup from being performed on utterances such as "number~~~". Atmark Plus Display the next page of the preview. @minus Display the previous page of the preview. At mark At mark Close the preview. The above is an example of a voice command. By using such voice commands, it becomes possible to display pages based on the user's emotions. Furthermore, the user learning unit 17, which will be described later, can suggest candidates for the next page based on the flow of the conversation, thus enabling the output of materials that more accurately reflect the user's emotions.

[0026] User Learning Section 17 Preferably, the system further includes a user learning unit 17 that learns the user's selection instruction information. This relates to the processing of the next page output unit 13 and the selected page output unit 15. Figure 5 is a conceptual diagram illustrating the user learning unit. Information including the user's selection instruction information of System 1 is used as learning data and input into an artificial intelligence model to construct a trained model that can estimate candidates for the next page. The learning data of the trained model may include the flow of topics (the order in which related words appear) and the selection instruction information at each branch. In addition, the learning data may further include either or both speaker information and audience information. By inputting various data as learning data into the artificial intelligence model, it becomes possible to effectively output materials based on the user's emotions (emotions such as which page they want to display as the next page). At that time, by including the flow of topics (the order in which related words appear) before and after the input of selection instruction information as learning data, the relationship between the flow of topics and selection at branches can be effectively learned. For example, the flow of topics shown in Figure 3 may be modified as appropriate by the user learning unit 17. Then, when data corresponding to the training data (for example, information about the flow of topics before branching) is input to the user learning unit 17 constructed in this manner, it becomes possible to effectively output information about the next page. In this way, by using the flow of topics based on the order in which related words appear, related to multiple pages, as training data to construct a trained model, it becomes possible to construct a user learning unit 17 that can accurately grasp the user's thoughts or feelings and suggest (output) the next page in accordance with the flow of topics.

[0027] Information about the speaker and / or audience may be stored in the memory of System 1. Examples of such information include employee number, name, gender, age, sales performance, job title, place of origin, presentation evaluation, years of service, and years in charge. Examples of information about the audience are the size of the hospital, the location of the hospital, whether it is a lecture or for a single doctor, and (if for a doctor) information about the doctor. Other examples of information about the audience are the region of the lecture, the level of the attendees, the academic year of the attendees, the number of attendees, the occupation of the attendees, the job duties of the attendees, the years of service of the attendees, and the job title of the attendees.

[0028] When constructing an artificial intelligence model, it is preferable to use RAG (Search Augmentation Generation). Using Search Augmentation Generation, an artificial intelligence model can be constructed as follows. Step 1: Data Preparation Collection of document corpora First, collect a knowledge base (document corpus) to be used for searching. Examples of a knowledge base include the flow of past topics and branching selection information at branching points. The collected information may be divided as needed. Step 2: Preparing the Retriever Model For example, a Dense Retriever could be used to compute embedding vectors between the training data and the output. Specifically, pairs of training and output data could be used to train the model to maximize the similarity between the question embedding and the correct embedding. However, vector computation is not required. The model could be trained by feeding back the correct and incorrect suggestions for the next page or candidate pages from System 1 as answers. Step 3: Preparing the Generator Model Selection of a pre-trained generative model The model is trained to take search results as input and generate answers corresponding to the questions. Step 4: Integrating the RAG Model Integration of search and generative models

[0029] An invention described in this specification relates to a computer program and a computer-readable non-temporary information recording medium (such as a CD-ROM) that stores the program. This program is basically a computer-readable program that enables the computer to function as one of the above-described explanatory material presentation assistance systems. For example, when this program is installed on a computer, it can enable the computer to function as an explanatory material presentation assistance system 1 having a data storage unit 3, a related word storage unit 5, a topic storage unit 7, an input term analysis unit 11, a next page output unit 13, and a selected page output unit 15.

[0030] The following describes an example of the process for providing explanatory materials using the explanatory material presentation assistance system. As explained earlier, the explanatory material presentation assistance system 1 includes a material storage unit 3, a related word storage unit 5, a topic storage unit 7, an input term analysis unit 11, a next page output unit 13, and a selected page output unit 15. Furthermore, it is preferable that system 1 further includes a user learning unit 17. The user learning unit 17 may be constructed using the method described earlier.

[0031] Suppose a speaker is explaining page A using their terminal 21. The user learning unit 17 has learned the flow of topic A on page A. The speaker's pronunciation is input to system 1, which analyzes it and analyzes the flow of conversation about page A, including related words. System 1 inputs information about the flow of conversation to the user learning unit 17. The user learning unit 17 then outputs only one candidate for the next page. It is likely that in the past, when a similar flow of conversation occurred, a specific next page was often used. In this case, the next page output unit 13 of system 1 outputs the next page relating to topic B, which is the next topic after topic A. This next page (page B) is also output to the audience's terminal. Page B is then displayed on the audience's terminal monitor. If this selection is correct, the speaker can, for example, input a message indicating that the selection is correct. This further improves the accuracy of the user learning unit 17. On the other hand, if page B is displayed as a thumbnail in the speaker's display area and page B is incorrect, the speaker will pronounce something like "at sign minus". System 1 will then perform processing related to the selection instruction "at sign minus". In this case, since the thumbnail of the next page candidate was incorrect, the system may display multiple related words for the next page, or display thumbnails for multiple candidates for the next page.

[0032] Furthermore, suppose a speaker is explaining page C using the speaker's terminal 21. Suppose page C is about topic C. For simplicity, in this example, each page is assumed to be about only one topic. The flow of the conversation then branches into topics D and E after topic C. Then, the next pages after page C are candidates for page D and page E. The selected page output unit 15 may further display thumbnails 25 of page D and page E on the display unit of the speaker's terminal 21 where page C is displayed, or it may display related words related to page D and page E in the related word display field 27. In this way, the speaker can be effectively allowed to select the next page. As explained earlier, the candidates for page D and page E may also be output data by inputting the flow of a speaker's conversation (and therefore the selection information) as input data to the user learning unit 17.

[0033] Furthermore, by storing explanatory sounds related to each page in the memory unit, it becomes possible to provide appropriate audio explanations based on the user's page selection command. For example, when trying to understand something (on one's own) based on the materials, the user can select the appropriate page by speaking a selection command based on the selection information displayed at branching points (e.g., thumbnails, related words), and receive an appropriate explanation that follows the flow of the conversation and corresponds to the user's emotions (the emotions of wanting this kind of information). [Industrial applicability]

[0034] This invention can be used in information-related industries because it learns the flow of past data utilization and proposes established data based on the current flow of data utilization. [Explanation of symbols]

[0035] 1. System to assist in presenting explanatory materials 3. Data Storage Section 5. Related Word Memory Section 7 Topic memory section 11 Input Term Analysis Unit 13 Next Page Output Section 15. Output section for selected page 17 User Learning Department 21 Speaker's terminal 23 Audience terminals 25 thumbnails 27 Related Words Display Field

Claims

1. A data storage unit (3) for storing explanatory materials, A related word memory unit (5) that stores one or more related words in relation to each page of the aforementioned explanatory material, A topic memory unit (7) that stores a sequence of topics related to the aforementioned explanatory materials in relation to the aforementioned related words, wherein the sequence of topics relates to the order in which the related words appear, An input term analysis unit (11) analyzes the input term, If there are multiple pages of the explanatory material related to the next topic of the current discussion, the selected page output unit (15) outputs the page of the explanatory material based on the input selection instruction, wherein the input selection instruction is a pre-registered voice input related to selection, A system for assisting the presentation of explanatory materials (1).

2. The explanatory material presentation assistance system (1) according to claim 1, The aforementioned selection page output unit (15) is an explanatory material presentation assistance system (1) that cancels out the output of audio related to the selection when audio related to the selection is input.

3. An explanatory material presentation assistance system (1) according to claim 2, wherein the audio related to the selection is not included in the related words.

Images