Information processing system, information processing device, information processing method, and program

Abstract

To propose an appropriate topic for current conversation.SOLUTION: The present invention provides an information processing system that comprises: an acquisition unit that acquires speech audio from a participant who participates in a communication and other participants; a speech recognition unit that converts the speech audio into conversation data by performing speech recognition of the speech audio; a search unit that searches for conversation patterns where a first word contained in the conversation data corresponds to two or more different words; and a topic providing unit that provides a conversation screen displaying a second word associated with the first word in the conversation pattern containing the first word matching the search.SELECTED DRAWING: Figure 23

Description

[Technical Field]

[0001] The present invention relates to an information processing system, an information processing device, an information processing method, and a program. [Background technology]

[0002] Remote communication, which involves sending and receiving audio and / or image content data between multiple terminal devices, is becoming increasingly common. However, remote communication can be more difficult to achieve smoothly than face-to-face communication.

[0003] A technology for facilitating communication has been devised (see, for example, Patent Document 1). Patent Document 1 discloses a technology for determining the active state of a conversation from audio or images during a call and outputting keyword information stored in a database. Summary of the Invention [Problem to be solved by the invention]

[0004] However, conventional technologies have a problem in that it is difficult to suggest appropriate topics for the current conversation. For example, simply presenting keywords that often appear as material for creating topics, as in the past, does not necessarily provide appropriate topics for the current conversation.

[0005] In view of the above-mentioned problems, the present invention provides a technique for suggesting appropriate topics for a current conversation. [Means for solving the problem]

[0006] In view of the above problems, the present invention provides an information processing system having an acquisition unit that acquires speech from participants participating in communication and other participants, a speech recognition unit that converts the speech into conversation data by speech recognition, a search unit that searches for conversation patterns in which a first word included in the conversation data is associated with two or more different words, and a topic provision unit that provides a conversation screen that displays a second word associated with the first word in the conversation pattern including the first word that matches the search. [Effects of the Invention]

[0007] The present invention can provide a technique for suggesting appropriate topics for a current conversation. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 1 is a diagram for schematically explaining how a topic is provided by an information processing system. [Figure 2] FIG. 10 is a diagram illustrating an outline of the operation of the information processing system to provide topics that may interest Person A. [Figure 3] FIG. 1 illustrates an example of a system configuration of an information processing system. [Figure 4] FIG. 10 is a diagram illustrating another example of the system configuration of the information processing system. [Figure 5] FIG. 1 illustrates an example of a hardware configuration of a computer. [Figure 6] FIG. 2 illustrates an example of a hardware configuration of a terminal device. [Figure 7] FIG. 1 illustrates an example of a functional configuration of an information processing system. [Figure 8] FIG. 2 is a diagram illustrating an example of a functional configuration of a data processing unit. [Figure 9] 1A is a diagram schematically showing the database creation phase, and FIG. 1B is a diagram schematically showing the topic provision phase during conversation. [Figure 10] FIG. 10 is a flowchart illustrating an example of a method for creating a database. [Figure 11]FIG. 10 is a diagram showing an example of conversation data (csv file). [Figure 12] FIG. 10 is a flowchart illustrating an example of a process for extracting a conversation pattern by a pattern extraction unit. [Figure 13] FIG. 10 is a diagram for schematically explaining a method for extracting a conversation pattern. [Figure 14] FIG. 10 is a diagram showing an example list of noun-noun pairs. [Figure 15] FIG. 10 is a diagram showing an example list of nouns, noun pairs, and compound episodes. [Figure 16] FIG. 10 is a diagram showing an example list of nouns, noun pairs, complex episodes, and recursively expanded complex episodes. [Figure 17] FIG. 2 is a diagram showing an example of conversation data that is the basis for creating a conversation pattern. [Figure 18] FIG. 18 is a diagram showing an example of nouns and conversation patterns extracted from the conversation data of FIG. 17. [Figure 19] FIG. 10 is a diagram showing an example list of conversation patterns extracted from a conversation of about 10 minutes. [Figure 20] FIG. 10 is a diagram showing an example list of highly related conversation patterns. [Figure 21] FIG. 10 is a diagram showing a list of examples of unexpected combinations of conversation patterns. [Figure 22] FIG. 10 is a diagram showing an example list of conversation patterns including proper nouns. [Figure 23] FIG. 10 is a flowchart illustrating an example of a process in which an information processing device provides topics in real time. [Figure 24] FIG. 10 is a diagram illustrating an example of a conversation screen displayed by the terminal device. [Figure 25] FIG. 10 is a diagram showing an example of a conversation screen displaying three topics extracted from three databases. [Figure 26] FIG. 10 is a diagram showing an example of a conversation screen displaying a database selection dialog. DETAILED DESCRIPTION OF THE INVENTION

[0009] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An information processing system and an information processing method performed by the information processing system will be described in detail below with reference to the accompanying drawings as embodiments of the present invention.

[0010] <Outline of operation of information processing system> Fig. 1 is a diagram for explaining the provision of topics by an information processing system. In Fig. 1, a user U is a recipient of a service provided by the information processing system, and is remotely communicating with a conversation partner (person A).

[0011] (1) The terminal device 100 operated by the user U transmits a speech to the information processing device 20.

[0012] (2) The information processing device 20 analyzes the speech and provides topics that may interest Person A to the terminal device 100. In this embodiment, the topics are, for example, nouns.

[0013] (3) The terminal device 100 displays topics, and the user views the provided topics and incorporates them into the conversation, thereby realizing smooth communication.

[0014] FIG. 2 is a diagram illustrating the general operation of an information processing system that provides topics that may interest Person A. The information processing system of this embodiment has a database 907 in which a plurality of conversation patterns are stored. In this embodiment, the information processing system uses EMMA (Episode Mining using Memory Anchor) processing as an example of a pattern extraction method. EMMA processing is a frequent episode mining technique that extracts frequent patterns from time-series events. A conversation pattern is a pattern in conversation, and indicates a regular type, type, or a plurality of frequently occurring phrases, etc., in actual conversations or corpora. As an example, in this embodiment, a conversation pattern will be described as a list of two or more nouns.

[0015] STEP 1: The information processing system performs real-time speech recognition of the spoken voice to create conversation data. For example, the information processing system removes unnecessary words and performs morphological analysis. The information processing system also extracts mainly "nouns" from the conversation data.

[0016] STEP 2: The information processing system calculates the scores of the nouns extracted by the morphological analysis and estimates the noun with the highest score as the current topic. The score means a number or points, and in this embodiment, it is a value that represents the desirability of the topic.

[0017] STEP 3: The information processing system searches the database 907 for the noun estimated to be the current topic. The information processing system can search the database 907 for one or more of nouns that are highly related to the noun estimated to be the topic, unexpected nouns that seem unrelated, and proper nouns. Note that the database 907 may be created in real time.

[0018] STEP 4: The information processing system supports smooth communication by displaying nouns that match the search as topics on the screen during online meetings and online business negotiations.

[0019] In this way, the information processing system of this embodiment searches for conversation patterns using the current topic and provides nouns associated with the current topic as topics, so it can provide topics that expand on the current topic (topics with a high degree of relevance) or change the direction of the topic (unexpected topics).In addition, the information processing system can provide proper nouns as topics, so it can provide specific topics.

[0020] <System configuration example> FIG. 3 is a diagram illustrating an example of the system configuration of an information processing system. As shown in FIG. 3, the information processing system 1 includes, for example, a conference server 10, an information processing device 20, multiple terminal devices 100a, 100b, 100c, 100d, and an administrator terminal 101, all of which are connected to a communication network 2 such as the Internet or a LAN (Local Area Network). In the following description, the term "terminal device 100" will be used to refer to any of the multiple terminal devices 100a, 100b, 100c, 100d, and the like. The number of multiple terminal devices 100 illustrated in FIG. 3 is merely an example. The terminal devices 100 only need to be connected to the communication network 2 during communication, and do not need to be connected all the time.

[0021] The information processing system 1 is a system that supports various communications, such as business negotiations, meetings, medical treatments, lessons, and counseling, by transmitting and receiving at least audio between a terminal device 100 and one or more other terminal devices 100. Here, the "support" of communications supported by the information processing system 1 includes support for checking the content of the communications later. As an example, the following description will be given assuming that the information processing system 1 transmits and receives video including audio between a user of a terminal device 100 and a user of another terminal device 100, and supports checking the content of a web conference for business negotiations later.

[0022] The terminal device 100 is a general-purpose information terminal such as a personal computer (PC), tablet terminal, or smartphone used by a user participating in a web conference. Alternatively, the terminal device 100 may be an electronic device with web conference functionality, such as a video conferencing device or an electronic whiteboard. An electronic whiteboard is an electronic whiteboard with a blackboard function that allows for mutual communication, and is also called an electronic information board or electronic whiteboard. Here, the following description will be given assuming that the terminal device 100 is a general-purpose information terminal, as an example.

[0023] Participants in a web conference can join the web conference by, for example, using a web conference application installed on the terminal device 100 or a web browser, accessing the conference address provided by the conference server 10.

[0024] The conference server 10 is an information processing device having a computer configuration or a system including multiple computers. The conference server 10 provides a web conference service for transmitting and receiving audio or video including audio between multiple terminal devices 100. In this embodiment, the web conference service provided by the conference server 10 may be any web conference service.

[0025] The information processing device 20 is, for example, an information processing device having a computer configuration or a system including multiple computers. The information processing device 20 provides the communication support service according to this embodiment.

[0026] The administrator terminal 101 is an information terminal such as a PC, tablet terminal, or smartphone that is used by an administrator or the like who uses the communication support service according to this embodiment. The administrator or the like uses the communication support service by using, for example, a web browser provided in the administrator terminal 101 or an application program (hereinafter referred to as an app) for the information processing system 1 that is executed by the administrator terminal 101.

[0027] Furthermore, participants who have joined the web conference can use the communication support service in the same way as the administrator, for example, by using the terminal device 100 used by the participants or the administrator terminal 101 or the like.

[0028] Note that the system configuration of the information processing system 1 shown in Fig. 3 is one example. For example, the information processing system 1 may not have the conference server 10, as shown in Fig. 4. In this case, the terminal device 100 may hold a web conference using a web conference service external to the information processing system 1. Alternatively, the information processing device 20 may have the functions of the conference server 10, and the terminal device 100 may hold a web conference using a web conference service provided by the information processing device 20.

[0029] <Hardware configuration> <<Computer hardware configuration>> The conference server 10, the information processing device 20, and the administrator terminal 101 each have the hardware configuration of a computer 700 as shown in Fig. 5. The conference server 10 and the information processing device 20 may be realized by a plurality of computers 700. The terminal device 100 may also have the hardware configuration of a computer 700 as shown in Fig. 5, for example.

[0030] Fig. 5 is a diagram showing an example of the hardware configuration of a computer according to an embodiment. A computer 700 includes, for example, a central processing unit (CPU) 701, a read-only memory (ROM) 702, a random access memory (RAM) 703, a hard disk (HD) 704, a hard disk drive (HDD) controller 705, a display 706, an external device connection interface (I / F) 707, a network I / F 708, a keyboard 709, a pointing device 710, a digital versatile disk rewritable (DVD-RW) drive 712, a media I / F 714, and a bus line 715, as shown in Fig. 5.

[0031] Furthermore, when the computer 700 is the terminal device 100, the computer 700 further includes, for example, a microphone 721, a speaker 722, an audio input / output I / F 723, a CMOS (Complementary Metal Oxide Semiconductor) sensor 724, and an image sensor I / F 725.

[0032] Of these, the CPU 701 controls the overall operation of the computer 700. The ROM 702 stores programs used to start up the computer 700, such as an IPL (Initial Program Loader). The RAM 703 is used, for example, as a work area for the CPU 701. The HD 704 stores programs such as an OS (Operating System), applications, and device drivers, as well as various data. The HDD controller 705 controls the reading and writing of various data from and to the HD 704, for example, under the control of the CPU 701. The HD 704 and the HDD controller 705 are examples of storage devices provided in the computer 700.

[0033] The display 706 displays various types of information such as a cursor, a menu, a window, characters, or an image. The display 706 may be provided outside the computer 700. The external device connection I / F 707 is an interface for connecting various external devices to the computer 700. The network I / F 708 is an interface for connecting the computer 700 to the communication network 2 and communicating with other devices.

[0034] The keyboard 709 is a type of input means having a plurality of keys for inputting characters, numbers, various instructions, etc. The pointing device 710 is a type of input means for selecting and executing various instructions, selecting a processing target, moving a cursor, etc. The keyboard 709 and pointing device 710 may be provided outside the computer 700.

[0035] The DVD-RW drive 712 controls reading and writing of various data from and to a DVD-RW 711, which is an example of a removable recording medium. The DVD-RW 711 is not limited to a DVD-RW, and may be other recording media. The media I / F 714 controls reading and writing (storing) of data from and to a medium 713 such as a flash memory. The bus line 715 includes an address bus, a data bus, various control signals, and the like for electrically connecting the above components.

[0036] The microphone 721 is a built-in circuit that converts sound into an electrical signal. The speaker 722 is a built-in circuit that converts the electrical signal into physical vibrations to produce sound such as music or voice. The sound input / output I / F 723 is a circuit that processes the input and output of sound signals between the microphone 721 and the speaker 722 under the control of the CPU 701.

[0037] The CMOS sensor 724 is a type of built-in imaging means that captures an image of a subject (e.g., a self-portrait) and obtains image data under the control of the CPU 701. Note that the computer 700 may have an imaging means such as a CCD (Charge Coupled Device) sensor instead of the CMOS sensor 724. The imaging element I / F 725 is a circuit that controls the driving of the CMOS sensor 724.

[0038] <<Example of hardware configuration of terminal device>> FIG. 6 is a diagram illustrating an example of the hardware configuration of a terminal device according to an embodiment. Here, an example of the hardware configuration of the terminal device 100 will be described when the terminal device 100 is an information terminal such as a smartphone or a tablet terminal. Note that the terminal device 100 may be, for example, a video conferencing device or an electronic device having a computer configuration such as an IWB and a web conferencing function. Furthermore, the administrator terminal 101 may have the hardware configuration of the terminal device 100 as shown in FIG. 6.

[0039] In the example of Figure 6, the terminal device 100 includes a CPU 801, a ROM 802, a RAM 803, a storage device 804, a CMOS sensor 805, an image sensor I / F 806, an acceleration / direction sensor 807, a media I / F 809, a GPS (Global Positioning System) receiving unit 810, etc.

[0040] Of these, the CPU 801 controls the overall operation of the terminal device 100 by executing a predetermined program. The ROM 802 stores a program used to start up the CPU 801, such as an IPL. The RAM 803 is used as a work area for the CPU 801. The storage device 804 is a large-capacity storage device that stores programs such as an OS and applications, various types of data, and the like, and is realized by, for example, an SSD (Solid State Drive), a flash ROM, or the like.

[0041] The CMOS sensor 805 is a type of built-in imaging means that captures an image of a subject (mainly a self-portrait) under the control of the CPU 801 to obtain image data. Note that the terminal device 100 may have an imaging means such as a CCD sensor instead of the CMOS sensor 805. The imaging element I / F 806 is a circuit that controls the driving of the CMOS sensor 805. The acceleration / azimuth sensor 807 is one of various sensors, such as an electronic magnetic compass or gyrocompass that detects geomagnetism, and an acceleration sensor. The media I / F 809 controls the reading or writing (storage) of data from a medium (storage medium) 808, such as a flash memory. The GPS receiver 810 receives GPS signals (positioning signals) from GPS satellites.

[0042] The terminal device 100 also includes a long-distance communication circuit 811, an antenna 811a of the long-distance communication circuit 811, a CMOS sensor 812, an image sensor I / F 813, a microphone 814, a speaker 815, an audio input / output I / F 816, a display 817, an external device connection I / F 818, a short-distance communication circuit 819, an antenna 819a of the short-distance communication circuit 819, and a touch panel 820.

[0043] Of these, the long-distance communication circuit 811 is a circuit that communicates with other devices, for example, via the communication network 2. The CMOS sensor 812 is a type of built-in imaging means that captures an image of a subject and obtains image data under the control of the CPU 801. The image sensor I / F 813 is a circuit that controls the driving of the CMOS sensor 812. The microphone 814 is a built-in circuit that converts sound into an electrical signal. The speaker 815 is a built-in circuit that converts an electrical signal into physical vibrations to generate sound such as music or voice. The sound input / output I / F 816 is a circuit that processes the input and output of sound signals between the microphone 814 and the speaker 815 under the control of the CPU 801.

[0044] The display 817 is a type of display means such as a liquid crystal display or organic electroluminescence (EL) display that displays an image of a subject, various icons, etc. The external device connection I / F 818 is an interface for connecting various external devices. The short-range communication circuit 819 includes a circuit for performing short-range wireless communication. The touch panel 820 is a type of input means that allows a user to operate the terminal device 100 by pressing the display 817.

[0045] The terminal device 100 also includes a bus line 821. The bus line 821 includes an address bus, a data bus, and the like for electrically connecting the components such as the CPU 801 shown in FIG.

[0046] 6 is an example of the hardware configuration of the terminal device 100. The terminal device 100 may have various other hardware configurations as long as it has a computer configuration, a communication circuit, a display, a microphone, a speaker, and the like.

[0047] <Functional configuration> Next, a description will be given of an example of the functional configuration of the information processing system 1. Fig. 7 is a diagram showing an example of the functional configuration of the information processing system 1 according to an embodiment.

[0048] <<Functional configuration of terminal device>> Here, the following description will be given assuming that the terminal device 100 has the hardware configuration of a computer 700 as shown in FIG.

[0049] The terminal device 100 realizes functional configurations such as a communication unit 911, a conference control unit 912, an audio transmission unit 913, a display control unit 914, and an operation reception unit 915 by the CPU 701 executing a predetermined program. Note that at least a part of the above functional configurations may be realized by hardware. In this embodiment, the terminal device 100b may have any functional configuration as long as it can participate in the same conference as the terminal device 100a via the conference server 10.

[0050] The communication unit 911 connects the terminal device 100 to the communication network 2 using, for example, the network I / F 708, and executes communication processing for communicating with, for example, the conference server 10, the information processing device 20, or other terminal devices 100, etc.

[0051] The conference control unit 912 executes a series of processes related to a web conference, such as connecting to the conference, sending and receiving conference video (or conference audio), and inputting and outputting conference video (or conference audio). Note that the processes executed by the conference control unit 912 may be the same as those executed in a general web conference. Note that a conference is an example of communication according to this embodiment.

[0052] The voice transmitting unit 913 acquires the speech of a participant participating in the conference (for example, a user using the terminal device 100a) and the speech of another participant participating in the conference (for example, a user using the terminal device 100b), and executes a voice transmitting process to transmit the speech to the information processing device 20. For example, the voice transmitting unit 913 acquires speech such as speech acquired by a microphone and speech output from a speaker from the sound input / output I / F 723 or the like, and transmits the acquired speech (voice data) to the information processing device 20. This enables the information processing system 1 to support communication not limited to web conferences, but also interactive conferences and the like.

[0053] However, this is not limited to this, and the voice transmitting unit 913 may acquire the speech of the participant and the speech of other participants from the conference control unit 912 and transmit the acquired speech (voice data) to the information processing device 20.

[0054] The display control unit 914 performs a display control process to display a display screen on a display unit such as the display 706. The operation reception unit 915 performs an operation reception process to receive operations from participants using an input device such as the keyboard 709 or the pointing device 710. Note that the conference control unit 912, the display control unit 914, and the operation reception unit 915 may be realized by, for example, a web browser included in the terminal device 100.

[0055] <<Functional configuration of information processing device>> The information processing device 20 realizes, by one or more computers 700 executing a predetermined program, functional configurations of, for example, a communication unit 901, an acquisition unit 902, a data processing unit 903, a search unit 904, and a topic providing unit 905. Note that at least a part of the above functional configurations may be realized by hardware.

[0056] The information processing device 20 also implements a storage unit 906 using storage devices such as the HD 704 and the HDD controller 705, for example.

[0057] The communication unit 901 uses, for example, the network I / F 708 or the like to connect the information processing device 20 to the communication network 2, and executes communication processing for communicating with the terminal device 100, the administrator terminal 101, and the like.

[0058] The acquisition unit 902 executes an acquisition process to acquire the speech of a participant participating in a conference (communication) and the speech of other participants participating in the same conference. For example, the acquisition unit 902 acquires the speech of the participant and the speech of other participants that are transmitted to the information processing device 20 by the speech transmission unit 913 of the terminal device 100 used by the participant.

[0059] The data processing unit 903 performs data processing on the speech sounds (speech data) of the participant and other participants acquired by the acquisition unit 902. The data processing unit 903 will be described with reference to FIG.

[0060] The search unit 904 performs a search process to search a database 907 stored in the storage unit 906 for a noun that is estimated to be the current topic. The method of creating the database 907 will be described later. The topic providing unit 905 also performs a topic providing process to provide the terminal device 100a with the topic that the search unit 904 has searched for from the database 907.

[0061] The storage unit 906 stores, for example, the speech voice acquired by the acquisition unit 902, data such as conference voice created by the data processing unit 903, and a database 907 of conversation patterns. The database 907 has databases 1 to 3, where database 1 stores conversation patterns with high relevance, database 2 stores conversation patterns of unexpected combinations, and database 3 stores conversation patterns including proper nouns. Note that the database 907 may be located on a network accessible by the information processing device 20.

[0062] <<Conference server functional configuration>> The conference server 10 realizes functional components such as a communication unit 921, a conference management unit 922, and a relay unit 923 by executing a predetermined program on one or more computers 700. Note that at least a part of the above functional components may be realized by hardware.

[0063] The communication unit 921 connects the conference server 10 to the communication network 2 using, for example, the network I / F 708 and executes communication processing for communicating with the terminal devices 100a, 100b, . . .

[0064] The conference management unit 922 executes a conference management process that allows multiple terminal devices 100 that access the same web conference URL (Uniform Resource Locator) to participate in the same conference (session).

[0065] The relay unit 923 executes a relay process to relay content data including audio and images between multiple terminal devices 100 participating in the same conference. Note that the functions of the conference management unit 922 and the relay unit 923 may be those of an existing web conference system.

[0066] In this embodiment, the conference server 10 may be any web conference service external to the information processing system 1.

[0067] <<Function configuration of administrator terminal>> The administrator terminal 101 realizes a communication unit 931, a display control unit 932, an operation reception unit 933, and the like, for example, by the computer 700 executing a predetermined program. Note that at least a part of the above functional configurations may be realized by hardware.

[0068] The communication unit 931 connects the administrator terminal 101 to the communication network 2 using, for example, the network I / F 708, and executes communication processing for communicating with, for example, the information processing device 20, etc.

[0069] The display control unit 932, for example, executes a display control process to display a display screen provided from the information processing device 20 on a display unit such as the display 706. The operation reception unit 933, for example, executes an operation reception process to receive operations, etc., from an administrator or the like on the display screen displayed by the display control unit 932. Note that the display control unit 932 and the operation reception unit 933 may be realized by, for example, a web browser or the like provided in the administrator terminal 101. For example, the administrator terminal 101 may be a general-purpose information terminal or the like provided with a web browser.

[0070] 7 is an example. For example, the functions of the conference server 10 may be provided by the information processing device 20, or may be provided by a computer 700 external to the information processing system 1, a cloud service, or the like. At least a part of the functional configuration of the information processing device 20 may be provided by the terminal device 100, the administrator terminal 101, or the like. Furthermore, at least a part of the functional configuration of the terminal device 100 may be provided by the information processing device 20, the conference server 10, or the like. In short, each functional configuration of the information processing system 1 shown in FIG. 7 may be provided by any of the devices included in the information processing system 1.

[0071] <<Data processing section>> 8 is a diagram illustrating an example of the functional configuration of a data processing unit 903 according to an embodiment. The data processing unit 903 includes, for example, a conference speech creation unit 1001, a speech recognition unit 1002, a preprocessing unit 1003, a morphological analysis unit 1004, a score calculation unit 1005, a pattern extraction unit 1006, and a database creation unit 1007.

[0072] The conference voice creation unit 1001 executes a conference voice creation process that synthesizes the speech voice of the participant acquired by the acquisition unit 902 with the speech voice of other participants to create conference voice that includes the speech voice of the participant and the speech voice of other participants.

[0073] The speech recognition unit 1002 performs speech recognition processing to distinguish between the speech of the participant and the speech of other participants and convert the speech into conversation data by speech recognition. The conversation data is a collection of text data divided into appropriate lengths.

[0074] The preprocessing unit 1003 performs preprocessing such as removing unnecessary words that are listed in a dictionary of unnecessary words from the conversation data. Unnecessary words are nouns that are unlikely to be used in conversation, such as "honma" (really), "toki" (time), "there," numbers, and symbols.

[0075] The morphological analysis unit 1004 performs morphological analysis on the participant's conversation data and the conversation data of other participants, respectively, and performs morphological analysis processing to break down the data into words with parts of speech. In this embodiment, the morphological analysis unit 1004 extracts nouns from the results of the morphological analysis, but it is also possible to extract words with other parts of speech.

[0076] The score calculation unit 1005 performs a score calculation process to calculate for each noun a score that increases each time the same noun appears from conversation data created in real time. The score calculation unit 1005 calculates scores by assigning different weights to nouns contained in the speech of other participants and to nouns contained in the speech of the participant, thereby making it easier to identify words that other participants are talking about.

[0077] The pattern extraction unit 1006 performs a pattern extraction process to extract conversation patterns from the nouns obtained by the morphological analysis. In this embodiment, the pattern extraction unit 1006 uses EMMA processing as an example of a pattern extraction method. Details of pattern extraction using EMMA processing will be described later.

[0078] The database creation unit 1007 performs a database creation process to create a database 907 that uses the multiple conversation patterns extracted by the pattern extraction unit 1006 as data. The database 907 can be created for each source speaker, such as for each individual participant, for the same "participant and other participants," or for all arbitrary participants. For example, when the same participant and other participants have multiple business negotiations, if a database 907 is created for the same "participant and other participants," the database 907 will be enriched with each business negotiation, making it easier for the topic provision unit 905 to provide an appropriate topic (noun searched for by the conversation pattern) for the current topic.

[0079] <Processing details> The processing performed by the information processing system 1 will be described in detail below. First, as shown in FIG. 9, the information processing system 1 1. Creating a database 2. Providing topics for conversation It has two phases.

[0080] 9(a) shows a schematic diagram of the database creation phase, and FIG. 9(b) shows a schematic diagram of the conversation topic provision phase. In the database creation phase, the pattern extraction unit 1006 analyzes the noun appearance patterns from the conversation data 231 between the participant and other participants using EMMA processing, and creates conversation patterns 232 that are determined to be specific appearance patterns. The database creation unit 1007 saves the multiple conversation patterns 232 in the database 907.

[0081] First, in the conversation topic provision phase, a topic 235 is estimated from real-time conversation data 234 of the participant and other participants. A search unit 904 searches a database 907 using this topic 235 to identify a conversation pattern. The topic provision unit 905 provides, as a topic 236, a noun that has not been used in the search in the identified conversation pattern.

[0082] The information processing system 1 can also create conversation patterns 232 and create database 907 during the conversation topic provision phase. As a result, even if a participant converses with another participant for the first time, a topic can be provided once time has passed and the information processing system 1 is able to create database 907.

[0083] <<Create a database>> Next, a method for creating database 907 will be described in detail with reference to Fig. 10 to Fig. 16. Fig. 10 is a flowchart illustrating a method for creating database 907. Database 907 may be created from any past conversation data (or speech data as is), or may be created from conversation data created in real time.

[0084] First, the preprocessing unit 1003 acquires conversation data and stores it in a csv file (S11). FIG. 11 shows an example of conversation data (csv file). In the conversation data, an ID is assigned to each conversation sentence. A conversation sentence may be determined, for example, by the timing of a speaker change, a certain amount of silent time, or context. Note that the file format may be other than csv.

[0085] Returning to Fig. 10, the preprocessing unit 1003 sets the initial value of the ID of the csvfile to the variable i (S12). While the variable i is less than the length of the csvfile (the number of utterance sentences), steps S14 to S16 are executed (Yes in S13). When the variable i becomes equal to or greater than the length of the csvfile (the number of utterance sentences) (No in S13), steps S17 and S18 are executed.

[0086] In step S14, the preprocessing unit 1003 removes unnecessary words from each conversation sentence (S14). Unnecessary words are words that are not appropriate for the topic and are listed in the dictionary of unnecessary words. Examples of unnecessary words include "honma" (really), "toki" (time), "there," numbers, and symbols.

[0087] Next, the morphological analysis unit 1004 performs morphological analysis on the conversation sentence to identify nouns (S15). However, even if they are nouns, the stems of adjectives, such as "anzen" (safety) and "decisive" (decisiveness), and the indicative nouns, such as "today" and "for the time being," are omitted because their meanings change depending on the context. Note that either step S14 or S15 can be executed first.

[0088] The preprocessing unit 1003 stores the conversation sentences in the csvfile that have been processed as described above in a list (S16).The preprocessing unit 1003 also increments the variable i by one.

[0089] When processing of all conversation sentences of the conversation data stored in the csvfile has been completed and the variable i is greater than the length of the csvfile (No in S13), the pattern extraction unit 1006 extracts conversation patterns from the conversation data in the list (S17). Details of the extraction of conversation patterns will be explained with reference to Figs. 12 to 16.

[0090] The database creation unit 1007 stores the conversation patterns in the database 907 (S18).

[0091] <<How to extract conversation patterns>> 12 is a flowchart illustrating the process of extracting a conversation pattern by pattern extraction unit 1006. Each process in FIG. 12 will be described below with reference to FIGS.

[0092] S101 FIG. 13 is a diagram that schematically explains how to extract conversation patterns. In FIG. 13, the passage of time is shown to the right. FIG. 13 shows the correspondence between conversation sentences (bottom row) and nouns (top row). t1 to t11 are conversation sentences, and a to d are nouns. The conversation sentence in FIG. 13 is a sentence of conversation data saved in list in the flowchart of FIG. 11. Conversation sentences t1 to t11 are recorded over time. a to d are nouns contained in the conversation sentences. In other words, it can be seen that conversation sentence t1 contains the nouns a and c.

[0093] In step S101, first, the pattern extraction unit 1006 counts the number (total number) of nouns appearing in each conversation sentence. {a}=5, {b}=3, {c}=2, {d}=1 The pattern extraction unit 1006 retains only the nouns that appear frequently. For example, if the threshold is set to 2, {d} is deleted. {a}=5, {b}=3, {c}=2 S102 Next, the pattern extraction unit 1006 records the appearance position of the noun. The appearance position is the sentence in which the noun appears. Therefore, the appearance position is one of 1 to 11. Location list (a) = {1, 2, 3, 6, 7} Location list (b) = {3,7,9} Location list (c) = {1,8} ·S103 Next, the pattern extraction unit 1006 combines the nouns that appear to find pairs of nouns that frequently appear in a single conversation. For example, focusing on the noun {a}, the pattern extraction unit 1006 finds conversations that include the noun pair {a} and noun {b}. The pattern extraction unit 1006 finds conversations that include the noun pairs {a} and noun {c} and noun {a} and noun {d}. The pattern extraction unit 1006 also finds conversations that include the noun pairs {b}, noun {c}, and noun {d} along with other nouns. However, overlapping noun pairs are excluded (if {a} and {b} are extracted, {b} and {a} are not extracted).

[0094] The pattern extraction unit 1006 also counts the number of noun pairs that are found. For example, when the noun {a} and the noun {b} are combined to form the noun pair {a,b}, the number of occurrences is two because the nouns appear in sentences t3 and t7. When the noun {a} and the noun {c} are combined to form the noun pair {a,c}, the number of occurrences is one because the nouns appear in sentence t1. The pattern extraction unit 1006 deletes {a,c} by setting a threshold of, for example, 2.

[0095] S104 Next, as shown in FIG. 14, the pattern extraction unit 1006 creates a list of noun-noun pairs. In FIG. 14, nouns and noun pairs whose occurrence count is equal to or exceeds a threshold are registered. In FIG. 14, ID is identification information of an episode. An episode is a noun or noun pair extracted from conversation data. As will be described later, an episode may contain three or more nouns. The number is the number of times the episode appears in the conversation data. Bound-list is identification information of the first and last conversation sentences in which the episode appears. The difference between the first and last conversation sentences represents the time interval in which the episode appears.

[0096] S105 Next, the pattern extraction unit 1006 finds a complex episode, as shown in FIG. 15. FIG. 15 shows a list of nouns, noun pairs, and complex episodes. A complex episode is an episode in which {a} is followed by any of {a} to {d} in different conversation sentences. For example, in FIG. 13, {a} appears in conversation sentence t1 and {a} appears in conversation sentence t2, so the complex episode is {{a},{a}}=[1,2]. Similarly, {{a},{a}}=[2,3] [6,7] is found.

[0097] In addition, for example, in Figure 13, {a} appears in conversation sentence t2 and {b} appears in conversation sentence t3, so the composite episode is {{a},{b}}=[2, 3]. Similarly, {{a},{b}}=[6, 7] is found.

[0098] In Figure 15, composite episodes whose number is less than the threshold are deleted. For this reason, {{a}, {c}}, etc. are not registered in Figure 15.

[0099] S106 Furthermore, as shown in FIG. 16, the pattern extraction unit 1006 recursively expands the episodes obtained in FIG. 15. FIG. 16 shows a list of nouns, noun pairs, compound episodes, and recursively expanded compound episodes. Recursion refers to combining different types of episodes, such as a noun-noun pair, a noun and a compound episode, or a noun pair and a compound episode, among the episodes obtained in FIG. 15. For example, when creating a compound episode of {a} and {a, b}, since {a} appears in conversation sentence t2 and {a, b} appears in conversation sentence t3, the compound episode of {a} and {a, b} is {{a},{a, b}}=[2, 3]. Similarly, {{a},{a, b}}=[6, 7] is found.

[0100] S107 The pattern extraction unit 1006 extracts, as conversation patterns, episodes including multiple nouns from among the episodes obtained as shown in Fig. 16. That is, among the episodes in Fig. 16, noun pairs, compound episodes, and recursively extended compound episodes are the conversation patterns extracted in this embodiment. Note that a conversation pattern may be one or more of noun pairs, compound episodes, and recursively extended compound episodes.

[0101] <<Examples of conversation patterns using specific conversation data>> Next, an example of a conversation pattern using specific conversation data will be described with reference to Fig. 17. Fig. 17 shows the conversation data that was the basis for creating the conversation pattern. Fig. 17 shows general conversation data as an example. The conversation data in Fig. 17 is conversation data between close university students on the topic of "meals," adopted from the Japanese Topic-Specific Conversation Corpus.

[0102] FIG. 18 shows the nouns and conversation patterns extracted from the conversation data of FIG. (1) The preprocessing unit 1003 performs unnecessary word removal, morphological analysis, etc. on the conversation data 251 to extract a list 252 of nouns such as sushi, neta, salmon, and salmon. (2) The pattern extraction unit 1006 performs EMMA processing on the noun list 252 to extract a conversation pattern 253 of {sushi, topping} {sushi, salmon} {topping, salmon}.

[0103] FIG. 19 shows a list of conversation patterns extracted from a conversation lasting approximately 10 minutes. Each conversation pattern is associated with a number of occurrences. A list of such conversation patterns is stored in a database 907. Two or more nouns are associated with a conversation pattern, and the search unit 904 searches conversation patterns such as those shown in FIG. 19 for a noun that is estimated to be the topic of the conversation, and provides the other noun paired with the noun that matches the search as a topic. For example, if the noun estimated to be the topic of the conversation is "cooking," the search unit 904 determines that [cooking, sushi], [cooking, eating out], or [cooking, salmon] matches the conversation pattern. The search unit 904 obtains one or more of the nouns "sushi," "eating out," and "salmon" (examples of second words) that are not "cooking" (an example of a first word) used in the search, and the topic providing unit 905 provides these to the participants.

[0104] The number of occurrences is the frequency with which a conversation pattern appears in a conversation. Since the number of occurrences tends to increase as the conversation duration increases, it may be converted into the frequency of occurrence per unit time or normalized to a certain range such as 0 to 100.

[0105] Furthermore, the database creation unit 1007 can divide the list of conversation patterns such as that shown in FIG. 19 into conversation patterns with high degrees of association, conversation patterns of unexpected combinations, and conversation patterns including proper nouns.

[0106] FIG. 20 shows a list of highly related conversation patterns. A highly related conversation pattern means that the concepts of the two nouns that make up the conversation pattern are similar. One method for determining whether two nouns are similar in meaning is to represent the nouns as vectors. This representation of a noun using a vector is called a distributed representation. A distributed representation of a noun represents a noun as a fixed-length vector. Because the nouns are vectorized, the distance (similarity) between nouns in vector space can be calculated. The database creation unit 1007 vectorizes the two nouns that make up a conversation pattern, calculates the distance between the nouns, and extracts conversation patterns where the distance is less than threshold A as highly related conversation patterns (threshold A is an example of a first threshold, and being less than threshold A corresponds to a degree of relevance equal to or greater than the first threshold).

[0107] Note that, although BERT (registered trademark), word2vec, etc. are known as technologies for converting words containing nouns into distributed representations, any method of conversion into distributed representations may be used. Furthermore, when a conversation pattern contains three or more nouns, if the relevance of any combination of two of the three or more nouns is high, the conversation pattern is determined to be highly relevant.

[0108] A conversation pattern with a high degree of noun relevance is preferable as a topic to be provided because it contains nouns related to the current topic.

[0109] FIG. 21 shows a list of conversation patterns with unexpected combinations. A conversation pattern with unexpected combinations is one in which the concepts of the two nouns that make up the conversation pattern are significantly different. Whether the concepts of the two nouns are significantly different can also be determined by the distance (similarity) between the nouns in vector space. The database creation unit 1007 vectorizes the two nouns that make up the conversation pattern, calculates the distance between the nouns, and extracts conversation patterns with a distance equal to or greater than threshold B as conversation patterns with unexpected combinations (threshold B is an example of a second threshold, and being equal to or greater than threshold B corresponds to a degree of association being less than the second threshold). Note that threshold B is greater than threshold A.

[0110] In addition, when a conversation pattern includes three or more nouns, if the relevance of any two of the three or more nouns is low, the conversation pattern is determined to be an unexpected combination.

[0111] Unexpected combinations of conversation patterns can change the current topic, and are therefore preferable as topics to offer when the conversation seems to have stalled or reached a dead end.

[0112] 22 shows a list of conversation patterns that include proper nouns. The database creation unit 1007 extracts conversation patterns that include at least one proper noun from the list of conversation patterns. Whether a conversation pattern includes a proper noun is determined by comparing it with a dictionary of proper nouns.

[0113] A further explanation will be given for the case where a proper noun is provided in the conversation topic provision phase. When the topic (noun) estimated in the conversation topic provision phase is a proper noun, the search unit 904 acquires a noun or proper noun associated with the proper noun that is the topic in the conversation pattern. When the topic estimated in the conversation topic provision phase is a noun, the search unit 904 acquires a proper noun associated with the noun that is the topic in the conversation pattern. Alternatively, the participant may set the information processing device 20 to provide only proper nouns. In this case, if the noun acquired by the search unit 904 from the conversation pattern is not a proper noun, the topic is not provided. Alternatively, the participant may set the information processing device 20 to search for a conversation pattern only when the topic is a proper noun.

[0114] It should be noted that a similar proper noun can be provided as a topic even if the exact same proper noun is not in database 907. For example, if a place name (proper noun) related to ramen is in database 907, topic providing unit 905 can search for ramen restaurants in the place names where other participants live and provide a topic about ramen restaurants in the place names where the participants live.

[0115] <Processing for providing topics during conversation> Next, the topic provision phase during conversation will be described in detail with reference to Fig. 23. Fig. 23 is a flowchart illustrating the process in which the information processing device 20 provides a topic in real time. The process in Fig. 23 starts when a participant and another participant start remote communication. In Fig. 23, it is assumed that a topic is provided to a participant.

[0116] First, the acquiring unit 902 acquires speech sounds from the terminal devices 100a and 100b (S31). The acquiring unit 902 determines from which of the terminal devices 100a and 100b the speech sounds have been acquired.

[0117] Next, the speech recognition unit 1002 recognizes the speech of the participant and other participants and converts it into conversation data (S32). The conversation data may be the same as in FIG. 11 and may be divided into conversation sentences.

[0118] The preprocessing unit 1003 removes unnecessary words from the conversation data, and the morphological analysis unit 1004 performs morphological analysis to extract nouns (S33). The removal of unnecessary words and morphological analysis are the same processes as those in the database creation phase.

[0119] Furthermore, even during communication, database creation unit 1007 may extract conversation patterns through EMMA processing and create database 907 (S34). In this way, even when a participant who has never spoken to another participant before is having a conversation for the first time, database 907 will gradually be enriched, making it possible to provide topics of conversation. Note that it is not essential that database 907 be created in real time.

[0120] Next, the score calculation unit 1005 weights the nouns extracted in step S33 and calculates the scores of the nouns (S35). (a) The weight of a noun that has never appeared in the past is "1". (b) The weight of a noun that appears more than once is as follows: If the participant speaks, the current weight + "0.1" - For other participants' utterances, current weight + "0.3" Since nouns contained in the speeches of other participants have a higher weight, it is possible to increase the weight of nouns that other participants are discussing. Note that the score calculation unit 1005 may calculate scores only for nouns contained in the speeches of other participants. Nouns with scores above a threshold are estimated to be the current topic. Note that weights such as 0.1 and 0.3 are just examples.

[0121] Next, the search unit 904 searches the databases 907 for nouns with scores equal to or greater than a threshold (S36). In this embodiment, there are three types of databases 907 (highly related, unexpected combinations, and proper nouns), so participants may select a database 907, or three databases 907 may always be searched for one noun.

[0122] Furthermore, the conversation patterns in the database 907 used for the search may be conversation patterns extracted by EMMA processing from data of past conversations between the participant and other participants. In this case, the database 907 is created in association with the user ID of the participant and the user IDs of the other participants. The search unit 904 identifies the database 907 using the user IDs of the participant who logged in to the information processing device 20 and the other participants.

[0123] The conversation patterns in database 907 used for the search may be conversation patterns extracted from conversation data between the participant currently participating in the communication and other participants, or may be conversation patterns created from general-purpose conversation data such as a corpus.

[0124] The search unit 904 acquires from the conversation patterns that match the searched noun a noun that is not the noun used in the search (S37). For example, if the noun used in the search is a and the matched conversation pattern is {a, b}, the search unit 904 acquires b.

[0125] There may be cases where multiple conversation patterns match the noun used in the search. For example, if the searched noun is a and the matching conversation patterns are {a, b} {a, c} {a, d}, the topic providing unit 905 selects the noun from the conversation pattern that appears most frequently (S38).

[0126] Furthermore, if the conversation pattern that matches the search has three or more nouns, the search unit 904 retrieves one or all of the remaining two or more nouns that are not the noun used in the search. To select one of them, if the database 907 stores conversation patterns with high relevance, the noun with the highest relevance to the noun used in the search can be selected. If the database 907 stores conversation patterns with unexpected combinations, the noun with the lowest relevance to the noun used in the search can be selected.

[0127] The topic providing unit 905 provides a topic by transmitting the noun included in the conversation pattern that is not the noun used in the search to the participant's terminal device 100a (S39). The communication unit 911 of the terminal device 100a receives the topic (noun). The display control unit 914 displays the topic on the conversation screen.

[0128] 23, the search unit 904 can search three databases 907. However, it is also possible to use only one database 907 and, after the search, determine whether the conversation pattern is highly related, a conversation pattern with an unexpected combination, or a conversation pattern containing a proper noun. In this case, the search unit 904 converts the two nouns contained in the conversation pattern that matches the search into a vector representation and determines whether the conversation pattern is highly related or a conversation pattern with an unexpected combination based on the distance between the vectors. The search unit 904 also checks the nouns contained in the conversation pattern that matches the search in a dictionary of proper nouns to determine whether the conversation pattern contains a proper noun.

[0129] <Screen example> Next, a description will be given of the conversation screen 200 displayed by the terminal device 100a. Fig. 24 is an example of the conversation screen 200 displayed by the terminal device 100a. In Fig. 24, the following two topics are provided. "Your conversation partner may be interested in XXX." "Your conversation partner might be interested in 'XXX'." The "XXX" 201 and "XXX" 202 are topics, that is, the nouns in the conversation pattern that are not used for search. Participants can utter conversations such as "Have you eaten XXX recently?" or "I also went to XXX the other day" to other participants.

[0130] The topic providing unit 905 may not only provide topics but also provide conversation examples using the topics. For example, the topic providing unit 905 may provide "XXX is nice, isn't it?" to the terminal device 100a. To create such example sentences, the topic providing unit 905 may use LLMs (Large Language Models).

[0131] As shown in Fig. 25, the conversation screen 200 may simultaneously display three topics retrieved from three databases 907. Fig. 25 shows a conversation screen that displays three topics extracted from three databases 907. The conversation screen 200 in Fig. 25 simultaneously provides three topics: a highly relevant topic 211, an unexpected topic 212, and a topic 213 using a proper noun. Participants can select a topic that is likely to spark a lively conversation from among the different topics and speak to the other participants.

[0132] Also, as shown in FIG. 26, it is preferable that participants can select which topic they would like to be presented with on the conversation screen 200, from among highly relevant, unexpected combinations, or proper nouns. FIG. 26 shows the conversation screen 200 displaying a database selection dialog 221. The database selection dialog 221 has a message 222 saying "You can select the topic to search," as well as check boxes associated with "highly relevant," "unexpected," and "contains proper nouns." Participants can select a database 907 (i.e., a topic to search) by checking one or more check boxes. The search unit 904 searches only the database 907 selected by the participant.

[0133] <Major Effects> The information processing system of this embodiment searches for conversation patterns in the current topic and provides nouns associated with the topic as topics, so it can provide topics that expand on the current topic (highly relevant topics) or change the direction of the topic (unexpected topics).In addition, the information processing system can provide proper nouns as topics, so it can provide specific topics.

[0134] [Other application examples] The best mode for carrying out the present invention has been described above using examples, but the present invention is not limited to these examples in any way, and various modifications and substitutions can be made within the scope that does not deviate from the gist of the present invention.

[0135] For example, in this embodiment, the conversation pattern is a list of two or more nouns, but the conversation pattern may also be composed of a list of two or more adjectives, a list of two or more adjectival verbs, a list of two or more adverbs, etc.

[0136] Furthermore, a conversation pattern may be a phrase including a noun, an adjective, or an adjectival verb. In other words, the pattern extraction unit 1006 provides topics in units of phrases including nouns, etc. Participants can easily use the nouns, etc. in conversations because they can understand the context in which the nouns, etc. are used.

[0137] In addition, in this embodiment, conversation patterns are extracted using EMMA processing, but conversation patterns may be created using a method different from or similar to EMMA processing. In other words, conversation patterns may be created using any method that can extract one or more of the following: conversation patterns with high correlation, conversation patterns of unexpected combinations, and conversation patterns containing proper nouns.

[0138] For example, the information processing device 20 may use a server that executes natural language tasks using LLMs (Large Language Models). The information processing device 20 transmits a prompt, such as "Please extract highly related noun pairs, unexpected combinations of nouns, and noun pairs containing proper nouns from this conversational text." The server extracts highly related conversational patterns, unexpected combinations of conversational patterns, and conversational patterns containing proper nouns from the conversational text and returns them to the information processing device 20.

[0139] The same applies to topic extraction, and the information processing device 20 can transmit conversation data progressing in real time to a server that executes a natural language task, and receive the current topic in return.

[0140] Furthermore, the functional configuration examples in Fig. 7 and the like are divided according to main functions to facilitate understanding of the processing by the terminal device 100 and the information processing device 20. The present invention is not limited by the way in which the processing units are divided or the names of the processing units. The processing of the terminal device 100 and the information processing device 20 can also be divided into more processing units depending on the processing content. Furthermore, one processing unit can also be divided so that it includes more processes.

[0141] Additionally, the devices described in the examples are merely illustrative of one of several computing environments for implementing the embodiments disclosed herein. In one embodiment, information processing apparatus 20 includes multiple computing devices, such as a server cluster, configured to communicate with each other via any type of communication link, including a network, shared memory, etc., and to perform the processes disclosed herein.

[0142] Furthermore, the information processing device 20, the terminal device 100, and the administrator terminal 101 can be configured to share the disclosed processing steps, for example, the processes shown in Figures 10, 12, and 23, in various combinations. For example, a process executed by the information processing device 20 can be executed by the terminal device 100 or the administrator terminal 101. Also, a process executed by the terminal device 100 and a process executed by the administrator terminal 101 can be executed by the information processing device 20. Furthermore, each element of the information processing device 20 may be integrated into one device or may be divided into multiple devices.

[0143] Each function of the above-described embodiments can be realized by one or more processing circuits. Here, the term "processing circuit" in this specification includes a processor programmed to perform each function by software, such as a processor implemented by an electronic circuit, as well as devices such as an ASIC (Application Specific Integrated Circuit), a DSP (Digital Signal Processor), an FPGA (Field Programmable Gate Array), and conventional circuit modules designed to perform each of the above-described functions.

[0144] <Aspect> [Claim 1] an acquisition unit that acquires speech sounds of participants participating in communication and other participants; a speech recognition unit that converts the uttered voice into conversation data by speech recognition; a search unit that searches for a conversation pattern in which two or more different words are associated with a first word included in the conversation data; a topic providing unit that provides a conversation screen that displays a second word associated with the first word in the conversation pattern that includes the first word that matches the search; An information processing system having the above. [Claim 2] The conversation pattern is associated with an occurrence frequency of the conversation pattern in the conversation; When a plurality of conversation patterns match the search, the search unit acquires the second word from the conversation pattern with the highest occurrence frequency. 2. The information processing system according to claim 1. [Claim 3] a score calculation unit that calculates a score for each of the first words, the score increasing each time the first word appears in the conversation data; the score calculation unit calculates the score by weighting the first words uttered by the other participants more heavily than the first words uttered by the participant; the search unit searches the conversation pattern for the first word whose score is equal to or greater than a threshold. 3. The information processing system according to claim 1 or 2. [Claim 4] The conversation pattern is associated with two or more words whose relevance is equal to or greater than a first threshold value, the topic providing unit provides a conversation screen that displays the second words whose relevance to the first words is equal to or greater than a first threshold value; 4. The information processing system according to claim 1. [Claim 5] The conversation pattern stores two or more words associated with each other and having a degree of association less than a second threshold value, the topic providing unit provides a conversation screen that displays the second word whose relevance to the first word is less than a second threshold value; 4. The information processing system according to claim 1. [Claim 6] The conversation pattern stores two or more words, at least one of which is a proper noun, in association with each other; the topic providing unit provides a conversation screen that displays, when the first word is a proper noun, the second word, which is a noun or a proper noun associated with the first word; providing a screen during a conversation that displays the second word, which is a proper noun associated with the first word, when the first word is not a proper noun; 4. The information processing system according to claim 1. [Claim 7] The conversation pattern is associated with two or more words whose relevance is equal to or greater than a first threshold, two or more words whose relevance is less than a second threshold, and two or more words, at least one of which is a proper noun; the topic providing unit provides a conversation screen that simultaneously displays the second words whose degrees of association with the first words are equal to or greater than a first threshold, the second words whose degrees of association with the first words are less than a second threshold, and the second words that are proper nouns; 4. The information processing system according to claim 1. [Claim 8] the topic providing unit displays, as a topic, the second word whose relevance to the first word is equal to or greater than a first threshold value; Whether the second word whose relevance to the first word is less than a second threshold is displayed as a topic; and Displaying the second word, which is a proper noun associated with the first word, as a topic; providing the in-conversation screen for receiving one or more selections of: 8. The information processing system according to claim 1. [Claim 9] a pattern extraction unit that extracts the conversation pattern by performing EMMA (Episode Mining using Memory Anchor) processing on any conversation data and associating two or more words; a database creation unit that creates a database in which a plurality of conversation patterns extracted by the EMMA process are stored; 9. The information processing system according to claim 1. [Claim 10] The conversation patterns include conversation patterns extracted by EMMA processing from conversation data that has been previously conducted between the participant and the other participant; the search unit searches for the conversation pattern created from conversation data that has been conducted in the past between the participant currently participating in communication and the other participant; 10. The information processing system according to claim 9. [Claim 11] the pattern extraction unit creates the conversation pattern from conversation data between the participant currently participating in communication and the other participant; the search unit searches for the conversation patterns created during communication. 10. The information processing system according to claim 9. [Claim 12] the first word and the second word registered in the conversation pattern are nouns; 12. The information processing system according to claim 1. [Explanation of symbols]

[0145] 1. Information Processing Systems 2. Communication Network 10 Conference Server 20 Information processing equipment 100, 100a, 100b, 100c, 100d terminal devices 101 Administrator terminal (terminal device used by administrator) [Prior art documents] [Patent documents]

[0146] [Patent Document 1] Japanese Patent Application Laid-Open No. 2015-207905

Claims

1. an acquisition unit that acquires speech sounds of participants participating in communication and other participants; a speech recognition unit that converts the uttered voice into conversation data by speech recognition; a search unit that searches for a conversation pattern in which two or more different words are associated with a first word included in the conversation data; a topic providing unit that provides a conversation screen that displays a second word associated with the first word in the conversation pattern that includes the first word that matches the search; and An information processing system having the above.

2. The conversation pattern is associated with an occurrence frequency of the conversation pattern in the conversation; When a plurality of conversation patterns match the search, the search unit acquires the second word from the conversation pattern with the highest appearance frequency. The information processing system according to claim 1 .

3. a score calculation unit that calculates a score for each of the first words, the score increasing each time the first word appears in the conversation data; the score calculation unit calculates the score by weighting the first words uttered by the other participants more heavily than the first words uttered by the participant; the search unit searches the conversation pattern for the first word whose score is equal to or greater than a threshold.

3. The information processing system according to claim 1 or 2.

4. The conversation pattern is associated with two or more words whose relevance is equal to or greater than a first threshold value, the topic providing unit provides a conversation screen that displays the second words whose relevance to the first words is equal to or greater than a first threshold value; The information processing system according to claim 1 .

5. The conversation pattern stores two or more words having a degree of association less than a second threshold in association with each other, the topic providing unit provides a conversation screen that displays the second word whose relevance to the first word is less than a second threshold value; The information processing system according to claim 1 .

6. The conversation pattern stores two or more words, at least one of which is a proper noun, in association with each other; the topic providing unit provides a conversation screen that displays, when the first word is a proper noun, the second word, which is a noun or a proper noun associated with the first word; providing a conversation screen that displays the second word, which is a proper noun associated with the first word, when the first word is not a proper noun; The information processing system according to claim 1 .

7. The conversation pattern is associated with two or more words whose relevance is equal to or greater than a first threshold, two or more words whose relevance is less than a second threshold, and two or more words, at least one of which is a proper noun; the topic providing unit provides a conversation screen that simultaneously displays the second words whose degrees of association with the first words are equal to or greater than a first threshold, the second words whose degrees of association with the first words are less than a second threshold, and the second words that are proper nouns; The information processing system according to claim 1 .

8. the topic providing unit displays, as a topic, the second word whose relevance to the first word is equal to or greater than a first threshold value; Whether the second word having a degree of association with the first word less than a second threshold is displayed as a topic; and Displaying the second word, which is a proper noun associated with the first word, as a topic; providing the in-conversation screen for receiving one or more selections of: The information processing system according to claim 1 .

9. a pattern extraction unit that extracts the conversation pattern in which two or more words are associated with each other by performing EMMA (Episode Mining using Memory Anchor) processing on any conversation data; a database creation unit that creates a database in which a plurality of conversation patterns extracted by the EMMA process are stored; The information processing system according to claim 1 .

10. The conversation patterns include conversation patterns extracted by EMMA processing from conversation data that has been previously conducted between the participant and the other participant; the search unit searches for the conversation pattern created from conversation data that has been conducted in the past between the participant currently participating in communication and the other participant; The information processing system according to claim 9 .

11. the pattern extraction unit creates the conversation pattern from conversation data between the participant currently participating in communication and the other participant; the search unit searches for the conversation patterns created during communication. The information processing system according to claim 9.

12. the first word and the second word registered in the conversation pattern are nouns; The information processing system according to claim 1 .

13. an acquisition unit that acquires speech sounds of participants participating in communication and other participants; a speech recognition unit that converts the uttered voice into conversation data by speech recognition; a search unit that searches for a conversation pattern in which two or more different words are associated with a first word included in the conversation data; a topic providing unit that provides a conversation screen that displays a second word associated with the first word in the conversation pattern that includes the first word that matches the search; and An information processing device having the above.

14. An acquisition process for acquiring speech sounds of the participants participating in the communication and other participants; a speech recognition process for converting the uttered voice into conversation data by speech recognition; a search process for searching for a conversation pattern in which two or more different words are associated with a first word included in the conversation data; a topic provision process for providing a conversation screen that displays a second word associated with the first word in the conversation pattern that includes the first word that matches the search; An information processing method executed by a computer.

15. An acquisition process for acquiring speech sounds of the participants participating in the communication and other participants; a speech recognition process for converting the uttered voice into conversation data by speech recognition; a search process for searching for a conversation pattern in which two or more different words are associated with a first word included in the conversation data; a topic provision process for providing a conversation screen that displays a second word associated with the first word in the conversation pattern that includes the first word that matches the search; A program that causes a computer to execute the following.

Images